The blossom images of traditional Chinese medicinal herbs were captured by Google search. The images were divided into twelve categories: (1) Syringa, (2) Bombax malabarica, (3) Michelia alba, (4) Armeniaca mume, (5) Albizia julibrissin, (6) Pinus massoniana, (7) Eriobotrya japonica, (8) Styphnolobium japonicum, (9) Prunus persica, (10) Firmiana simplex, (11) Ficus religiosa and (12) Areca catechu. The number of original images is 1716, and the total number of images acquired after data augmentation was 12538. The dataset provide a collection of blossom images on traditional Chinese herbs help Chinese pharmacist to classify the categories of Chinese herbs. In addition, the dataset can serve as a resource for researchers who use different algorithms of machine learning or deep learning for image segmentation and image classification.