Objective To solve the problems of the machine-picked fresh tea leaves mixing with different grades of tea leaves, high mixing degree and low classification accuracy of physical characteristics.Method Using the random forest classification model, a method based on the fusion of color and edge feature was proposed. We collected three different grades of fresh tea leaves, and processed the original images with cropping, size normalization and denoising, and then extracted the color features and edge features. Through parameter modification and testing, the optimal random forest classification model was constructed, and the comparison experiment was performed with the K-nearest neighbor and SVM classifier.Result After feature fusion, the classification accuracy of random forest model reached 99.45%, which was 7.14 and 9.34 percentage points higher than those of single color feature and single edge feature, 15.38 and 5.49 percentage points higher than those of K-nearest neighbor model and SVM classifier respectively.Conclusion The established method can accurately separate single bud, one bud and one leaf, and one bud and two leaves of fresh tea leaves.