Aiming at the problems of low recognition rate of surface defects in industrial aluminum product and inaccurate location of small defects, etc. of the traditional detection algorithm, an improved deep learning network called Faster RCNN is proposed to detect 10 kinds of aluminum product surface defects. Firstly, after the data is enhanced, the feature pyramid network (FPN) structure is added to the backbone network to enhance the feature extraction ability for small defects, and then the ROI Align algorithm is used to replace ROI Pooling algorithm to obtain more accurate defect location information. Finally, the K-means algorithm is added to cluster the defect data to obtain an anchor more suitable for aluminum product defects. The experiment shows that the mean of the average precision (mAP50) of the improved network for aluminum product surface defect detection is 91. 20% , which is 16% higher than that of the original Faster RCNN network, and the detection ability of aluminum product small defects is also improved obviously.