Recognition of steel surface defect is essential to promote the improvement of steel production quality. However, the traditional image processing and recognition methods have low accuracy and are easily affected by factors such as light. However, the emerging algorithms based on deep learning have problems such as large amount of model parameters and difficulty in deployment, which cannot be widely used in practical production. In this article, a lightweight partial depth mixture separable network ( PDMSNet) is proposed, which is small model size and floating-point operations (FLOPs) and is easy to deploy on resource-constrained platforms. The test results of the standard strip steel surface defect data set NEU-CLS show that the performance of the proposed model is better than other defect classifiers in strip steel surface defect detection. The recognition accuracy reaches 99. 78% , and the number of parameters is only 0. 17 M and 272 M FLOPs. The average time of an image recognition on a single low-end GeForce MX 250 GPU is 0. 47 ms, which can meet the requirements of real-time detection in industrial field.