The mechanical fault of the contact system for a conventional circuit breaker is a process from slight to severe. The accurate identification of its operating state can greatly improve the reliability of the circuit breaker. In this article, a single signal input and multi-task output MTL-SEResNet model is proposed for fault diagnosis and degree evaluation. Firstly, the raw vibration signals of the contact system are analyzed using a continuous wavelet transform. And the corresponding two-dimensional time-frequency images are obtained. Secondly, the improved ResNet18 network is combined with the SENet structure, and the multi-task learning sharing mechanism is used to formulate the MTL-SEResNet model. The model is optimized by adjusting the weight ratio of the two task loss functions for fault classification and degree evaluation. Finally, the proposed method is verified by experiments with simulated fault data of the contact system. The results show that the proposed model has better performance with 99. 78% and 99. 36% accuracy in type and degree, respectively, which can effectively evaluate the fault degree of the conventional circuit breaker.