To address the problem that the traditional milling surface roughness prediction method relies excessively on signal processing knowledge to extract features and has low prediction accuracy, a surface roughness prediction method based on a deep residual shrinkage network improved by the inception module ( IDRSN) and a bidirectional long-short-term memory network ( BiLSTM) is proposed. Firstly, the input signal is noise reduced using the soft thresholding structure and attention mechanism in the deep residual shrinkage network. Secondly, the Inception module is introduced to build IDRSN to enhance the multiscale information acquisition capability of the network for adaptive multiscale feature extraction. Then, a bidirectional recurrent network structure is introduced to construct a BiLSTM prediction network, which utilizes both positive and negative LSTM to improve the network′s ability to capture complete information about the past and the future. Finally, experiments verify the effects of four methods of extracting features, IDRSN, DRSN, BiLSTM and manually extract features, and the prediction accuracy of four surface roughness prediction models, BiLSTM, CNN, DRSN, and CNNLSTM, are compared respectively. It is shown that the proposed method has a high prediction accuracy and establishes a method basis for surface roughness prediction of milling machining.