A siamese network model is proposed for fault diagnosis of rolling bearings under small samples and strong noise. First, a series of time-frequency images are obtained from fault signals by the continuous wavelet transform, and the convolutional neural network is introduced to realize the pattern recognition. Secondly, the small samples are recombined with each other to form new sample pairs through cross matching. Thus, the number of fault samples are increased dramatically. Thirdly, a siamese network model including two sub-models is formulated, which uses the new sample pairs. Finally, a new classifier is designed for the siamese network model to realize fault classification with small samples under strong noise. The proposed faulty diagnose method is evaluated by using fault samples from both noise database and experimental measurement. The accuracy vaules are 96. 25% and 97. 08% , respectively. Results show that one fault can be identified by the proposed siamese network model using only 20 samples, which is less than the samples required by CNN model to reach a similar accuracy.