The high-speed camera tends to produce noise when it works under an ultra-high frame rate (>10 000 FPS). The distribution of the produced noise is complex. Thus, it is very difficult to obtain the noisy-clean image pairs. To address this problem, a training scheme for the convolutional denoising networks based on the non-ideal noisy-clean pairs is proposed. Firstly, the cameras with high and low frame rates are leveraged to capture the images of the same scenes, through which the noisy images and the corresponding non-ideal paired clean images are achieved. Then, a deep denoising model based on a convolutional neural network is formulated to achieve supervised learning with the non-ideal noisy-clean image pairs by utilizing the brightness consistency and image alignment methods, contributing to removing imaging noise. Finally, a model quantization technique is introduced to quantize the values of parameters and activations, which helps them to transform the 32 bit floating-point number to an 8 bit fixed-point number, and thus greatly reduces the model size, memory consumption, and running time. Experimental results show that the proposed denoising method can effectively remove the imaging noise of a high-speed camera. Compared with other methods, the peak signal-to-noise ratio and structural similarity of the denoised image are improved by at least 1. 96 dB and 1. 95% , respectively. In addition, with the help of the model quantization technique, the model size is reduced by 4 times, and the memory consumption and running time are decreased by 45. 62% and 37. 5% , respectively.