The rate-distortion optimization plays a key role in video coding, which aims to achieve a tradeoff between compression efficiency and video quality distortion. The existing rate-distortion optimization algorithms mainly aim to eliminate time and space redundancy, which ignore subjective perception of video content and result in a large amount of perceptual redundancy in video. To address these issues, a rate distortion algorithm based on visual perception is proposed in this article. Firstly, the Lagrangian multiplier factor is obtained based on the data-driven just noticeable distortion prediction mode, which is more in line with the perception of the human eye. Secondly, the Lagrangian multiplier weight coefficient is based on salient model. Finally, the fusion of two models is applied to rate-distortion optimization, and SW-SSIM is used to evaluate video quality and achieve perceptual video coding optimization. Compared with the third generation audio and video coding standard algorithm, experimental results show that the proposed algorithm reduces bitrate by 12. 15% averagely, and the salience weighted-structural similarity index metric increases by 0. 004 3. Furthermore, the proposed algorithm reduces the perceptual redundancy in video content, and improves the video perceptual quality and coding performance.