The magnetic compensation process of single beam SERF atomic magnetometer has the problem of three-axis magnetic field coupling. To address this issue, a method of reverse calibration of three-axis sequential compensation value of magnetometer is proposed to reduce the coupling magnetic field in the process of three-axis compensation. Firstly, a three-axis magnetic field coupling mathematical model is formulated, which is a 3×3 to describe the magnetic field coupling relationship between three axes. The triaxial coupling coefficient of the laboratory magnetometer prototype is tested. Then, a method of reverse calibration of triaxial sequence compensation value by using the triaxial coupling coefficient is proposed. Finally, the magnetic compensation effects before and after calibration are compared. Experimental results show that after the reverse calibration of triaxial sequential compensation, the average response linewidth of the magnetometer is narrowed by 2~ 10 Hz and the sensitivity is improved by 3~ 5 fT/ Hz 1/ 2 . The effectiveness of this method is verified, which lays a foundation for further optimizing the magnetic compensation technology.