To reduce the random error of the microelectromechanical system (MEMS) gyroscope, a new method combining the improved empirical mode decomposition (EMD) with the traditional modeling and filtering method is proposed. Firstly, the traditional EMD algorithm is used to decompose the signal into a finite number of intrinsic mode functions (IMF). Based on Pearson correlation coefficient criterion and statistics of noise, a screening mechanism is proposed to divide IMFs into three categories, including noise dominated IMFs, mixed IMFs, and signal IMFs. Then, the timeseries model of the mixed IMFs is formulated. Kalman filtering algorithm is fitted after the modeling is finished. Finally, the mixed IMFs after modeling and filtering and signal IMFs are reconstructed to obtain the denoised signal. Experimental analysis results show that the proposed method has obvious advantages in suppressing the effect of random errors, which can significantly improve the signal quality and the accuracy of the inertial navigation system.