There are great nonlinearity and uncertainty in the Global Positioning System/Inertial Navigation System (GPS/INS) integrated navigation system composed of low cost devices. In order to improve this issue, a filtering method with Sliding Mode Observer (SMO) introduced is proposed in this paper. Firstly, in this method, the integrated navigation system model is established, the calculation process of Extended Kalman Filter (EKF) is introduced and its shortcomings are analysed. Then, the basic principle of SMO is introduced, and the Sliding Mode Observer is constructed according to the system. Finally, the implementation procedure of the EKF integrated navigation algorithm with SMO is explained. The SMO integrates the model error, state estimation and mean variance into EKF algorithm to correct the output of the system. The trajectory simulation experiment and vehicle test prove that the proposed method is superior to the traditional EKF method, and has higher filtering accuracy. In the vehicle test, when the satellite signal is out of lock for 15 s, compared with those of the EKF method, the eastbound and northbound position errors are reduced by 53%, 37%, respectively. The result proves that the proposed method can effectively suppress the error divergence of GPS/INS integrated navigation, and provides certain reference value for future engineering practice.