To address uncertainties which have influence on trajectory tracking of quadrotor unmanned aerial vehicle (UAV), such as internal and external disturbances, model errors, this article proposes an integral sliding mode control program based on the improved extended state observer. In particular, we firstly regard the uncertainty factors in the quadrotor UAV system, including model error, internal and external disturbances as lumped disturbances. The improved extended state observer is used to observe them. Furthermore, on this foundation, the control continuity of the quadrotor UAV system additionally is considered. An integral sliding mode controller is designed, which is based on trajectory error, velocity error, attitude angle error and attitude angular velocity error of the quadrotor UAV. The system stability is analyzed. The numerical simulation and real machine experiment are conducted. Results show that the tracking errors of each state in numerical simulation do not exceed 1% and the tracking accuracy is the highest. In actual machine experiments, the position tracking error can be controlled below 20% by using the proposed algorithm. For this reason, the proposed method in this article is efficacious and workable.