To address the issue of poor tracking performance caused by target appearance changes during tracking, a target tracking method under appearance change based on improved ToMP network is proposed. Firstly, a target appearance state discrimination module was added to the ToMP network, using a cascaded Long Short-Term Memory (LSTM) network to output the target appearance state discrimination information. Secondly, the online sample storage criterion of the network was improved by adding the normal appearance discrimination information to the confidence score-based evaluation. This optimizes the model weights using reliable samples, enhancing the network′s classification ability for the target. Then, the mechanism for utilizing online samples during appearance changes was improved, updating the model weights with the latest samples to enhance classification performance for newly appeared targets. Finally, a center point trajectory prediction was used to weight the target response score generated by the network, improving the target feature mapping while reducing interference from similar objects, thereby stabilizing target tracking. The accuracy on public datasets reached 93.9% and 68.9%, respectively, outperforming other methods and feasibility is further confirmed through robotics experiments.