To improve the accuracy and robustness of the speaker tracking system in noisy and reverberant environments, an adaptive unscented particle filter (AUPF) algorithm based on multi-feature is proposed. The multi-feature of the speech signal is regarded as the observation information in this algorithm, where the multi-hypothesis and frequency selection function is applied to the mechanisms of time delay selection and beam output energy optimization. Subsequently, the likelihood function is constructed by combining these two mechanisms, which makes up for the deficiency that noise and reverberation cannot be restrained simultaneously by a single feature. Considering the randomness of speaker motion, a new proposal distribution is utilized in the particle filter algorithm, which combines the unscented Kalman filter (UKF) and the robust estimation theory based on the adaptive constant speed model to improve the adaptability of the model. The simulation and experimental results show that based on AUPF, the position average RMSE of multi feature algorithm is reduced by more than 18% compared with that of SBFSRP, and under multi-feature observation, the position average RMSE of AUPF algorithm is reduced by more than 14% compared with that of CV algorithm. It has the characteristics of high tracking accuracy and strong numerical stability.