Based on the statistical moment theory derived from the single degree of freedom system, aiming at the large gap betweenidentifying the structure damage with model-free method and the practical application, ineorporates the generalized patterm searchalgorithm,proposes a new two-step hierarchieal rapid damage detection method for frame strueture, which takes fusing thefourth-order displacement moment and the eighth-order aceeleration moment as the index. The generalized pattern search algorithmcombines with the model-free rapid damage location theory to form a new damage deteetion method proposed. The numeriealsimulations of the 12-story frame structure numerical model with or without noise were carried out, and the deviations from thestandard results are all within 3%, which proves the accuracy and the anti-noise effect of this method. This method was comparedwith the two-norm optimization method and the quasi-Bayesian thinking method of statistical moments under the same index, theresults verify the superiority of the new method in accuracy, stability and fast deteetion calculation time of damage detection. Withthe test data of the 12-story standard frame vibration table, three typical working cases were selected for damage diagnosis, and theresults were compared with the actual test report and analyzed, the results show that compared with other methods, this method canrefleet the accumulated damage level change caused by the test working case accumulation better, which highlights the reliability of the new method in engineering detection.