Structural displacement response is the basic data for structural health monitoring (SHM) and safety condition evaluation. To make full use of the advantages of vision-based sensors, overcome its shortcomings, and improve the accuracy of structural displacement, an accurate structural displacement monitoring method by fusing tilt photogrammetry-based and accelerometer measurement is proposed in this article. On the one hand, the dynamic displacement at higher frequencies is reconstructed by the measured acceleration. On the other hand, the acceleration reconstructed and vision-based displacements in the same frequency band can be used to calculate the timedomain variable scaling factor, which can reduce the error caused by the conversion from pixel displacement to real displacement. Experimental tests on a self-anchored suspension model bridge and field tests on a simply supported beam bridge are carried out to explore the efficiency of the proposed method. The results of experimental tests show that, compared with the linear variable displacement transducer results, the maximum normalized root mean square error of the proposed method is 2. 70% which is about 60% lower than the traditional vision-based approaches. The proposed method can promote the further application of computer vision in the field of bridge structural health monitoring.