The relationship between the pointing error and the underlying structural parameters of Alt-Azimuth photoelectric telescopes, which are crucial instruments in astronomical observations and laser communications, has not been fully understood. To establish the connection between the underlying structural parameters and the pointing error, and to provide a theoretical basis for the design and compensation of instrumental errors, this paper first introduces the typical Alt-Azimuth structure and proposes the classification and definition of two-level error sources for pointing errors. A top-down all-link pointing error model, which extends from the underlying structural parameters to the system’s optical axis, is developed using error synthesis theory. The proposed model is then verified using open calibration data from the literature. Subsequently, a sensitivity analysis is conducted on nine processing parameters and nine design parameters within the proposed all-link pointing error model. The results indicate significant variations in the contributions of different underlying structural parameters to pointing errors. It is determined that these structural parameters should be designed differently according to their sensitivities. Furthermore, it is found that the structural parameters significantly influence the nonlinear effects of pointing errors. Finally, based on the proposed model, the pointing error of a 500 mm aperture laser communication Alt-Azimuth photoelectric telescope is compensated. The maximum values of the azimuth and pitch components of the pointing error are reduced from 218. 83″ and - 208. 66″ to 16. 45″ and 6. 89″, respectively, which is more than 12 times better. The all-link pointing error model proposed in this paper reveals the underlying structural factors of pointing errors and their mechanisms. It demonstrates the superiority of the model for error compensation and provides a theoretical basis for the design of instrumental errors and systematic error compensation of Alt-Azimuth photoelectric telescopes, which is of great significance.