To address the challenges of extracting defect features and low signal-to-noise ratio (SNR) of conventional ultrasonic phased array in the detection of highly attenuated thick-walled structures, an ultrasonic signal processing method based on the improved sparse representation is proposed to solve the problems in this article. Firstly, the ultrasonic full matrix capture is preprocessed to obtain segmented signals and their corresponding time-frequency parameters. Building upon the parameters, the adaptive Gabor sub-dictionaries are constructed. Then, the segmented signals are sparsely decomposed and reconstructed by the improved support matching pursuit algorithm. Compared with the conventional orthogonal matching pursuit algorithm, the improved support matching pursuit algorithm combines the adaptive Gabor sub-dictionaries and l p -norm (0 < p < 1). Atoms of the sub-dictionaries are optimized to better match the characteristics of ultrasonic signals, while a more flexible approach to measuring sparsity is provided by l p -norm. They reconstruct ultrasonic signals more accurately through compact dictionaries, enhancing the quality of reconstructed signals. Finally, the total focusing method is applied to the processed full matrix capture to produce ultrasonic images. Experimental results show that the improved sparse representation can accurately extract defect signals and achieve high SNR, improving the ultrasonic image quality of internal defects contained in highly attenuated thick-walled structures.