The performance of an analog-to-digital converter (ADC) determines the quality of the entire acquisition system. As system sampling rates and bandwidths increase, nonlinear errors become significantly more detrimental than linear errors. To address frequency-dependent nonlinear errors in ADCs, this paper proposes a novel digital post-calibration method based on an artificial neural network (ANN). This method first conducts spectral analysis on the single-tone signal sampled by the ADC to remove nonlinear harmonics, and takes this result as the reference true value to train the artificial neural network from the perspective of the frequency domain. The proposed method takes the amplitude information of the signal spectrum as the training object of the artificial neural network, retains the phase information, and compensates the output amplitude result after calibration. The amplitude result is reconstructed with the retained phase information to form a complex frequency spectrum, and an inverse Fourier transform is performed to restore the time-domain signal. For experimental validation, a time-interleaved (TI) ADC system is utilized as the application scenario, where various inter-channel mismatch errors and the non-linear error of the ADC are jointly calibrated. Using multiple sets of single-tone signals with different frequencies, the data is divided and the neural network training is conducted by combining the strategy of frequency-layered sampling and time-segmented sampling. Furthermore, the generalization performance of the multi-tone signals is also verified. The universality of the proposed method in ADC nonlinear calibration applications is demonstrated. The network is also verified on a 4-channel TIADC hardware platform with a sampling rate of 20 GSPS, resulting in an overall improvement in the spurious-free dynamic range of the system by about 36 dB while ensuring that the multi-tone signals could maintain the correct phase relationship.