A new nondestructive testing method is proposed to fast scan and detect the macroscopic defects of materials using the variation characteristics of the differential permeability of ferromagnetic materials. Firstly, the ferromagnetic material, such as steel plate, is placed under the bias magnetic field, and the defects in the material will inevitably lead to the distortion of its internal magnetic flux. Secondly, the magnetic flux distortion is further reflected in the difference of the differential permeability distribution on the surface of the tested material. Thirdly, using a specially designed probe, the distribution of differential permeability on the surface of tested material is rapidly scanned. Finally, from the difference of the differential permeability distribution, the discontinuity information in the material can be obtained, so as to realize the nondestructive testing of macroscopic defects. Series of experiments were carried out, and the results show that, compared with traditional magnetic flux leakage method, the new method has the advantages of low magnetic intensity, less flux leakage and stable detection signals. Moreover, the detection signals for the frontside and backside defects have obviously different distribution characteristics, and the method also has obvious advantages in the field of defect depth identification.