In order to detect weak signal with unknown frequency, this paper presents a novel method for detecting the unknown frequency signal by combining the variance peak value of the Duffing oscillator and genetic algorithm (GA). Firstly, the impact of periodic driving signal with different frequencies, initial phase angle and noise on the system operating state are analyzed. Second, the corresponding relationship between the system output variance and the system running state is studied. The effect of the test signal frequency, as well as the phase difference between the periodic driving signal and the test signal on the state variables variance and the state transition time are discussed. Finally, the new method is proposed, which using a number of Duffing oscillators with different initial phases to cover all phase signal detection. Moreover, the proposed method combines with genetic algorithm to obtain the detected signal frequency by optimizing the calculation of peak value of system output varaince under different input frequency signals. This method resolves the limitation of existing weak signal detection method based on the chaotic oscillator, in which the signal frequency needs to be known. Experimental results show that the method can accurately and flexibly detect the frequency of the test signal and has strong adaptability. This workt provides a novel approach for the weak signal detection.