Noise reduction is normally carried out in time domain and works well with the real-valued signal. The reduction of noise can also be performed in frequency domain. However, the signal must be transformed into complex random variable. The widely linear distortionless (WLD) filter has already been used for the speech enhancement. The WLD filter utilizes the Short Term Fourier Transform (STFT) to transform real valued signals into complex random variables in the frequency domain. The designed filter exclusively depends on the non-circularity coefficient of the non-stationary signal. Moreover, WLD filter works well even in the very noisy environments where the input signal-to-noise ratio (SNR) is far down below 0 dB. Visual Evoked Potential (VEP) is known as non-stationary signal and its characteristic shows that this signal also has very low SNR such as 10 dB. In this paper, we have proposed the frequency domain based WLD filter for VEP estimation. The results demonstrate that the WLD filter has been able to better estimate the VEP signals, as it effectively suppressed the unwanted VEP signal peaks. Moreover, it also raised the amplitudes of the desired peaks to considerable levels for their easy detections. © 2011 IEEE.