In this paper a simple audio visual information based speaker tracking technique is proposed for indoor environment. Specifically, a Kalman filter based image processing technique is used to extract visual information, and Fast Fourier Transform (FFT) based approach is used to extract audio information for speaker tracking. Finally, a decision tree has been used to estimate the location of the speaker based on audio and visual information. One of the main advantages of the proposed technique is the use of a built-in microphone of the tracking camera; which makes this technique cost effective and simple. We have examined our method with case studies from the online SPEVI database. The proposed technique shows the best detection and works properly even when the speaker is not visible. © 2006-2016 Asian Research Publishing Network (ARPN).