Due to the ubiquitous ness of the digital media including broadcast news, documentary videos, meeting, movies, etc. and the progression in the technology and the decreasing outlay of the storage media leads to an increase in the data production. This explosive proliferation of the digital media without appropriate management mimics its exploitation. Presently, the multimedia search and retrieval are an active research dilemma among the academia and the industry. The online data repositories like Google, You Tube, Flicker, etc. provides a gigantic bulk of information but findings and accessing the data of interest becomes difficult. Due to this explosive proliferation, there is a strong urge for the system that can efficiently and effectively interpret the user demand for searching and retrieving the relevant information. In order to cope with these problems, we are proposing a novel technique for automatic query interpretation known as the Semantic Query Interpreter (SQI). SQI interprets the user query both lexically and semantically by using open source knowledge bases i.e. WordNet and ConceptNet. Effectiveness of the proposed method is explored on the open-benchmark image data set the LabelMe. Experimental results manifest that SQI shows substantial rectification over the traditional ones. © 2011 IEEE.