The number of smart phones being fitted with accelerometers and other motion sensitive sensors are increasing by time as the technologies continue to grow. Currently this sensor is used in providing physical gesture interaction in a gesture-based application or software. The function of this new extendibility in mobile phone can be practice in the lecture surrounding. For this research, it is to study the use of mobile phone in the learning environment and proposing interactive learning based on handwriting recognition. It will create a new wayofinteractionduringlecturesessioninvolvingthe mobile phone by using the gesture applied on it to be recognized as the handwriting recognition. Motion will be applied on the phone as the medium of interaction between the lecturer and the student where the gesture applied will be recognized and projected on the lecture hall's screen. Thus, it is believed to help the lectures to be more interactive with the students and create a new way of learning in a lecture environment. © 2010 IEEE.