The human face is an important human body part which plays an extraordinary role in the human to human or human to machine communications. As such, it is important to design robust emotion detection system for real world applications like human decision making and effective human computer interaction. Facial expression provides non-verbal communication for human computer interactions. This study identifies the problem of loss of data in the feature extraction scheme based on limited number of positions of facial muscles. To improve detection performance, relative sub-image based features are proposed. Classifications have been done using the support vector machine to implement an automated emotion detection system for facial expressions. The results show that the proposed relative sub-image based features enhance the classification rates. © 2012 IEEE.