Machine Learning Based Approach for Facial Expression Classification

Abstract

Facial expressions deliver intensive information about human emotions and the most valuable way of social collaborations, despite difference in ethnicity, culture, and geography. These differences addresses the three main problems, which are; facial appearance variation, facial structure variation, and inter-expression resemblance. Due to these problems the existing facial expression recognition techniques are very inconsistent. This study presents several computational algorithms to handle these problems in order to get high expression recognition accuracy. We proposed a novel ensemble classifier for cross-cultural facial expression recognition. The proposed ensemble classifier consists of three stages; base-level, meta-level and predictor, where binary neural network adopted as base-level classifier, neural network ensemble (NNE) collections as meta-level classifier and naive Bayes (NB) with Bernoulli distribution as predictor. The NB classifier takes the binary output of NNE collections and classifies the sample image as one of the possible facial expressions. The Viola-Jones algorithm is used to detect the face and expression concentration region. The acted still images of three databases JAFFE, TFEID, and RadBoud originate from four different cultures are combined to form multi-culture facial expression dataset. Three different feature extraction techniques LBP, ULBP and PCA are applied for facial feature representation. Further, boosted NNE collections are developed to enhance the facial expression recognition accuracy. The proposed boosting technique combines multiple NNEs which are complement to each other. The combination of boosted NNE collections with HOG-PCA feature vector perform significantly better than NNE collections. Later on the multi-culture dataset is extended by adding more cultural diversity from KDEF and CK+ databases, which is used to train the SVM based ensemble collections. The introduction of SVM ensemble collections at meta-level provides strong generalization ability to learn the vast variety of cultural variations in expression representation. Moreover, sensitivity analysis and inter-expression resemblance analysis are performed to quantify the level of complexity in cross-cultural facial expression recognition. It shows that expressions of happiness, surprise and anger are easy to recognize as compare to expressions of sadness and fear. It proves that these expressions are innate and universal across all cultures with minor variations. The experimental results demonstrate that proposed cross-cultural facial expression recognition techniques perform significantly better than state of the art techniques.