Performance Comparison of Facial Emotion Recognition: Introducing a Model within the Driver Assistance Framework based on Deep Learning with LBP Feature Extraction for In-Vehicle Applications

چکیده مقاله

This study proposes a descriptor-based approach combined with deep learning, which recognizes facial emotions for safe driving. Paying attention to the driver's facial expressions is crucial to address the increasing road accidents. This project aims to develop a Facial Emotion Recognition (FER) system that monitors the driver's facial expressions to identify emotions and provide instant assistance for safety control. In the initial stage, Viola-Jones face detection was employed to detect the facial region, followed by Butterworth high-pass filtering to enhance the identified region for locating the eye, nose, and mouth regions, using Viola-Jones face detection. Secondly, the Local Binary Patterns (LBP) feature descriptor is utilized to extract features from the identified eye, nose, and mouth regions. Using 3 RGB channels, the extracted features from these three regions are fed into RessNet-50 and EfficientNet deep networks. The outputs of the two deep learning models' classifiers are combined and integrated using two ensemble methods: ensemble maximum voting and ensemble mean. Based on these combining classifier rules, the performance was evaluated on the JAFFE and KMU-FED databases. The experimental results demonstrate that the proposed method can effectively and with higher accuracy than other competitors recognize emotions in the JAFFE and KMU-FED datasets. The novelty and originality of this paper lie in its significant application in the automotive industry. Implementing our proposed method in a system capable of high accuracy and precision can help mitigate numerous driving hazards. Our approach has achieved 99% and 98% accuracy on the JAFFE and KMUFED databases, respectively. This high level of accuracy, coupled with its practical relevance, underscores the innovative nature of our work.