Hybrid Model for Human Emotion State Recognition using Deep Learning

Software application development for human emotion recognition is yet to optimize for utilization in medical applications to diagnose mental health. This proposed hybrid deep learning model evaluates the maximum score using CNN for facial emotion recognition and MFCC for voice emotion recognition. The maximum score function provides a unified result observed from two different recognition models. We use the FER dataset for facial and IEMOCAP dataset for speech emotion recognition. In this work, the real time video frames are extracted, segregated into image frames and audio blocks. The CNN model is used for facial expression recognition on video frames and MFCC is used for recognizing speech emotions. Both models execute independently and outcome convergence to produce combined results for recognizing human emotion state. The result shows that this new model performs better, and multiple test cases ensured the same by comparison to individual CNN for facial recognition and MFCC for speech emotion recognition