A Smart System for Recognizing Human Emotions based on Audio and Text in Multi-Faceted Environments

Recognizing emotions from speech and text data in multi-faceted environments is a challenging task due to the ambiguous nature of emotion itself. To improve the modeling of different aspects of conduct and capture emotional content in various styles of speaking, the project utilizes multiple deep learning models to extract robust features from speech data and various lexical and prosodic features from text data. To achieve robustness and context sensitivity, it employs multi-faceted networks that are insensitive to outliers. The evaluation of models is done using various metrics such as accuracy, F-score, precision, and recall. In addition to deep learning models and multi-faceted networks, it also explores ensemble methods that combine different models to improve the overall accuracy and robustness of the emotion recognition system. These ensemble methods include combinations of random forest, XGBoost, multilayer perceptron, naive Bayes, logistic regression, and long short-term memory (LSTM) models. By combining multiple models, the project can leverage the strengths of each model while mitigating their weaknesses. Despite the diverse datasets used in the project, lighter machine learning-based models trained on a few hand-crafted features can achieve comparable performance to state-of-the-art deep learning-based methods for emotion recognition. Overall, the project aims to advance the field of emotion recognition in multi-faceted environments by improving the modeling techniques, utilizing multi-faceted networks, and evaluating the models with various metrics