Enhancing Accessibility: An AI-Powered Web Application for Sign Language to Text Conversion

There has always been a significant communication gap between sign language and non-sign language individuals. This sign language barrier can hinder the inclusion of deaf and hard-of-hearing individuals in social interactions on a daily basis, education, and the workplace. To fill this communication gap, we propose an AI-based Sign Language Recognition system that translates sign language into readable text in real time. The proposed system is based on a twopart deep learning model. The initial part, Gesture Recognition and Sensing System, utilizes Convolutional Neural Networks (CNNs) to handle the visual information—fingertips, hand shapes, finger movements, and facial expressions—of sets of images. The Sequential interpretation module, the second part of the system, employs Recurrent Neural Networks (RNNs) and is responsible for learning the timing and sequence of the gestures and how they accommodate and are interrelated with one another. To make the translation even more meaningful, we also applied Natural Language Processing (NLP) algorithms to translate the captured gestures into grammatically correct readable text. This ensures that the output is not only technically accurate but also understandable and user-friendly. The final system was deployed as a web application that allows individuals to communicate in real time by translating sign language into text through a camera interface. This accessible platform offers a practical solution for both signers and non-signers to communicate more easily, highlighting how AI can help make the world more accessible.