Hybrid Sensor and Vision Assisted Gesture-to-Voice Conversion System for Real-Time Assistive Communication

Speech-impaired individuals are suffering from communication barriers which are leading to intelligent assistive systems capable of translating sign gestures into understandable speech. This work recommends a Gesture-to-Voice Conversion System which combines flex sensor-based glove recognition with vision-assisted machine learning-based gesture classification suitable for real-time speech synthesis. The system employs flex sensors which changes its resistance with bending, Arduino board for processing, Bluetooth communication, and Android text-to-speech interface. The software subsystem utilizes KNN-SVM assisted gesture recognition to enhance flexibility and scalability. Experimental results show gesture recognition accuracy of 92.8%, and end-to-end response latency below 750 ms. Implementation cost of the system is very low. Novelty in the work is dual-mode sensing, hybrid recognition, and real-time mobile speech integration.