A Voice-Driven Intelligent Coding Assistant using Speech Recognition and Large Language Models

In this study, we are working to create a voice enabled intelligent coding assistant with the goal of simplifying and enhancing the efficiency of programming. Typically, in programming environments, programmers must do continuous typing and often move back and forth between different tools such as documentation, coding editors, and debugging environments which interrupts the programming process and ultimately decreases programmer productivity. The intent of the proposed coding assistant would be to allow programmers to create, modify or understand their programming code via voice commands thus increasing the accessibility of programming not only for professional coders but for students, newcomers, teachers, and others that would like to program with voice or have a hands-free computing experience. This coding assistant will provide the user with a speech recognition and natural language processing interface combined with large language models to convert voice instructions into executable code. The coding assistant uses a ReactJS web interface for the front-end development and has a FastAPI application for the back end and connects to the Google Gemini for processing the code. The MongoDB database will be used to store the conversational history, and the Monaco Code Editor will be utilized as an interactive development environment (IDE) for displaying and modifying the generated code.