A Multi-Modal Framework for Emergency Vehicle Prioritization using Visual Detection and Acoustic Classification

Traffic jams in cities often make it tough for emergency services to reach people in need, especially during the golden hour, when every minute counts. This issue arises from long distances and slow-moving vehicles. Most of the technologies that are currently used for managing traffic struggles to address this problem effectively because it typically depends on just one type of sensor. Challenges like blocked views, poor lighting, and background noise can make this approach unreliable. In our study, we introduced a new method that combines different types of data to give priority to emergency vehicles. We use a visual detection system and an audio classification system together, which has proven to be more reliable, especially in busy urban environments. Specifically, we have used a computer vision model called YOLOv8 to spot emergency vehicles through cameras placed on traffic signals. Alongside that, we have employed a sound recognition model named PANNs CNN14, which can pick up sirens and differentiate that from other sounds. Both visual and audio data are processed simultaneously and blended together smartly. This allows us to quickly recall either sound or sight information in critical situations. The Use of regular traffic cameras and microphones instead of expensive equipment makes it easier to use and more reliable, which is the best part.