Transformer-Integrated YOLO–SSD for Real-Time Traffic Monitoring and Pedestrian Detection

Robust real-time perception systems are critical to the future of autonomous vehicles and ensuring safety and compliance with regulations. Pedestrian detection and traffic sign recognition rank among the most important perception tasks that must operate with a high degree of accuracy in dynamic traffic systems. Challenges hindering robust recognition include diverse lighting conditions and backgrounds, occluded objects, and small-sized traffic signs. The proposed model introduces an integrated real-time object detection framework that incorporates Multiple-Object Object Detection model YOLO v11, SSD, and Transformer Based to perform simultaneous pedestrian and traffic sign recognitions. The proposed framework addresses both live-sourced video streams and recorded static images, including optimized preprocessing of image data, parallel extraction of features from images, confidence-based feature fusion algorithms, and optimized post-processing of feature populations. Evaluation of this proposal against various benchmark datasets and custom-created road-scene datasets showed that this hybrid framework achieved an overall mean average precision (MAP) of 89.6 percent compared to YOLOv11 82.4 %, SSD 85.7 % and the standalone transformer-based object detector 86.9%. While providing comparable performance across the various benchmarks, the hybrid framework maintains a real-time inference capability of 29 frames per second (FPS). The proposed hybrid detection system performed at the same time as the models and consistently outperformed the individual model detections while still providing real-time results.