Reliability-Aware CNN–SNN-Inspired Fusion with Adaptive Modality Dominance for Road-user Classification

Vision-based traffic participant classification systems often rely on a single visual cue, making them vulnerable to real-world degradations such as motion blur, occlusion, or poor illumination. Spatial models such as convolutional neural networks (CNNs) excel under clear conditions but degrade with image corruption, while temporal models such as spiking neural networks (SNNs) capture motion dynamics but fail under temporal disruption. We propose Adaptive CNN–SNN Fusion (ACSF), a framework that dynamically balances spatial CNN and temporal SNN-inspired experts on a per-input basis through a learnable fusion gate. Unlike static fusion, ACSF learns input-dependent weights reflecting each modality’s reliability. Experiments on custom dashcam and BDD100K datasets show that the fusion gate adapts its weighting under controlled degradations, maintaining accuracy within 1% drop while outperforming single-modality and fixed-fusion baselines, confirming adaptive expert fusion’s effectiveness for robust traffic perception.