Adaptive Traffic Signal Optimization using Reinforcement Learning: A SUMO-based Study

Traffic congestion which occurs in urban areas has become a severe issue within the contemporary world as the conventional time based traffic signal controllers are not responsive to the fluctuating issue of traffic on the basis of time. This paper shows a smart and adaptive traffic signal control framework based on the Reinforcement Learning (RL) applied in a Simulation of Urban Mobility (SUMO) simulator and Traffic Control Interface (TraCI). An actual single intersection case study with eastbound (EB) and southbound (SB) approaches is modeled using six lane-area (E1) detectors so that accurate lengths of queues of different distances can be measured. It methodically compares three controllers based on a fixed time base after traditional 30-30-second cycles, (1) fixed time simple controller, (2) tabular Qlearning controller and (3) Deep Q-Network (DQN) Neural network approximation modeling Q-values estimation. The RL agents maximize some formulation of reward that will penalize total queue length that is above 3000 simulation steps. In-depth analysis based on the metrics of traffic performance (vehicle delay total, average and maximum queue lengths, time of travel distributions (mean, 90th percentile, maximum) and vehicle throughput) shows that the learning based improvements were made by controllers. Findings indicate that the DQN controller only causes just about 40 % reduction in the overall delay and 12 % increase in throughput that it provides compared to the baseline which is static, and it also performs better than the tabular Q-learning in terms of stability and convergence speed. These results confirm deep reinforcement learning as a strong base in learning scalable adaptive traffic signal control and possible utilization in multi intersection networks.