RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control

First and foremost, we sincerely thank you for pointing out the issues, as your suggestions are invaluable in enhancing the quality of this paper.

1. Scalability: The paper mentions that the computational cost increases with the number of intersections. Could the authors elaborate on potential strategies to scale RobustLight for large-scale urban networks with hundreds of intersections?

Here, we need to clarify that our method can be deployed in either centralized or decentralized scenarios. In the case of centralized deployment, the model inference speed may slow down as the number of intersections increases.

Distributed Deployment: Specifically, if a distributed deployment is chosen, our diffusion model can be trained on a central server to fully leverage data from different intersections, while model parameter updates are performed on edge computing devices, similar to a federated learning architecture.

Centralized Deployment: If a centralized deployment is preferred, scalability and real-time requirements for hundreds of intersections must be addressed. On the hardware side, we can procure high-performance servers or employ inference acceleration techniques, such as data partitioning and parallelized inference, to enhance the diffusion model's inference performance. On the algorithmic side, we can utilize techniques like DDIM [1] to accelerate the diffusion model's inference process.

In summary, by carefully selecting the deployment strategy and leveraging advanced hardware and algorithmic optimizations, we can ensure the robustness, scalability, and real-time performance of our Robust TSC system while maintaining its security against potential attacks.

[1] Denoising Diffusion Implicit Models.

2. Real-World Deployment: While the experiments are conducted on real-world datasets, the paper does not discuss the challenges of deploying RobustLight in real-world TSC systems. What are the practical considerations and potential barriers to real-world implementation?

Our framework could leverage DDIM and high-performance GPUs for accelerated inference. Denoising supports decentralized deployment with centralized training, while data missing scenarios use centralized inference. DDIM acceleration times on one A100:

As shown, the total processing time is less than 1 second meet real-time control. However, it is important to note that in decentralized deployment scenarios, the repaint algorithm will not be able to address the kriging missing (full-intersection failure) issue within data missing. Distributed deployment significantly improves algorithm performance due to the reduction in the batch size of inference data.

We recommend deploying the centralized solution to address both kriging (full-intersection failure) and random missing (sensor-specific-direction-single-intersection-failure) data issues using data missing algorithm, while employing data noise algorithms on the edge side using low-cost hardware to achieve accelerated denoising.

First and foremost, we sincerely thank you for pointing out the issues, as your suggestions are invaluable in enhancing the quality of this paper.

1. For the experiment part, my concern is whether this method is still effective under some potential adaptive attacks, such as attacks effective on Diffusion Models.

Diffusion Models are generally robust due to their iterative denoising process, which inherently introduces noise and reduces the impact of adversarial perturbations. However, like other deep learning models, they are not immune to adaptive attacks specifically designed to exploit their weaknesses.

This situation resembles a Russian nesting doll, but based on our experience in TSC deployments, most TSC control systems are deployed within internal networks. While outdoor sensing devices are prone to noise interference and sensor damage, the TSC control module and our diffusion module can be deployed in either a distributed manner (with control devices at each intersection connected via an internal network) or a centralized manner (deployed within the internal network). This network isolation effectively protects the diffusion module from potential attacks.

Distributed Deployment: Specifically, if a distributed deployment is chosen, our diffusion model can be trained on a central server to fully leverage data from different intersections, while model parameter updates are performed on edge computing devices, similar to a federated learning architecture.

Centralized Deployment: If a centralized deployment is preferred, scalability and real-time requirements for hundreds of intersections must be addressed. On the hardware side, we can procure high-performance servers or employ inference acceleration techniques, such as data partitioning and parallelized inference, to enhance the diffusion model's inference performance. On the algorithmic side, we can utilize techniques like DDIM [1] to accelerate the diffusion model's inference process.

In summary, by carefully selecting the deployment strategy and leveraging advanced hardware and algorithmic optimizations, we can ensure the robustness, scalability, and real-time performance of Robust TSC system while maintaining its security against potential attacks.

[1] Denoising Diffusion Implicit Models.

2.The latency issue of this method because of the usage of diffusion models and whether it can handle traffic problems in a real-time way.

Our framework could leverage DDIM and high-performance GPUs for accelerated inference. Denoising supports decentralized deployment with centralized training, while data missing scenarios use centralized inference. DDIM acceleration times on one A100:

As shown, the total processing time is less than 1 second meet real-time control. However, it is important to note that in decentralized deployment scenarios, the repaint algorithm will not be able to address the kriging missing (full-intersection failure) issue within data missing. Distributed deployment significantly improves algorithm performance due to the reduction in the batch size of inference data.

We recommend deploying the centralized solution to address both kriging (full-intersection failure) and random missing (sensor-specific-direction-single-intersection-failure) data issues using data missing algorithm, while employing data noise algorithms on the edge side using low-cost hardware to achieve accelerated denoising.

First and foremost, we sincerely thank you for pointing out the issues. Your suggestions are invaluable in enhancing the quality of this paper. Below is our answer to your questions.

1. Experimental results on some datasets are not reported.

Due to space constraints, data noise results for JN2 and HZ2 were omitted from the main text. The partial snapshot below shows RobustLight’s superior performance. Additional dataset results are included in the appendix and link.

Advanced-CoLight Noise Scale 3.5

2. Other related data generation methods are introduced to reflect the innovation of the proposed data generation method.

HINT[1] proposed using Transformer combined with GAN for data reconstruction, while CaPaint[2] employs diffusion for data generation. However, these methods require extensive offline data for pre-training and only address data missing, not data noise. DiffLight[3] tackles random missing and kriging missing but uses separate algorithms for each and underperforms SOTA on clean data. In contrast, our method employs a single model to simultaneously resolve data missing and noise, handling both random missing and kriging missing with a unified algorithm. Moreover, our approach acts as a plug-in, optimizing SOTA algorithms without altering their core structure.

[1] HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention

[2] Causal Deciphering and Inpainting in Spatio-Temporal Dynamics via Diffusion Model

[3] DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data

3. Missing baseline and quantitative analysis.

We fine-tuned the HINT model on an offline dataset and evaluated it on the TSC task, quantifying performance using PSNR (higher is better) and MAE (lower is better). Our algorithm surpasses HINT in data generation capability.

Data Generation Performance Comparison

To validate our approach, we setup involved randomly masking data from Kriging Missing (12.5%) and Random Missing (12.5%). AMPR (Advanced-MaxPressure-Robustlight), ACR (Advanced-Colight-Robustlight).

Performance Compared with DiffLight

4. The difference with other diffusion model.

Key modifications: (1)use a new beta schedule (2) optimized with new loss function. (3) employ new action gradient method to improve data noise and missing.

5. New link: https://anonymous.4open.science/r/RobustLight-72B2/README.md

6. More baseline results refer to qV9c and code link.

7. Refer to Jqz8 and ACd7, our theoretical explanation based Lemma 3.1

First and foremost, we sincerely thank you for pointing out the issues, as your suggestions are invaluable in enhancing the quality of this paper.

W1: https://anonymous.4open.science/r/RobustLight-72B2/README.md

W2：Traffic movements and TSP are defined in Figure 1 and Section 2.1, with cyan coloring used for aesthetic design. D() represents the norm distance.

W3&W6:

We conduct the experiments for Difflight[1] and MissingLight[2]. [1] uses offline RL and diffusion models for two missing-data cases: random missing (single-intersection-sensor failure) and kriging missing (full-intersection failure), requiring separate treatments. To validate our approach, we setup involved randomly masking data from Kriging Missing (12.5%) and Random Missing (12.5%).

Data Missing , Our is Based On Advanced-Colight

Data Noise Scale 3.5

As can be seen, our method not only effectively handles data missing scenarios in both (kriging missing and random missing) but also performs well in data noisy condition.

W6：Without Data Missing and Data Noise

Under clean data, RobustLight preserves the optimal performance of the TSC algorithm without unnecessary denoising, it simplifies integration and enhancing system stability and reliability.

Compared to [3], which uses adversarial pre-training to improve robustness of RL, but it cannot repair corrupted inputs and risks overfitting to specific attacks, our diffusion model handles diverse disturbances without attack-specific training, directly denoising states for robust RL inputs as a plug-and-play module.

[1] DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data. NIPS 2025

[2] Reinforcement Learning Approaches for Traffic Signal Control under Missing Data. IJCAI 2023

[3] Explainable Deep Adversarial Reinforcement Learning Approach for Robust Autonomous Driving. TIV 2024

W4:

Our framework could leverage DDIM and high-performance GPUs for accelerated inference. Denoising supports decentralized deployment with centralized training, while data missing scenarios use centralized inference. DDIM acceleration times on one A100:

As shown, the total processing time is less than 1 second meet real-time control. Further deployment discussion can be found in reviewer Jqz812.

W5:

Our RobustLight addresses challenges in TSC deployment, where TSC RL inputs often suffer from noise or missing values, impacting control. To handle the dynamic nature of traffic flow, we propose an online training framework using diffusion models as the upper level RL policy during clean data periods. When noise or missing data is detected, our trained DSI agent employs denoising and repainting algorithms to restore data quality, improving lower level TSC RL performance.

W6&W7：

[1] introduced TP-FDS, which identifies anomalies by comparing new data distributions with historical data from the same period, achieving an AUC of 96% and an F1 score of 76%. Minor changes like 3-5 have minimal impact on system efficiency. Anomalies can also be detected by cross-referencing data from multiple sensors, such as cameras and radar. Rule-based methods are another option; for example, a queue increase from 3 to 5 during a north-south green light would signal an anomaly. To simulate real-world scenarios, we conducted experiments with detection rates of 80% and 60%:

MAD Scale 3.5

It sustains high performance with different detection rates.

Refer to Jqz8 and ACd7, our theoretical explanation based Lemma 3.1

[1] Traffic Anomaly Detection: Exploiting Temporal Positioning of Flow-Density Samples