Role-aware Multi-agent Reinforcement Learning for Coordinated Emergency Traffic Control

First and foremost, we sincerely thank you for pointing out the issues. Your suggestions are invaluable in enhancing the quality of this paper. Below is our answer to your questions.

W1&Q1. The paper introduces role-aware reinforcement learning, but it is unclear how different roles actually lead to different behavior patterns in agents.

Thank you for your comments. We address them from two perspectives:

• Role Influence on Policy: Dynamic roles affect RL behavior in two ways: role embeddings are combined with local states as inputs to the policy network, enabling EMV-aware decisions. Additionally, a role-conditioned intrinsic reward encourages agents to take EMV-friendly actions. Together, these mechanisms guide both behavior and coordination, improving training efficiency.
• Case Study: Following the reviewer’s comment, we showcase how the dynamic role influences the reinforcement learning process with the following case. TL 1, one hop ahead of the EMV, has a high cosine similarity (0.49) to the EMV feature and maintains its green phase to facilitate passage. TL 2, with no nearby EMV, shows low similarity (0.17) and changes its signal independently. This shows role embeddings capture EMV proximity and influence signal behavior accordingly.

Action indicates the signal phase before and after the decision. Each character corresponds to a lane direction: 'g' denotes green and 'r' denotes red. For TL 1, the EMV enters when the signal is green, and the phase remains unchanged to allow passage.

W2&Q2. The multi-agent modeling does not provide enough details, especially about how the Dynamic Role Learning module is trained, making it hard to fully understand or reproduce.

Thank you for your comments.

• Role Embedding Learning: We first train the Heterogeneous Role Convolution module using the role learning objective in Eq.9. This module captures spatial-temporal interactions in the heterogeneous traffic graph and outputs dynamic role embeddings for each agent. These embeddings are learned to reflect agent role, especially in relation to nearby EMVs.

• Role-aware Policy Learning: The learned role embeddings are then concatenated with the agent’s state to form role-aware observations. These enhanced inputs are fed into the policy networks, allowing each agent to condition its decisions on both local traffic state and semantic role information. Role learning and policy optimization proceed in an alternating training loop to ensure consistency and coordination.

W3&Q3. Some formulas in the manuscript are incomplete or simplified. For example, Eq.11 assumes only one EMV in the scenario, but the paper does not clarify how the intrinsic reward would be computed when multiple EMVs are present.

Thank you for your valuable suggestion. Our method encodes the relations between each agent and multiple EMVs through a heterogeneous graph, learning role embeddings via mutual information maximization. This design ensures that agent roles dynamically reflect their varying relevance to different EMVs based on spatial positions. Additionally, we propose a role-similarity-based reward redistribution mechanism to enable soft prioritization when multiple EMVs coexist. We have revised Eq.11 to the following form to support multi-EMV scenarios:

$r\_{n,\text{intr}}^{(t)} = \sum\_{i=1}^{N_{emv}} s\left(h\_n^{(t)}, h\_{\text{emv},m}^{(t)}\right) \cdot r\_{\text{emv},i,\text{extr}}^{(t)}, \quad \forall n \in \{ I_{\text{rev}}, I_{\text{tl}} \}$

We will update Eq.11 and the corresponding explanations in the manuscript to clearly present this multi-EMV reward computation.

First and foremost, we sincerely thank you for pointing out the issues. Your suggestions are invaluable in enhancing the quality of this paper. Below is our answer to your questions.

W1. Limited modeling as EMVs are part of the system and treated as such, instead of allowing for individual decisions.

Thank you for the comment. In our framework, EMVs are modeled as independent learning agents with their own observations, policies, and rewards. This allows each EMV to make individual decisions rather than being passively controlled by the environment or rule-based logic.

W2 & Q1. Scalability and Real-World Applicability

Thank you for your comment. We address this from the following four perspectives:

• New experiment on Larger traffic Network Size. Following prior works[1,2,3], we conduct our main experiments on commonly used traffic networks. To further assess scalability with respect to the number of intersections, we introduce a larger 5×5 grid network comprising 25 intersections. The results show that our method consistently maintains effective performance and coordination as the network size increases.
• Computational Overhead. We report episode training time and inference time averaged over 100 episodes for traffic networks with 8, 16, and 25 traffic signals. As shown in the table below, the episode time of RMTC increases from 122.12s (8 signals) to 160.90s (16 signals) and 239.92s (25 signals). The inference time similarly grows from 0.0275s to 0.0424s and 0.0507s. Despite the increasing network size, the growth trend remains close to linear, demonstrating that RMTC can scale efficiently with larger traffic scenarios.
• Impact of GNN and MI on Computation: Removing the GNN module reduces training time by 35.7% (16-signal case), and the MI component increases training time by 13%, with minimal impact on inference. Despite the added cost, our full model achieves the highest final reward among all variants, demonstrating better learning performance without hindering practical deployment.
• Scalability to Large Real-World Scenarios. In practice, EMV dispatch is organized by region, as emergency services such as fire stations and hospitals are distributed across districts. Our method supports region-wise training and deployment, making it well-suited for scalable application in large cities, rather than requiring training over the entire city.

Comparison on Grid 5×5

Efficiency experiment

[1]Jiang, Haoyuan, et al. "X-Light: cross-city traffic signal control using transformer on transformer as meta multi-agent reinforcement learner." Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 2024.

[2]Wei, Hua, et al. "Colight: Learning network-level cooperation for traffic signal control." Proceedings of the 28th ACM international conference on information and knowledge management. 2019.

[3]Wei, Hua, et al. "Presslight: Learning max pressure control to coordinate traffic signals in arterial network." Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019.

First and foremost, we sincerely thank you for pointing out the issues. Your suggestions are invaluable in enhancing the quality of this paper. Below is our answer to your questions.

W1. The related work section is not comprehensive and overlooks many recent studies.

Thank you for your helpful comment. In the revised version, we will expand the related work section to include recent advances in traffic signal control and studies specifically addressing emergency vehicle scenarios[1,2,3], which are closely related to our problem setting.

W2 & Q2 & Q1. How are dynamic role embeddings learned and how do they influence the reinforcement learning process?

Thank you for your comments. We address them from three perspectives:

• Role Embedding Learning: Our approach learns compact role embeddings that focus on essential EMV-related information by maximizing the mutual information (MI) between the embeddings and the local EMV state. This ensures the roles capture task-critical semantic cues, serving as informative context variables that help the policy network concentrate on critical features. Prior works such as ROMA [4], RODE [5], and ACORM[6] demonstrate that maximizing mutual information between role embeddings and key state features improves policy learning and efficiency. This supports our use of EMV-centered mutual information to enhance role semantics and policy performance.

• Role Influence on Policy: Dynamic roles affect RL behavior in two ways: role embeddings are combined with local states as inputs to the policy network, enabling EMV-aware decisions. Additionally, a role-conditioned intrinsic reward encourages agents to take EMV-friendly actions. Together, these mechanisms guide both behavior and coordination, improving training efficiency.

• Case Study: Following the reviewer’s comment, we showcase how the dynamic role influences the reinforcement learning process with the following case. TL 1, one hop ahead of the EMV, has a high cosine similarity (0.49) between its role embedding and the EMV feature, which reflects the level of EMV-awareness and contextual relevance. TL 2, with no nearby EMV, shows low similarity (0.17) and changes its signal independently. This shows role embeddings capture EMV proximity and influence signal behavior accordingly.

  Agent	Relative to EMV	Cosine Sim (Role vs EMV feat)	Action
  TL 1	EMV 1 hop ahead	0.49	gGGGgrrrrgGGGgrrrr->gGGGgrrrrgGGGgrrrr
  TL 2	No EMV nearby	0.17	gGGGgrrrrgGGGgrrrr->grrrgGGGGgrrrgrrrr

Action indicates the signal phase before and after the decision. Each character corresponds to a lane direction: 'g' denotes green and 'r' denotes red. For TL 1, the EMV enters when the signal is green, and the phase remains unchanged to allow passage.

W3&W4. Baselines and performance stability

Thanks for your suggestion. To address recent concerns, we include EMV-specific baselines, and report mean ± standard deviation over 3 random seeds for our method.

W5. The number of agents used in each task is not specified. If each vehicle requires an independent policy, the computational burden could be substantial. This important detail should be clearly addressed in the paper

Thank you for pointing this out.

• Vehicle agent number setting. In our framework, each vehicle and traffic light is modeled as an agent. The number of vehicles is determined by the vehicle generation rate and episode duration. Specifically, the Grid and Cologne datasets contain approximately 600 vehicles, while the Arterial and Fenglin datasets include around 1000 vehicles per episode.

• Shared policy. All agents share a centralized policy network (parameter sharing). This design enables agent-specific behavior while avoiding the computational burden of maintaining independent policies for each agent, ensuring scalability and learning efficiency in large-scale environments. We will include a detailed explanation of this design choice in the final version

Q3. The interaction and potential conflicts between the different loss components (e.g., role consistency vs. responsiveness to EMVs) are not analyzed or theoretically grounded

Thank you for the suggestion. We address the concern from two aspects:

• Temporal stability: The temporal consistency loss is designed to ensure smooth role transitions across adjacent time steps. It does not constrain adaptation to major changes. When an EMV changes its road segment, the mutual information loss actively guides the role embedding to adapt, ensuring responsiveness. Thus, the two objectives are functionally complementary rather than conflicting.

• Experiment: We experimented with different values of the weight $\alpha$ in the role loss function, which is defined as follows.

  $\mathcal{L}\_{\text{role}} = \sum\_{t=1}^{T}\left( - \alpha \left( I^{(t)}(h^{(t)};h^{(t)}\_{\text{emv}}) + I(h^{(t)}; g\_{\text{emv}}^{(t)}) \right) + \mathcal{L}\_{\text{cons}}^{(t)}\right)$​

  The results show that setting $$\alpha$= 1$ yields the best balance, achieving the lowest EMV travel time and vehicle travel time. This indicates a stable and effective interaction between the mutual information terms and the consistency loss.

Q4. While the proposed framework involves reinforcement learning as a core optimization engine, the paper lacks critical evidence of actual policy learning. There are no training curves, no sample efficiency comparisons, and no analyses of training stability, convergence, or the effects of reward shaping.

Thank you for the valuable comment. Below, we provide a detailed analysis of RMTC’s training stability, convergence, and sample efficiency:

• The first table shows the episodic rewards of RMTC and its ablations across different training stages. RMTC consistently achieves higher rewards and faster convergence. Stability is measured by the standard deviation of evaluation rewards, where RMTC exhibits the lowest standard deviation (1.480), indicating superior stability.
• The second table compares RMTC and MAPPO across different timesteps. RMTC achieves better performance at each stage, especially in early training (e.g., at 5k and 10k timesteps), demonstrating stronger sample efficiency and faster learning.

Reward Ablation Study on RMTC Components Across Episodes

Sample efficiency Comparison Between RMTC and MAPPO

[1]Yao, Jiarong, et al. "Incorporating vision-based artificial intelligence and large language model for smart traffic light control." Applied Soft Computing (2025): 113333.

[2]Wang, Lijuan, et al. "An adaptive traffic signal control scheme with Proximal Policy Optimization based on deep reinforcement learning for a single intersection." Engineering Applications of Artificial Intelligence 149 (2025): 110440.

[3]Alruwaili, Madallah, et al. "LSTM and ResNet18 for optimized ambulance routing and traffic signal control in emergency situations." Scientific Reports 15.1 (2025): 6011.

[4]Wang, Tonghan, et al. "ROMA: multi-agent reinforcement learning with emergent roles." Proceedings of the 37th International Conference on Machine Learning. 2020.

[5]Wang, Tonghan, et al. "RODE: Learning Roles to Decompose Multi-Agent Tasks." International Conference on Learning Representations.

[6]Hu, Zican, et al. "Attention-Guided Contrastive Role Representations for Multi-agent Reinforcement Learning." The Twelfth International Conference on Learning Representations.

First and foremost, we sincerely thank you for pointing out the issues. Your suggestions are invaluable in enhancing the quality of this paper. Below is our answer to your questions.

W1. Incremental contribution and Novelty.

Thank you for your comment. We will clarify our key contributions in the revised version.

• New Problem Setting: We formulate emergency traffic coordination as a multi-agent problem involving EMVs, REVs, and traffic lights—going beyond prior TSC works that only control signals.
• Framework Design: Instead of combining existing techniques, we customize MI-based role learning, heterogeneous temporal graphs, and role-conditioned RL to address behavior differentiation and cross-agent interaction in emergency scenarios.
• EMV-aware Role Learning: We further design three losses based on EMV states and trajectories to enable dynamic role representation, and introduce a role-aware intrinsic reward to balance global traffic efficiency with EMV priority.

W2. Unclear evaluation:

Dataset Splitting for Training and Testing

Method Applicability in Normal Traffic Scenarios

Thank you for your comment.

• Evaluation Protocol: Our evaluation is conducted in the SUMO simulation environment. Unlike supervised learning, reinforcement learning does not rely on static training/testing splits. We evaluates performance by varying initial environment conditions. This approach is widely adopted in traffic control tasks[1,2].
• Degraded Setting: In scenarios without emergency vehicles, our method naturally reduces to a basic actor-critic model without role embeddings or intrinsic rewards.

W3 & Q2. Scalability, computational overhead, and applicability to large-scale real-world scenarios

Thank you for your comment. We address this from the following four perspectives:

• New experiment on Larger traffic Network Size. Following prior works[1,2,3], we conduct our main experiments on commonly used traffic networks. To further assess scalability with respect to the number of intersections, we introduce a larger 5×5 grid network comprising 25 intersections. The results show that our method consistently maintains effective performance and coordination as the network size increases.
• Computational Overhead. We report episode training time and inference time averaged over 100 episodes for traffic networks with 8, 16, and 25 traffic signals. As shown in the table below, the episode time of RMTC increases from 122.12s (8 signals) to 160.90s (16 signals) and 239.92s (25 signals). The inference time similarly grows from 0.0275s to 0.0424s and 0.0507s. Despite the increasing network size, the growth trend remains close to linear, demonstrating that RMTC can scale efficiently with larger traffic scenarios.
• Impact of GNN and MI on Computation: Removing the GNN module reduces training time by 35.7% (16-signal case), and the MI component increases training time by 13%, with minimal impact on inference. Despite the added cost, our full model achieves the highest final reward among all variants, demonstrating better learning performance without hindering practical deployment.
• Scalability to Large Real-World Scenarios. In practice, EMV dispatch is organized by region, as emergency services such as fire stations and hospitals are distributed across districts. Our method supports region-wise training and deployment, making it well-suited for scalable application in large cities, rather than requiring training over the entire city.

Comparison on Grid 5×5

Efficiency experiment

W4. Assumes perfect sensing, V2X comms, and strict driver compliance, which may limit its real-world application.

We appreciate the reviewer’s concern. Our framework assumes V2X-based communication, where vehicles can exchange information with traffic lights and nearby vehicles. This assumption is widely adopted in EMV coordination research. Additionally, we consider each vehicle’s destination, which allows integration with real-world navigation systems to assist both emergency and regular drivers in cooperative routing during emergencies.

W5. Related Work: Related work lacks recent attention-based or graph-meta RL traffic control methods and possibly role-based MARL

Thank you for your comment. We will add a discussion of recent works on attention-based or graph-meta reinforcement learning for traffic control [4,5] and role-based multi-agent reinforcement learning methods [6,7,8] to the related work section in the revised version. These methods differ from ours in both problem formulation and data modalities, and thus were not directly compared in our experiments.

Q1 & Q3. EMV Priority Baseline Design and Comparison Protocol

Thank you for your valuable comment. We provide detailed clarifications as follows:

• Baseline Settings: Our baselines consist of existing traffic signal control methods, each implemented with the same settings and hyperparameters as reported in their original works to ensure fair comparison.
• EMV Priority in Baselines: All baselines incorporate a rule-based EMV priority(switching lights when EMV is detected under safe conditions), reflecting common practical strategies for emergency vehicle prioritization.
• Greedy EMV Baseline: We follow the suggestion to add a rule-based baseline that switches the signal to green as soon as an EMV is detected and keeps it green until the EMV leaves. While this design minimizes EMV travel time, it significantly increases delay for regular vehicles, showing poor overall efficiency balance.

Q4. How to handle contentions among many simultaneous emergencies?

Thank you for the insightful question.

• Our method encodes the relation between each agent and multiple EMVs using a heterogeneous graph, and learns role embeddings via mutual information maximization. This ensures that agent roles reflect varying importance to different EMVs based on their spatial positions. For example, if an agent is spatially closer to EMV A than EMV B, its role embedding will more strongly reflect EMV A’s features, guiding behavior accordingly. We further design a role-similarity-based reward redistribution mechanism, enabling soft prioritization when multiple EMVs coexist.

• We tested our model in scenarios with multiple concurrent EMVs, as reported in the main paper (Table 2). The results demonstrate that our method maintains high EMV efficiency while preventing excessive delay for REVs.

[1]Wei, Hua, et al. "Colight: Learning network-level cooperation for traffic signal control." Proceedings of the 28th ACM international conference on information and knowledge management. 2019.

[2]Wei, Hua, et al. "Presslight: Learning max pressure control to coordinate traffic signals in arterial network." Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019.

[3]Jiang, Haoyuan, et al. "X-Light: cross-city traffic signal control using transformer on transformer as meta multi-agent reinforcement learner." Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 2024.

[4]Li, Zhenyu, Tianyi Shang, and Pengjie Xu. "Multi-Modal Attention Perception for Intelligent Vehicle Navigation Using Deep Reinforcement Learning." IEEE Transactions on Intelligent Transportation Systems (2025).

[5]Jiang, Qize, et al. "Dynamic lane traffic signal control with group attention and multi-timescale reinforcement learning." IJCAI, 2021.

[6]Wang, Tonghan, et al. "ROMA: multi-agent reinforcement learning with emergent roles." Proceedings of the 37th International Conference on Machine Learning. 2020.

[7]Wang, Tonghan, et al. "RODE: Learning Roles to Decompose Multi-Agent Tasks." International Conference on Learning Representations.

[8]Hu, Zican, et al. "Attention-Guided Contrastive Role Representations for Multi-agent Reinforcement Learning." The Twelfth International Conference on Learning Representations.

Hi authors, thank you for your rebuttal. I think there are two remaining concerns:

1. W2 Unclear evaluation protocol is not addressed by the rebuttal

Datasets. Our datasets include two synthetic traffic scenarios, Grid4×4 and Avenue, as well as208two real-world scenarios, Cologne8 and FengLin [10, 11]. The synthetic scenarios are constructed209with idealized grid layouts containing uniformly distributed four-arm intersections. The real-world210scenarios feature a mix of two-arm, three-arm, and four-arm intersections. To simulate realistic211emergency conditions, we schedule the EMVs to depart from the middle point of the traffic episode.212Details about these datasets are shown in Appendix C.1.

You mentioned that you used a dataset, although I understand the general reinforcement learning protol, I wonder how these dataset were used to split into train&test

2. Novelty

I understand the MI role-based MARL has existed for a while, but the entirely missing the related work section on the specific topic makes your experiment design incomplete without situating your work in the previous work and setting yourself apart from the prior work. Instantiating the problem with a real-world problem is OK, but it should be considered an application paper instead of framing it as a new contribution to MARL.