ControlAgent: Automating Control System Design via Novel Integration of LLM Agents and Domain Expertise

We appreciate the reviewers' valuable comments and have provided our responses below. We realize that the reviewer has misunderstood our contributions, and provide extensive clarifications to address this. In addition, based on the reviewer’s suggestions, we have also added new results on the more robust evaluations with pass@k metric and sample numbers needed for each control problem type. We hope these responses address your concerns, and we hope you can reevaluate our paper based on our new results and we also welcome any further feedback.

Q1. While this paper is decently presented and seems to achieve decent results, I am uncertain about recommending it for ICLR. Primarily, the paper seems highly domain-specific and engineering-focused, rather than more general cutting-edge academic research. Still, it is a good engineering system.

Thanks for the positive feedback on the presentation and for acknowledging our results. We really believe that our research is cutting-edge academic research that is particularly relevant to ICLR. As stated by the ICLR 2025 Reviewer Guide, a paper brings value to the ICLR community when it convincingly demonstrates new, relevant, impactful knowledge. A key academic contribution of our paper is the novel integration of two major disciplines - control engineering and large language models. Control engineering is not merely a domain-specific application, but rather a fundamental discipline that underpins modern technology and civilization (from aerospace and robotics to power systems and autonomous vehicles) and has rich mathematical foundations and design principles developed over decades. Our paper presents the first approach to systematically bridging these two disciplines by showing how to encode complex control engineering principles into LLM agents and combine control-oriented computation tools with LLM reasoning to enable ControlAgent to automatically navigate the fundamental performance-robustness tradeoffs in control design. This is beyond a naive integration of control and LLMs. Our key contributions include:

1. Iterative Refinement Mechanism: Mimicking control engineers' decision-making processes to navigate performance-robustness trade-offs.
2. Structured Memory Management: Allowing LLMs to learn from prior designs while respecting context limitations.
3. Multi-Agent Architecture: Enabling specialized LLM agents to collaborate effectively, ensuring technical correctness and efficiency.

Just as the integration of deep learning with physics, chemistry, or biology has led to significant advances in both AI and these respective fields (e.g., AlphaFold), our work shows how the marriage of LLMs with control engineering can push the boundaries of both disciplines. The insights gained from enabling LLMs to understand and apply control engineering principles could inform how we approach the integration of LLMs with other engineering disciplines.

In addition, the ICLR community encourages applications to various domains such as robotics, autonomy, and planning, etc. We see no reason why engineering applications should be excluded from consideration. In fact, prior ICLR has accepted papers focused on LLM applications in application domains such as operations research and geospatial tasks [1,2].

[1] Xiao, Z., Zhang, D., Wu, Y., Xu, L., Wang, Y.J., Han, X., Fu, X., Zhong, T., Zeng, J., Song, M. and Chen, G., 2023. Chain-of-Experts: When LLMs Meet Complex Operations Research Problems. ICLR.

[2] Manvi, R., Khanna, S., Mai, G., Burke, M., Lobell, D. and Ermon, S., 2023. Geollm: Extracting geospatial knowledge from large language models. ICLR.

Our research addresses core challenges in AI/ML research, particularly how to enable LLMs to perform reliable technical reasoning and design in disciplines requiring deep theoretical understanding and sharp engineering insights. The solutions we develop - including the structured integration of engineering knowledge, iterative refinement mechanisms, and multi-agent coordination - represent important advances in LLMs for engineering.

Our empirical validation via ControlEval and the new added results on real-world applications (see our general response, part 2), systematically demonstrates ControlAgent’s effectiveness compared to traditional tools and baseline LLM approaches. The strong performance highlights the value of integrating rigorous engineering principles with LLM reasoning, showcasing a practical path toward leveraging AI for real-world, complex problem-solving. We also believe that our work offers useful insights beyond control by demonstrating how LLMs can be leveraged to solve complex, real-world engineering problems.

To sum up, the fundamental nature of both control engineering and LLMs, combined with the technical depth and novelty of our approach, makes this work relevant to ICLR, which has a strong tradition of publishing papers that advance ML methodology through innovative integration with other disciplines.

Q7. I also don't see reporting of how many samples are taken to achieve the reported success rates. How much sampling is done of LLM-generated designs? e.g. is the budget 10 designs?

For each problem in ControlEval, ControlAgent is run on five independent trials. For each trial, ControlAgent iteratively refines the controller parameters in a feedback manner with multiple conversation rounds (each conservation round requires calling the LLM API once and does not involve any human intervention). Each trial is terminated once a successful design is achieved (as determined automatically and exactly by the Python computation agent) or when ControlAgent reaches the predefined maximum number of conversation rounds per trial. Based on the difficulty level of each problem class, the maximum number of conversation rounds per trial is set up accordingly, and the details are provided in Appendix D.1 paragraph Parameter Setup for ControlAgent (this number is set to be 10 for simpler problem class (stable 1st/2nd-order systems), and 30 for the difficult problem class (e.g. higher order)). For the number of samples (rounds) to achieve the reported success rates, we have reported the averaged iteration number in Table 4, where we show that for first-order stable systems, for each task, on average the ControlAgent only needs less than 3 iterations for all cases. In the table below, we present the average number of conversation rounds (sample size) for each type of control problem:

The results provide an average estimate of the sample efficiency of ControlAgent. As expected, the sample number needed increases as the problem type becomes more difficult. To offer a more comprehensive analysis, we also provide a breakdown of the conversation round distribution for each problem type below.

Q6. While the method is interesting, it seems to be an incomplete solution to a highly domain-specific problem, so I'm unsure about the larger impact of the work, e.g. the paper doesn't give much insight into designing general LLM-based systems.

We thank the reviewer for the feedback and appreciate the recognition of our method as an interesting approach. We respectfully disagree with the assessment that our framework is limited to a highly domain-specific problem. On the contrary, we believe that ControlAgent offers a versatile and generalizable solution that can be adapted to a wide range of engineering domains and applications.

ControlAgent can solve many real-world problems: Control engineering is not merely a domain-specific application, but rather a fundamental discipline that underpins modern technology and civilization, from aerospace and robotics to power systems and autonomous vehicles. Our ControlAgent can be readily adapted to the control of real-life applications including flight control, DC motor, high speed train control, hard disk drive control, etc. We have added a new application category in ControlEval to demonstrate the utility of ControlAgent in such applications. More details can be found in Appendix Section C in the revised paper.

General Framework Beyond a Specific Domain and Insight into LLM-based System Design: While our primary evaluation focuses on control design, the core principles of our framework, design, evaluation, feedback, and iterative improvement, are common features across many engineering domains. We believe that our work not only provides a concrete case study on how to extend LLM capabilities beyond standard reasoning tasks into real-world engineering design scenarios, but also can serve as a blueprint for future AI systems targeting a wide range of complex engineering design problems. Our paper brings the following key insights into designing general LLM agent systems for serving as AI engineers.

1. Our study sheds light on the importance of using domain expertise to address context window limitations via selectively storing and providing only the essential historical information to the LLMs based on the domain expertise in the specific targeted engineering field. For ControlAgent, the memory module only retains key data such as design parameters, performance metrics, and feedback from previous iterations, while excluding unnecessary details from the LLMs’ responses. This ensures that the memory buffer remains compact and efficient. Such insights can be potentially useful for developing LLM agent systems to target other engineering design problems such as circuit design and structure design.

2. Our paper also brings general insights on how to build LLM agents to mimic human engineers. Specifically, one has to combine the design recipes from human engineers with the LLM agents.

3. Our paper highlights the importance of integrating an exact evaluation agent (in ControlAgent, it is the Python Computation Agent which gives the exact evaluations of the performance and robustness of the designed controllers) for building LLM agents in the context of engineering design.

To summarize, we believe that it is fair to claim that our ControlAgent paper does bring important insights into designing general LLM agent systems for serving as AI engineers.

Regarding the relevance of our paper to the ICLR community, please also see (Author Response, Part 1).

Q4. Clarify about "a system is considered successfully designed if at least one of the multiple independent trials results in a successful design". It seems this would greatly skew the statistics, since failures are being filtered out.

We thank the reviewer for the constructive comment. We have added a new experiment to evaluate a more robust evaluation with pass@k metric in Section D.4.2 in our revised paper. However, we would like to mention that our main evaluation metric is averaged successful rate (ASR), which is computed as the fraction of successful design among multiple independent trials. Formal definitions of ASR can be found in Section D.2 in our revision. ASR is robust and should not skew the statistics. From Table 1 (we reported ASR not AgSR) in our main paper, we have shown the superior performance of ControlAgent to the baseline methods. AgSR is designed in an ensemble manner, similar to metrics used in previous works such as [1]. This approach is meaningful in practical applications, particularly in control system design, where it is often feasible to use the Python Computation Agent to automatically filter out failed designs and pick the best design solution.

We agree that AgSR could potentially introduce high variance. To address this, we employed the advanced pass@k metric as presented in [2], which is designed to be unbiased and reduces variance. We computed pass@k with k = {1, 3, 5} and n = 5, as detailed below:

It can be seen that ControlAgent is able to achieve high pass@k rates, even for complex tasks, highlights its robustness and effectiveness in handling diverse control problems.

[1] Kulal, S., Pasupat, P., Chandra, K., Lee, M., Padon, O., Aiken, A. and Liang, P.S., 2019. Spoc: Search-based pseudocode to code. NeurIPS.

[2] Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P.D.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G. and Ray, A., 2021. Evaluating large language models trained on code.

Q5. Given the unpredictable and error-prone nature of LLMs, I am skeptical that the overall system can work without a human in the loop or method for filtering correct answers. I would expect it to be rather fragile.

We emphasize that ControlAgent is fully automated and there is no human in the loop. We even provide a real DC motor example in C.2 to showcase that ControlAgent can work for real problems without human in the loop. Please see our general response. More explanations are provided to explain why our approach is not fragile and can work reliably.

Constrained Reasoning Space for LLMs: We acknowledge the reviewer’s skepticism about the unpredictability of LLMs. However, in our framework, the LLM’s reasoning space is constrained by the problem context and task-specific requirements. By distilling control knowledge in loop shaping as system prompts for LLM agents, the likelihood of generating irrelevant or incorrect content is reduced.

Robust Python Computation Agent: Our Python Computation Agent is exact and deterministic, free from any generative hallucinations. This agent handles all numerical computations and logical verifications, ensuring precise and reliable outcomes. The determination of whether a design is successful or not is made solely by the Computation Agent, not the LLM. Thus, this step is exact, leaving no room for hallucinations to affect the overall system.

Accurate Feedback Mechanism: The feedback module in our framework is designed to be exact and accurate. Intermediate designs generated by the LLM are passed to the Computation Agent for validation, which also rigorously checks the current design against task-specific requirements and provides precise feedback to guide subsequent LLM designs. By continuously validating and refining the LLM’s designs, we mitigate potential hallucinations and ensure robust system performance.

Iterative Design Process: Although the risk of hallucination is inherent in LLMs, our framework additionally mitigates this issue through a multi-step reflection process. We employ an iterative design and verification process, where the LLM reflects on its previous designs and adjusts accordingly, based on exact feedback. This iterative approach also enhances robustness/reliability.

The superior performance of ControlAgent on ControlEval (and extra applications in Appendix C) supports our claim that ControlAgent has addressed the potential issues from LLM hallucinations and can operate reliably without requiring a human in the loop.

Q2. The proposed method is essentially a domain-specific application of LLM-modulo, e.g. an interative prompt with a verifier and critiques [1].

We thank the reviewer for bringing the LLM-modulo framework [1] to our attention. We have included this reference in our updated paper. While we agree that there are similarities between ControlAgent and LLM-modulo frameworks, we would like to clarify the key differences and highlight the unique aspects of our approach. Specifically, our ControlAgent addresses the unique challenges specific to engineering design. The verifier and critique framework from LLM-modulo does not tell us how to re-design the engineering solutions based on all the past outcomes from the verifier and critiques. Our ControlAgent combines domain expertise with LLM reasoning to perform such iterative design processes that mimic how human control engineers iterate control system design. In addition, ControlAgent tackles design tasks for a wide range of dynamic systems from stable first-order systems to unstable higher-order systems, creating a fully automated design pipeline (we emphasize that there is no human intervention). By integrating domain-specific instructions, implementing an iterative design process with accurate feedback, conducting a robust, deterministic evaluation, and introducing the ControlEval evaluation dataset, ControlAgent sets itself apart from other approaches and demonstrates its capability in handling complex, real-world engineering tasks.

[1] Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., ... & Murthy, A. B. Position: LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks. In Forty-first International Conference on Machine Learning.

Q3. It seems guarantees would be desirable when working with control systems, and I assume the problem requirements are meant to be guarantees. However, I feel the paper would be made a lot stronger by discussing guarantees at length.

We thank the reviewer for this suggestion. We have added more detailed discussions on the guarantees in Section B.3, B.4, and B.5. Yes, the designed controllers from ControlAgent have strong performance guarantees in term of settling time and strong robustness guarantees in term of phase margins. As commented by Reviewer AbjT, the iterative design process of ControlAgent is noteworthy for its theoretical soundness. Our ControlAgent framework is the first LLM agent framework in automating the process of designing controllers that can rigorously meet pre-specified requirements on performance (settling time) and robustness (phase margin) and effectively navigate the fundamental performance/robustness trade-off in classic control design. ControlAgent uses a Python computation sub-agent to automatically and exactly evaluate the stability, settling time, and phase margin of the designed controllers, and put such exact information into a memory module such that the LLM sub-agents exactly know whether the current designed controller meets the user-specified performance/robustness requirements or not. The Python computation sub-agent is integrated as a module of ControlAgent and works in a fully automated manner to ensure the ControlAgent always know the true settling time and phase margin of its own designed controllers. Whenever ControlAgent gives a successful design, it has to pass the verification of the Python computation agent such that the performance guarantees on settling time and the robustness guarantees on phase margins are both ensured.

I greatly appreciate the Author's detailed response, and apologize for my original review which, honestly, was preliminary and should've listed a lower confidence.

Given the level of effort demonstrated by the authors, I will be raising my score.

Thanks for the feedback. As noted by the other two reviewers, we believe our presentation is clear. It is unclear to us why the reviewer stated that our paper lacks an introduction section, background information, and an approach section. Our submission includes all of these elements. Specifically, we have a preliminary section that provides background information. Section ControlAgent outlines our approach. One minor typo we have in the submission is that we mistakenly named the "introduction" section as "instruction". However, it is quite obvious that the first section in our paper is an introduction to our paper.

That being said, we have made additional efforts to further improve the paper and address potential concerns. Specifically, we have:

1. Expanded Section B to include more detailed background information.
2. Added real-life application scenarios to Section C to better illustrate the practical relevance of our framework.
3. Included additional experimental details in Section D to further support our methodology.

We hope these updates strengthen the paper and address the reviewer’s concerns.

Q4. The framework's heavy reliance on proprietary LLM models raises questions about reproducibility and practical deployment. The authors don't thoroughly explore how the system's performance might vary with different base LLMs or how it might degrade with smaller, more practical models.

We thank the reviewer for the thoughtful comments regarding the reliance on proprietary LLM models and concerns about reproducibility and practical deployment. To address this, we have included additional experiments using a more accessible, open-source LLM backbone: Llama-3.1-70b.

We evaluated ControlAgent with the Llama model on four representative tasks: three first-order stable systems (with fast, moderate, and slow response modes) and a more challenging task involving higher-order systems. The results are provided below, where we report the metrics ASR, AgSR, and pass@k with $k=1,3,5$ (the formal definition of pass@k metric can be found in Section D.2). Additionally, we include the averaged number of samples required, stability margins designed by ControlAgent for completeness.

It can be seen that ControlAgent with Llama-3.1-70b is also effective for simpler, first-order control tasks but faces challenges with more complex, higher-order systems. The $pass@1$ rate is only 0.300, indicating that the model struggles to solve the problem on the first attempt. The $pass@3$ and $pass@5$ rates improve to 0.446 and 0.480, respectively, but still remain below 50%, suggesting that the task is considerably more challenging for the model.

In addition, the average number of iterations required for first-order stable systems is relatively low, with moderate response mode requiring the fewest iterations (2.016), followed by slow (2.824) and fast (3.053). In contrast, the higher-order system requires a significantly higher average number of iterations (24.5), reflecting the increased complexity and difficulty of the task.

Q5. How does ControlAgent handle model uncertainty? While you discuss robustness through phase margin, could you elaborate on whether the framework considers parametric uncertainties or unmodeled dynamics?

Thank you for your thoughtful question regarding how ControlAgent handles model uncertainty, including parametric uncertainties and unmodeled dynamics. ControlAgent heavily relies on loop-shaping, which is well-known to yield good gain margins (for parametric uncertainty), and phase margin (for phase variations and tolerance of time delays). In addition, the famous loop-shaping theorem states that loop-shaping also yield a general disk margin for a value at least at 0.4. The disk margin provides a comprehensive robustness measure by simultaneously addressing both gain margin and phase margin, providing robustness against non-parametric unmodeled dynamics. To summarize, although phase margin and settling time are used as tuning knobs for ControlAgent, reasonably good gain margins and disk margins are also automatically addressed by ControlAgent. We have added a new disk margin analysis of ControlAgent in Section D.4.4.

The following table demonstrates the disk margins for controllers designed by ControlAgent, showing robust stability across various control problems.

These results show that ControlAgent maintains adequate robustness margins even under varying conditions, reinforcing its capability to handle parametric uncertainty and unmodeled dynamics effectively.

Q2 (Part 1). The comparison with baselines is somewhat limited. The paper primarily compares against relatively simple LLM-based approaches (zero-shot, few-shot) and a single traditional tool (PIDtune). Modern control design often employs more complex methods like robust control, model predictive control, or optimization-based approaches, which are notably absent from the comparison.

We agree with the reviewer that modern control design often involves advanced methodologies such as robust control (H-infinity and mu-synthesis) and MPC. However, our current focus is on classic control on PID and loop-shaping, which represents the majority of control design in industrial applications (there is a well-accepted folklore that 90% of controllers in the control industry are PID or loop-shaping). For such classic control that dominates the control industry, PID tuning methods would be the most direct competitor and we have compared ControlAgent against PIDTune, which is popular for industry use.

Our work also motivates the need of future study on developing LLM agents to design H-infinity robust controllers and MPC schemes with the right cost function and time horizon for the remaining 10% of advanced control tasks in industry. For such future study, it makes sense to compare to existing robust control or MPC methods that require human experts for setting up (for example, traditionally, human experts need to carefully set up the so-called "weighting functions" such that robust control can be applied to a specific problem). We emphasize that the use of robust control methods or MPC methods even require more human expertise and computational resources than PID tuning, and this is the main reason why they are much less popular than classic PID/loop-shaping control methods for control industry.

Q2 (Part 2). The performance metrics are also relatively basic, focusing mainly on settling time and phase margin while overlooking other important characteristics like disturbance rejection and noise sensitivity.

Phase margin, settling time, and stability are fundamental metrics that are widely used in control industry. These metrics are well-defined over a wide class of problems and in general less problem-dependent, capturing the performance/robustness trade-off in classic control design. In contrast, disturbance rejection and noise sensitivity are more problem-dependent in the sense that disturbance and sensor noise can have very different patterns for different application tasks (IID Gaussian assumptions are not widely used in control industry, and the evaluations on disturbance and noise rejection eventually require hardware experiments). Given the current resources we have, we added the study on noise sensitivity for a real DC motor in the physical real world. We show that the controller designed by ControlAgent achieves good reference tracking and noise rejection simultaneously on a real DC motor under real sensor noise. Please see our discussions in Appendix C.1 of our revised paper.

Q3. The iterative design process lacks theoretical guarantees of convergence or optimality. The paper doesn't provide analysis of when or why the iteration process might fail, nor does it establish bounds on the number of iterations needed for convergence.

Thank you for raising this important concern about the theoretical guarantees of convergence and optimality in the iterative design process. We want to argue that even human control engineers cannot guarantee that they can always converge to a successful PID control design. Since ControlAgent mimics how human practicing controllers to design control systems, it seems natural to expect that ControlAgent will not have theoretical convergence guarantees. We agree with the reviewer that we should provide analysis for when or why the iteration process of ControlAgent might fail. We provide such failure analysis in Section D.4.5 of our revised paper. Please also see our response to Q7 below in (Author Response, Part 5).

Representativeness of Real-World Control Applications

As mentioned above, to improve the representativeness of ControlEval on real-world control application, we have added two new sets of important results to our revised paper. Now details are given below.

First, we expanded ControlEval with a new application category consisting of 10 real-world control design application problems from diverse resources. In addition, for these 10 real-world physical systems, we randomly generated the design requirements for settling time (for real-world applications, only upper bounds are needed to ensure fast tracking) and phase margin within proper range to reduce human bias. The information of these application problems are briefly given below.

The details of these applications and related resources are provided in Appendix C.1 of our revised paper. We have also obtained the evaluation results for ControlAgent and baseline methods as below:

One can clearly see that ControlAgent outperforms all the baselines in this new class of real-world control application problems. More details can be found in Appendix C.1 of our revised paper.

Implement ControlAgent with a real-world DC motor control task.

Secondly, we evaluated ControlAgent and all the baseline methods on a real-world DC motor control task, see Figure 7 in the revised paper for demonstrations. The model used for design is a third-order model in the form of $T(s)=\frac{K }{s\left((L_a s+R_a)(J s+b)+K_\tau K_v \right)},$ where there is a gap between the model and the real motor that the control is deployed. Specifically, we implement the designed controllers from ControlAgent and all the baseline methods on a real DC motor and provide real-world data on the controller performances. The results in Figure 8 again demonstrate that ControlAgent gives much better design than all the baseline methods when the designed controllers are deployed on a real system (details are given in Appendix C.2 of our revised paper).

We believe that our new results have convincingly demonstrated the practical use of ControlAgent across a wide range of real-world control tasks, significantly improving the empirical soundness of our work.

We sincerely thank the reviewer for the constructive feedback and have provided detailed responses below to address all the reviewer’s comments carefully. Based on the feedback, we have made several significant updates to the paper: clarified higher-order dataset generation process, expanded ControlEval with new results on 10 real-world control application problems, demonstrated ControlAgent on a real DC motor in the physical real world, evaluated the performance of ControlAgent using a more accessible and smaller LLM (Llama-3.1-70B), and provided additional robustness evaluation results for controllers designed by ControlAgent. We hope these updates address the reviewers' concerns and invite you to reevaluate our paper based on these new results. We welcome any further feedback and suggestions for improvement.

Q1. The paper doesn't clearly justify the distribution of these tasks or demonstrate their representativeness of real-world control problems. The generation process for higher-order systems is particularly problematic – the authors admit to manually designing these cases, which could introduce bias and may not reflect the true complexity of higher-order system control. What criteria guided your selection?

A1: Thank you for this insightful comment. For first-order and second-order systems, we randomize important system parameters such as dominant pole, DC gain, natural frequency, and damping ratio to ensure the representativeness of the selected problems. For higher-order systems, it is true that we manually designed 50 stable and unstable higher-order systems along with their associated control requirements for ControlEval. For representativeness, we manually add two or three extra poles to 1st/2nd order systems. We make sure that the problems cover both the case where the added poles are far away from the original 1st/2nd-order poles and the case where the added poles are close to the original poles. We use our own control domain expertise to ensure that the resulting systems remained controllable and adhered to practical design requirements, such as there existing a controller for the designed system to achieve desired settling time and phase margin. A subtle point that should be emphasized is that the generation of higher-order systems cannot be completely random since there are higher-order systems that are fundamentally difficult to control (e.g., the system with the right half plane zero and the right half pole being close to each other is fundamentally difficult to control) and such systems are avoided by practical system design so that we should not include them in our evaluation set. Therefore, it is quite difficult to have a fully randomized generation process for higher-order systems. We definitely agree with the reviewer that we need to improve the representativeness of real-world control problems. To achieve this, we have added two new sets of important results: i) we expanded ControlEval with a new application category consisting of 10 real-world control design application problems from diverse resources, ii) we evaluated ControlAgent and all the baseline methods on a real DC motor control task in the physical real world. More discussions are given in our next response (Part 2).

Q6. For the history and feedback module, how do you handle the context window limitations of LLMs? Could you provide more details about the memory management strategy?

Thank you for your question regarding our approach to memory management and handling context window limitations. In ControlAgent, we address these limitations by selectively storing and providing only the essential historical information to the LLMs based on our own control expertise. Specifically, we retain key data such as design parameters, performance metrics, and feedback from previous iterations, while excluding unnecessary details from the LLMs’ responses. This ensures that the memory buffer remains compact and efficient. For example, here is an illustration of two historical designs stored in the memory buffer for one specific task:

### Design 1
Parameters:
- omega_L = 5.5  
- beta_b = 3.162  
- beta_l = 3.162  

Performance:
- phase_margin = 15.89  
- settling_time = 3.20  
- steadystate_error = 0.0  

Feedback:
- Phase margin should be at least 52.82 degrees.  

### Design 2
Parameters:
- omega_L = 6.5  
- beta_b = 4.0  
- beta_l = 4.0  

Performance:
- phase_margin = 30.06  
- settling_time_min = 3.60  
- steadystate_error = 0.0  

Feedback:  
- Phase margin should be at least 52.82 degrees.

Only the above summarized history is fed into the LLM for the next iteration. This strategy allows the LLM to focus on refining the design based on the key feedback and performance metrics, without exceeding the context window limitations. From our observations, this approach effectively prevents context window overflow while maintaining the iterative design process to be effective. Additionally, it ensures memory efficiency by retaining only the critical information required to improve upon previous designs. We have added the above discussions in Section D.3.

Q7. Could you provide a more detailed analysis of failure cases, particularly for higher-order systems where performance was lower? Understanding these cases would help assess the framework's limitations.

Thank you for this thoughtful question. We have added a new failure mode analysis of ControlAgent for higher-order system design in Section D.4.5. Specifically, we identified several failure modes in the LLM's approach to controller design, each revealing challenges in reasoning and parameter adjustment strategies:

1. Calculation Errors: One notable failure occurred with a marginally unstable system featuring a double integrator. The LLM incorrectly calculated the minimum loop bandwidth as below: "The fastest unstable pole is at 0, so we initially chose $\omega_L = 2.5 \times 0 = 2.5.$" This calculation was incorrect and did not align with proper design principles.

2. Incomplete Parameter Adjustments: The LLM often adjusted only two parameters ($\omega_L$ and $\beta_l$), neglecting $\beta_b$, which is crucial for balancing the settling time and phase margin. For example, in one design, the final parameters were $\omega_L = 60$, $\beta_b = 0.8$, and $\beta_l = 1000$, with $\beta_b$ remaining unchanged throughout iterations. This limited adjustment scope hindered optimal design.

3. Hallucination Errors: Another failure involved misidentifying the dominant pole. In one instance, the LLM incorrectly identified -50 as the dominant pole instead of the actual dominant poles at $-2.1 \pm 2.142$: "The poles at -50 and $-2.1\pm2.14242853j$ suggest a relatively fast response due to the dominant pole at -50." This misunderstanding led to incorrect design decisions.

These examples highlight key areas where the LLM's reasoning and parameter optimization strategies need improvement. Addressing these failure modes is a priority for future iterations of the framework.

Dear Reviewer,

We want to thank you again for your valuable comments and constructive feedback that helped us improve our paper significantly. In response to your suggestions, we have made significant improvements to the paper including: adding more evaluations with open-sourced model; expanding ControlEval with 10 more tasks related to real-world applications; demonstrating ControlAgent on the hardware of a real DC motor control system in the physical real world; adding more evaluations with disk margins. We believe that such revision has significantly improved our paper, and we hope that all your concerns have been addressed. With the discussion period closing soon, please feel free to let us know if you have any further questions or feedback. Thanks so much!

Sincerely,

Authors