ABNet: Adaptive explicit-Barrier Net for Safe and Scalable Robot Learning

We really appreciate the reviewer for all the positive and helpful comments. We address the remaining comments below.

(1) The part of references/related works is okay, but it would be nice to have more recent papers included, like the ones published in 2023 and 2024.

Response: We will add more recent references as suggested by other reviewers, especially those in safe RL and CBFs for manipulation.

Weakness 1 Novelty. The explicit barrier can be seen as a novelty, but it is not significant at all. The way of linear combined control output by different heads is not new. And thus the novelty of the whole paper is concerning.

Response: The proposed explicit barrier is significant in improving the computational efficiency in scalable training, as demonstrated in Fig. 3, we can significantly improve the computation time with our method compared to the benchmark (dQP) from both the batching and training perspectives. Computational efficiency is very important in large safe foundation models, especially when those models are trained in a scalable way (as proposed in this paper).

The linear combined control output is simple, but also very effective. However, proving the property (e.g., safety) of the combined control is non-trivial and challenging. The main contribution of our work is to show the safety of the combined control (Thms 3.1 and 3.2, and their proofs in Appendix B), which has not been done in the literature. We showed the existence of a new HOCBF constraint from the combined control, and thus the idea or approach of safety proof is indeed novel.

Weakness 2 Regarding novelty, as the authors want to push this paper towards a " safe foundation model", I would suggest considering adding a self-attention layer for the outputs by different heads. In this way, even if some heads' output actions are not safe (due to learning errors, etc), the self-attention auto-weighting has the ability to correct the final output.

Response: Thanks a lot for the constructive suggestion. Adding a self-attention layer for the outputs is indeed very interesting. However, proving the safety of the combined control is non-trivial and challenging with such self-attention layers. We will further explore this. One possibility is to consider the linear attention mechanism (e.g., [1] Transformers are rnns: Fast autoregressive transformers with linear attention). We will add the discussion in the revision.

Question 1 Are there existing works of 'safe foundation model'? If yes, has this paper discussed them?

Response: We found some survey papers regarding the safety of foundation models (e.g., On the opportunities and risks of foundation models), and we will discuss them in our revision, especially from the perspective of the importance of safety in foundations models, which will make our work stronger. Please let us know if the reviewer has any other suggestions for references.

We thank the reviewer for all the positive and constructive comments. We address the remaining comments below.

(1) The paper needs a clarification on the scope of systems. The target dynamics is of a general control affine form while an assumption is made on relative degree m...

Response: The proposed method can be applied to general nonlinear systems. The problem that the coefficient of the control in the CBF constraint becomes 0 at some state is the so-called ``singularity’’ problem (if it is zero for all states, then the relative degree should be increased by one until it is non-zero). This problem can be solved by defining a CBF that avoids those states (e.g., see [1] High-order barrier functions: Robustness, safety, and performance-critical control). When there are multiple controls, there may be the so-called mixed relative degree problem, and we can address this by defining auxiliary dynamics to make all the controls show up (e.g., see [2] Control barrier functions for systems with multiple control inputs). When we have non-affine control systems, we can also define auxiliary dynamics to ensure that the method can still work (e.g., see [3] On the forward invariance of neural ODEs). We will make this clear in the revision.

(2) Limited to low-dimensional system and action space...

Response: We totally agree with the reviewer. The scalability is mainly in terms of the safety of the robot, the size of the safe robot learning models, and the computational cost of the model. Our method can also work for high DOF scenarios (e.g., manipulation) by replacing the corresponding system dynamics with the high DOF dynamics (e.g., manipulator dynamics) in the model. We will discuss this in the revision.

(3) The paper seems to take visual observations as piece-wise constant input, ...

Response: We thank the reviewer for pointing this out. Although we are taking piece-wise constant input for visual observations, the proposed model works for continuous visual observation as well. This is indeed a very interesting direction that involves differentiating the image input (e.g., in the form of optical flow). We will add this discussion in the revision.

Question 1

Response: Our method works for dynamics without a globally constant relative degree as well. There are many approaches to deal with such problems, e.g., by defining a CBF that avoids those states (e.g., see [1] High-order barrier functions: Robustness, safety, and performance-critical control). In fact, we only need to care about the states at the boundary of the unsafe sets that may make the coefficient of control in HOCBFs become zero, and safety can still be preserved in such cases.

Question 2

Response: The implication to other applications is that we always consider the most threatening factors in guaranteeing safety, and it may not necessarily just be related to the activeness of constraints, but also related to the importance of those constraints. For example, autonomous vehicles should always follow traffic rules, and it should always consider the most important rule (e.g., ensure collision free to pedestrians) rather than less important rules (e.g., lane or road keeping). We do need to care on how to partition the constraints when the problem scales up, and this can be taken care of by leveraging other factors (such as ethics and local culture in driving), e.g., see [1] Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks [2] Rule-based optimal control for autonomous driving.

Since the model outputs controls in real time, we can definitely change the activeness of constraints in real time as well. We implement the model in discrete time, thus it won’t cause any differentiability problem since we can address the inter-sampling issue (safety in continuous time) using event-triggered approaches (e.g., [3] Event-triggered control for safety-critical systems with unknown dynamics). Another way to address this differentiability problem is to use the soft min approach (as discussed right before equation (4)) to combine and consider all the constraints.

Question 3

Response: This is really a good point. Since we always consider the two most important constraints at each time step, and we can get the closed-form solution, therefore, we do not find any infeasibility issue for now (however, there may be other problems that are worth further exploring in future work). This provides a promising way to address the infeasibility problem, and we will add the discussion in our revision.

Question 4

Response: We also have some state- net to predict the vehicle and obstacle states from image observations (as also demonstrated in the BNet). The demonstrated safety is based on the assumption that the state is reliably predicted. In cases where there are some uncertainties for the prediction of the states, we can use robust CBFs in our framework. We will add the discussion in the revision.

We appreciate the reviewer for all the helpful and constructive comments. We address all the concerns below.

(1) Claims are not well-supported by experiments, writing is not particularly clear.

Response: Our main claim is safety guarantee of the model, and this is supported by the SAFETY or CRASH in Tables 1-4. The computational efficiency of our method is demonstrated in Fig. 3 with comparison to benchmarks. The performance improvement in scalable learning is shown in Figs. 4-6. We will improve the writing with preliminary on CBFs and BNet in the revision.

(2) Fundamental distinction with Safe RL and optimization approaches based on BFs and Lagrangian

Response: There is indeed a fundamental distinction between our method and Safe RL. In safe RL, safety is usually taken as a component of reward function, and thus, it can only improve safety without guarantees. While our method can formally prove safety, as shown in Thm. 3.1 and 3.2. Safety guarantees are also demonstrated in Tables 1-4. We will add discussions on safe RL and include the references in the revision.

(3) Complexity of environments not well-articulated. Could the authors provide clarification on the environments?

Response: The complexity of the experimental environments is given in Appendix section C. In summary, we consider different nonlinear dynamics and constraints across three different tasks. We have one complex vision-based end-to-end autonomous driving experiment in which the model directly takes the front-view image as input, and outputs the safe control. All the experiments are done in VISTA, and it is a sim-to-real driving simulator that generates driving scenarios from real driving data [Amini, et.al 2022]. The VISTA allows us to train model with guided policy learning. This learning method has been shown to work for model transfer to a full-scale real vehicle. This is also given in Appendix section C.4.

(4) Does this method remain effective in environments requiring complex contact handling?

Response: Our method is still effective in environments requiring complex contact handling, in which case we just need to replace the dynamics by the ones of manipulator. The safety may be different in contact scenarios (e.g., force constraint instead of collision avoidance). The CBF method has been widely used in manipulation in complex environments (e.g., Safe Multi-Robotic Arm Interaction via 3D Convex Shapes), since our model is based on the CBF method, we can apply our model to those complex scenarios when learning is involved.

(5) Lack of clarity in the Background and Related Work.

Response: We will add preliminaries on CBFs and BNet to improve the clarity, and discuss more on safe RL and other related works in Related Works section (now the related work is given in Sec. 5).

(6) Some related work is not discussed in the paper.

Response: We will add the references suggested by the reviewer and other reviewers (e.g., safe RL), and discuss them.

(7) What are the key advantages of using Barrier Functions in the proposed approach?

Response: The key advantages of using barrier functions in the proposed method is that we can formally train models that have safety guarantees in a scalable way, as shown in Thms. 3.1 and 3.2 and Tables 1-4. Another advantage is the high efficiency of our proposed method, as shown in Fig. 3.

(8) Applicability constrained by known and differentiable constraint functions. Uncertainties, noise, and sensor errors may further degrade performance, potentially leading to failures.

Response: There are many related works that learn differentiable constraint functions from demonstrations, such as [1] Learning control barrier functions from expert demonstrations and [2] Synthesis of control barrier functions using a supervised machine learning approach. Therefore, we can combine existing literature with our ABNet to learn constraints in scenarios that the safety is not predefined. This has been made clear in the future work section 6. When there are uncertainties, noise or sensor errors, we can employ robust CBF methods in our model (e.g., Fault tolerant neural control barrier functions for robotic systems under sensor faults and attacks). We will make this clear in the revision.

(9) large language models (LLMs) (safely aligned): preventing harmful content—requires defining constraint functions for “human safety,” How do the authors view this issue in the context of their approach?

Response: Thanks a lot for pointing out the safety problem of LLMs, we have been actively working on this and on other models or problems where safety is not clearly defined. In LLMs, in order to avoid toxic language, we may learn a differentiable risk function according to the harmfulness of the content, and then we can take this risk function as a CBF in our approach such that we ensure that the harmfulness of generated content be below some level. We will discuss this in the revision.

We appreciate the reviewer for all the positive and constructive comments.

(1) Specific technical challenges

Response: There are two main technical challenges: (a) The training and testing efficiency of the scalable robot learning model; (b) Formal proof for the safety of the composed model in scalable robot learning;

(2) Innovative solutions to overcome these challenges

Response: For challenge (a), we proposed the explicit-barrier approach (in which a closed-form solution for the QP is given) to significantly improve the efficiency of the model, as shown in Fig. 3. For (b), we show the existence of a new HOCBF condition from the composed model, which has not been done in the literature ((21) of Appendix sec. B).

(3) Distinctive advantages

Response: The most distinctive advantages of the method is the safety guarantee, as shown in Tables 1-4 (SAFETY). This is also the main contribution of our work. We also show the improvement of performance with the increase of heads of explicit-barrier, as given in Figs. 4-6 and Table 3 (PASS).

(4) Experiment are relatively simplistic, and lack qualitative analysis.

Response: We have two intuitive experiments that are easy to understand (from dynamics, safety perspective). We also have one complex vision-based end-to-end experiment to show how our methods can work in realistic scenarios. We have shown qualitative analysis in Fig. 4, Fig. 8, 9, 11. All the qualitative results are from videos, and we will attach them in the paper revision (the rebuttal only allows pictures).

(5) Optimal performance: Is there a mechanism to balance performance and safety?

Response: The objective of this paper is to show the safety instead of optimality. However, as we solve dQPs, we can indeed ensure optimal task performance. For applications where safety is not critical, we can relax the first condition in (3) with a slack variable, and minimize it in the cost (2). In this way, we can balance task performance and safety. We will add this in the revision.

(6) Criteria for selecting penalty functions: how does the method balance conservatism and optimality?

Response: The penalty functions in (3) are output of previous NNs or trainable parameters, and we do not need to select them by hand. Smaller penalty functions will make the robot more conservative. Our method finds optimal functions through training of the model to achieve a desired balance between conservatism and performance (given by the training data). We will make it clear in the revision.

(7) Figure 6: performance differences between ABNet and methods like MPC or BNet. Table 3 does not provide a comparison with MPC.

Response: There is a clear improvement of ABNet over BNet as trajectories with BNet stop near the obstacle (shown by blue trajectories in Fig. 6), while the ones of ABNet can safely pass obstacle. The MPC is the ground truth (that is computationally expensive, cannot be applied in real time), and thus it is not included in Table 3. We will make this clear in the revision.

(8) Scalability to dynamic obstacles, and testing in more advanced simulators.

Response: Our ABNet can be scaled to dynamic environments as we just need to incorporate the dynamics and states of obstacles in (3). However, identifying dynamics and states of obstacles requires more powerful models, which does not rely on ABNet (not a limitation of ABNet). The ABNet mainly provides safety guarantees, and we can use existing literature in dynamics and state estimation to better augment ABNet in dynamic environments. We will explore it in future, and make it clear in the revision.

Weakness: Dependence on predefined, differentiable functions.

Response: There are many related works learning differentiable constraints from demonstrations, e.g., [1] Learning control barrier functions from expert demonstrations, [2] Synthesis of control barrier functions using a supervised machine learning approach. Therefore, we can combine existing literature with ABNet to learn constraints in cases that safety is not predefined.

Suggestion: Large transformer models with safety planning. How do the authors view online planning, and could barrier functions be integrated to enhance safety?

Response: Existing approaches with large models can improve safety, however, there are no guarantees. Our method can be integrated into them to formally guarantee safety. The transformer models can be integrated either in the upstream or downstream of ABNet (e.g., via linear attention such that we can still prove the safety of the model).

Question: Can the method handle complex contact scenarios?

Response: Our method can handle complex contact scenarios by replacing the dynamics with the ones of humanoid. The safety may be different in contact scenarios (e.g., force constraint instead of collision avoidance). This would be another important application of our model, and we will discuss it in the revision.