We thank the reviewer for their thoughtful feedback. We address the concerns below.

---

Quality 1: Chosen divergence and impact of $\beta$

Thank you for highlighting the chosen divergence in UOT. This work uses the well-studied KL divergence, which finds diverse applications in information theory, machine learning and optimal transport. Exploring other metrics or divergences is of course interesting and left as our future work.

Thank you for highlighting the parameter $\beta$. We have added a thorough discussion on the impact of $\beta$ from both the theoretical and practical perspectives. The theoretical discussion can be found in Common Response C2.3. Practically, we conducted additional experiments on linear classification to assess the model's performance across a broader range of $\beta$. Since changes in $\beta$ affect the function values, we used accuracy as the performance measure. As shown in the new Table 3, our method achieves the accuracy above 90% across the range of $\beta$ values from 0.5 to 20 and $\lambda$ values from 5 to 20. This performance is better than that of standard DRO with accuracy 66%, and OR-WDRO with accuracy 79%. Besides, as analyzed in Common Response C2.3, the model's performance degrades when $\beta$ is either too small or too large. Overall, these simulation results show that our method is relatively insensitive to the choice of $\beta$ and easy to be tuned.

---

Quality 2: Explanation of our method and large $\lambda$

In lines 215-225, we explained our method using the case of large $\lambda$ because it simplifies the function, making it easier to explain. We would like to clarify that our method UOT-DRO has outlier robustness regardless of the $\lambda$ value. Please see Common Response C1 for a general explanation.

In Section 5.1 on linear regression, we show that Eq (15) can be simplified to Eq (16) if $\lambda\geq \lambda_2 + \kappa(\theta)$. This condition is satisfied when we select $\lambda=10$, $\lambda_2=5$, since the true $\theta$ we use to generate data has a norm $1$ and $\kappa(\theta) = \left\| [\theta,1]\right\|_*\leq 2$. We apologize for any lack of clarity previously. The influence of outliers is reduced through assigning a small weight thanks to the function $h$, see lines 215-225 and Common Response C1 for more discussions.

---

Clarity

Thank you for your suggestion regarding clarity. We will enhance the clarity in the revised draft. The cost functions for linear regression and classification are the same as those in [17] to facilitate fair comparison. The cost function of logistic regression is taken from the seminal work [15].

---

Evaluation and Q4: Comparison with approaches in [1,3,4]

Through this comparison, our goal is to highlight the advantage of our method in handling outliers by utilizing unbalanced optimal transport. We did not compare our method with the approaches in [1] and [3] because the DRO problem they address differs from ours, making a direct comparison infeasible. Specifically, references [1] and [3] use the KL divergence to construct the ambiguity set, resulting in a KL DRO problem, whereas [2] and this paper employ Wasserstein distance, leading to a Wasserstein DRO problem. The comparison between this paper and [1] or [3] would be unfair since there are two key differences: the distance used to build the ambiguity set and method to deal with outliers. Any observed performance differences could not be solely attributed to the outlier-handling methods, making the comparison inconclusive. While it might be possible to adapt our unbalanced method for outlier handling to the KL DRO framework, this would constitute a different problem altogether, requiring a complete reevaluation, which we have left as future work due to time constraints. Regarding reference [4], it is a classical paper on CVaR that does not propose any method for handling outliers. Though the dual of CVaR can be seen as a DRO problem, it cannot be applied to handle outliers directly.

---

Q1: Choice of cost function $c(\xi,\zeta)=\left\|\xi-\zeta\right\|_1$

We adopt the cost function for linear regression from [17] to ensure a fair comparison between our method and theirs.

---

Q2: Parameter selection

The parameters are tuned based on the parameter selection guideline as outlined in Common Response C2. Moreover, we conducted a thorough sensitivity analysis in three tasks. As shown in Tables 3 and 4 for linear regression, Table 9 and new Table 3 for linear classification, new Tables 1 and 2 for logistic regression, our method performs well across a wide range of parameters and is not sensitive to the choice of parameters. Please see Common Response C3 for a detailed discussion on sensitivity analysis.

---

Q3: Performance of UOT-DRO for contamination $C=30$ and $C=100$

Recall that our method reduces the effect of outliers by assigning them a small (but non-zero) weight. When the contamination is high, the outlier points are significantly distant from the normal data. In this case, outlier points, despite their minimal weighting, may slightly influence model performance due to their extreme deviation from the normal data. To assess the impact of increasing $C$, we conducted additional experiments on linear classification, keeping the parameters fixed. The results indicate an accuracy of 95.8% for $C=30$, 95.2% for $C=100$, 95.0% for $C=300$, 94.6% for $C=600$, and 93.4% for $C=1000$. Given that $C=1000$ represents a substantially large contamination level relative to the variance of the data distribution, which is $\sqrt{10}$, these findings suggest that our method is not sensitive to $C$ and performs well even at high contamination levels.

---

Minor Limitations

Thank you for pointing out these typos. We have refined them in the updated draft.

We thank the reviewer for their thoughtful feedback. We address the concerns below.

---

W1: Excess risk

Thank you for raising this point. Analyzing the excess risk of UOT-DRO is quite challenging due to the existence of unbalanced optimal transport. Moreover, while [17] gives the analysis of excess risk, the distance used in the excess risk analysis is different from the distance used to obtain the DRO tractable reformulation. Analyzing excess risk is of course interesting and left as our future work. This work mainly focuses on computational efficiency compared to previous methods.

---

W2: Parameter selection

We agree that this warrants further discussion. We have added thorough discussions on the selection of all the parameters $\lambda,\beta,\lambda_2$. Please see Common Responses C2 and C3.

---

Q1: Sensitivity to $\varepsilon$

Thank you for providing the additional interesting reference [A]. Upon careful review, we find that the analysis in [A] cannot be applied to assess the sensitivity to $\varepsilon$ in [17], since the distance that the reference [17] uses to define the DRO problem is different from that analyzed in [A]. The numerical experiments in [17] show that the performance of OR-WDRO deteriorates when $\hat{\varepsilon}$ is much smaller than the true $\varepsilon$, as illustrated in Figure 2 in [17]. This finding aligns with the simulation results presented in Tables 6 and 8 of our paper.

The mechanism behind OR-WDRO is that it cuts off the leftmost $\hat{\varepsilon}$ percentile of the random value \sup _ {\xi \in \Xi} \left\\{ l(\theta,\xi) - \lambda_1 \left\| \xi - \xi_0 \right\|^2- \lambda c(\xi,\zeta)\right\\} in Eq (12), which is induced by outliers, and then computes the average of the remaining $(1-\hat{\varepsilon})$ percentile. If $\hat{\varepsilon}$ is much smaller than the true $\varepsilon$, it results in inefficient outlier exclusion and thus poor performance of OR-WDRO. Therefore, OR-WDRO depends on a relatively accurate estimate of $\varepsilon$. In contrast, our unbalanced method does not rely on cutting off outliers but instead re-weights samples, offering a smoother approach. As a result, our method does not require an accurate estimate of $\varepsilon$. For more details about how our method deals with outliers, please see Common Response C1.

---

Q2: Convergence of Algorithm 1

We run Algorithm 1 in linear regression and the convergence result is shown in the new Figure 1. Since $F^*$ is hard to compute, we plot the evolution of $F(\theta_t)$.

---

Q3: Parameter tuning for the proposed method and the baseline method

Thank you for raising this significant point. The guidelines for parameter selections can be found in Common Response C2. Following the guidelines, we tune the parameters and we find that the performance is not sensitive to parameter selections in three tasks. For more details about sensitivity analysis, please see Common Response C3 and the attached PDF file, where we conducted additional experiments on linear classification and logistic regression.

The baseline method proposed in [17] requires tuning the parameters $\lambda,\sigma,\epsilon$. We adopted the parameter selections from the original implementation of [17], which we find is nearly optimally tuned. Specifically, it sets $\sigma = \sqrt{d}$, see discussions in [17]; $\epsilon=0.1$, aligning with the 10% data corruption rate; $\rho = 0.1$, corresponding to the size of Wasserstein perturbation. We will include these in the updated draft.

We thank the reviewer for your thoughtful feedback. Please see the detailed response below.

---

W1(Q1): Reference [JGX2023]

We thank the reviewer for bringing our attention to the reference [JGX2023] and other related references about regularized optimal transport distance. We will add these references and additional comparative analyses into our revised manuscript.

The fundamental difference between regularized and unbalanced optimal transport distance lies in whether the marginal distributions of the couplings are allowed to be different from the empirical distribution. Take Sinkhorn Wasserstein DRO (WDRO) in [JGX23] as an example. Both Sinkhorn WDRO and unbalanced WDRO use the KL divergence, albeit in fundamentally different ways. In Sinkhorn WDRO, the KL divergence acts as a regularization term that facilitates computational efficiency at the expense of computation error. Importantly, the coupling $\gamma \in \Gamma(\hat{\mathbb{P}},\mathbb{P})$ in Sinkhorn WDRO must have at least one marginal distribution that is identical to the empirical distribution $\hat{\mathbb{P}}$. In contrast, unbalanced WDRO employs KL divergence to penalize discrepancies between marginal distributions, allowing for the coupling $\gamma\in \Gamma(\bar{\mathbb{P}},\mathbb{P})$ to have a different marginal distribution $\bar{\mathbb{P}}$. One of the resulting technical differences, as the reviewer pointed out, is the relative entropy regularization between two couples of distributions.

The strong duality of Lagrangian penalty formulation of several related problems is as follows:

Traditional WDRO: $\mathbb{E}_{\zeta \sim \hat{\mathbb{P}}} \Big[ \sup _ {\xi} \\{ L(\xi)-\lambda c(\xi,\zeta) \\} \Big],$

Sinkhorn WDRO: $\lambda \beta \mathbb{E} _ {\zeta\sim \hat{\mathbb{P}}} \Big[ \log \mathbb{E} _ {\xi\sim \nu } \Big[{\rm{exp}} \left(\frac{L(\xi)-\lambda c(\xi,\zeta)}{\lambda\beta} \right) \Big]\Big],$

Unbalanced WDRO: $\lambda \beta \log \mathbb{E} _ {\zeta\sim \hat{\mathbb{P}}} \Big[ {\rm{exp}}\left( \frac{\sup _ {\xi} \\{L(\xi)-\lambda c(\xi,\zeta)\\}} {\lambda\beta}\right)\Big].$

From these dual formulations, Sinkhorn WDRO modifies traditional WDRO by substituting the $\sup$ with a log-sum-exp smoothing function, thereby enhance the computational efficiency. In contrast, unbalanced WDRO employs the log-sum-exp function differently. Here, it is used to re-weight samples from $\hat{\mathbb{P}}$ rather than replacing the $\sup$ operation. As a result, outliers exert a reduced impact on the decision-making process. Computationally, unbalanced is more challenging as it involves an inner-maximization problem, similar to traditional WDRO.

---

Q2: Restrictive assumptions in Theorem 1

Compared to Sinkhorn DRO in [JGX2023], the additional assumption about the differentiability and concavity of $L(\theta,\zeta)$ is due to the existence of an inner sup in the dual formulation, which is smoothed out in Sinkhorn DRO. Another assumption about $\lambda^*=0$ is discussed in Q3. Relaxing these assumptions is left as our future work.

---

Q3: Strictly positive dual variable is not necessary

It may happen that $\lambda^*=0$ when we select a sufficiently large radius. However, this scenario is uncommon in practice because choosing a very large radius typically results in overly conservative outcomes that are generally less appealing. We agree that removing this assumption is promising under additional conditions. We plan to address this in our future work as it is not straightforward by applying techniques from current DRO literature, e.g., [JGX2023].

---

Q4: Implicit assumption $c(\zeta,\zeta)=0$

Thank you for raising this point. We have added it in the updated version.

---

Q5: Extension to non-convex smooth optimization

Thank you very much for providing these two related references. We will incorporated them in our revised draft. Thank you for proposing the non-convex smooth optimization problem. This is definitely interesting and left as our future work.

---

Q6: Neural network implementation

Thank you for affirming the potential of our method for use with neural networks (NNs). We believe our method can be applied to NN loss functions, and we plan to explore this in our future work.