We sincerely thank the reviewer for their valuable feedback and constructive comments. We are pleased that the reviewer found our work correct, transparent, and an important direction for relaxing smoothness assumptions in variational inequalities. Next, we address the concerns raised by the reviewer and respond to their questions:

• p-Quasi Sharpness: We respectfully disagree with the reviewer’s assessment that the derived results are unsurprising. The main difficulty of our analysis lies in the generalized smoothness assumption, which makes the problem fundamentally different from smooth SVI. Even in the stochastic optimization case, one-sample clipped SGD fails to converge to a solution with diminishing step sizes (Koloskova et al., 2023 [1]). To the best of our knowledge, this is the first work to establish almost sure convergence and unbiased convergence rates for stochastic methods under generalized smoothness assumptions for SVIs. Additionally, SVIs differ significantly from optimization problems, as one cannot bound operator norms using function values (as it was done in Li et al., 2023 [2]; Koloskova et al., 2023 [1]; Xi, 2024 [3]). To address these challenges, we developed a novel technique demonstrating that a series of stochastic clipping step sizes is almost surely not summable.

• Motivation for Extragradient: Both the projected gradient method and the extragradient method are widely used for SVI problems, and investigating their performance under generalized smoothness provides a comprehensive understanding of their behavior in more relaxed settings. While both methods exhibit the same convergence rate with respect to  $k$ , the extragradient method's convergence guarantees for bilinear or smooth convex-concave problems make it a valuable addition to this study of the generalized smooth SVIs. We agree with the reviewer that further exploration of extragradient methods under the Minty condition ($μ=0$) could be an interesting direction for future research.

• Two Samples for Gradient Clipping: The use of two samples for gradient clipping, while unconventional, is  one of our key insights and contributions to our work. We do agree with the reviewer that removing the second sample would be an interesting problem for further research. However, Theorems 3.1, 3.2 in Koloskova et al. 2023 suggest that there exists an operator (which does satisfy our assumptions) for which there exists a fixed point of standard clipping SGD, which is not a solution, which leads to unavoidable bias for one sample clipped SGD. This approach allows us to overcome unavoidable bias issues present in one-sample clipped SGD, as shown in Koloskova et al. (2023), and ensures both almost-sure convergence and unbiased convergence rates. Please see lines 280–290 in the revised manuscript for further clarification.

• Minor Issues: Thank you for noticing these issues, we fixed them. We also corrected all noted typographical errors.

• Answers to Questions:

  • We added the stochastic Popov method and its clipping step sizes in the Experiments section (lines 449–453). However, we are uncertain why stochastic clipped Popov performs better in the solution neighborhood.
  • The analysis of the stochastic clipped Popov method presents additional challenges due to the complexity of its clipping steps. We are open to including convergence results for this method if deemed beneficial.
  • Based on our experiments, a large value of $q$ improves the performance of the methods in the $\sigma$-neighbourhood of the method. This can be explained by the fact that the stepsizes decrease faster for the large choice of $q$. Thus, larger $q$ makes stepsizes smaller, leading methods to converge to a solution's smaller neighborhood.
  • Regarding Theorem 2.5 of Koloskova et al. (2023), we believe a direct comparison is not entirely appropriate due to differences in assumptions and settings. In our stochastic setting, decreasing stepsizes are necessary, while the stepsize in Theorem 2.5 in Koloskova et al [1] is fixed. Also, it is well-known that while the rate of the projection (gradient) method for strongly monotone Lipshcitz VIs and strongly convex smooth optimization is linear, the factor inside $\exp$ is different: $\frac{L}{\mu}$ in optimization and $\frac{L^2}{\mu^2}$ in VIs. However, for $p=2$ when using fixed stepsizes in equation (76) of the proof of Theorem 3.4, it can be shown that the convergence rate to the neighborhood of a solution is also linear.

[1] Revisiting gradient clipping: Stochastic bias and tight convergence guarantees, Anastasia Koloskova, Hadrien Hendrikx, and Sebastian U Stich, ICML 2023

[2] Convex and non-convex optimization under generalized smoothness. Haochuan Li, Jian Qian, Yi Tian, Alexander Rakhlin, and Ali Jadbabaie, NeurIPS 2023

[3] Delving into the convergence of generalized smooth minimax optimization, Wenhan Xian, Ziyi Chen, and Heng Huang. , ICML 2024.

We thank the reviewer for their answer.

1. Yes, to the best of our knowledge, our two-sample approach is the first unbiased result of clipped SGD, even for optimization.

2. For smooth strongly convex strongly concave problems, which are a special case of $p=2$, we recover $O(\frac{1}{k})$ rate for stochastic clipped EG, which matches the rate for stochastic EG obtained in Kannan & Shanbhag, 2019 [1]. However, convex-concave problems do not satisfy the $p$-quasi sharpness assumption and are out of the scope of this work. We are not aware whether it is possible to show that the proposed variant of EG converges in the general setting of $\mu=0$, but it is an interesting open question for further research.

[1] Aswin Kannan and Uday V Shanbhag. Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variants. Computational Optimization and Applications, 2019.

We sincerely thank the reviewer for their valuable feedback and constructive recommendations, which have helped us improve the quality and clarity of our paper. We also corrected all noted typos.

• Rate Comparison with Related Works: We have added detailed discussions comparing our rates with related works after Theorems 3.4 and 4.4 (see lines 329–334 and 438–443). Additionally, we included a recommended discussion on Xian et al. 2024 [1] in lines 84–89. Observe that Xian et al. 2024 [1] considered generalized smooth stochastic nonconvex strongly-concave min-max problems and provided $\mathcal{O}(\frac{1}{k^{1/4}})$ convergence rates for variants of stochastic gradient sescent ascent (SGDA) with normalized and adaptive stepsizes. The specific structure of the minmax problem and the fact that the gradient of the corresponding function is nonmonotone in one variable and strongly monotone in another makes it difficult to directly apply their results to SVIs. While both settings are interesting, direct comparisons of results from Xian et al 2024 and ours are challenging due to these differences. One possible example that satisfies the assumption from Xian et al. 2024 [1] and our assumptions is a generalized smooth, strongly convex strongly concave min-max problem. For this problem, the rate of SGDA from Corollary 4.16 is $\mathcal{O}(k^{-1/4})$, while ours is $\mathcal{O}(k^{-1})$ (since $p=2$) which is much better. However, we acknowledge that the differences in assumptions and problem structures limit this comparison. We might consider including this example problem in our manuscript, but we believe the fundamental distinctions between the works should be considered.

• Two samples: Theorems 3.1, 3.2 in Koloskova et al. 2023 [2] suggest that there exists an operator (which does satisfy our assumptions) for which there exists a fixed point of standard clipping SGD, which is not a solution, which leads to unavoidable bias for one sample clipped SGD. Using two samples helps us to overcome unavoidable bias issues present in one-sample clipped SGD, as shown in Koloskova et al. 2023 [2], and ensures both almost-sure convergence and unbiased convergence rates. Please see lines 280–290 in the revised manuscript for further clarification. Furthermore, utilization of two samples per iteration does not affect dependency on the number of iterations $k$. However, developing a method that requires only one sample per iteration would be an interesting further direction of research.

• Comparison of Projection and Korpelevich Methods: We have included a comparison of the rate for stochastic clipped projection and Korpelevich methods after Theorem 4.4. Both methods exhibit the same dependency on $k$ for convergence rates, in cases where $p=2$ and $p\geq 2$.

• Bounded Variance Assumption: We agree with the reviewer that the bounded variance assumption is strong and could potentially be relaxed. However, even under this assumption, the analysis in the generalized smooth setting presents significant challenges. To achieve convergence even under bounded variance assumption, one needs to use or develope new techniques such as bounding gradients over trajectories (Li et al., 2023 [3]) or demonstrating the almost sure non-summability of the series of step sizes, as in our work. It is worth noting that even in the original work by Chen et al. 2023 [4], the authors adopted a stronger assumption than bounded variance. Relaxing this assumption remains an interesting direction for future research.

• Comparison in Theory and Experiments: Based on both theoretical results and experimental findings, the stochastic clipped projection and Korpelevich methods are comparable under the assumptions of $p$-quasi sharpness and $\alpha$-symmetry.

[1] Delving into the convergence of generalized smooth minimax optimization, Wenhan Xian, Ziyi Chen, and Heng Huang. , ICML 2024.

[2] Revisiting gradient clipping: Stochastic bias and tight convergence guarantees, Anastasia Koloskova, Hadrien Hendrikx, and Sebastian U Stich, ICML 2023

[3] Convex and non-convex optimization under generalized smoothness. Haochuan Li, Jian Qian, Yi Tian, Alexander Rakhlin, and Ali Jadbabaie, NeurIPS 2023

[4] Generalized-smooth nonconvex optimization is as efficient as smooth nonconvex optimization, Ziyi Chen, Yi Zhou, Yingbin Liang, and Zhaosong Lu, ICML, 2023

• We thank the reviewer for their valuable feedback and comments. We are delighted that the reviewer found our work important, novel, and correct.

Below, we address the concerns raised by the reviewer and respond to their questions:

• Two Samples per Iteration: We have added a discussion in lines 329–332 addressing the use of two samples per iteration. While this approach requires twice as many oracle calls to achieve an $\epsilon$-solution, it does not affect the rate dependency on the number of iterations  $k$ . Using two samples per iteration allows us to achieve an unbiased rate, meaning that we can obtain an arbitrarily small  $\epsilon$ by decreasing the step sizes. In contrast, as shown in Koloskova et al. (2023), one-sample clipped SGD cannot achieve arbitrarily small  $\epsilon$ -solutions even with diminishing step sizes. This advantage highlights the trade-off between computational cost and achieving unbiased convergence guarantees, which we believe is an important contribution to the literature.
• Parameters: We agree with the reviewer that the knowledge of the parameters  $L_0,L_1,K_0,K_1,K_2$  is crucial for our analysis. However, we note that the step sizes  $\beta_k$  for the clipped stochastic projection method in Theorem 3.4 (Case 2,  $p\geq2$ ) do not depend on problem-specific parameters. Furthermore, we believe that deriving optimal rates without prior knowledge of  $L_0$ ​ and ​ $L_1$  even in deterministic optimization—is an open problem. For reference, see Huber et al. (2024) [1].
• Best Iterate Convergence in Stochastic Cases: We acknowledge the reviewer’s concern that best-iterate convergence in stochastic cases is less tractable. To address this, we have modified the proof and demonstrated that the same rate can be achieved for the average-iterate convergence. Specifically, we show that the weighted point $\bar{u} = \frac{1}{\sum_{t=0}^k \beta_t} \sum_{t=0}^{k} \beta_t u_t$ sattisfies $\mathbb{E}[\rm{dist}^2( \bar{u}_k)] \leq \mathcal{O}(\frac{1}{k^{2(1-q)/p}})$ . This modification provides a more practical convergence guarantee. The result has been added to Theorems 3.4 and 4.4 in the revised manuscript.

[1] Parameter-agnostic optimization under relaxed smoothness, Florian Hübler, Junchi Yang, Xiang Li, Niao He, AISTAT, 2024

Dear Reviewer,

We addressed all your concerns and improved our paper based on your recommendations. We kindly ask you to reconsider your evaluation of our paper.

• We thank the reviewer for their valuable feedback and comments. We have addressed all typos and missing or incorrect references, provided additional details in the abstract as recommended, and uploaded a revised version of our paper.

• Firstly, we respectfully disagree with the reviewer’s concern regarding the correctness of Proposition 2.2. Proposition 1(a, b) in Chen et al. (2023) is general enough to apply to any vector field, not just gradient operators, as the proof does not use any information specific to gradient fields. Furthermore, Eq. (9) in Chen et al. (2023) is valid for  $\alpha$ -symmetric operators, as demonstrated in Lemma 2 (Appendix A) of their work, where $h(\theta) = L_0 + L_1 || F(\theta w' + (1 - \theta )w)||$ . To enhance clarity, we are willing to include additional clarifications or proofs of these results from Chen et al. (2023) in our manuscript, although we believe this might be redundant given their existing availability.

• We have also corrected the citations for the Extragradient and Popov methods, removed the duplicate reference to Chen et al. (2023), and incorporated all other feedback regarding references and citations.

• Secondly, we would like to highlight some contributions of our work that the reviewer might have overlooked. Our paper introduces a relaxation of the standard Lipschitz continuity assumption, presenting what we believe to be the first convergence results in the literature for generalized smooth stochastic variational inequalities. To the best of our knowledge, this work also provides the first almost-sure convergence results for stochastic clipping methods in a generalized smooth setting, even within optimization. Additionally, we introduced a novel analysis by demonstrating that the series of stochastic clipping step sizes is almost surely unsummable. For both methods considered, we established new unbiased convergence rates. These rates can't be achieved by the technique from Li et al 2024 or Xian et al 2024 since in SVIs one can not upperbound the gradient norm by the function residuals.

• We hope that our responses and the revisions made to the manuscript address all of the reviewer’s concerns. We respectfully request the reviewer to reconsider their evaluation, as we believe the contributions of this work, particularly the novel analysis of generalized smoothness and clipped stochastic methods, significantly advance the field. We are looking forward to a discussion.

Dear Reviewer,

We have addressed all your concerns and improved our paper based on your recommendations. However, we believe the only technical weakness you provided is not valid, as Proposition 1 in (Chen et al., 2023a) applies to any vector field. We kindly ask the reviewer to reconsider their evaluation in light of these clarifications.

Unfortunately, the last day that authors may upload a revised PDF was November 26, while we provided our response on November 19. While we cannot upload a new version at this stage, we will include these clarifications and results in the final version of the paper.

The first equivalence (eq. 9) in (Chen et al. 2023a) already fails for vector fields. Also, since the proof for (9) in (Chen et al. 2023a) also relies on ODEs it involves solving the integral involving therein that contains a vector field. Could you please elaborate?

The proof of (9) in Chen et al., 2023a, is provided in Appendix A, specifically in Lemma 2. To show that the $\alpha$-symmetric assumption implies (9), the authors relied on (i) the triangle inequality and (ii) the $\alpha$-symmetric assumption. These steps are valid for any vector field, which demonstrates that eq (9) is valid for any vector field.

We kindly ask the reviewer to reconsider their evaluation in light of these clarifications.