1. Unclear Novelty: The novelty of GeoDP remains unclear. While the authors claim their method remedies the direction bias of gradients, they still adopt gradient clipping, which inherently introduces bias. Furthermore, the gradient direction is still bounded by the parameter $\beta$.

   To clarify the novelty, it would be helpful if the authors could provide a more detailed comparison of the bias introduced by GeoDP’s bounding versus traditional gradient clipping. Specifically, how does the noise introduced by GeoDP differ in terms of bias compared to DP-SGD, given that both methods use some form of bounding? A deeper analysis of this would help demonstrate how GeoDP introduces "unbiased" noise or mitigates bias more effectively.

2. Weak Evaluation: The evaluation lacks depth, particularly due to the omission of the privacy parameter $\epsilon$, which hinders comparison with other DP methods in the literature. Including the value of $\epsilon$ in the evaluation would allow for a clearer comparison of privacy guarantees across methods.

   Additionally, since GeoDP introduces the new hyperparameter $\beta$, it would be beneficial to provide a sensitivity analysis of both $\beta$ and the gradient clipping parameter $C$. This analysis should illustrate how tuning each parameter impacts model performance. Such a comparison would clarify which parameter plays a more critical role in optimizing the algorithm's effectiveness.

Incorrect Implementation Affecting Understanding and Validity of Results:

After reviewing the authors' submitted code, I found that when training the CNN with PyTorch, the clipping operation was applied to the averaged gradient. However, in a correct DP-SGD implementation, each sample’s gradient should be computed first, then clipped individually before averaging. This process is crucial to DP-SGD, as it limits each sample’s contribution to the gradient, distinguishing DP-SGD from standard SGD. Previous studies [1] have shown that per-sample clipping has a substantial impact on optimization results, so this is a significant oversight. This incorrect implementation raises concerns about the authors' understanding of DP-SGD and renders the subsequent experimental results unreliable. I suggest that the authors use a correct implementation and re-run the experiments, potentially utilizing the Opacus package to compute per-sample gradients in pytorch.

Unclear Determination of Direction Range:

The algorithm uses a “Direction range” (line 318), but how the values for \Gamma_1 and \Gamma_2 were chosen is not clearly explained in the paper. I also could not locate this implementation in the code (though I may have missed it). Without a proper range for controlling direction, the sensitivity calculation may be incorrect, implying that the noise added could be insufficient to meet DP requirements, leading to an uncorrectable privacy error. I recommend that the authors clarify the determination of \Gamma_1 and \Gamma_2 and ensure their implementation is reflected in the code.

Limited Experimental Comparisons:

The experimental section lacks comparisons with recent DP research, failing to show results near SOTA performance. As DP-SGD is highly sensitive to hyperparameter choices, comparing the proposed method with recent research would lend more credibility than showing specific cases, such as \sigma = 10 or \sigma = 1, which may not be representative. I recommend that the authors benchmark their method against more recent SOTA DP approaches to robustly validate their improvements [2].

Lack of Privacy Budget Reporting:

The paper does not report the privacy budget parameters \epsilon and \delta. These parameters are critical to understanding the strength of the privacy guarantees offered by the method. I suggest that the authors include the privacy budget in their experimental results to provide a more complete picture.

[1] Bu, Z., Wang, H., Dai, Z., & Long, Q. (2023). On the convergence and calibration of deep learning with differential privacy. Transactions on machine learning research, 2023. [2] De, S., Berrada, L., Hayes, J., Smith, S. L., & Balle, B. (2022). Unlocking high-accuracy differentially private image classification through scale. arXiv preprint arXiv:2204.13650.

Despite the idea being very promising, I think there are several critical issues in the current version of the manuscript:

• Most importantly, lack of privacy analysis. What are the DP guarantees of the algorithm? Do we have to count each gradient update as an application of two Gaussian mechanisms? If not, why? Without answers to this, we cannot compare to standard DP-SGD with the same level of noise as the privacy guarantees are likely different at the same noise scale. The paper promises a privacy analysis in Section 4.3, but it is not there.
• Insufficient evaluation:
  • MNIST is not a representative dataset, so we cannot really know how much better is GeoDP in many practical scenarios. I would suggest including at least CIFAR10 and, e.g., language model finetuning on MMLU.
  • I would also suggest to systematically compare with different clipping strategies.
• The proofs of formal results are not included. The appendix is missing in the paper, and there is no supplement.

Minor issues:

• The wording switches between efficiency and deficiency seemingly. It would help to have consistent naming.
• There are multiple typos. The manuscript would benefit from careful proofreading.
• Theorem 2 has a broken reference as "Theorem .2"

In sum, I think the idea is very promising, but the paper would need a formal privacy analysis, additional experimental evaluations, and fix multiple presentation issues to pass the bar.

1. The setting of $\beta$ parameter is not well discussed.
2. The related works are not well discussed.
3. The presentation needs to be improved. It's not clear enough in Algorithm 1 how the dimensions are decided.

It seems to be an LLM-generated paper ...

The proposed algorithm's biggest problem is that there's no proof of the privacy guarantee. I don't see how Step 2 in the algorithm in Sec 4.2 restricts the sensitivity. The paper is poorly written. I took a look at Theorem 1 and I think it is completely useless. The proof seems to assume SGD also uses a clipped gradient, and both DP-SGD and SGD start with w_t. I suspect the entire theorem is generated by GPT.