Personalized Federated Learning via Tailored Lorentz Space

Q4: Does the method work for larger numbers of clients, without full client participation each round?

R: Thank you for the good question. We have conducted experiments on the image dataset to evaluate the results of these scenarios in the response of C2.

 

---

Q5: Statistically significant?

R: Our method's significance is validated using p-values. Additionally, the standard deviation values are shifted to two decimal places to save space. We will provide clarification on this in the revised manuscript for better understanding.

 

---

Q6: Embedding dimension?

R: Thank you for pointing out. Yes, the embedding dimension is the hidden dimension of the model. We will further refine our writing and try to provide as much clear explanation as possible within the limited space.

Thank you for your valuable comments and suggestions. Below, we will summarize the concerns with "C:" and our responses with "R:"

 

---

C1: Regarding the theoretical part

R: Thanks for your comments. To clarify, this theoretical analysis is not clamied as our main contribution, it serves to establish the mathematical rigor of our novel personalized federated learning strategy in hyperbolic space. Let us elaborate on the importance of our analysis

1. Corollary 1 is important because we are the first to apply Lorentz Neural Networks to the federated learning setting. This means that after each aggregation step, the parameters are updated, changing the model. The $\mathbf{N}$ in our case should be $\mathbf{M}_c - \eta \sum \frac{|\mathcal{D}_c|}{N} \nabla \mathbf{M}_c$, where $\nabla \mathbf{M}_c$ is the gradient of the shared parameters $\mathbf{M}$, $N$ is the total number of samples. Due to hyperbolic space's non-Euclidean nature, parameter changes can directly affect the validity of output representations. Thus, ensuring output representations remain in the hyperbolic space after parameter updates is important for the correctness of hyperbolic neural networks, an issue unique to this setting. While not claimed as a core contribution, the proof of Corollary 1 is essential to address this challenge specific to the federated learning of hyperbolic models.

2. The importance of other analysis: The goal of the remaining part is to show the rationale behind our proposed method from different perspectives, as stated in the paper. Our emphasis is on analyzing our approach from the perspective of hyperbolic geometry. For example, we perform a non-linear debiasing operation on the input data. Moreover, we provide interpretable insights drawing an analogy from special relativity, treating each client as a distinct space-time.

In the paper, we have added an analysis of the convergence analysis of our proposed method in Appendix B.5. We show that the decoupling strategy does not impede the convergence of the model. In future work, we plan to further investigate the specific impact of the hyperbolic curvature on model performance, such as its effect on generalization error and other metrics. We believe this would be an interesting theoretical research direction about how geometry can directly influence the models’ performance in various tasks.

C2: Regarding the experiments part

R: We appreciate the reviewer's comments on our empirical evaluation. We would like to clarify several important points:

 

1.The reason we focus on graph data:

• Our focus on graph data is theoretically motivated. Graphs inherently exhibit non-Euclidean structures and significant heterogeneity, as demonstrated in previous theoretical analyses [1-2].
• More importantly, there is a theoretical correlation between the heterogeneity of graph distributions and hyperbolic curvature [1]. In contrast, for other domains, such as images or text, hyperbolic geometry has been studied more thoroughly in graph data, where explicit complex structures of heterogeneity are present, serving as an excellent testbed for interpreting how hyperbolic geometry handles hierarchical relationships and heterogeneous structures. Therefore, it provides a clear theoretical foundation for validating our method's effectiveness in handling complex geometric properties.

 

2. Applicability to other domains and settings:

We further conducted experiments with an increased number of clients (50 clients) in the Cora dataset, which represents a large client pool configuration of federated learning scenarios in graph datasets. The results demonstrate that our method maintains its effectiveness even with an expanded client base. The below table illustrates the performance comparison between FedAvg and FlatLand under different participation rates on the Cora dataset with 50 clients. (We added the results in Appendix C4 in revised paper).

The experimental results show that FlatLand exhibits remarkable robustness in a larger number of clients and across various participation rates, even with only 10% client participation (5 clients). These findings confirm that FlatLand can maintain high performance even under low client participation scenarios, demonstrating its practical value federated learning applications where full client participation may not always be feasible.

 

Our method can be readily transferred to other fields that use linear neural networks. In the following, we provide additional experiments on widely used image dataset, MNIST (also considering a larger number of clients). The results are promising compared with the strong baselines [3-5]:

We also conduct additional experiments with varying client participation rates from [0.1-0.7] based on the above setting with 100 clients, and FlatLand still works in this setting.

 

It shows that our method can also work and perform well without full client participation in each round. Note that our method is orthogonal and non-conflicting with methods that utilize partial strategies, allowing for complementary integration with such approaches. This is NOT the main focus of this paper.

 

---

References:

[1] Krioukov, Dmitri, et al. "Hyperbolic geometry of complex networks." Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 82.3 (2010): 036106.

[2] SINCERE: Sequential Interaction Networks representation learning on Co-Evolving RiEmannian manifolds, WebConf 2023

[3] Exploiting shared representations for personalized federated learning, ICML 2021

[4] Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration, ICCV 2023

[5] Gpfl: Simultaneously learning global and personalized feature information for personalized federated learning, ICCV 2023

Thank you for your questions. Next, we will summarize the questions with "Q:" and our responses with "R:"

 

---

Q1: Why do the $v$ parameters not appear anywhere in Equation (4)?

R: The parameter $v$ is introduced to enable a more generalized formulation, supporting advanced non-linear operations with learnable parameters. While dropout is presented as a simple example without $v$ , the parameter can play a role in other minor operations within hyperbolic space. Specifically, $v$ can function as learnable parameters in normalization operations, constraining the norm via $\sigma(v^T x)$ to prevent excessive growth of hyperbolic embeddings, or as bias terms added to $x$. In later sections, the presentation is simplified to focus on the core linear transformation aspects (lines 283–284), as these additional operations do not influence the fundamental principles of our method. We will add this explanation in the latter version.

 

---

Q2: How does Lorentz space actually model heterogeneity? Provide theoretical or empirical evidence.

R: From a theoretical perspective. There is a direct correlation between graph distributions and the curvature of hyperbolic geometry [1]. Specifically, the stronger the power-law distribution of graph degrees, the more the data deviates from Euclidean space, corresponding to a larger curvature of Hyperbolic space. This makes graph data particularly well-suited for validating our method's ability to address complex geometric properties, supported by strong theoretical motivation and guarantees. For data in other domains, there is also a series of works to empirically show the relationship between the hyperbolic curvature and the distribution [6-7].

The theoretical guarantee of our decoupling strategy stems from the working principles of the Lorentz neural network. As shown in Equation (4), only the time-like dimension of the representation ($x_t$) is directly influenced by the curvature $-K$. Since curvature reflects the overall distribution of the data, it is straightforward and reasonable to assume that $x_t$ captures the heterogeneity of the data.

Moreover, in Section 6 (Perspective on Lorentz transformations), we provide a relativistic perspective on the distinction between $x_t$ and $x_s$. Specifically, Lorentz space with different curvature represents varying spacetime intervals, and changes in the same event across different intervals primarily manifest in the time-like dimension. This further supports our hypothesis that $x_t$ effectively encodes data heterogeneity.

From an empirical perspective. We conducted a series of experiments to validate the ability of our method to decouple data heterogeneity. First, we aggregated personalized data during training and observed significant performance fluctuations, with results deteriorating as shown in Figure 4. Second, we applied the same approach in Euclidean space, but it failed to achieve significant improvements. These observations help validate the effectiveness of our method in leveraging Lorentz space to handle heterogeneous data.

 

---

Q3: Why do you decouple the parameters the way you do?

R: Thank you for your question. Our decoupling strategy focuses on separating the parameters at each layer of the model. As discussed in our response to Q2, the key component that captures data heterogeneity is the time-like dimension ($x_t$) of the hyperbolic embedding. After each layer of the Lorentz neural network, the input vectors are transformed into a new hyperbolic embedding, with the time-like dimension updated as $x_t^{(l+1)} = \sqrt{||mx_t^{(l)} + \mathbf{M}\mathbf{x}_s^{(l)}||^2+K}$. This means that the component at layer $(l+1)$ that carries the heterogeneity information is transferred as $x_t^{(l+1)}$. Thus, the parameters associated with $x_t^{(l+1)}$ are naturally aligned with the heterogeneity information. This ensures that our decoupling strategy is consistent with the theoretical derivation and can be effectively applied across multiple layers.

 

---

References:

[1] Krioukov, Dmitri, et al. "Hyperbolic geometry of complex networks." Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 82.3 (2010): 036106.

[6] Curvature Generation in Curved Spaces for Few-Shot Learning. Gao et al. (2021). ICCV

[7] Curvature-Adaptive Meta-Learning for Fast Adaptation to Manifold Data. Gao et al. (2023). TPAMI

Our work introduces a novel geometric perspective in personalized federated learning, particularly for addressing data heterogeneity. As the first work exploring this direction, we provide valuable insights for the field, as acknowledged by multiple reviewers.

• Reviewer 4tQH noted that "the idea in this paper is interesting and inspiring,"

• Reviewer Wtgh highlighted "the novelty of increasing dimensionality to embed client data in Lorentz space,"

• Reviewer Dc6W recognized our work as "an innovative approach to personalized federated learning by leveraging hyperbolic geometry, specifically through the use of Lorentz spaces with tailored curvatures.”

 

Overall, the proposed decoupling strategy has been demonstrated to effectively mitigate data heterogeneity challenges and enhance the performance of clients’ local Lorentz neural network models with strong motivation and theoretical support. And our approach achieves superior performance, particularly on tree-like structures and power-law distributed data such as graphs, where the inherent hierarchical nature aligns well with hyperbolic geometry. We believe our analyses and ablation experimental results strongly validate the effectiveness of our proposed approach.

 

Though our method can be readily transferred to other fields that use linear neural networks, we want to clarify that our primary goal is not to achieve SOTA performance across all data and tasks, as it also depends on the performance of the local hyperbolic backbone. Hyperbolic space may not be universally optimal for all underlying data distributions, as its effectiveness depends on whether the data can be appropriately modeled by varying negative curvature. For instance, certain datasets may exhibit inherent positive curvature characteristics [8-9]. This observation suggests a promising future research direction: modeling more complex data structures in mixed-curvature spaces. We will explore this method in the future, as claimed in our conclusion section.

 

---

References:

[8] Learning mixed-curvature representations in product spaces. ICLR. 2019

[9] Pseudo-Riemannian Graph Convolutional Networks. NeurIPS 2022

Thank you very much for your following-up questions.

 

Metric used in Participation Rate: For the graph dataset, we are using classification accuracy, which is consistent with the main experiment table. For MNIST, we are using Accuracy (Acc) as the metric.

 

How AUC is calculated? We directly followed the computation method used in the PFLib library [1]. After examining the source code, the specific implementation is:

auc = sklearn.metrics.roc_auc_score(y_true, y_prob, average='micro')

 

About the baselines: Due to time constraints, some baselines require hyperparameter tuning, and we are still conducting experiments. We will continuously update and include them in the final version. However, based on our presented experiments, we can observe that lowering the participation rate does not significantly impact our performance, which indicates that our method still works and performs well in this setting. Therefore, we believe this would NOT be a major concern for this work.

 

Thank you again for all your valuable suggestions and questions. Please do not hesitate to let us know if there are any other questions and concerns.

 

---

References:

[1] PFLlib: Personalized Federated Learning Algorithm Library.

Dear Reviewer KA7L,

 

We sincerely thank you for your efforts in helping us improve our paper! With only two days remaining, we are kindly reaching out to request your further feedback. We believe we have thoroughly addressed all your concerns and respectfully hope you consider raising your score.

Thank you once again for your efforts in ensuring the high standards of academic excellence. We look forward to your response and remain available for any further discussions.

 

Best Regards,

The Authors

 

3. Decoupling strategy.

The heterogeneity information is not simply captured by the input $x_t$, but rather by the time dimension representation in the hyperbolic space during the Lorentz Transformation process. The key component that captures data heterogeneity is the time-like dimension of the hyperbolic embedding rather than the original input $x_t$. The time-like component of the output representation is calculated using the norm of the space-like dimensions and the curvature, as shown in Equation (4) of our manuscript. Its special computation method determines its uniqueness on the hyperbolic manifold. Therefore, after multiple layers of output, the 0-th dimension of the embeddings truly captures the heterogeneity information we hypothesize, and this is entirely consistent with our derivation. We have provided a detailed explanation of this in the rebuttal (Response (3/4) Q2 and Q3). We are confident that this is an intuitive and easily understandable concept within the field of hyperbolic neural networks.

Most importantly, our extensive experimental results, including thorough ablation studies, validate the significance and effectiveness of our proposed decoupling strategy. These empirical results provide concrete evidence supporting the merits and performance of our method, which we believe deserves proper acknowledgment and recognition. This is also acknowledged by your previous comments: "The authors provide ablation studies examining the impact of different components of the method". Additionally, this is also mentioned by

• Reviewer nTmN: Experimental study of the paper looks convincing
• Reviewer Wtgh: Experiments are extensive

 

---

In summary, in our opinions, all the issues you mentioned above are not the core contributions of our paper, and some are not even core research problems in the field of personalized federated learning (such as late-joining clients and partial participation). This does not affect the integrity and contribution of our work. We would like to re-emphasize that :

1. The core contribution of our work is being the first to leverage geometric information for decoupling personalized information, providing new insights into this field, as highlighted by

   • Reviewer 4tQH: "I think this work can inspire more ideas in federated hyperbolic learning problems."
   • Reviewer Dc6W: "I still find this approach offers valuable insights, particularly in new domains, and I will maintain my score."

2. Our choice to conduct experiments on graphs is well-supported by theoretical and intuitive justifications, better validating our hypotheses (Response 2/4 1). Furthermore, our experimental scopes are consistent with those of previous methods. Our work is coherent and complete within the scope of our claimed contributions.

3. We acknowledged that hyperbolic spaces might not be effective for ALL data types (which is not our contribution or focus). We have provided sufficient reasoning and potential future research directions regarding this. "Note that hyperbolic space is not universally optimal for all data distributions — some exhibit positive curvature — highlighting the need to model complex data structures in mixed-curvature spaces."

4. Partial training participation is not a key point in the point of personalized federated learning. We have already thoroughly verified the feasibility of our method for handling partial training participation in graph dataset including all strong baselines. For extreme scenarios, we believe our method still has great potential, as we explained before, which could be explored in future work, although it is not a core focus of the personalized federated learning field.

Thank you again for all your comments. We would greatly appreciate a reconsideration based on these clarifications and the actual merits of our work.

 

Best regards,

The Authors

 

---

References:

[1] Subgraph Federated Learning with Missing Neighbor Generation

[2] An efficient federated learning framework for graph learning in hyperbolic space

Dear Reviewer KA7L,

We appreciate your time and effort in reviewing our manuscript. However, we respectfully disagree with your follow-up evaluation and would like to address some misconceptions.

 

1. Theoretical Contribution.

First, regarding the theoretical contribution, specifically Corollary 1: We NEVER claimed this as our primary contribution even in the original manuscript. Rather, it serves as a necessary foundation in hyperbolic networks, which is not as trivial as suggested in your review. A thorough understanding of hyperbolic geometry would reveal its significance. The series of analyses we provided, including but not limited to Corollary 1, contributes to the completeness and correctness of our entire work. We also add a convergence analysis accordingly in the revised version. These theoretical foundations are essential components that support our main contributions and methodological innovations.

And what is important is that our method is designed and evaluated supported by strong theoretical motivation, as we explained in responses 1/4 and 3/4.

 

2.1 Experimental Settings.

Regarding experimental validation, we want to clarify that we NEVER claimed our method works universally across ALL datasets. We explicitly acknowledged limitations and future directions in both our original submission and revised version. And explanations are also provided in the rebuttal (Response 2/4 The reason we focus on graph data) and we also added this clarification in the Future Work Section that "Note that hyperbolic space is not universally optimal for all data distributions — some exhibit positive curvature — highlighting the need to model complex data structures in mixed-curvature spaces".

 

Regarding the experimental settings with up to 50 clients: We NEVER claim that work in a cross-device setting is our main contribution. Instead, we follow the graph federated learning settings to solve personalized FL problems, which are seldom considered in this scenario. This should not be considered a major concern, as **it aligns with current standards in federated graph learning research, as demonstrated in accepted papers $1-2$ **. More importantly, our comprehensive experimental results demonstrate excellent performance across these settings even in low partial participation rates in graph dataset, which strongly validates our hypotheses and the effectiveness of our proposed method.

 

2.2 Partial participation scenarios.

We respectfully strongly disagree with your assertion that our method cannot work in partial participation scenarios for the following reasons:

From the experimental results, our experimental results demonstrate strong and consistent performance, particularly in our emphasized settings, yet these significant results seem to have been overlooked in your evaluation. While you highlighted the MNIST results, we want to emphasize that the image domain dataset was not our primary focus, as we clearly stated in our response section. Due to time constraints, we haven't exhaustively explored all possible parameter configurations to find the optimal settings in image datasets (the results have been updated). We will explore more possibilities in future work.

Moreover, the key issue you mentioned is the scenarios where "the personalized portions of the parameters for the later joined clients are not updated." This problem is not the focus of the research on personalized federated learning (PFL). In fact, the extreme situation you mentioned occurs for all parameter decoupling methods, including FedPer, which is one of the most widely used and popular methods in PFL. This is a separate research problem, and our method is orthogonal to it. We believe relevant strategies for handling this issue are not in conflict with our method. Our decoupling strategy, similar to FedPer, effectively handles this issue by separating heterogeneous and homogeneous information but with enhanced interpretability and better heterogeneity handling.

Nevertheless, we are willing to provide a solution of how our method can solve this issue easily. When a new client joins, we can directly use the global shared parameters as the initialized shared parameters. Compared to FedPer, an advantage of our method is that we can pre-estimate the curvature of the client's data and directly fetch the well-trained personalized parameters of other clients with similar data curvature and use them as initialization. This not only avoids parameter optimization performance issues but also ensures privacy. In contrast, FedPer cannot effectively tell which personalized parameters are better or more useful for new clients. This, in turn, highlights a potential advantage of our method in this scenario compared to ordinary parameter decoupling methods, as we utilize geometry information, further underscoring the inspiring nature of our approach.

Thank you for your valuable comments and suggestions. Below, we will summarize the concerns with "C:" and our responses with "R:"

 

---

C1: Motivation about focusing on graph data and generalizability to other types of data and models.

R: In this work, we specifically chose to focus on graph data because graphs inherently exhibit non-Euclidean structures and significant heterogeneity, as demonstrated in previous theoretical analyses. Besides, the heterogeneity is directly related to the curvature of hyperbolic geometry [1]. This makes graph data particularly suitable for validating our method's effectiveness in handling complex geometric properties with a clear motivation and theoretical guarantee.

The main reason that we didn't compare with FedRep in the main paper is due to their lack of graph-related implementations. With the limited time, we have conducted additional experiments on MNIST to demonstrate our method's broader applicability. The results are as follows:

These results demonstrate that FlatLand performs competitively even on image data. We plan to explore FlatLand more for image and text datasets and analyze the detailed relation between the curvature and the specific data properties in future work.

 

---

Reference:

[1] Krioukov, Dmitri, et al. "Hyperbolic geometry of complex networks." Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 82.3 (2010): 036106.

[2] Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration, ICCV 2023

[3] Gpfl: Simultaneously learning global and personalized feature information for personalized federated learning, ICCV 2023

C2: Computational efficiency. “The method introduces additional complexity through hyperbolic geometry and requires curvature estimation, which could make implementation and tuning more challenging in practice. The computational overhead of working in Lorentz space compared to Euclidean space is not thoroughly discussed”

R: Let us clarify the complexity analysis of FlatLand compared to FedAvg:

1. Local updates. the additional operations in FlatLand (curvature estimation and exponential map). The curvature estimation does not introduce significant computational overhead since it only needs to be computed once for each client's data distribution. This value can be pre-computed and reused. Similarly, the exponential map transformation only requires a single non-linear mapping operation based on input sample norms with a complexity of $O(1)$, which can also be pre-computed and cached for efficiency.

2. Aggregation, FlatLand maintains the same time complexity as FedAvg when using equivalent hidden dimensions. While FlatLand introduces time-like space parameters, it only aggregates shared parameters while maintaining personalized parameters locally. Importantly, as shown in Section 7.3, FlatLand performs better in low dimensionality, potentially reducing communication costs in practice.

3. Storage requirements, FlatLand requires $O(d+1)$ additional storage per client compared to FedAvg, accounting for the time-like dimension and curvature parameter, where $d$ is the hidden dimension. This overhead is minimal since d is typically small, and FlatLand's superior performance in low-dimensional settings further mitigates practical storage concerns.

Overall, while FlatLand adds a few steps compared to FedAvg, the computational overhead is limited due to pre-computation and efficient constant-time operations. Consequently, this does not affect the scalability of the method. Moreover, our experiments show that its benefits outweigh the slight complexity increase, particularly for managing heterogeneous data in resource-constrained, low-dimensional settings. We have added this analysis in Appendix B.6.

 

---

C3: Numerical stability. “The paper doesn't address potential numerical stability issues that often arise when working with hyperbolic spaces.”

R: Regarding numerical stability, our method specifically employs fully Lorentz neural networks, which were designed to address the numerical instability issues commonly associated with hyperbolic neural networks. Traditional hyperbolic approaches often face instability due to frequent projections between tangent and hyperbolic spaces [5]. However, fully Lorentz neural networks perform operations directly in Lorentz space without any further mapping functions, significantly reducing these stability concerns.

 

---

C4: Differences with previous decoupling methods, particularly FedRep

R: We would like to further highlight our novelty. While FedRep does propose a parameter decoupling approach in Euclidean space, our method differs fundamentally in both motivation and implementation.

Our decoupling strategy is uniquely motivated by the geometric properties of Lorentz space, where the time-like dimension naturally captures heterogeneous information. This geometric interpretation, as illustrated in Figure 1(b), allows us to separate client-specific features (manifested in the time-like dimension) from shared information (in space-like dimensions) without requiring explicit similarity calculations or additional modules.

In contrast, FedRep focuses on finding shared representations in Euclidean space through optimization constraints. While both methods aim to separate shared and personalized components, our approach leverages the inherent structure of hyperbolic geometry to achieve this separation more naturally. This is evidenced by our theoretical analysis in Section 6, which shows how the time-like parameter inherently captures client-specific information through the gradient calculations (Equations 11-13).

Furthermore, our experimental results demonstrate the effectiveness of this geometry-driven approach, particularly in scenarios with strong non-Euclidean properties, as shown in the curvature analysis (Figure 6). We also conduct experiments that compare our method with FedRep in MNIST in the response to C1.

 

---

References:

[5] Hyperbolic graph convolutional neural networks. Chami, I., et al. (2019). NeurIPS

Q2: Potential challenges or modifications needed when applying your method to other domains.

R: Different data modalities exhibit varying levels of non-Euclidean characteristics. While graph data naturally possesses strong non-Euclidean properties (as shown by our Ricci curvature analysis in Figure 6), other domains like images and text also demonstrate heterogeneous structures. For instance:

1. In vision tasks, the manifold of natural images often exhibits hierarchical relationships (e.g., different scales of visual features) that can benefit from hyperbolic representation. The heterogeneity often comes from varying visual styles, lighting conditions, or camera angles.
2. In text data, semantic hierarchies and power-law distributions in word frequencies naturally align with hyperbolic geometry's ability to embed tree-like structures. The heterogeneity might arise from different writing styles, topics, or languages.

While our work represents a promising start in leveraging geometric information for personalized federated learning, we acknowledge that hyperbolic space might not be optimal for all types of data. Analyzing the scenarios and types of heterogeneity where hyperbolic geometry is most advantageous remains a promising direction for future work. Besides, we can explore spaces with mixed curvature (including positive and negative curvatures simultaneously) to better accommodate different data geometries. $\mathsf{FlatLand}$'s fundamental approach to addressing heterogeneity through geometric modeling provides a solid foundation for these extensions.

 

---

Q3: Scalability considering the additional overhead.

R: We provide a detailed analysis of the time complexity cost and memory usage of our method compared to the simplest FedAvg method, as addressed in our response to C2. This analysis highlights the scalability of our approach. Additionally, we conducted experiments on data from 100 clients, further validating the extensibility of our method.

 

---

We hope we've addressed your key concerns, particularly regarding the generalizability of our method beyond graph data, the complexity of curvature estimation, and computational efficiency. We demonstrated that FlatLand performs well on benchmarks like MNIST, with minimal computational overhead due to pre-computation (Appendix B.6.). Additionally, we clarified that curvature estimation is stable and does not impact convergence (Appendix C.2. and Appendix B.5.).

Thank you for your questions. Below, we will summarize the questions with “Q:” and our responses with "R:"

 

---

Q1: Curvature estimation implementation.

R: For curvature estimation, we employ Forman-Ricci curvature as described in Section 5.1. The motivation behind this choice is that Forman-Ricci curvature effectively captures how significantly a graph's structure deviates from Euclidean geometry, and has a theoretical connection with the curvature of hyperbolic geometry [1]. As shown in Figure 6, different clients exhibit varying curvature values, reflecting their distinct structural properties. For practical implementation, we treat curvature as a learnable parameter initialized using the Forman-Ricci estimate. Specifically, we use σ(K) + 0.5 to ensure the curvature remains negative and within a well-performing range of [0.5, 1.5]. This range has been empirically validated in previous hyperbolic learning literature and helps maintain numerical stability while accommodating heterogeneous data distributions [4-7].

The effectiveness of this approach is demonstrated in our ablation study (Figure 4), where comparing against a fixed curvature setting ("w/o TS") shows that adaptive curvature significantly improves performance and does not lead to heavy fluctuation, which further shows the stability of this method.

Moreover, we theoretically analyze the convergence rate of FlatLand and demonstrate that the parameter $K$ has almost no impact on the convergence rate. Detailed analyses are added in Appendix B.5.

 

---

References:

[4] Fully hyperbolic neural networks. Chen, W., et al. (2021). ACL

[5] Hyperbolic graph convolutional neural networks. Chami, I., et al. (2019). NeurIPS

[6] HGCF: Hyperbolic graph convolution networks for collaborative filtering. Sun, J., et al. (2021). WWW

[7] Discrete-time temporal network embedding via implicit hierarchical learning in hyperbolic space. Yang, M., et al. (2021). KDD

Dear Reviewer Dc6W,

 

We sincerely thank you for your positive feedback and efforts in helping us improve our paper! With only two days remaining before the deadline and having not yet received a response to our carefully formulated point-to-point replies, we kindly request your feedback. If you have any further questions or suggestions, we welcome the opportunity to discuss and address them.

We believe we have thoroughly addressed all the concerns raised in your initial review. And we would be grateful if you would consider raising your score accordingly.

Thank you once again for your efforts in ensuring the highest standards of academic excellence. We look forward to your response and remain available for any further discussions.

 

Best Regards,

The Authors

Thank you for your valuable comments and suggestions. Below, we will summarize the weaknesses with "W:" and our responses with "R:"

 

---

W1: Baselines selection.

R: Thank you for this suggestion. Note that our primary goal was to highlight and evaluate the benefits of integrating hyperbolic geometry in federated learning. Hence our comparisons focused on baselines with strong performance on graph data to ensure a fair evaluation. FedCAC and GPFL perform well in image datasets but lack graph-related implementations. Therefore, we compare our method with FedCAC and GPFL in image dataset; please check the response to W2.

We will incorporate these clarifications into the revised manuscript to provide a clearer context for our focus and methodology.

 

---

W2: This paper primarily focuses on graph datasets. Can FlatLand effectively perform on commonly used benchmarks in PFL?

R: Thank you for your suggestions. We first would like to demonstrate the reason we chose graph data to conduct experiments and analysis.

1. The reason for choosing graph data.

In this work, we chose to focus on graphs because graphs inherently exhibit non-Euclidean structures and significant heterogeneity, which are directly related to curvature. This makes graph data particularly suitable for validating the effectiveness of our hyperbolic geometry-based approach in addressing data heterogeneity.

• Theoretical foundation: The use of hyperbolic spaces to model varying graph structures is well-supported by theory. As demonstrated in [1], the relationship between graph structure and hyperbolic space provides a strong mathematical basis for our method's effectiveness.

• Clear demonstration of benefits: Graph data is ideal for showing our method’s ability as they generally have different curvatures. It also allows us to demonstrate the effectiveness of our parameter decoupling strategy in a setting where heterogeneity is more evident.

2. Applicability to other domains.

Our method can be easily applied to other data as our method directly adds one extra dimension to the linear layer. With limited rebuttal time, we have conducted additional experiments on MNIST to primarily demonstrate our method's broader applicability. The results are promising:

We plan to conduct more experiments and analysis on more domain and tasks. This is also mentioned in the "Future Work" section of our manuscript.

 

---

Reference:

[1] Krioukov, Dmitri, et al. "Hyperbolic geometry of complex networks." Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 82.3 (2010): 036106.

Dear Reviewer Wtgh,

 

We sincerely thank you for your positive feedback and efforts in helping us improve our paper! With only two days remaining before the deadline and having not yet received a response to our carefully formulated point-to-point replies, we kindly request your feedback. If you have any further questions or suggestions, we welcome the opportunity to discuss and address them.

We believe we have thoroughly addressed all the concerns raised in your initial review. And we would be grateful if you would consider raising your score accordingly.

Thank you once again for your efforts in ensuring the highest standards of academic excellence. We look forward to your response and remain available for any further discussions.

 

Best Regards,

The Authors

Thank you very much for your valuable comments and suggestions. Below, we will summarize the concerns with "W:" and our responses with "R:"

---

W1: Typos.

R: Thank you for your comments. We have corrected the identified typos and will conduct a thorough review to address any remaining issues.

 

---

W2 and Questions: Ambiguous concepts.

R: Thank you for your careful review. We appreciate your feedback and will address the ambiguity by adding clarifications to ensure the concepts are more precisely defined in the revised manuscript. Please let us address each point:

1. Regarding the operation function $f(Wx, v)$ and the usage of $v$:

The parameter $v$ is introduced to enable a more generalized formulation, supporting advanced non-linear operations with learnable parameters. While dropout is presented as a simple example without $v$ , the parameter can play a role in other minor operations within hyperbolic space. Specifically, $v$ can function as learnable parameters in normalization operations, constraining the norm via $\sigma(v^T x)$ to prevent excessive growth of hyperbolic embeddings, or as bias terms added to $x$. In later sections, the presentation is simplified to focus on the core linear transformation aspects (lines 283–284), as these additional operations do not influence the fundamental principles of our method.

2. Regarding Equation (4) and the usage of the first row of $\hat{M}^{(l)}$:

The first row of $\hat{M}^{(l)}$ provides a generalized formulation for our defined LT operation, ensuring the correct input-output matrix size. The seemingly "missing" usage arises because our focus was on presenting the key transformations (lines 283–284). Typically, the first row’s parameters support minor operations. We will clarify this in the revised manuscript for better transparency.

3.Regarding $K$ and its implementation:

Yes, $K$ is a learnable scalar parameter. To ensure the curvature remains negative (as required for hyperbolic space), we implement it as $\text{sigmoid}(K) + 0.5$. This design also keeps curvature $- K$ within an effective range of $[0.5, 1.5]$, which prior work has shown to be an ideal range for most hyperbolic models [1-4]. Additionally, this approach maintains numerical stability while satisfying the need for a heterogeneous space.

Thanks for bringing these points to our attention. We have revised the manuscript to include these clarifications in Appendix C2, making the technical details more transparent and accessible to readers.

4. Regarding no parameter decoupling strategy:

Yes. In the aggregation process, all parameters are aggregated including the curvature scalar ensuring consistency with our approach. We will clarify this in the revised manuscript.

 

---

W3: Implementation of the graph classification task.

R: Thank you for your comments. LGCN serves as the backbone for our graph learning framework, combining Lorentz linear layers (Equation 2) with graph aggregation operations, similar to how Euclidean counterparts like GCN and GIN integrate linear layers with graph aggregation. Each layer applies a Lorentz transformation followed by neighbor aggregation using the adjacency matrix to get the node representations. The "3-layer hyperbolic encoder" employs the same architecture with three stacked layers to learn node representations within each graph. For node classification, node representations are used directly, while graph classification employs mean pooling for graph-level representations.

As noted in Section 5.3, the parameter decoupling strategy remains valid since "the aggregation operation does not involve any parameters." This allows us to directly apply our methods to the parameters of the Lorentz linear layers without modification.

Thank you for the suggestion to include more technical details in the main text. Due to time constraints and the page limit, moving more content to the main pages at this stage is challenging. We will prioritize reorganizing the main text in the final version to better integrate these details.

 

---

W4: Submit code as supplementary materials.

R: Thank you for your suggestion. We have now included our code as part of the supplementary materials for easier access and review.

 

---

References:

[1] Fully hyperbolic neural networks. Chen, W., et al. (2021). ACL

[2] Hyperbolic graph convolutional neural networks. Chami, I., et al. (2019). NeurIPS

[3] HGCF: Hyperbolic graph convolution networks for collaborative filtering. Sun, J., et al. (2021). WWW

[4] Discrete-time temporal network embedding via implicit hierarchical learning in hyperbolic space. Yang, M., et al. (2021). KDD

Dear Reviewer 4tQH,

 

Thank you very much for your positive feedback. We're glad our work resonated. Your comments motivate us to explore the potential of hyperbolic geometry in solving various challenging federated learning problems further, aiming to inspire more research in this emerging area.

 

Best Regards,

The Authors

Thank you very much for your valuable comments and suggestions. Below, we will summarize the concerns with "W:" and our responses with "R:"

---

 

W1: Some arguments in the paper are vague.

R: Thank you for your comment. We will further refine our writing and try to provide as much explanation as possible within the limited space. Let us clarify these mentioned points:

• The first passage highlights that PFL approaches address heterogeneity through three main strategies during aggregation: (1) splitting models into shared and personalized components, (2) analyzing weights or gradients to evaluate client similarities, and (3) incorporating additional modules for client-specific customization. Importantly, all these methods operate in Euclidean space.

• The second passage emphasizes the challenge of embedding data from diverse clients into a fixed Euclidean space, as it complicates the interpretability of model parameters. In such settings, all parameters serve the same role during training, making it difficult to distinguish between those capturing client-specific heterogeneity and those representing shared patterns. This not only hinders meaningful model segmentation but also makes client similarity assessment more complex. Adding extra modules to address this further increases the system's complexity and reduces its flexibility.

Thanks again for your comments. We have made modifications to the original text (lines 33–36 and lines 48–52) to address this ambiguity.

 

---

W2: Additional operations from the Lorentz model.

R: Thanks for your comment. We would like to clarify this contradiction:

The computational complexity concerns raised in our original statement refer specifically to assessing similarities between client models during the aggregation phase, not the initial data projection. While FlatLand does indeed require projecting data into Lorentz space, this operation is computationally efficient for several reasons:

1. The exponential map is a single input mapping function (Equation 1) that depends only on the norm of the input: $\mathbf{x}^K = \left( \cosh \left(\frac{\||\mathbf{v}^E\||_2}{\sqrt{K}}\right), \sqrt{K} \sinh\left(\frac{\||\mathbf{v}^E\||_2}{\sqrt{K}}\right)\frac{\mathbf{v}^E}{\||\mathbf{v}^E\||_2} \right)$

This projection has a constant time complexity of $O(1)$ per data point and can be optimized through pre-computation of input norms before training begins.

2. No additional projections are required during training after the initial mapping.

3. Most importantly, FlatLand eliminates the need for expensive similarity computations during aggregation. While other methods require computing pairwise model similarities or maintaining additional computational modules, our method performs a simple aggregation operation with complexity equivalent to standard FedAvg.

4. The effectiveness of FlatLand in low-dimensional settings (as demonstrated in Section 7.3) further reduces practical computational and communication costs.

To make this clearer in the paper, we have added a detailed complexity analysis in Appendix B.6 that quantifies these trade-offs. The limited overhead from the initial projection is substantially outweighed by the computational savings during model aggregation and the reduced communication costs from effective low-dimensional representations.

Dear Reviewer nTmN,

 

We sincerely thank you for your positive feedback and efforts in helping us improve our paper! With only two days remaining before the deadline and having not yet received a response to our carefully formulated point-to-point replies, we kindly request your feedback. If you have any further questions or suggestions, we welcome the opportunity to discuss and address them.

We believe we have thoroughly addressed all the concerns raised in your initial review. And we would be grateful if you would consider raising your score accordingly.

Thank you once again for your efforts in ensuring the highest standards of academic excellence. We look forward to your response and remain available for any further discussions.

 

Best Regards,

The Authors