EduLLM: Leveraging Large Language Models and Framelet-Based Signed Hypergraph Neural Networks for Student Performance Prediction

We sincerely thank the reviewer for the constructive feedback and for acknowledging the novelty and strengths of our proposed framework. Please find our detailed responses below:

• LLM Selection, Fine-tuning, and Task Adaptation: We appreciate the reviewer’s observation and agree that further analysis of LLM choices could be valuable in broader contexts. However, to clarify the scope of our work: LLMs in EduLLM are primarily used as a semantic information extraction tool to generate initial node features for questions in the student-question signed hypergraph, rather than being the core focus of technical innovation. One reason why we do not study in detail the effect of different LLM selections is that, as we would like to note, this submission is flagged under the ''Application-Driven Machine Learning'' category, where the focus is on solving a specific real-world task and not on advancing LLM development itself. Another reason is that we aimed to ensure fairness in the performance comparison with the baseline models, so we used the same LLM module and processing pipeline to generate the preprocessed semantic embeddings. This manner ensures that any observed improvements are due to the merits of the proposed framework, particularly the signed hypergraph setting and advantages of FraS-HNN, rather than differences in the LLM processing itself.

  Alternatively, to approximately assess the impact of LLM-based embeddings on model performance, we conducted a robustness study presented in Section 4.7. The results, shown in Figure 3, demonstrate that the model’s performance degrades smoothly and only slightly at higher noise levels. This robustness suggests that EduLLM is relatively insensitive to variations in the LLM-induced embeddings, implying that replacing the current LLM backbone with a different one is unlikely to significantly affect task performance.

• Large-scale Dataset: We agree that evaluating on larger-scale datasets would strengthen the practical validation of EduLLM. Our current datasets are widely used and shared with baseline models, ensuring fairness and comparability. Meanwhile, our lab is constructing new large-scale datasets with MCQ texts from various subjects, which will allow us to further assess scalability and real-world applicability in future work.

• Theoretical Clarity and Intuition: While detailed proofs are provided in the appendix, we agree that offering more intuitive explanations would help clarify the motivation and benefits of our approach. In a future updated version, we will provide more intuitive explanations of FraS-HNN’s advantages over existing methods, which we believe will also benefit researchers working on (signed) hypergraph learning.

• Comparisons with Existing Hypergraph and KT Models: We would like to clarify that our problem formulation, predicting student performance at the question level using signed hypergraphs, is not entirely equivalent to the KT task, which (although relates to student performance prediction) typically focuses on modeling students' evolving mastery over concepts across time. We have added additional evaluations against representative hypergraph neural networks (HNNs), including HGNN, HyperGCN, AllDeepSets, AllSetTransformer, ED-HNN, SheafHyperGNN. Due to space constraints, the detailed results are provided in our response to Reviewer stfm, demonstrating clearly FraS-HNN's effectiveness in capturing signed high-order interactions.

• Computational Complexity and Scalability: We provide a comparative analysis of training complexity for several representative HNNs and our proposed FraS-HNN, summarized in the table below. In our formulation, $k$ (the number of high-pass filters), $J$ (the scale level in FraS-HNN), and $K$ (the largest number of non-zero values in the framelet transform matrices) are constants independent of specific hypergraph structures. In practice, $k$ and $J$ are small, and due to the sparsity of hypergraph framelets, $K$ is typically small and often comparable to $\|H\|_0$. As a result, FraS-HNN's overall complexity is comparable to models such as AllDeepSets and ED-HNN, without introducing significant overhead. | Model | Computational Complexity | |---------------------|----------------------------------| | UniGCNII | $\mathcal{O}(TL(N+M+\|H\|_0)d + TLNd^2)$ | | Deep-HGCN | $\mathcal{O}(TLM'd + TLNd^2)$ | | AllDeepSets | $\mathcal{O}(TL\|H\|_0d + TL(N+M)d^2)$ | | ED-HNN | $\mathcal{O}(TL\|H\|_0d + TL(N+M)d^2)$ | | FraS-HNN (ours) | $\mathcal{O}(TL(kJ+1)Kd + TL(N+M)d^2)$ |

  Here, $N$ and $M$ denote the number of nodes and hyperedges, respectively; $\|H\|_0$ is the number of non-zero entries in the incidence matrix; $T$, $L$, and $d$ denote the number of epochs, layers, and feature dimensions; $M^{'}$ is the number of edges in the clique expansion (when transforming the hypergraph into a graph).

We sincerely thank the reviewer for the encouraging feedback on our work. We are pleased that you recognized the novelty and technical contributions of EduLLM, including its theoretical soundness, methodological innovation, and comprehensive experimental validation. For your key concerns, please find our responses below:

• Generalization Beyond Educational Domains & Alternative Wavelet Designs: Thank you for noting these issues. For detailed clarifications regarding the potential generalization of FraS-HNN beyond educational applications and the discussion on exploring alternative wavelet designs, please kindly refer to our responses to Reviewer Lfh6 (who is also curious about this). In addition, we would like to emphasize that, as flagged in our submission type, this work is submitted under the Application-Driven ML Submissions category. Our particular focus is specifically on the student performance prediction task, a well-recognized and meaningful problem in the educational domain. The proposed EduLLM framework is newly developed for solving this specific problem. Therefore, we did not engage in studying broader generalization across domains, as that falls outside the intended scope of our application-driven submission. For more context, please refer to the Supplementary Guidelines for Reviewing Application-Driven ML Submissions (see https://icml.cc/Conferences/2025/ReviewerInstructions).

  Hopefully, this clarifies the position of our work within the specific context of this year’s ICML submission categories..

• Framelets for Directed and Signed Hypergraphs: We appreciate the reviewer’s insightful question regarding the extension of framelet theory to directed and signed hypergraphs. Although there is very recent work defining the notion of directed hypergraphs (see: https://openreview.net/forum?id=h48Ri6pmvi), to the best of our knowledge, a formal and unified definition of directed and signed hypergraphs has not yet appeared in the literature. Developing framelet transforms for such structures would require extending spectral theory to handle both edge directionality and polarity, potentially involving non-symmetric Laplacians or incidence-based spectral operators. Key challenges include defining appropriate inner product spaces, preserving desirable mathematical properties (e.g., tightness and localization), and ensuring the interpretability of the resulting representations. While this extension is beyond the scope of the current work, we view it as an important direction for future investigation. Of course, we hope that our current work, particularly the key module FraS-HNN, can motivate further follow-up studies on directed and signed hypergraphs, whether through model development or advanced applications.

Again, we sincerely thank you for recognizing the merits of our proposed FraS-HNN module (also recognized by Reviewer Lfh6), which we believe contributes not only to the student performance prediction task but also holds broader value for advancing (signed) hypergraph learning in general. In response to Reviewer stfm's suggestions, we have conducted additional empirical studies that include comparisons with several representative hypergraph neural network (HNN) baselines, i.e. HGNN (AAAI, 2019), HyperGCN (NeurIPS, 2019), AllDeepSets (ICLR, 2022), AllSetTransformer (ICLR, 2022), ED-HNN (ICLR, 2023), and SheafHyperGNN (NeurIPS, 2023), by replacing the FraS-HNN backbone of EduLLM with each of these modules. Please refer to our response to Reviewer stfm for further details. These supplementary results further validate the effectiveness of the high-pass and low-pass filters in FraS-HNN, as positively noted in your comments. For future work, we plan to construct more challenging signed hypergraph datasets beyond the educational domain (e.g., in social networks, biology, traffic systems) to support the development of this emerging direction in terms of theoretical understanding, model design, and real-world applications. We believe our framework, along with the core FraS-HNN module, can benefit other complex tasks and applications where signed hypergraphs (or their variants) align well with the problem formulation.

We hope the above responses have clarified your questions and comments. We welcome any further discussion or suggestions.

We sincerely thank the reviewer for the thoughtful evaluation. We are pleased that the novelty and effectiveness of EduLLM and FraS-HNN are recognized, along with the theoretical soundness, strong empirical results, and broader contributions to hypergraph learning and educational data mining. Below we provide point-by-point clarifications regarding the concerns raised:

• Generalization Beyond Educational Domains: We agree that it is important to consider the applicability of FraS-HNN beyond the student performance prediction task. Although the current work is tailored for modeling educational data, the proposed signed hypergraph formulation and framelet-based representation learning are inherently general and applicable to domains involving higher-order and polarity-sensitive interactions. For instance, tasks such as signed social network modeling, sentiment-based recommendation, or misinformation spread detection can similarly benefit from the ability to distinguish positive and negative multi-way relations. The design of FraS-HNN does not rely on domain-specific assumptions, which supports its potential transferability.

• Motivation for Haar-type Filter: Mathematically, the choice of the Haar-type filter is based on its efficiency, orthogonality, and suitability for decomposing signals into coarse (low-frequency) and detail (high-frequency) components. In the context of signed hypergraphs, this allows the model to simultaneously capture common learning behaviors and student-specific deviations. While other filters (such as Daubechies or spline-based filters) may offer smoother basis functions or better frequency localization, the Haar-type was selected as a starting point due to its simplicity and proven utility in prior work on multiscale graph signal processing. Exploring alternative filters remains a promising direction for future extension.

• Visual Comparison with Signed Graphs: We appreciate the suggestion to better highlight the advantages of signed hypergraph modeling. To support this, we will provide illustrative examples showing how signed hypergraphs can represent multi-way interactions (e.g., groups of students responding to related questions) with polarity, in contrast to signed graphs that are limited to pairwise relations.

• Further Insights and Clarification for the Theoretical Properties: The intention behind our theoretical analysis is to reveal how the proposed FraS-HNN effectively captures both low-pass and high-pass components of node signals in signed hypergraphs, enabling the model to differentiate shared and individualized patterns within complex educational interactions. To make these insights more accessible to the readers, we will summarize the key takeaways in the main text (in the updated version), such as: (1) how the framelet transform enables multiscale analysis on signed hypergraphs, (2) the specific role of positive and negative hyperedges in modulating spectral responses, and (3) how this facilitates richer representation learning compared to traditional graph-based or unsigned hypergraph approaches.

Again, we sincerely thank you for recognizing the merits of our proposed FraS-HNN module, which we believe contributes not only to the student performance prediction task but also holds broader value for advancing (signed) hypergraph learning in general. In response to Reviewer stfm's suggestions, we have conducted additional empirical studies that include comparisons with several representative hypergraph neural network (HNN) baselines, i.e., HGNN (AAAI, 2019), HyperGCN (NeurIPS, 2019), AllDeepSets (ICLR, 2022), AllSetTransformer (ICLR, 2022), ED-HNN (ICLR, 2023), and SheafHyperGNN (NeurIPS, 2023), by replacing the FraS-HNN backbone of EduLLM with each of these modules. Please refer to our response to Reviewer stfm for further details. These supplementary results further validate the effectiveness of the high-pass and low-pass filters in FraS-HNN, as positively noted in your comments. For future work, we plan to construct more challenging signed hypergraph datasets beyond the educational domain (e.g., in social networks, biology, traffic systems) to support the development of this emerging direction in terms of theoretical understanding, model design, and real-world applications. We believe our framework, along with the core FraS-HNN module, can benefit other complex tasks and applications where signed hypergraphs (or their variants) align well with the problem formulation.

We hope the above responses clarify the key concerns raised in your comments and questions. Please feel free to reach out with any further suggestions or points for discussion. We are happy to engage further to improve the clarity and impact of our work.

We sincerely thank the reviewer for carefully reading our paper and providing detailed feedback. We respectfully offer the following clarifications:

• Further Clarification on Motivation: While cognitive diagnosis models have explored learner-element relationships using graphs, our goal is to provide a new perspective by modeling student-question interactions using signed hypergraphs. Specifically, in the context of student performance prediction, where responses can be either correct or incorrect, we introduce positive and negative hyperedges to explicitly encode this polarity in group-level interactions. We believe this formulation brings a fresh and principled structural view to the problem, which forms the core of our motivation.

• Clarifying the Role of LLMs: We would like to note that, this submission is flagged under the Application-Driven Machine Learning type, where the focus is on solving a specific real-world task rather than developing LLMs itself. In our framework, LLMs are used as a semantic feature extraction tool to generate question embeddings, serving as initial node features in the signed hypergraph. One reason we did not explore different LLM selection or prompt strategies is that we aimed to ensure fairness in performance comparison with the baseline models. We kept the use of the same LLM module and processing pipeline for generating the preprocessed semantic embeddings. This manner ensures that any observed improvements are due to the merits of the proposed framework, particularly the signed hypergraph setting and advantages of FraS-HNN, rather than differences in the LLM processing itself.

  For your concern about stability and proportional contribution of LLM-induced embeddings, we actually conducted a robustness test in Section 4.7, where additive Gaussian noise was introduced to the semantic embeddings to simulate perturbations. The results (in Figure 3) indicate that performance degrades smoothly and moderately as the noise level increases, suggesting that EduLLM is generally robust to fluctuations in LLM outputs. To a certain extent, this empirical evidence supports the stability of the model with respect to semantic input variations. In addition, the advantage of using LLM-induced semantic embeddings, as opposed to initial embeddings, has already been validated in related works such as SBCL and LLM-SBCL.

• Comparisons and Baseline Selection: We would like to clarify that our problem formulation, i.e., modeling student-question interactions via signed hypergraphs, is fundamentally different from traditional knowledge tracing or cognitive diagnosis tasks. Specifically, cognitive diagnosis methods are designed around concept-level mastery modeling over time and typically require a fine-grained concept-question mapping. In contrast, our approach focuses on question-level prediction using signed hyperedges to represent correctness of responses, which is not directly supported by the data structures or assumptions of cognitive diagnosis datasets. Therefore, direct comparison would be misaligned and may lead to unfair conclusions. That said, we acknowledge the broader relevance of cognitive diagnosis research in educational modeling and will incorporate appropriate discussion in a future version. Additionally, based on your suggestion, we have included comparisons with hypergraph neural network (HNN) baselines, including HGNN, HyperGCN, AllDeepSets, AllSetTransformer, ED-HNN, and SheafHyperGNN, by replacing the FraS-HNN backbone of EduLLM with each of these HNN modules. As shown in the table below, EduLLM (with FraS-HNN) consistently outperforms the variants equipped with each HNN module across all datasets, demonstrating the effectiveness of FraS-HNN in modeling signed high-order interactions and its potential to benefit future research in (signed) hypergraph learning.