Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing

Thank you very much for your thoughtful review and for recognizing the potential of our work!

Your decision to support this submission gives us the opportunity to bring a foundational contribution to the community. We strongly believe that ExRec opens up a scalable and modular pathway for future research in personalized education, enabling researchers to plug in their own datasets, KT models, or RL policies and push the boundaries of exercise recommendation systems. We truly appreciate your feedback in helping us refine and strengthen this vision.

Below, please find our clarifications to the points you raised.

W1. The limitations specified in the introduction “they often rely on ID-based embeddings, which neglects the semantics of question”—is not correct. The first DKT paper was on question ID-based embeddings. But after that, the weakness has been addressed by the community. Most DKT algorithms rely of embeddings of KC, such as AKT [1]. Embeddings of questions has been explored by researchers’ way before the LLM era, see a 2018 paper: EERNN[2], a 2020 paper: RAKT[3], a 2022 paper: SRL-KT [4].

Thank you very much for your thoughtful comment! We agree that there has been meaningful progress in incorporating question and KC semantics into KT models, and we appreciate your references to EERNN, RAKT, and SRL-KT. However, we respectfully believe that the broader problem of relying on ID-based embeddings is far from solved, especially when viewed in the context of exercise recommendation or more generalizable KT frameworks.

Indeed, our framework differs from prior approaches not just in using semantic information, but in how we use it. The cited models typically rely on shallow integration, such as pre-trained word embeddings or static text encoders. In contrast, we train our question representations end-to-end via a contrastive learning module that explicitly aligns questions (and their solution steps) with corresponding knowledge concepts. This yields richer, more interpretable embeddings that better support both generalization and downstream tasks like RL-based recommendation.

Further, even the most recent and advanced KT models [e.g., 1,2,3,4,5,6] still primarily rely on ID-based embeddings or even only KC embeddings. This highlights that, while promising work exists, the field has not fully moved beyond this limitation. We appreciate the opportunity to clarify this distinction and will revise our framing accordingly.

[1] Li, Qing, Zhijun Huang, Jianwen Sun, Xin Yuan, Shengyingjie Liu, and Zhonghua Yan. "HKT: Hierarchical structure-based knowledge tracing." Information Processing & Management. 2025.

[2] Cheng, Weihua, Hanwen Du, Chunxiao Li, Ersheng Ni, Liangdi Tan, Tianqi Xu, and Yongxin Ni. "Uncertainty-aware Knowledge Tracing." AAAI. 2025.

[3] Bai, Youheng, Xueyi Li, Zitao Liu, Yaying Huang, Mi Tian, and Weiqi Luo. "Rethinking and improving student learning and forgetting processes for attention based knowledge tracing models." AAAI. 2025.

[4] Hou, Mingliang, Xueyi Li, Teng Guo, Zitao Liu, Mi Tian, Renqiang Luo, and Weiqi Luo. "Cognitive Fluctuations Enhanced Attention Network for Knowledge Tracing." AAAI. 2025.

[5] Bai, Youheng, Xueyi Li, Zitao Liu, Yaying Huang, Teng Guo, Mingliang Hou, Feng Xia, and Weiqi Luo. "csKT: Addressing cold-start problem in knowledge tracing via kernel bias and cone attention." Expert Systems with Applications. 2025.

[6] Shen, Xiaoxuan, Fenghua Yu, Yaqi Liu, Ruxia Liang, Qian Wan, Kai Yang, and Jianwen Sun. "Revisiting knowledge tracing: A simple and powerful model." ACM International Conference on Multimedia. 2024.

W2: The KC annotation in Section 4.1 is a problematic, in my opinion. Prior deep learning era, KC annotation was performed by domain experts with multiple rounds of review. The proposed approach annotated KC by a sampling method through LLM with no expert intervention. The step can be performed by a machine-teaching or human-in-the loop fashion. Similarly, the solution step generation is done by LLM, but no quality assurance is performed.

Thank you for raising this important point! We respectfully believe that the absence of expert intervention in our KC annotation process is not a drawback, but a key strength. Our aim is to reduce reliance on time-consuming and potentially inconsistent expert annotations to enable scalable and adaptable pipelines for new datasets and domains. That said, our framework is modular by design: teams with access to high-quality expert-labeled KCs can bypass this step and proceed directly to the subsequent modules.

Regarding quality assurance, we build upon recent studies [1,2] that rigorously evaluated the use of LLMs for KC annotation. These works conducted both automated and human evaluations, comparing LLM-generated annotations with those provided in standard KT datasets. In both cases, the LLM-based annotations were consistently preferred, particularly for their clarity, generalizability, and semantic coherence: they often outperformed the dataset-provided expert labels.

Finally, we agree that quality control remains important, and our approach naturally accommodates optional human review or refinement where needed. Our contribution lies in demonstrating that high-quality KC annotations can be generated automatically and reliably. This way, we significantly expand the accessibility and applicability of personalized educational systems.

Q1: Do the authors have any plan how to ensure QA of the generated solutions by LLMs?

Thank you very much for raising this important point! While enhancing the mathematical reasoning abilities of LLMs is beyond the scope of our work, we fully agree that ensuring the quality of generated solution steps is essential for maintaining the validity of our framework.

To this end, we leveraged the fact that the XES3G5M dataset includes ground-truth answers. After generating solution steps via the LLM, we conducted a second round of LLM-based evaluation, comparing the final computed answer within the generated steps against the correct answer. Since extracting the answer from free-form output is non-trivial, this LLM-assisted comparison proved both scalable and reliable. We observed a correctness rate of over 98%, with most discrepancies involving minor rounding issues.

Crucially, in our framework, what truly matters is not the final numeric accuracy, but the verbal description of the strategy expressed in the solution steps. This is because our contrastive learning and semantic embedding modules compute representations based on the textual and conceptual content of entire sentences, where numerical tokens contribute minimally to the embedding. Therefore, as long as the reasoning is sound and well-articulated, minor arithmetic deviations do not undermine the learning signal.

Still, for flagged errors, we prompted the LLM again (this time providing the correct answer) to encourage better alignment. We further validated the QA process by manually reviewing 100 random samples marked as correct by the evaluator, and confirmed that they were indeed accurate.

While not central to our core contribution, we will describe this procedure in detail in the implementation section. We appreciate your suggestion to include it!

L1:The authors have used only one dataset. They can also explore relevant K-12 math dataset on their proposed model, such as the ASSISTments datasets.

Thank you for the suggestion. We initially considered using ASSISTments, which exists in several versions across different years. However, in the versions accessible to us, the question content was either completely unavailable or embedded in highly noisy HTML, often rendering the items unreadable even for human annotators. Given that our framework critically relies on understanding the semantic content of questions for KC annotation and exercise representation, ASSISTments was unfortunately not a viable option for a meaningful evaluation.

That said, we took your suggestion seriously and extended our evaluation to another large-scale online math learning dataset, EEDI, which offers clean and verified textual content. This dataset includes 47,560 students, 4,019 unique questions, and a total of 2,324,162 interactions. It shares many real-world characteristics with the XES3G5M dataset and is provided by the same organization behind the NIPS34 benchmark. Its questions were OCR-transcribed and reviewed by EEDI’s internal team, making it well-suited for semantic analysis.

Task 1: Global Knowledge Improvement. (Reported: % Max. Improvement as in paper)

Note: The reported improvements on the EEDI dataset appear lower than those on XES3G5M because the maximum possible absolute improvement is much larger (0.56 vs. 0.21). As a result, students must achieve larger absolute knowledge gains to reach full mastery. In absolute terms, the improvements on EEDI are comparable to those on XES3G5M, even though the percentages appear smaller.

Thanks to the flexibility of our framework, we were able to seamlessly apply our entire pipeline to EEDI, and we report the results in the revised version.

Thank you for your thoughtful and constructive review! We appreciate your recognition of the novelty and potential of our work.

Our goal with ExRec is to provide a foundational, extensible framework that bridges knowledge tracing and reinforcement learning for personalized education. Built for flexibility, modularity, and scalability, ExRec allows researchers to plug in their own datasets, use advanced KT models, and test alternative RL strategies in a unified, semantically grounded environment.

We believe NeurIPS is the ideal venue to catalyze the next wave of research in personalized education, and we have addressed all your suggestions to further strengthen and clarify our contributions.

Q1: Missing several related works

Thank you for your valuable suggestions. Our original submission focused on works aiming to improve knowledge state using RL-based frameworks built on simulated student behavior, which aligns with our central goal: learning a recommendation policy that optimizes long-term knowledge improvement through semantically grounded KT.

Following your suggestion, we now broaden our discussion to include a wider range of exercise recommendation approaches. While relevant, these works do not directly optimize knowledge state, nor do they develop a trainable RL-based policy. Most rely on rule-based heuristics, optimize simpler proxy metrics (e.g., difficulty, novelty), and generally overlook semantic representations of questions and KCs.

We elaborate on each paper below in the order provided.

Exercise recommendation method based on knowledge tracing and concept prerequisite relations: This method recommends KCs rather than actual exercises, treating all exercises within a KC as interchangeable. It relies on manually defined prerequisite graphs and uses only KC IDs, ignoring the semantic content of both KCs and questions. The recommendation is rule-based and static. It neither learns from data nor simulates how student knowledge evolves over time.

Exercise recommendation based on knowledge concept prediction: This work models KC mastery and recommends KCs based on predicted gains. The recommendation logic is rule-based and not trainable. It does not model question semantics, nor does it use RL or simulate student behavior. It focuses on proxy objectives like difficulty or novelty, whereas ExRec directly optimizes for actual knowledge state improvement.

Pedagogical interventions in SPOCs: Learning behavior dashboards and KT support: This work applies KT to estimate students’ knowledge but does not perform exercise recommendation. Instead, it provides a dashboard interface for instructors to decide what to recommend. The KT model relies on hand-crafted features, unlike ExRec, which operates directly on question semantics and automates the full pipeline.

Enhanced personalized learning exercise question recommendation based on KT: This approach predicts question difficulty and filters exercises accordingly, based on instructor-specified difficulty ranges. It applies KT but does not simulate student learning or use RL. It assumes external input of difficulty to guide recommendation and does not attempt to optimize knowledge improvement directly.

MLKT4Rec: Enhancing Exercise Recommendation Through Multitask Learning with KT: MLKT4Rec casts recommendation as a multitask supervised problem, jointly predicting correctness and recommending exercises to maximize immediate success. Therefore, it does not prioritize long-term knowledge gain. It does not incorporate RL, simulate knowledge gain, or model semantic content or KCs. In contrast, ExRec optimizes for long-term knowledge improvement, incorporates LLM-based KC annotations, and enables generalization to unseen questions.

Knowledge modeling via contextualized representations for LSTM-based personalized exercise recommendation: This method uses LSTM and attention to recommend exercises based on historical interactions. It focuses on predicting immediate performance without RL, semantic modeling, or KC representations. ExRec advances this line by using traced knowledge states, semantically enriched representations, and RL to guide students toward lasting knowledge gains.

-> We will incorporate these works into our extended Related Work section to clearly position ExRec in relation to these approaches and to highlight our unique contributions.

Q2: Limited dataset evaluation: The proposal is validated on only one knowledge tracing dataset. Several well-established benchmark datasets are available, such as NIPS34 and EdNet. The authors are encouraged to add these representative datasets to provide more comprehensive validation of their proposal.

Thank you for the suggestion. While we agree that broader dataset evaluation is important, our framework specifically leverages the semantic content of questions, which is central to our contribution. Unfortunately, the suggested datasets are incompatible with this requirement:

• EdNet does not provide question text — only question IDs — making it unusable for semantically grounded recommendation.

• NIPS34 includes only image-based questions, many with geometric content. Converting them to meaningful text requires reliable multi-modal understanding, which is beyond this paper’s scope. However, we appreciate this suggestion and will include it as a limitation and a future direction.

To address your feedback, we extended our evaluation to the EEDI dataset, developed by the same creators as NIPS34. EEDI provides OCR-transcribed and manually verified textual question content. Therefore, it is well-suited for our framework. It also reflects real-world conditions, with 47,560 students, 4,019 unique questions, and over 2.3M interactions.

Thanks to ExRec’s modular design, we seamlessly applied our full pipeline to this dataset. Please find the results below:

Task 1: Global Knowledge Improvement. (Reported: % Max. Improvement as in paper.)

Note: The reported improvements on the EEDI dataset appear lower than those on XES3G5M because the maximum possible absolute improvement is much larger (0.56 vs. 0.21). As a result, students must achieve larger absolute knowledge gains to reach full mastery. In absolute terms, the improvements on EEDI are comparable to those on XES3G5M, even though the percentages appear smaller.

We are grateful for the opportunity to demonstrate that our framework generalizes easily to other large-scale math datasets. This has been a great opportunity to demonstrate its practical applicability and extensibility.

Q3: Lack of mutual impact analysis: While the proposal is designed for exercise recommendation, it would be valuable to investigate whether the proposal could, in turn, improve knowledge tracing performance.

Thank you very much for your valuable (and quite interesting) suggestion! As a disclaimer, please correct us if we misinterpreted your suggestion — if that’s the case, we are more than happy to investigate it more deeply!

Following your suggestion, we treated the generated trajectories through our RL environment as the synthetic dataset (i.e., the new question-answer pairs). We then appended this synthetic dataset to the original KT dataset and performed another round of KT training.

After the above procedure, we found that the AUROC score of KT model dropped from 81.65 to 81.32. We believe the main reason is that the original dataset was large enough to train our KT model and that the synthetic dataset propagated the errors of a KT model that caused a decrease in the performance.

To confirm the above hypothesis, we trained another KT model (from scratch) with only 10% of the existing students in the dataset and repeated the above procedure of producing synthetic data through our RL environment. This time, we observed an increase in the prediction performance from 79.41 to 79.98 AUROC.

These findings suggest that our framework can serve not only as a recommendation policy but also as a data augmentation tool to improve KT performance. This is particularly relevant in low-data regimes.

-> We will incorporate this analysis into the paper as an additional use case, especially highlighting its value when the number of available student interactions is limited.

Q4: Insufficient efficiency analysis: The authors are encouraged to report the inference time performance, as it directly impacts the feasibility of real-time implementation in educational systems.

Thank you very much for this insightful suggestion! While we had already reported the training time and hardware requirements of our computationally intensive modules in Appendix B, we agree that inference time is an important factor for real-time deployment in educational systems.

In response, we conducted additional experiments to measure inference efficiency. Below, we report:

• KT training takes around 2.5 hours on a single NVIDIA Quadro RTX 6000 GPU with 32 GB of memory.

• KT inference per question takes 8.7 milliseconds on a single NVIDIA Quadro RTX 6000 GPU with 24 GM of memory.

• RL inference per exercise recommendation takes 78.5 milliseconds on a single NVIDIA Quadro RTX 6000 GPU with 24 GM of memory.

We will add these details to our Appendix B.

Thank you for your thoughtful and constructive review! We appreciate your recognition that our work brings a new perspective by combining student modeling with sequential decision-making, a direction we believe has strong potential for personalized education.

Our goal with ExRec is to lay a foundation for this emerging area, and we see NeurIPS as the ideal venue to foster further exploration and community engagement. We are grateful for the opportunity to clarify and strengthen our work based on your feedback.

W1: The title suggests personalized exercise recommendation, but the experiments mainly evaluate a KT + RL framework’s impact on knowledge improvement — without assessing the accuracy or relevance of the recommended exercises.

Thank you for the comment. By personalized exercise recommendation, we mean selecting exercises tailored to each student’s knowledge needs to improve their knowledge state. Since experimenting on real students is infeasible (ethical and scalability issues), we simulate learning through KT. While this is bounded by KT model’s quality, our modular design allows future integration of more advanced KT models.

Regarding “accuracy,” there is no single “correct” action in RL; we instead measure effectiveness via long-term knowledge improvement, as captured in our reward design.

Yet, to address this, we add a clairvoyant-style metric: % of times the chosen exercise falls in the top-10, top-20, or top-50 actions based on immediate (1-step) reward. For each of 1024 students and each step, we compute rewards for all 7652 possible questions.

We believe this metric complements our main results by showing that our trained policies (especially SAC w/ MVE) consistently select high-reward exercises significantly more often than a random policy.

W2: The dataset has KC annotations, so why add more via steps and LLMs?

Thank you for the observation. While KC annotations exist, many are noisy or overly specific—e.g., “Single person speed change problem” or “Application question module–Chicken rabbit cage problem–Hypothesis method–Prototype question”. Such labels are hard to generalize or align with learning standards.

Instead of using these as-is, we generated concise, semantically grounded KCs with LLMs, aligned with Common Core standards. Recent studies [1,2] show this approach produces more interpretable and preferred annotations, yielding better semantic representations and a stronger foundation for recommendation.

W3: Since Module 1 requires LLMs pre-processing, it is difficult to judge whether the model is an end-to-end structure.

Thank you for the comment. While Module 1 uses LLMs for KC annotation, it is fully automated and requires no manual input. This aligns with common definitions of “end-to-end,” as the system operates directly from raw question texts and student logs to exercise recommendations.

Importantly, Module 1 is optional: if reliable KC labels are available, it can be skipped. We include it to enhance flexibility and reduce reliance on manual or noisy annotations, which are often missing in real-world datasets.

W4: The contrastive learning module design is unclear. Why is question–step alignment not used, given the stepwise structure from the LLM?

Thank you. ExRec is the first KT-based framework to learn embeddings for both questions and KCs simulatenously, aiming for a shared semantic space. Instead of collapsing the full question (all steps) into a single embedding aligned with all KCs, we use step–KC alignment for two reasons:

1. Avoiding representation collapse: Full question–KC alignment forces the encoder to compress diverse concepts into one vector, which prior work [2] and our experiments show is ineffective.

2. Finer-grained supervision: Aligning each step only with its relevant KCs provides a clearer learning signal, improving representation quality and downstream performance. This design produces more generalizable embeddings, critical for handling unseen questions at test time.

W5: The paper lacks external validation. Results are based solely on KT model simulations, with no comparison to real student behavior, expert suggestions, or human judgments.

Thank you for raising this important point. We fully agree that real-world validation is the next, but non-trivial next step. Note that this is not unique to our paper, but such external validation is infeasible and therefore generally missing in related work. However, we took your suggestion to heart and compared 100 student traces against expert suggestions. We found that ExRec’s recommendations were preferred 92% of the time with respect to targeting the knowledge concepts students had yet to master.

For example, when a student lacked understanding of both “Addition of decimal numbers” and “Alignment of decimal numbers”, ExRec first recommended a question that practiced both:
What is the result of 42.62 + 4.1 = ?
After the student answered incorrectly, ExRec adapted and recommended a simpler question that still targeted “Addition of decimal numbers” but removed the alignment requirement:
What is the result of 1.5 + 0.3 = ?

As one of the less-preferred cases, when a student struggled with “Multiplication of decimal numbers”, ExRec recommended a relevant question but one that also involved a unit conversion. Our experts suggested that a simpler question should have been given first.

We will include this external validation as additional evidence supporting the practical relevance of ExRec.

Q1: Line 27, add the literature on personalized practice recommendation through KT, such as [1] or more.

We couldn’t find [1] in your review, but we are happy to incorporate it in our paper once we have it!.

Q2: Footnote [1], there are some new KT reviews that have conducted in-depth analysis from different perspectives, and it is recommended to add references.

Thank you for your contribution! Upon your suggestion, we further incorporated [3] and [4], and we are happy to incorporate if you have other recommendations.

Q3: Why do you think the KT model can simulate the RL environment?

Thank you for the question. We address it in two parts: (1) why KT is well-suited for simulating an RL environment, and (2) why prior attempts failed to scale.

1. Many KT works (e.g., those cited in line 26) are motivated by the idea that online learning benefits from personalization. This requires modeling how a student’s knowledge evolves in response to different questions, a natural fit for simulation and policy learning. However, most of these works stop at next-question prediction and do not explore how KT can actually be used for personalized recommendation.

2. Only a few works (e.g., cited in line 30) attempted to use KT for student simulation and RL-based recommendation. Yet they failed to scale to realistic datasets: their state representations often exceeded memory limits, and reward computation required expensive inference over all questions at each time step. To address this, we propose an entirely new framework that is scalable by design.

Our framework overcomes these limitations by: (1) enabling recommendation of unseen questions, (2) supporting various RL algorithms, and (3) integrating into a scalable, public library for broader use -> a practical contribution we believe is often underappreciated.

Q4: Line 132, which LLM Encoder is used?

As described in Appendix B line 559, we used and finetuned BERT encoder through our contrastive learning module.

Q5: Contrastive learning optimizes (1) questions and concepts (2) steps and concepts. What is the consideration for doing so? Why not questions and steps?

Thank you for your questions! Please see our response to your W4.

Q6: Does the representation of the problem (Formula (10)) come from the average representation of the problem and steps?

You are correct. The representation used in downstream KT and RL modules is obtained by averaging the embeddings of the question text and its solution steps, as described.

We avoided alternatives like concatenation or more complex fusion methods for the following reason:

• To address Limitation 3 (inefficient reward computation), we enable the KT model to perform inference not only on questions (as in prior work) but also directly on knowledge concepts (KCs), a novel feature in our framework.

• After Module 2 (contrastive learning), question texts, steps, and KCs share a unified embedding space. For KT to process both exercises and KCs seamlessly, they must also have same dimensionality. Thus, in Eq. (10), we average question and step embeddings to match the size of a KC embedding.

We hope this clarifies our motivation for the design in Eq. (10).

Q7: As far as I know, there is a certain percentage of picture questions in XES3G5M dataset. How do you handle this type of input?

Thank you for your question. For module 1, we leveraged gpt-4o’s visual understanding to annotate solution steps and KCs. For the LLM encoder to work on question content at module 2 (contrastive learning), we first prompted gpt-4o to write the description of the visual in the question, and appended this description to the question content as input.

References

[1] Moore, Steven et al. “Automated generation and tagging of knowledge components from multiple-choice questions”” Learning@ Scale, 2024.

[2] Ozyurt, Yilmazcan et al. “Automated knowledge concept annotation and question representation learning for knowledge tracing”. arXiv, 2024.

[3] Liu, Zitao et. al. “Deep Learning Based Knowledge Tracing: A Review, A Tool and Empirical Studies”, Knowledge and Data Engineering, 2025.

[4] Zhou, Yiyun et al. "Revisiting applicable and comprehensive knowledge tracing in large-scale data." arXiv, 2025.

We sincerely thank the reviewer for their thoughtful and constructive feedback! We truly appreciate your recognition of the novelty and integration quality of our framework, and we’re grateful that you took the time to engage deeply with our work.

Your comments helped us better articulate the strengths and scope of our contributions. As your review rightly notes, educational personalization is a complex challenge, and we believe that presenting ExRec at a venue like NeurIPS is vital to spark meaningful discussion and catalyze progress at the intersection of large language models, reinforcement learning, and education.

We hope that our responses have clarified the key distinctions and practical innovations introduced in our framework, and we would be truly grateful for your support in helping bring this work to the broader research community.

W1: The motivation could be refined. The manuscript claimed that current related methods "typically support only a single RL algorithm". IMHO, these methods could conveniently extend their methods with other RL algorithms.

Thank you for raising this point. Our claim is not that existing methods theoretically cannot adopt multiple RL algorithms, but rather that they are not implemented in a modular or scalable way to make this practical.

Many prior works rely on custom, tightly coupled environments and pipelines, making integration of alternative RL methods non-trivial and error-prone. Most did not release code, and their scalability remains unclear, especially on large-scale datasets.

In contrast, ExRec is fully built on the standardized Tianshou library, supporting plug-and-play use of various RL algorithms. We also addressed critical bottlenecks such as memory-efficient state representations and concept-based reward computation.

We will revise the manuscript to clarify that our contribution lies in enabling scalable, efficient, and modular integration of diverse RL policies, which has not been realized in prior work.

W2: Heavy reliance on LLMs for KC annotation and solution step generation raises concerns about robustness and generalizability across domains/languages without fine-tuning.

Thank you for this point. Our motivation for using LLMs goes beyond quality—it is also about scalability and efficiency. Manual annotation is costly, while our approach scales to large systems and new domains.

On quality, prior work [1,2] shows LLM-generated annotations outperform expert tags in coherence and generalizability. Original dataset labels were often noisy and hard to reuse (e.g., “Application question module–Chicken rabbit cage problem–Hypothesis method–Prototype question”), unlike our concise labels (e.g., “Linear Equations”, “Area of Triangle”).

We acknowledge LLM outputs can be inconsistent. Our contrastive learning module (Module 2) mitigates this by aligning question/step embeddings with concepts, reinforcing consistency.

To test generalizability, we applied the pipeline—unchanged—to the EEDI dataset (2.3M interactions, verified question content) and observed that it transfers well across math domains.

These results show our framework is robust and adaptable, while still allowing for domain-specific validation where needed.

W3:For educational applications, understanding why a particular exercise is recommended can be crucial for educators and students. Although the paper mentions interpretable student learning trajectories, it does not delve into the interpretability of the RL policy's recommendations themselves.

Thank you for raising this point. While interpretability was not the paper’s primary focus, we fully agree it is important for real-world use, and we designed ExRec with this in mind.

Our RL policy operates in the same embedding space as the knowledge concepts (KCs), so one can analyze the action embedding’s proximity to KC embeddings. For example, if a student struggles with “multiplication” and “factoring out a common factor,” the RL policy will produce an action embedding close to these KCs, therefore making it possible to infer which concepts are being targeted and select a question accordingly.

Interpretability also extends beyond RL: our calibrated KT module supports real-time, concept-level inspection, which allows educators to query student knowledge on any KC at any time step. We will make these aspects clearer in the final version.

Action: While we are not allowed to add figures here, we will add an example with visualizations to the revised paper.

Q1: Is the claim that existing methods “typically support only a single RL algorithm” fully accurate, and could the authors clarify why those methods cannot be easily extended to support multiple RL algorithms?

Thank you very much for your question! We would like to kindly refer to our response to your W1 for our detailed answer!

Q2: How robust and generalizable are the LLM-generated KC annotations and solution steps across different mathematical domains or languages, especially in the absence of additional fine-tuning?

Thank you for the question. We clarify that our framework does not require any LLM fine-tuning to enable easy deployment across new datasets. This is supported by prior research: [1] demonstrated the effectiveness of LLM-based KC annotation in domains like Chemistry and Physics. Further our own experiments on the EEDI math dataset confirm generalizability within Math.

We still acknowledge that LLM outputs should be evaluated before large-scale annotation, especially in new subjects or languages.

Our modular pipeline allows flexible integration of custom prompts, LLMs, or expert-curated annotations. To apply ExRec in other domains, we offer the following guidance:

• If questions lack multi-step solutions, skip step generation and directly use KC annotation.

• Use the resulting annotations for representation learning with question-level loss.

• The rest of the pipeline (KT + RL) remains unchanged.

Finally, public KT datasets with full question content are rare, mostly limited to Math. We hope our work encourages broader dataset releases to support cross-domain research.

Q3: To what extent is the RL policy itself interpretable, and can the authors explain how educators or students might understand why specific exercises are recommended?

Thank you very much for raising this important concern. We would like to kindly refer to our response to your W3 above.

Q4: The paper uses a fixed horizon of 10 steps for RL agent interaction. How does the performance of the RL policies, especially those with MVE, scale with a significantly longer recommendation horizon, and what challenges might arise in such scenarios?

Thank you very much for the thoughtful question! We now performed additional experiments with varying horizons, particularly for 20, 50 and 80 steps.

Task 4: Knowledge Improvement in Weakest KC. (Reported: % Max. Improvement as in paper.)

Original Corpus

Extended Corpus

We observe that our framework continues to provide notable gains at longer horizons. In fact, our MVE framework benefits from longer horizons, albeit improvements being marginal.

One challenge we noticed is that, even with MVE, the model may need to revisit the same weakest KC approximately 30–40 steps after it was first addressed. We hypothesize that the KT model tends to lower the knowledge state of certain KCs over time, particularly when no relevant exercises are encountered for an extended period. We believe that future KT models could explicitly account for this forgetting behavior, better distinguishing when a knowledge state should (or should not) decrease if a KC is left unpracticed for a long time.

We will incorporate these new findings in the paper. Thank you very much for helping us improve our submission!

Q5: How generalizable is ExRec to other domains where KCs are less structured and LLM annotation may be harder?

Thank you for this question. Applying ExRec to non-math domains can pose challenges, especially with less-structured KCs, but we believe our framework is well-suited.

First, datasets with full question text are very limited, restricting cross-domain experimentation. We hope our work encourages broader dataset releases.

Importantly, ExRec does not require hierarchical KC structures or expert ontologies. It infers semantically meaningful KCs directly from question text, lowering barriers in less-structured domains.

While math is considered one of the most demanding domains for LLMs, we believe areas like reading comprehension or social sciences may be easier to model semantically. In those cases, LLM-based annotation may face fewer obstacles and still produce usable KC representations for downstream learning and recommendation.

We will reflect this discussion in the revised limitations section.

References

[1] Moore, Steven et al. “Automated generation and tagging of knowledge components from multiple-choice questions”” Learning@ Scale, 2024.

[2] Ozyurt, Yilmazcan et al. “Automated knowledge concept annotation and question representation learning for knowledge tracing”. arXiv, 2024.