We are deeply grateful to you for recognizing the strengths of our work, particularly the clear presentation of our methodology and the balanced trade-off CKA-RL achieves among plasticity, stability, and scalability. We sincerely appreciate your acknowledgment of our comprehensive experimental validation and the potential impact of our approach on the continual reinforcement learning community.

Q1. The experimental evaluation lacks statistical rigor (e.g., multiple seeds, standard deviations), and the absence of code availability limits reproducibility.

A1. Thank you for your valuable feedback on the statistical rigor of our evaluation. We will address this in the revised version, as detailed below:

• Multiple Random Seeds Used. Our experiments were conducted with 10 different random seeds, consistent with standard practice in the recent CRL method CompoNet. We acknowledge that this detail was not explicitly stated in our initial submission, which was an oversight.
• Statistical Significance Evidence. We have computed and included standard deviations for all reported results. The updated results table (see Tab. A) now shows mean performance $\pm$ standard deviation across all task sequences and methods. Notably, CKA-RL demonstrates lower variance compared to most SOTA methods.
• Code Availability Commitment. We confirm that our code will be released upon paper acceptance.

Tab. A: Summary of results across all task sequences and methods.

Q2. There is insufficient discussion of related work on model merging in CL. Is the main contribution the adaptation to CRL, or are there novel methodological innovations?

A2. We appreciate your insightful comments. While we referenced model editing techniques as inspiration for our knowledge vectors, we agree that a deeper analysis of the connection to model merging literature would strengthen our paper. We will expand the discussion on model merging in the revised version of our manuscript. While our work is indeed inspired by model merging techniques, CKA-RL introduces several substantive methodological novelties beyond simply adapting these techniques to RL:

• Dynamic Knowledge Adaptation Mechanism. Unlike traditional model merging that uses static pre-trained models, our Continual Knowledge Adaptation strategy dynamically determines how to utilize historical knowledge through learnable adaptation factors ($\alpha$). This allows the agent to flexibly select and weight relevant knowledge based on the current task, which is critical in non-stationary RL environments where task relationships are complex and evolving.
• Task-Specific Knowledge Vector Learning. In contrast to existing model merging methods that operate on full model parameters, we introduce a specialized knowledge vector structure that isolates task-specific adaptations from the base policy. This design specifically addresses RL challenges like policy interference and catastrophic forgetting by preserving a stable base policy while allowing task-specific adjustments.

Q3. The methods in literature typically use more tasks and longer horizons, making comparisons on plasticity challenging in the current setup.

A3. We appreciate your feedback regarding the evaluation of model plasticity. Specifically:

• Standard Benchmarking Practice. Our main experimental setup, including task count and evaluation protocol, follows standard practice in the continual reinforcement learning (CRL) literature. Like CompoNet [ICML 2024], our core experiments are conducted on 20-task sequences using widely accepted CRL benchmarks. This ensures a fair and consistent comparison across all methods.
• Comprehensive Performance Metrics. Beyond simple task count, we measure plasticity through multiple metrics: 1)Average performance; 2) Forward transfer; 3) Memory efficiency analysis (Fig. 5a), demonstrating our method parameter memory remains nearly constant after the 5th task, while competing methods like ProgNet and CompoNet show linear growth.
• CKA-RL maintains superiority on longer task sequences We conduct additional experiments with extended 22-task sequences on the Meta-World environment. The results demonstrate that even with longer task sequences, CKA-RL retains its performance advantage (CKA-RL vs CompoNet: (0.4601, -0.0042) vs (0.4098, -0.0073)). We will include these results in the revised manuscript to strengthen our plasticity evaluation.

Q4. The manuscript claims that superior initial performance is due to effective use of historical knowledge vectors, but in the second task, no such historical vectors are available.

A4. Thanks for your valuable comments. We will clarify in the revision. Below we address this concern with specific points:

• Clarification of Knowledge Vector. The historical knowledge vector $v_1$ is defined as a zero vector ($v_1 = \theta_1 - \theta_{base} = 0$) as stated in Eqn.(3) of our paper. Therefore, the superior initial performance observed in the second task is not solely due to historical knowledge vectors.
• Mechanism of Knowledge Transfer from the Third Task Onward For tasks beyond the second, historical knowledge vectors $(v_1, v_2)$ become non-zero and actively contribute to superior initial performance. From Fig. 4, the reward curve of CKA-RL demonstrates a higher starting point compared to baselines like CbpNet and CReLUs.

Q5. The performance and memory efficiency of CKA-RL under longer task sequences remain unclear.

A5. We would like to clarify the rationale behind our experimental design, while providing additional evidence for the scalability of CKA-RL.

• Standard Benchmarking Practice in CRL. While CompoNet presents a theoretical scalability analysis involving 300 tasks, by constructing models and measuring parameter growth without actual training, our scalability analysis is conducted on real tasks with full training, enabling a more realistic assessment of memory and performance under continual learning.
• Scalability Guarantee. The core of our scalability argument lies in the Adaptive Knowledge Merging mechanism. Unlike methods that accumulate parameters linearly (e.g., ProgNet, CompoNet), CKA-RL maintains a bounded knowledge vector pool through similarity-based merging.
• Additional Long-Sequence Experiments. Following CompoNet setting, we conduct supplementary experiments with extended task sequences (300 sequences). From Tabs B&C, CKA-RL maintains nearly constant parameter size and inference latency through vector merging, while other methods show linear growth and increasing computational overhead.

Tab. B: Total Parameter Memory Consumption (MB) in the Meta-World.

Tab. C: Inference latency (s) in the Meta-World.

Q6. How does the size of the knowledge vector pool affect performance, especially with increasing task diversity and quantity? Are there guidelines or heuristics for determining the optimal pool size in such scenarios?

A6. Thank you for your insightful feedback. We appreciate the opportunity to elaborate on this aspect.

• Empirical Analysis of Pool Size Impact. From Fig. 3, we evaluated the effects of different $K_{max}$ values on both performance and forward transfer. When $K_{max}$ is small, CKA-RL removes too many knowledge vectors during adaptive knowledge merging, thus being unable to utilize enough knowledge. When $K_{max}$ is too large, redundant or even conflicting knowledge may slow down the knowledge adaptation process, resulting in performance degradation.
• Task Diversity Considerations. When task diversity increases, the optimal Kmax tends to increase as well. We conduct experiments in Meta-World (which has higher task diversity), we find $K_{max}=8$ to be optimal (see Tab. D).
• Practical Guidelines for Determining $K_{max}.$ 1) Initial Estimation: Start with $K_{max} = 5$ as a baseline for moderate task diversity; 2) Task Similarity Analysis: If tasks are highly heterogeneous, increase $K_{max}$ proportionally to the task diversity; 3)Memory-Performance Trade-off: For resource-constrained environments, select the smallest Kmax where performance plateaus.

In future work, we plan to investigate dynamic pool sizing mechanisms that automatically adjust $K_{max}$ based on task similarity metrics and performance feedback.

Tab D. Effects of different pool sizes in Meta-World.

---

We sincerely hope our clarifications above have addressed your questions.

Q1. Softmax function for computing the weights $\beta$, it is not possible to enforce $\alpha_1=1$. Parameter conflicts remain unavoidable.

A1. We sincerely thank the reviewer for this insightful observation. Below we address the concern regarding the use of the softmax function for computing the weights $\beta$ and the potential for parameter conflicts:

• Intent and Mechanism Behind the Design Choice. The statement “by setting $\alpha_1 = 1$” was meant to convey the functional behavior of the model under specific learned conditions, rather than a hard-coded assignment. Specifically:
  • When learning the very first task ($k = 1$), there are no previous knowledge vectors. Therefore, the summation $\sum_{j=1}^{k-1} \alpha_j \mathbf{v}_j$ is empty, and the policy is constructed solely from the base parameters and the current task’s knowledge vector $\mathbf{v}_1$ (initialized to zero and then optimized).
  • In this case, the model effectively starts learning from scratch without interference from prior tasks, which is what we intended to capture with the idea of “setting $\alpha_1 = 1$.”
• Handling Subsequent Tasks and Emergent Behavior. For subsequent tasks, while the softmax does not allow any single $\alpha_j$ to be exactly 1 unless others are 0, our model learns to assign very high weight (close to 1) to beneficial knowledge vectors and near-zero weights to irrelevant or conflicting ones. From Fig. 11a, our learned knowledge vectors exhibit near-orthogonal structures, indicating minimal interference. This emergent behavior allows the model to effectively disregard harmful prior knowledge, which mirrors the intended behavior of "setting $\alpha_1 = 1$", i.e., preventing negative transfer.
• Clarification of Original Phrasing. We will revise the manuscript to clarify that this is an emergent property of the learned attention weights $\alpha_k$, not a fixed constraint. This will ensure that readers understand that the model adapts dynamically based on the learned task-specific knowledge.

Q2. Why the performance of CompoNet in this paper appear significantly lower than reported in its original publication?

A2. We appreciate your attention to the performance comparison between our implementation of CompoNet and its original publication. Clarify as follows:

• Consistent Performance Metrics. When comparing results rounded to two decimal places, our implementation of CompoNet shows performance metrics that align closely with those reported in the original CompoNet paper. Specifically, in Meta-World (0.41 vs 0.42) and SpaceInvaders (0.98 vs 0.99), the performance values are consistent within reasonable rounding precision.
• Code Implementation and Corrections. We built our evaluation upon the official CompoNet implementation released by the authors [28]. During our implementation process, we identified several inconsistencies in the original codebase, including mismatched MLP layer configurations compared to the baseline implementations. We have carefully corrected these issues while maintaining the core algorithmic principles of CompoNet to ensure fair and accurate comparisons across all methods.
• Reproducibility Commitment. We will release our complete codebase upon paper acceptance. This will allow the research community to verify our implementation and reproduce all experimental results, including the CompoNet baseline evaluations.

Q3. Is the overall parameters of the model the form $\theta_{base}x$ until the size of the knowledge pool in Fig. 3?

A3. Thanks for this important feedback regarding the parameter growth in our CKA-RL framework. We would like to clarify this point as follows:

• Overall Parameter Count in CKA-RL. The overall parameter count in CKA-RL remains bounded, regardless of the number of tasks. While each knowledge vector $v_j$ shares the same dimensionality as the base parameters $\theta_{base}$, the Adaptive Knowledge Merging mechanism (Sec. 4.3) ensures that the knowledge vector pool $V$ never exceeds a fixed maximum size $K_{max}$.
• Adaptive Knowledge Merging Mechanism. When the knowledge vector pool size $|V| > K_{max}$ (typically set to 5, see Fig. 3), our algorithm identifies the most similar pair of knowledge vectors $(v_m, v_n)$ using Equation (6) and merges them into a single compact representation: $v_{\text{merge}} = \frac{1}{2}(v_m + v_n)$. This merging process ensures that the essential knowledge is preserved while keeping the pool size bounded.
• Parameter Composition in CKA-RL. The total number of parameters in CKA-RL is composed of:
  • Fixed base parameters $\theta_{base}$ (which are never updated after initial training).
  • A bounded knowledge vector pool with at most $K_{max}$ vectors.
  • Additional lightweight parameters for the attention mechanism ($\beta^k$).

Q4. The standard forgetting metric is not reported in Tab. 1.

A4. We sincerely thank the reviewer for this insightful observation regarding the evaluation of catastrophic forgetting. We agree that including the standard forgetting metric would provide a more comprehensive assessment of our method's capabilities. Below is our response addressing this concern:

• Supplementing the Evaluation with Standard Forgetting Metric. We have supplemented our evaluation with the standard forgetting metric: Forgetting=before training on new task - after training on new task [r1], as shown in Tab. A.
• Key Insights from the Results. PackNet achieves the lowest forgetting rate (0.04), but this comes at the cost of significantly lower overall performance (0.2767).In contrast, CKA-RL achieves the highest performance (0.7923) and forward transfer (0.7429) among all methods, while maintaining a relatively low forgetting rate (0.45).

Tab. A: Results on Freeway.

[r1] Replay-enhanced Continual Reinforcement Learning, TMLR 2023.

Q5. For catastrophic forgetting, the weighted sum of history vectors may not be robust to conflicting knowledge across tasks.

A5. Thanks for this insightful feedback regarding potential knowledge conflicts in our Continual Knowledge Adaptation. Clarify as follows:

• Emergent Orthogonality of Knowledge Vectors. Softmax prevents exact assignment of $\alpha_1$, but our framework naturally learns to approximate this behavior. From Fig. 11a, the learned knowledge vectors exhibit near-orthogonal structure across tasks, with off-diagonal cosine similarities ranging from -0.24 to 0.12. This emergent property enables the model to assign weights extremely close to 1 for beneficial knowledge and near-zero for conflicting knowledge. In practice, when a new task conflicts with previous ones, the weights converge to values where $\alpha_j\approx0$ for incompatible vectors and $\alpha_1\approx1$ for compatible ones.
• Base Parameter Stability. The base parameters $\theta_{base}$ remain fixed, providing a stable foundation that prevents complete overwrite of fundamental capabilities. This ensures that even when historical knowledge vectors conflict, the core functionality of the agent remains preserved.
• Conflict Resolution via Adaptive Merging. Our Adaptive Knowledge Merging mechanism actively identifies and resolves conflicts by only merging highly similar knowledge vectors, while preserving distinct task-specific knowledge.
• Task Identity Preservation. Contrary to the concern about discarding task-specific identities, our merging mechanism actually preserves task-specific knowledge by only combining highly similar vectors. Additionally, our approach intentionally avoids task-ID information to reflect real-world scenarios where task boundaries are unknown.
• Empirical Validation. From Tab. A and Tab. 3, CKA-RL effectively balances the stability-plasticity trade-off—preserving sufficient knowledge from previous tasks while remaining adaptable to new tasks. Our method successfully mitigates catastrophic forgetting without sacrificing the ability to learn new tasks effectively.

Q6. The claim that the method “preserves and adapts critic model parameters,” seems overly strong.

A6. We appreciate this opportunity to clarify the mechanism of our approach.

• Clarifying "Preservation" Terminology. We acknowledge that our phrasing could be misinterpreted as suggesting strict preservation of exact parameter values. In reality, CKA-RL achieves functional preservation rather than literal parameter preservation. As demonstrated in Figure 11a, our learned knowledge vectors exhibit near-orthogonal structure, enabling the model to effectively preserve task-specific knowledge by assigning weights extremely close to 1 for beneficial knowledge and near-zero for conflicting knowledge. This emergent orthogonality allows the model to functionally disregard harmful prior knowledge while retaining useful information.
• Beyond Simple Initialization. While a stronger initialization does contribute to performance, our method provides more than just initialization benefits: 1) During training on task k, the model dynamically accesses relevant knowledge from previous tasks. 2) From Fig. 4, CKA-RL demonstrates consistent performance improvements throughout the entire training trajectory, not just at initialization.
• Knowledge Transfer Mechanism. Our adaptive merging mechanism specifically targets vectors with high similarity while preserving distinct task-specific knowledge. This selective merging enables effective transfer even across dissimilar tasks, as evidenced by our strong performance across diverse environments.

---

We sincerely hope our clarifications above have addressed your questions.

Q1. Why did you choose residual model parameters for knowledge vectors?

A1. Thank you for this insightful comment regarding our knowledge vector design. Clarify as follows:

• Optimal Stability-Plasticity Balance [18]. Unlike full policy parameters exhibiting high cross-task correlation, residual vectors $v_k$ capture task-specific adaptations in nearly decoupled directions (Fig. 11a, similarity: −0.24–0.12). This minimizes interference while enabling effective transfer, with $\theta_{base}$ providing stability and $v_k$ enabling plasticity.
• Comprehensive Policy Adaptation: The residual vector compactly encodes all necessary adjustments to the base policy’s behavior, as validated by near-perfect performance on complex tasks.
• Scalability Through Merging: Fig. 5a proves our method maintains constant memory (11.49MB after 10 tasks) while CompoNet requires 108.21MB.

Q2. How are betas updated?

A2. Thank you for your insightful comment on the learning mechanism of adaptation factors. The $\beta$ are directly optimized during the training of each new task through standard gradient-based methods. Specifically:

• For each new task, we initialize a set of learnable parameters $\beta_k=[\beta_1^k,\ldots,\beta_{k-1}^k]$.
• These $β^k$ parameters are directly optimized alongside the current task knowledge vector $v_k$ during the reinforcement learning process.
• The gradients for $β^k$ come from the RL objective function, just like gradients for policy parameters.

Q3. Design of similarity between knowledge vectors.

A3. Thanks for your comment. Clarify as follows:

• Cosine Similarity vs. Functional Similarity. Cosine similarity in parameter space does not directly capture the functional similarity between policies. However, task (or knowledge) vectors [18] are defined as the difference between fine-tuned and base models, exhibit meaningful structure: near-orthogonal vectors often correspond to disjoint tasks, and similar vectors often lead to shared behaviors. While not a perfect proxy, cosine similarity remains a practical measure of representation similarity, which correlates with functional behavior in many settings [18].
• Empirical results. The high cosine similarity between certain knowledge vectors suggests shared structure, and we confirm this with empirical results:
  • Our merging mechanism preserves performance across tasks, showing no degradation in previously learned policies.
  • From Fig. 11, the near-orthogonality of learned vectors helps reduce interference, supporting functional retention.

Q4. Effects of knowledge pool size.

A4. Thanks for your insightful comment. Please see Reviewer-dAWz, A6.

Q5. Which environment in Fig. 4?

A5. Fig. 4 is in the SpaceInavders environment.

Q6. The standard forgetting metric.

A6. Thanks for your insightful comment. Please see Reviewer-33rx, A4.

Q7. Memory efficiency and inference efficiency.

A7. Thanks. We have conducted thorough memory and inference efficiency evaluations on both Meta-World and Freeway environments. From Tabs A&B, CKA-RL maintains nearly constant parameter size and inference latency through vector merging, while other methods show linear growth and increasing computational overhead.

Tab. A: Analysis in the Meta-World.

Tab. B: Analysis in the Freeway.

Q8. The statistical significance results.

A8. Thanks for your valuable feedback. Please see Reviewer-Mgpw, A1.

Q9. Why can CKA-RL address conflicts between tasks?

A9. Thanks. Cross-task conflict in CRL primarily manifests as parameter overlap and interference. CKA-RL mitigates this through the knowledge vector.

• Orthogonal knowledge vectors. From Fig. 11a, CKA-RL’s task-specific knowledge vectors exhibit near-orthogonal structures in parameter space. This minimizes representational overlap between task-specific adaptations, directly reducing the potential for conflict when learning new tasks.
• Stable base parameters. The fixed base parameter preserves general knowledge across tasks, ensuring that foundational abilities are not overwritten by task-specific updates. This separation prevents new task parameters from disrupting core knowledge needed for earlier tasks.
• Adaptive knowledge weighting. $\alpha_k$ dynamically adjusts the contribution of historical knowledge vectors to new tasks, allowing the model to prioritize relevant historical knowledge while suppressing conflicting information.

Q10. Task Sequence Impact.

A10. Thanks for your comment. Clarify as follows:

• Task Sequence Impact. The choice of the first task as base parameter does matter. Results confirm that different task sequences lead to performance variations (about 3.7% difference in performance).
• Robustness Analysis. Despite this sensitivity, CKA-RL demonstrates greater robustness to task ordering than methods like CompoNet, as shown in Tab. C.

Tab. C: Different task orders on the Freeway.

Q11. Why does defining the null knowledge improve the flexibility?

A11. Thanks. The term "flexibility" refers to the unified mathematical formulation of knowledge adaptation across all tasks, including the first task $\tau_1$. By defining $v_1 =\theta_1 - \theta_{base} = 0$ and including it in the knowledge pool $\mathcal{V}$. In Eqn.(4), the adaptation factors $\alpha_j^k$are derived from learnable parameters $\beta^k$ through softmax. When $v_1 = 0$, the value of $\alpha_1^k$ becomes irrelevant to the final policy parameter, allowing the model to "learn" to ignore historical knowledge when it's not beneficial.

Q12. “The model can disregard previously learned knowledge when it is not beneficial for learning new tasks.” Why?

A12. Thanks. This statement refers to the role of the null knowledge vector $v_1$ in our framework.

• $\alpha_1$ is the adaptation factor corresponding to $v_1$, the null knowledge vector. Since the sum of all $\alpha^k_j$ (j = 1, ..., k-1) is constrained to 1, setting $\alpha_1 = 1$ forces all other factors $\alpha^k_2, ..., \alpha^k_{k-1}$ to be 0.
• Because $v_1 = 0, \alpha_1 = 1$ means the model’s parameter update for the new task $\tau_k$ depends only on the base parameter and the current task’s knowledge vector, with no contribution from other historical knowledge vectors $v_2, ..., v_{k-1}$. In this case, the model effectively disregards previously learned knowledge stored in these vectors. This design ensures the model avoids over-reliance on unhelpful historical knowledge.

Q13. Do you store learned betas and how are policies restored?

A13. Thanks. We do not store the $\beta^k$ parameters.

• Role of $\beta^k$ Parameters. The $\beta^k$ serve as temporary, task-specific variables during the training of a new task. Their role is to dynamically determine the adaptation factors $\alpha^k$ that weight historical knowledge vectors, facilitating the acquisition of the current task’s knowledge vector.
• Restoring Policies for Previously Learned Tasks. To restore the policy for a previously learned task $\tau_i$, we rely on the stored knowledge vector $v_i$ and the fixed base parameter $\theta_{base}$. The policy parameter is defined as: $\theta_i = \theta_{base} + v_i$. Since $v_i$ encapsulates all task-specific adaptations needed for $\tau_i$, restoring the policy only requires retrieving $v_i$ from the knowledge pool and combining it with $\theta_{base}$.

Q14. Why does performance improve with the adaptive knowledge merging mechanism?

A14. Thanks. Clarify as follows:

• Reducing interference from conflicting knowledge. By merging redundant or conflicting vectors, the model avoids carrying irrelevant or contradictory information into new task learning. This minimizes "noise" in knowledge transfer, allowing the model to focus on truly useful historical insights.
• Preserving and amplifying critical information. The merging process retains essential task-specific knowledge while consolidating similar vectors into more generalized representations. This strengthens the signal of transferable knowledge, enabling more effective adaptation to new tasks.

Q15. Why does CKA-RL lead to superior initial performance and rapid convergence?

A15. Thanks. The reason is that CKA-RL ability to leverage high-quality historical knowledge as a useful prior while minimizing interference during new task learning.

• Effective reuse of historical knowledge. CKA-RL initializes the current task’s knowledge vector to 0 and dynamically weights relevant historical vectors via $\alpha^k$, directly boosting initial performance.
• Refined knowledge signals from merging. The adaptive knowledge merging mechanism consolidates redundant vectors, strengthening the signal of transferable knowledge. This ensures the historical knowledge used to initialize new tasks is more relevant and less noisy, further speeding up learning.

We are deeply grateful to you for recognizing the strengths of our work, particularly the clear presentation of our methodology and the balanced trade-off CKA-RL achieves among plasticity, stability, and scalability. We sincerely appreciate your acknowledgment of our comprehensive experimental validation and the potential impact of our approach on the continual reinforcement learning community.

Q1. Significance of Empirical Results is Unclear: My biggest concern is the lack of statistical rigor in the experimental validation. The performance improvements reported in the main results (Table 1) appear to be quite marginal (e.g., ~1-4% in many cases). The authors state in the checklist that results are from a single run with a fixed seed and do not report standard deviations. For an empirical RL paper, this is a critical omission. Without error bars over multiple runs, it is impossible to determine if these small advantages are statistically significant or simply the result of run-to-run variance.

A1. Thank you for your valuable comment regarding the statistical rigor of our experimental evaluation. We will address this in the revised version, as detailed below:

• Multiple Random Seeds Used. Our experiments were conducted with 10 different random seeds, consistent with standard practice in the recent CRL method CompoNet. We acknowledge that this detail was not explicitly stated in our initial submission, which was an oversight.
• Statistical Significance Evidence. We have computed and included standard deviations for all reported results. The updated results table (provided in Tab. A) now shows mean performance ± standard deviation across all task sequences and methods. Notably, CKA-RL consistently demonstrates lower variance compared to baselines.
• Contextualizing the Magnitude of Improvements. While the percentage improvements may appear modest numerically, they represent meaningful advances in the context of continual RL. 1) The field of CRL has seen diminishing returns in performance gains, with recent SOTA methods (CompoNet, CbpNet) showing only average 1-3% improvements over prior work (CReLUs); 2) Our method achieves these gains while using significantly less memory (about 5.8MB vs CompoNet's about 108.21MB at the 10th task).

Tab. A: Summary of results across all task sequences and methods.

Q2. Moderate Methodological Novelty: The core idea of decomposing model updates into task-specific vectors is heavily inspired by prior work in model editing and task arithmetic. While its application to CRL is new, the fundamental mechanism is not.

A2. Thank you for your thoughtful comment regarding the methodological novelty of our work. We would like to clarify the substantive contributions of CKA-RL beyond its conceptual inspiration from model editing techniques.

• Acknowledgment of Inspiration. You are correct that our knowledge vector concept is inspired by prior work in model editing and task arithmetic, as explicitly acknowledged in Section 4.1 of our paper.
• Fundamental Novelty in CRL Context. The core contribution of CKA-RL lies not merely in applying this concept to continual reinforcement learning, but in developing a complete framework specifically designed for the unique challenges of CRL.
  • Dynamic Knowledge Adaptation Mechanism Unlike static model merging approaches in supervised learning that typically combine pre-trained models with fixed weights, our Continual Knowledge Adaptation strategy dynamically determines how to utilize historical knowledge through learnable adaptation factors. This allows the agent to flexibly select and weight relevant knowledge based on the current task, which is critical in non-stationary RL environments where task relationships are complex and evolving.
  • Adaptive Knowledge Merging for CRL. Our merging mechanism (Eqn. (8)) is specifically designed for the sequential nature of continual RL, where tasks arrive sequentially rather than simultaneously. The similarity metric and merging strategy we propose are tailored to the unique characteristics of RL knowledge transfer, considering both parameter similarity and performance impact.

Q3. For related work, could the authors also discuss more recent crl specific papers like 1. Prediction and Control in Continual Reinforcement Learning. Anand et al. 2. Knowledge Retention for Continual Model-Based Reinforcement Learning. Sun et al. etc. ?

A3. Thank you for your valuable suggestion regarding the inclusion of more recent CRL-specific papers in our related work section. We appreciate this insightful observation and will incorporate these important works in our revision. Below is our detailed response:

• Inclusion of Suggested Works. We sincerely thank you for bringing "Prediction and Control in Continual Reinforcement Learning" by Anand et al. and "Knowledge Retention for Continual Model-Based Reinforcement Learning" by Sun et al. to our attention. These represent significant recent contributions to the CRL field that were inadvertently omitted from our initial submission.
• Comparison with Anand et al.'s Work. Anand et al. propose decomposing the value function into permanent and transient components that update at different timescales. While their approach shares conceptual similarities with our work in addressing the stability-plasticity dilemma, there are fundamental differences: Anand et al. focus on value function estimation, while CKA-RL operates directly on policy parameters. Our knowledge vector approach decomposes policy parameters into stable base weights ($θ_{base}$) and task-specific knowledge vectors ($v_i$), rather than manipulating value functions.
• Relationship to Sun et al.'s DRAGO. Sun et al.'s DRAGO framework introduces innovative mechanisms for knowledge retention in model-based CRL. While highly relevant, our approach differs in scope aspects: DRAGO specifically targets model-based RL where tasks share state spaces and dynamics but differ in rewards, while CKA-RL is designed for both model-free and model-based settings with broader applicability across diverse task distributions.

---

We sincerely hope our clarifications above have addressed your concerns. We would be grateful if you could kindly reconsider the evaluation of our paper.