Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing

We thank the reviewer for the feedback and for taking the time to review our work. We are glad that the reviewer finds our qualitative and quantitative results take an important step towards building a mechanistic view of knowledge editing.

---

---

Main comments

I'd recommend experimenting on at least two models on each size--perhaps containing a mixture of uni-directional models (like GPT) and bidirectional models (like BERT).

Thank you for this suggestion! While we were not able to run two models of each size due to time constraints, we extended our experiments beyond GPT-2 by replicating key results with the Mamba-S4 2.8B model. The results are now added to Appendix F.5.3. Both the quantitative and qualitative metrics point to a distortion in the representation geometry, i.e., representation shattering is seen in other architectures (appendix F.5.3), as well as other knowledge editing methods (appendix F.5.2). We also note that we are open to adding further experiments with more models in the final version of the paper.

---

The quantitative explanation part can take up more space in the writing. The definition for describing the representation shattering effect should be introduced earlier.

We appreciate this suggestion and note that we have now added a new section (3.4) on representation shattering that more prominently defines the central hypothesis being explored, and presents a quantitative measure of representation shattering. In particular, we define $R(D_*) = ||D_* - D_\varnothing|| / ||D_\varnothing||$ as the degree of distortion, where the norm is the Frobenius norm, $D_\varnothing$ is the pairwise distances between the representations of entities of the unedited model, and $D_*$ is the pairwise distances between representations of the edited model. We have added this metric to all our results throughout the paper.

---

Currently only Figure 5(c) involves quantitative description of representation shattering, and other parts are qualitative

As we remarked above, we have now added a quantitative metric to supplement the qualitative pictures in the manuscript. In particular, Figure 6 and Figure 7 now report the representation distortion measure $R(D_*)$. Appendix F.5.2 contains an extended version of Table 1 with $R(D_*)$ reported alongside the factual recall accuracy. Across all figures and tables, we find that the distortion measure is consistent with the qualitative picture and is negatively correlated with the factual recall accuracy. Thank you for this suggestion!

---

Could you elaborate on the synthetic data generation process? Do A/B/C refer to the entities and the numbers refer to the edges?... I struggled at understanding how the nodes are connected, and e.g., what I/III/ABC refer to.

Thank you for raising this point! We have added a few paragraphs (highlighted in blue) to section 3.2 to further clarify our data-generating process. We summarize the same below.

We define a knowledge graph with 2048 entities (denoted by 1-2048) over which we define 3 cyclic orders (order I, II and III). The cyclic orders arrange the entities into a cycle and each cyclic order is a random permutations of the entities. We create 8 relations for each cyclic order totaling to 24 relations. The 8 relations correspond to the 1-hop, 2-hop, 3-hop and 4-hop neighbors in the clockwise and anti-clockwise directions in the cycle. The relations are named after a combination of the cyclic order (I, II, III), the neighbor’s distance (between 1-4) and the neighbor’s direction (Clockwise, Anti-clockwise). For instance, the relation “I_C2” denotes the 2-hop neighbor in the clockwise direction, with respect to cyclic order I.” The fact (5, II_C3, 23) tells us that entity “23” is the clockwise 3-hop clockwise neighbor of entity “5”.

---

Subsection 3.3 can be made clearer.

Thank you for the suggestion! We have rewritten this section and improved its organization as well. We would be happy to make changes to the paper if you find any particular parts of subsection 3.3 are still unclear.

---

---

Summary

We thank the reviewer for their feedback that has helped us substantially improve the clarity of our paper! Specifically, we have now added a precise quantification of representation shattering throughout our results and reworked Section 3 to improve the clarity of our data-generation process. We have also added several new experiments that confirm the robustness of our claims across different editing methods, model architectures, and graph structures. We hope these updates help address the reviewer's concerns, and, if so, that they would consider raising their score to support our work's acceptance!

Dear Reviewer,

We thank you again for your detailed feedback on our work. Given the discussion period ends soon, we wanted to check in if our provided responses address your concerns, and see if there are any further questions that we can help address.

Thank you!

Dear reviewer,

This is a gentle reminder that the discussion period is ending soon. We have implemented all of your valuable feedback in our latest revision, and we would greatly appreciate it if you could let us know your thoughts on the improvements we made.

Below is a brief overview of how we incorporated your specific feedback.

---

---

• Quantitative representation shattering measures

  • Introduced a metric: $R(D_*) = ||D_* - D_{\varnothing}|| / ||D_{\varnothing}||$.
  • Consistently report this measure in new results (Figures 5–7, Table 2).
  • Detailed in Section 3.4.

• Different models and editing methods

  • Added experiments with a larger, non-GPT-like architecture model (Mamba-S4 2.8B, in Appendix F.5.3).
  • Added experiments with MEMIT, AlphaEdit, and PMET (Appendix F.5.1 and F.5.2) and Mamba-S4 (Appendix F.5.3).

• Improved writing, figures, and captions

  • Restructured / re-wrote Section 3 (Formalizing Knowledge Editing)
  • Fixed typos, simplified confusing figures, and added clear notation definitions

---

---

Summary

• In our latest revision, we:
  • Defined a quantitative metric of representation shattering and updated our results to consistently rely on the metric.
  • Verified the reproducibility of representation shattering across additional KE methods and models and showed that our findings hold across different approaches and model architectures.
  • Thoroughly revised our manuscript to clarify confusing points and improve readability.

We hope these improvements address your concerns and merit a higher score to support acceptance!

We thank the reviewer for taking the time to give us feedback. We are glad that the reviewer found our synthetic setup to be well-motivated, convincing, and our qualitative results to be helpful in highlighting the phenomenon of representation shattering. We respond to specific comments below.

---

---

Main comments

The expected answers (or the ground truth answers) after knowledge editing is not clearly stated in the paper. For example in Figure 3 Ring I, after the editing of 1.I_C2 =3, is 1.I_C2=2 still true … or does this edge simply disappear?

We refer the reviewer to Section 3, where we detail our data-generating process and clarify how the ground truth for edits are created (for both corrective and counterfactual edits). To summarize, since the knowledge graph is defined by us procedurally, the ground truth corresponding to edits are also procedurally ascertained. That is, given any arbitrary node, following the data-generating process’s rules will help one procedurally arrive at the correct answer for an edit.

To specifically address the example raised by the reviewer, we note the edge (i.e., 1.I_C2=3) simply disappears in this case, and the edge (relation) to 2 no longer exists after the knowledge edit.

---

Which one of the following is considered the ground truth for evaluation after the editing: 1.I_C3=4 or 1.I_C4=4 ? In the proposed synthetic dataset, it could still have a consistent logic for LLMs even after knowledge editing. The paper should make the true facts after the knowledge editing more clear.

This is a great question! To appreciate why this is tricky to define, consider the following example: let us make a counterfactual edit that changes Lionel Messi into a professional basketball player. Did Lionel Messi still play football (in the team Barcelona), or is that no longer true? The challenge of knowledge editing is compounded by the ambiguity in defining what the underlying ground truth should look like and there is no single correct definition.

In our work, the ground truth after a counterfactual edit will be I_C3=3, i.e., the other facts are not changed. While this is not always the right choice of ground truth for all naturalistic scenarios, we adopt this choice for our synthetic KGs. We have now emphasized this in the manuscript.

---

The prompt templates and codes are not released

We appreciate the reviewer’s comment and emphasize that we take reproducibility seriously. In the manuscript, we describe the prompt templates used for experiments involving real LLMs (i.e., Lines 470-472). As for code, we promise we have a fairly well-written codebase planned for release after the paper is accepted.

---

can authors show some examples of the LLM outputs in natural languages after knowledge editing

We appreciate the thoughtfulness of this question. Previous works provide representative examples of KE algorithms’ failures with realistic edits in natural language. In particular, we recommend the reviewer explore “Appendix G: Generation Examples” of the ROME manuscript [1]. However, for completeness we describe below an example of a failure from our counterfactual calendar month editing task with GPT-2.

Setup. In this example, the unedited model initially correctly outputs that one month from February is March. Then, even after applying the counterfactual edit “One months from January is March” (a distance of 1), the model can still correctly reason that one month from February is March. However, after applying the counterfactual edit “One months from January is June” (a distance of 4), the model loses the ability to reason that March follows February (instead outputting “May”).

## Unedited Model
[Pre-ROME]:   One months from February is March and one month from...

## Counterfactual Edit Distance = 1
Executing ROME algorithm for the update: [Let's do some calendar math. One months from January is] -> [ March]
[Post-ROME]:  Let's do some calendar math. One months from February is March. So we can...

## Counterfactual Edit Distance = 4
Executing ROME algorithm for the update: [Let's do some calendar math. One months from January is] -> [ June]
[Post-ROME]:  Let's do some calendar math. One months from February is May, and one month...

We hope these pointers and examples above are helpful! We would also be happy to include these examples in the appendix in the final version of the paper.

[1] Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 35, 2022a.

---

---

[Continued below...]

More knowledge editing baselines should be included. In this paper ROME is the mainly considered knowledge editing methods. There are other 'locate and edit' methods such as PMET[1].

Thank you for this suggestion. In addition to ROME, we have added new experiments with three other knowledge editing methods: namely MEMIT, AlphaEdit, and PMET. We feature these methods in Appendix F.5.1 and F.5.2. We find that the newer editing methods generally degrade the model’s accuracy to a lesser extent than ROME. However, corrective edits still degrade accuracy and counterfactual edits cause greater drops in accuracy for greater distances, i.e., representation shattering is seen across all these knowledge editing methods.

---

Why is “1 IC_4 3 IIIA_2 7 IIIA_3 2 IIC_2 6” in line 235-236 a plausible sequence in figure 3?

Thank you for pointing this out! There was a typo here, and the correct plausible sequence is 1 IC_4 4 III_A2 8 II_ A3 3 II_C2 7. The above is a valid sequence because each fact is valid according to Figure 3. In particular (1 IC_4 4) is valid since the clockwise 4-hop neighbor of entity 1 in order I is entity 4. Similarly, the (4 III_A2 8) is valid since the anti-clockwise 2-hop neighbor of entity 4 in cyclic order III is entity 8. We can similarly reason about all the other facts in the sequence.

---

Some typos need to be fixed…

Thank you for pointing them out. We have fixed the typos line 191 and line 291.

---

---

Summary

We thank the reviewer for their feedback that has helped us substantially improve the clarity of our paper. Specifically, we have reworked Section 3 improve the clarity of our data-generation process, and also edited several other parts of the paper to emphasize the experimental protocol (e.g., adding prompt templates); these changes are marked in blue in the updated manuscript. We hope our updates help address the reviewer's concerns, and, if so, that they would consider raising their score to support our work's acceptance!

Dear Reviewer,

We thank you again for your detailed feedback on our work. Given the discussion period ends soon, we wanted to check in if our provided responses address your concerns, and see if there are any further questions that we can help address.

Thank you!

Dear reviewer,

This is a gentle reminder that the discussion period is ending soon. We have implemented all of your valuable feedback in our latest revision, and we would greatly appreciate it if you could let us know your thoughts on the improvements we made.

Below is a brief overview of how we incorporated your specific feedback.

---

---

• Additional KE methods

  • Added experiments with MEMIT, AlphaEdit, and PMET (Appendix F.5.1 and F.5.2) and Mamba-S4 (Appendix F.5.3).
  • Findings: Newer methods degrade model accuracy less than ROME, but representation shattering persists across all methods.

• Improved writing and documentation of details

  • Described prompt templates and experimental details
  • Responded to questions regarding counterfactual editing
  • Provided an example of post-edit model failures
  • Clarified commitment to releasing our codebase publicly if the paper is accepted
  • Fixed typos and re-wrote text to improve readability

---

---

Summary

• In our latest revision, we:
  • Verified the reproducibility of representation shattering across additional KE methods and models and showed that our findings hold across different approaches and model architectures.
  • Clarified our commitment to publishing relevant code if accepted.
  • Thoroughly revised our manuscript to clarify confusing points and improve readability.

We hope these improvements address your concerns and merit a higher score to support acceptance!

We thank the reviewer for taking the time to review our work and are glad that the reviewer finds our representation shattering hypothesis to be new. We have addressed all the reviewer’s concerns below and hope to engage in a discussion.

---

---

Main comments

Is the assumption still valid…

We emphasize that we make no assumptions in our work. Our proposed hypothesis, representation shattering, is a concept that we put forth based on extensive empirical evidence in synthetic and naturalistic settings. In fact, during the course of revisions we have now added several new editing protocols (4 in total), new models (Mamba variants), and new graph structures: our results generalize across all the novel settings as well!

---

on different model architectures, including encoder-decoder models like T5, and Mamba?

It would certainly be interesting to test our hypothesis for other model architectures! To this end, we have now added experiments analyzing the effects of editing Mamba-S4 via ROME (or, specifically, the variant of that algorithm that was specifically designed for editing Mamba-S4 by Sharma et al. [1]). Results are provided in Appendix F.5.3, and we find (both qualitatively and quantitatively) the same effects as our experiments with GPT-2 models: edits at a larger distance yield more representation shattering!

[1] https://arxiv.org/abs/2404.03646

---

Is the assumption still valid on larger LLM models like Lamma, Gemma, Phi 3.5 etc?

Thank you for the suggestion. In our aforementioned experiment with Mamba-S4, we used the Mamba-2.8B variant to scale up beyond GPT-2. Our results show that representations of concepts follow similar geometries regardless of model size. Furthermore, we find that the extent of representation shattering increases with the edit distance as expected.

---

The result seems hard to reproduce.

We appreciate the reviewer’s comment and emphasize that we take reproducibility seriously. We promise we have a fairly well-written codebase planned for release after the paper is accepted.

---

---

Summary

We thank the reviewer for their feedback that has helped us improve the robustness of our claims. Specifically, we have added experiments with a new model architecture to address reviewer's comment. We note we have also added experiments with 3 more editing protocols and a different graph structure, which may interest the reviewer. We hope these updates help address the reviewer's concerns, and, if so, that they would consider raising their score to support our work's acceptance!

Dear Reviewer,

We thank you again for your detailed feedback on our work. Given the discussion period ends soon, we wanted to check in if our provided responses address your concerns, and see if there are any further questions that we can help address.

Thank you!

Dear reviewer,

This is a gentle reminder that the discussion period is ending soon. We have implemented all of your valuable feedback in our latest revision, and we would greatly appreciate it if you could let us know your thoughts on the improvements we made.

Below is a brief overview of how we incorporated your specific feedback.

---

---

• Different models and editing methods

  • Added experiments with a larger, non-GPT-like architecture model (Mamba-S4 2.8B, in Appendix F.5.3).
  • Added experiments with MEMIT, AlphaEdit, and PMET (Appendix F.5.1 and F.5.2) and Mamba-S4 (Appendix F.5.3).

• Clarified commitment to reproducibility

  • We will release our codebase publicly if the paper is accepted.

---

---

Summary

• In our latest revision, we:
  • Verified the reproducibility of representation shattering across additional KE methods and models and showed that our findings hold across different approaches and model architectures.
  • Clarified our commitment to publishing relevant code if accepted.

We hope these improvements address your concerns and merit a higher score to support acceptance!

We thank the reviewer for their feedback and for taking the time to review our work! We are glad that the reviewer felt our mechanistic study on knowledge editing offers a unique perspective from the lens of representation geometry. We have added additional experiments which hopefully convince the reviewer that representation shattering extends beyond cyclic structures and to many other knowledge editing methods.

---

---

Main comments

Why not investigate the geometry of transformer representations in real-world KGs when the relation subgraphs are trees, and examine how KEs affect such geometry?

Mathematically describing what knowledge graphs look like in the real world is difficult. However, inspired by your suggestion to explore real-world knowledge graphs consisting of trees, we have added preliminary experiments with a knowledge graph of countries and cities represented as a small tree using a real language model (GPT-2). The results align with our main findings with our synthetic KGs: KE methods distort language models' representations in order to insert new facts or alter old ones (i.e., representation shattering), and the extent of representation shattering increases with the distance between the old fact and the desired new fact on the manifold. Furthermore, we found that counterfactual edits essentially collapse the representations of the countries and cities. For a detailed exploration, please kindly refer to Appendix F.6.

---

The paper only studies one knowledge editing method, ROME. The findings would be significantly strengthened if multiple KE methods were analyzed

Thank you for this suggestion. In addition to ROME, we have added new experiments with three other knowledge editing methods: namely MEMIT, AlphaEdit, and PMET. We feature these methods in Appendix F.5.1 and F.5.2. We find that the newer editing methods generally degrade the model’s accuracy to a lesser extent than ROME. However, corrective edits still degrade accuracy and counterfactual edits cause greater drops in accuracy for greater distances, i.e., representation shattering is seen across all these knowledge editing methods.

---

The construction of the knowledge graphs, particularly the use of cycles, is not very intuitive and is challenging to understand

We have significantly improved the writing in Section 3, with the salient changes highlighted in blue. In particular, we have added a more detailed description of the setup and the reasoning behind using 3 cyclic orders. We have addressed specific questions below.

---

what the exact cyclic orders are for the edit, retain, and test relations...

The 3 orders are defined over the 2048 entities and are sampled at random. All three orders are different from each other. This detail is now included in Section 3.2.

---

why the authors chose this specific construction, particularly the necessity of the use of different cyclic orders...

Thank you for raising this point! We have significantly expanded the text to address this question (see paragraph starting at Line 199). To summarize, we edit a relation that is associated with the first order. We need a second cyclic ordering of the entities to define a set of facts that remain unchanged after an edit. These facts are used by knowledge editing algorithms like ROME. We additionally define a third cyclic order which corresponds to facts that were neither edited, nor used by the knowledge editing algorithm. These serve as a held out set of facts for evaluation.

---

it lacks discussion on potential ways to mitigate this problem. Including some hypotheses or suggestions for future work.

This is a great question! However, we emphasize that the goal of our paper was to establish a mechanistic hypothesis for why knowledge editing protocols impact model capabilities. As we show, preserving representational structures underlying a model’s knowledge is likely crucial to avoid negative consequences of knowledge editing: distortion of such structures impacts a model’s broader capabilities. While a concrete answer to the question of how to design a knowledge editing protocol that can accommodate these structures is out of scope of our work, we believe that such a protocol should at least perform well on our synthetic knowledge graph setup. Failing even this simple, albeit systematically defined setting, likely implies the protocol should not be readily trusted or applied at scale. Following your recommendation, we have added a short discussion of these recommendations to Section 5 (Conclusion).

---

[Continued below...]

Additionally, could the Isomap visualization technique still be used effectively in this scenario to visualize a tree-like structure rather than rings?

Yes! Isomap constructs the geometry of the manifold using a local neighborhood graph and is not designed specifically for cycles: it can also be used with other structures, including ones that are tree-like. To demonstrate this, we have now added new experiments in Appendix F.6 where we show isomaps continue to work well beyond ring structures.

---

For the logical inference task, could you elaborate on how the "hold out" process works?

We refer the reviewer to Appendix B, where we describe in detail the hold-out process. To summarize, for the logical inference task, any facts about an entity’s clockwise neighbors can be inferred if it is known that the said entity is an anti-clockwise neighbor of another entity. For example, even if “Bob is the advisee of Alice” is held out during training, the fact “Alice is the advisor of Bob” can be inferred, so long as both relation directions are observed for other pair of individuals in pre-training (i.e., “Carol is the advisor of David” and “David is the advisee of Carol”). As we elaborate in Appendix B, we only show both directions of neighboring relations with a probability of 1/3rd. The model must use this 1/3rd fraction of the relations to infer the remaining 2/3rds of the facts.

---

---

Suggested improvements to writing

The Figure 1 is somewhat confusing, particularly in panel (b)

Thank you for pointing this out. To further clarify the figure, we have added additional annotations and expanded the caption. In brief, we note the bar graph in panel (b) represents the probability of generating different entities. The bar graph below is a probability distribution over the tokens and highlights that the edit changes the answer for “the neighbor to the right of entity #123”. However, this edit also degrades the model’s accuracy on direct recall, logical inference, and compositional inference.

---

in Section 3.2 (e.g., Line 202 states that the KG is defined to have 3 relations, but Line 205 later refers to 24 relations)

Thank you for pointing out this typo. We have fixed this in the updated manuscript. To clarify, the knowledge graph has 3 cyclic orders which are used to define 24 relations.

---

Figure 3, intended to provide more intuition and details, is difficult to interpret due to unclear labeling, such as the meaning of different arrow colors and which edges were edited

We apologize if the labelling was unclear. We have added more details to the caption and improved the quality of the Figure to make it easier to interpret. Figure 3 (top row) depicts an edit that changes the red arrow to the green arrow. The distance of the edit is 1, which is defined with respect to cyclic order I.

---

The abbreviation "DGP" (Figure 1 caption, line 67) is undefined.

We apologize for this! DGP stands for data generating process; we have fixed it in the latest version of the manuscript.

---

An explanation of the naming convention and its significance (e.g., whether the number of "I"s corresponds to hops in the KG) would be helpful.

We have added another paragraph to the main text explaining the notation and more prominently featured the text in the caption of Figure 3. In particular, I, II, and III are used to represent the 3 different cyclic orders. The entities are named by the ID number (1-2048). The relations are named using a combination of the cyclic order (I, II, III), the neighbor's distance (1-4), and the neighbor's direction (Clockwise / Anti-clockwise). For instance, the relation “I_C2” is the 2-hop neighbor in the clockwise direction.

---

Typo On line 291, the phrase "it changes fact"

Thank you for pointing this out. We have fixed it in the newer version of the manuscript.

---

Unclear Sentence: On line 366, the caption of Figure 5 reads: "We plot the mean drop in ac...

We have removed the text “in the”, from the sentence. Thank you for pointing out this typo.

---

Are all edit, retain, and test relations used during the pre-training of the transformer, and the test relations are only held out specifically for the KE process

Yes, all 3 relations are seen during pre-training to ensure the model internalizes these relations (else, editing it would be impossible). To make this clearer, we have added more discussion in Lines 216-217 as well as Lines 237-242.

---

---

Summary

We thank the reviewer for their feedback that has helped us improve the clarity of our paper and the robustness of our claims. Specifically, we have added experiments with 3 new editing methods and a different graph structure, along with several writing edits to improve clarity. We hope these updates help address the reviewer's concerns, and, if so, that they would consider raising their score to support our work's acceptance!

Dear Reviewer,

We thank you again for your detailed feedback on our work. Given the discussion period ends soon, we wanted to check in if our provided responses address your concerns, and see if there are any further questions that we can help address.

Thank you!

Dear reviewer,

This is a gentle reminder that the discussion period is ending soon. We have implemented all of your valuable feedback in our latest revision, and we would greatly appreciate it if you could let us know your thoughts on the improvements we made.

Below is a brief overview of how we incorporated your specific feedback.

---

---

• Realistic graph structures (trees)

  • Conducted experiments with a small tree-structured KG (countries and cities).
  • Results support our findings: KE methods distort representations (representation shattering), with greater shattering at larger fact distances. See Appendix F.6.

• Additional KE methods

  • Added experiments with MEMIT, AlphaEdit, and PMET (Appendix F.5.1 and F.5.2) and Mamba-S4 (Appendix F.5.3).
  • Findings: Newer methods degrade model accuracy less than ROME, but representation shattering persists across all methods.

• Improved writing, figures, and captions

  • Clarified the graph construction and data generation processes
  • Restructured / re-wrote Section 3 (Formalizing Knowledge Editing)
  • Fixed typos, simplified confusing figures, and added term definitions
  • Added discussion of recommendations for KE methods (Section 5)

---

---

Summary

• In our latest revision, we:
  • Further verified the hypothesis by extending the synthetic setup to incorporate small, realistic, tree-shaped KGs.
  • Verified the reproducibility of representation shattering across additional KE methods and showed that our findings hold across different approaches.
  • Thoroughly revised our manuscript to clarify confusing points and improve readability.

We hope these improvements address your concerns and merit a higher score to support acceptance!

We thank the reviewer for their valuable feedback and for taking the time to review our work! We are glad the reviewer finds that our work has valuable insights for knowledge editing, the tasks to be innovative and the paper to be well-written and accessible. We have run new experiments using other knowledge editing methods and tree-structured graphs, which we hope sufficiently addresses the reviewer’s concerns.

---

---

Main comments

Extend the synthetic setup to incorporate small, realistic KGs to further validate the hypothesis.

Inspired by your suggestion to explore small, realistic knowledge graphs, we conducted preliminary experiments with a knowledge graph of countries and cities represented as a small tree using a real language model (GPT-2). The results align with our main findings arrived at using our synthetic KGs: KE methods distort language models' representations in order to insert new facts or alter old ones (i.e., representation shattering), and the extent of representation shattering increases with the distance between the old fact and the desired new fact on the manifold. For more details, see Appendix F.6.

---

Only one knowledge editing (KE) technique, ROME, is tested…

Thank you for this suggestion. In addition to ROME, we have added new experiments with three other knowledge editing methods: namely MEMIT, AlphaEdit, and PMET. We feature these methods in Appendix F.5.1 and F.5.2. We find that the newer editing methods generally degrade the model’s accuracy to a lesser extent than ROME. However, corrective edits still degrade accuracy and counterfactual edits cause greater drops in accuracy for greater distances, i.e., representation shattering is seen across all these knowledge editing methods.

---

Rely more consistently on quantitative measures

Thank you for this suggestion. We have added a new section (3.4) on representation shattering that presents a quantitative measure of representation shattering. In particular, we use $R(D_*) = ||D_* - D_{\varnothing}|| / ||D_{\varnothing}||$, where the norm is the Frobenius norm, $D_{\varnothing}$ is the pairwise distances between the representations of entities of the unedited model and $D_*$ is the pairwise distances between representations of the edited model. We now consistently report $R(D_*)$ in our experiments (Figures 5, 6, 7, Table 2).

---

---

Summary

We thank the reviewer for their feedback and suggestions which have helped improve the robustness of our claims. In particular, we have now added experiments with (i) 3 new editing methods, (ii) a new model architecture (Mamba), and (iii) a different graph structure, along with (iv) a thorough quantification of the degree of representation shattering throughout the paper. We hope these updates to the manuscript help address the reviewer's concerns, and, if so, that they would consider raising their score to support our work's acceptance!

Dear Reviewer,

We thank you again for your detailed feedback on our work. Given the discussion period ends soon, we wanted to check in if our provided responses address your concerns, and see if there are any further questions that we can help address.

Thank you!

Dear reviewer,

This is a gentle reminder that the discussion period is ending soon. We have implemented all of your valuable feedback in our latest revision, and we would greatly appreciate it if you could let us know your thoughts on the improvements we made.

Below is a brief overview of how we incorporated your specific feedback.

---

---

• Realistic graph structures (trees)

  • Conducted experiments with a small tree-structured KG (countries and cities).
  • Results support our findings: KE methods distort representations (representation shattering), with greater shattering at larger fact distances. See Appendix F.6.

• Additional KE methods

  • Added experiments with MEMIT, AlphaEdit, and PMET (Appendix F.5.1 and F.5.2) and Mamba-S4 (Appendix F.5.3).
  • Findings: Newer methods degrade model accuracy less than ROME, but representation shattering persists across all methods.

• Quantitative representation shattering measures

  • Introduced a metric: $R(D_*) = ||D_* - D_{\varnothing}|| / ||D_{\varnothing}||$.
  • Consistently report this measure in new results (Figures 5–7, Table 2).
  • Detailed in Section 3.4.

---

---

Summary

• In our latest revision, we:
  • Further verified the hypothesis by extending the synthetic setup to incorporate small, realistic, tree-shaped KGs.
  • Verified the reproducibility of representation shattering across additional KE methods and showed that our findings hold across different approaches.
  • Defined a quantitative metric of representation shattering and updated our results to consistently rely on the metric.

We hope these improvements address your concerns and merit a higher score to support acceptance!