Model-diff: A Tool for Comparative Study of Language Models in the Input Space

Overall, the presentation is too unclear to understand the content of the paper. I recommend a thorough revision of the manuscript.

• The equations are significantly unclear. For example, the authors define $\mathcal{D}= NLL_{A, x}-NLL_{B, x}$, and $z$ represents $NLL$. Then, $D-(z_{A,x}-z_{B,x})$ should be $0$ for any $x$ in Equation (1) and (2)? Also, as the paper does not define $\lambda$, I cannot understand Equation (3) and the subsequent equations.

• The paper also has many grammatical flaws that are critical for understanding the proposed method. For example,

l155: Define A → B as the representative inputs XA from model MA are evaluated by model MB
l164: the larger ρA→B (D) means a larger number of inputs whose output differences are by D.

I am uncertain about the motivation behind this study. The authors suggest that Model-diff can be used to determine which model is better or to detect model plagiarism, both of which could be accomplished by comparing performance on a human-generated benchmark dataset rather than on text sampled from LMs. The authors manually annotated the quality of the sampled text to evaluate the LMs' performance, but this human annotation process is costly and its validity is unclear. Although the authors discuss the challenges of using benchmark datasets as follows, I believe that a more diverse and massive dataset could be constructed by gathering existing datasets, rather than by sampling and annotating text from LMs.

The challenge of using benchmark datasets in this case is twofold: (1) the tested perspectives are limited by the types of test sets, and (2) the variety of inputs from the same perspective is limited by the dataset size.

1. The explanation for methodology details is not very clear. For example, It is not very clear to me what the difference is between $\mathcal{D}$ and $(Z_{A,x} - Z_{B,x})$, as well as the meaning of $\rho_{A\rightarrow B}(\mathcal{D})$.

2. Although the paper demonstrates the correctness of their method using a toy example, it lacks effective quantitative metrics to measure the method’s effectiveness and to show that it outperforms other approaches.

3. The experiments are limited to a few autoregressive language models (GPT-2 and Llama) and generation tasks. Testing on more diverse models and tasks (e.g., classification, question-answering) would provide stronger evidence of Model-diff's effectiveness.

1. Lack deep survey of this field. The section on related work is superficial, lacking a systematic overview of the development of comparative approaches in language models. Some similar related work [1] [2] should be involved in related work section, but these should not be the only references. I recommend that the authors include additional related works discussing model comparison to provide a more comprehensive background.

[1] LMDIFF: A Visual Diff Tool to Compare Language Models

[2] MODELDIFF: A Framework for Comparing Learning Algorithms

2. The toy experimental results do not provide a definitive conclusion. While they demonstrate that the proposed sampling method approximates brute-force enumeration in the input space, they do not evaluate whether Model-diff is an effective metric for comparing models. Additional experiments could help clarify Model-diff’s effectiveness, for example: (a) comparing two models fine-tuned on different datasets from the same base model, (b) comparing two models trained on highly overlapping datasets, and (c) comparing a model with another that has been further trained on the same data. How does Model-diff perform under these scenarios?

3. The experimental section lacks comparison with existing methods or baselines, which undermines the credibility and feasibility of the proposed contributions. Adding comparisons with prior studies could strengthen the claims and demonstrate the practical value of the proposed method.

4. The paper’s writing style presents several challenges for the reader, especially Figure 1. Specifically, an overabundance of complex notations obscures key points, and there is no initial overview of the framework before discussing the methodology. Enhancing clarity and readability by reducing notational complexity and adding a high-level overview at the beginning of the paper is recommended.

The major drawback of the proposed methodology is its limited practical utility. Conceptually, it might be interesting to generate/sample representative examples and sift through the differences in the NLL values of two models, however, the paper does not convince me in terms of actual value or applications it affords. The Toy setup in Section 4.2, and the real examples in 4.3 and applications in 4.4 are too weak to be convincing evidences of the utility of this approach. For instance, page 7 and 8 discuss the differences in two models in terms of the quantities defined in the paper, but it is unclear why the differences are intrinsically interesting especially without making a clear connection towards some other desirable properties (e.g. performance, robustness, safety, etc.).

I felt that the motivation of the paper could be improved: it might help to clearly articulate the kinds of actionable insights that comparative analysis may offer and demonstrate positive evidence. It would also help to compare with baselines for each individual application (which the current paper misses). For instance there are already well-established approaches to compare which model is better, and whether a model was plagiarized.

Overall, the paper is not the easiest to read and there are several places where writing could be improved. As an example, Sections 2.2 and 4.4 could be made more clear by describing in detail the nature of human annotations, and how the annotations were collected in the first place.

Some writing/typographical suggestions:

• Line 95: differentvalues --> different values
• Line 146: higher NLL are human understandable --> higher NLL values are human understandable
• Line 258 could be rephrased as it is challenging to parse.