We thank the reviewer for the detailed and helpful comments. We address the reviewer’s concerns below.

We kindly request the reviewers to consider whether our clarifications and added discussions adequately address their concerns, and if so, we would greatly appreciate reconsideration of the scores to better reflect the revised assessment.

We highlighted the changes and the added new content in our latest submission to make it easier for the reviewers to track them. We will remove the highlights at the end of the discussion session.

[W1] How can a small nudging model effectively nudge a large base models on tasks that requires emergent abilities?

It is true that many task capabilities come with scale. In fact, many of the tasks we used, such as MMLU and GSM8K, are used to show the emergent abilities [1], and nudging works well on these datasets. One of the main insights from our work is that we can largely disentangle the alignment-related formatting abilities simply at the token level from the general task capabilities (such as many of the emergence capabilities). As illustrated in Table 7, as well as in the examples in the appendix, the nudging tokens are mostly transition or formatting tokens such as “1.” or “Great”, and our scaling-up analysis (section 4.5) suggests that we don’t need a very large nudging model to effectively generate these formatting tokens.

It is indeed interesting that even when the small aligned model does not have strong task capabilities it can still generate nudging tokens that guide the more capable base model to a good answer. Take the last-letter-concat (LLC) dataset for an example of the emergent abilities: for both Llama-2 and Gemma-2 models, the small chat models can only solve around 5% of the questions, while the large chat models perform significantly better (Table 3). As shown in Table 3, even themselves not able to perform the task, the small chat models can still effectively nudge the large base models. One analogy to understand this is to consider how humans write code using CoPilot, where we usually don’t deeply understand how to use each library, but simply through writing comments and defining proper variables and function names we can guide the CoPilot model to generate the code we want.

[W2] Clarification of the setups in Section 2.2 (Line 191)

We are sorry for the confusion about the experiment setups. The goal of section 2.2 is to show that at alignment-related positions, the aligned models of different sizes tend to have similar token distributions. This is verified in the following way: given the same prefix and generating the next token for a large base, a large aligned, and a small aligned model, when the large base and large chat disagree about the next token, we measure how often do the small chat and large chat models agree. Similar to Section 2.1, we compare the token distributions of three models on the large aligned models’ answers. As an example, consider the prefix: “Question: What is the smallest breed of Dog? The smallest breed of dog is Chiwawa.”. The next token generated by the large base model can be “Question”, where the base model follows the format and tries to generate another question. On the other hand, both a small and large chat model generate “On average …” to elaborate on their answers. So the next token position is alignment-related since the large base and large chat disagree, but the small chat and large chat agree on what to generate next. We have elaborated the setup in the paper.

[W3] Limitation section

We thank the reviewer for the suggestions of potential limitations of nudging. We have expanded Section 7, describing the potential limitations and future directions of this approach, such as the inference latency of the API-based implementation and learning a nudging model to better model customized alignment rules.

[W4&W5] Application of nudging in practice.

We thank the reviewer for pointing this out. We have addressed this concern in the general response.

[W6] Ethics and reproducibility statement.

Our work does not involve topics mentioned in the ICLR 2025 Author Guide such as “studies that involve human subjects or practices to data set releases”. Still, we have added an ethics and reproducibility statement to our paper. We thank the reviewer for the suggestion.

[1] Wei J, Tay Y, Bommasani R, et al. Emergent abilities of large language models[J]. arXiv preprint arXiv:2206.07682, 2022.

Is nudging only for “alignment” related instruction fine tuning? Authors mention that nudging can improve performance on GSM8k. Does it mean reasoning can be improved only by adding tokens like “Think” or “Great” or other formatting tokens?

We thank the reviewer for the great and deep question! One way to understand that the aligned models have a much better zero-shot performance on reasoning tasks like GSM8K is that after alignment, these models learn to elaborate on their reasoning steps (CoT) before outputting the final answer. Nudging and (few-shot) CoT prompting help the base models similar to this. Similar to (few-shot) CoT prompting, nudging leverages specific answer formatting to elicit the model’s latent reasoning abilities, rather than directly enhancing those abilities. But different from prompting, nudging more dynamically guides the base model to adopt a step-by-step CoT-style reasoning process by injecting tokens such as “Step 1,” “First,” or “1.,” at proper positions during inference. Another key insight here from nudging is that carefully selecting certain answer tokens (e.g., tokens with high uncertainty) can significantly improve performance on reasoning tasks. Recent work has also explored this idea [1].

Does nudging work on say GSM8k even when prompting base model with CoT and the standard fewshot GSM8k evaluation template?

We want to emphasize that one of the main goals of RLHF/instruction tuning is to get away from the few-shot examples since people seldom provide few-shot demonstrations when query these models with questions in practice. As a result, we mainly focus on the zero-shot setting in this paper, and comparing with few-shot is not consistent with the motivations and main arguments in our paper. Still, to partially address the reviewer's concern, we follow the reviewer’s suggestion and test the few-shot performance of Gemma-2-27b on GSM8K. We use the 8-shot CoT prompt from [2]. The 8-shot cot (78.2) has a similar performance to zero-shot nudging (74.6). This suggests that nudging can unlock the base model’s abilities similar to in-context learning but without requiring demonstrations. We want to point out that coming up with a setting to comprehensively and fairly compare these two methods is non-trivial. Such a comparison usually leads to a whole new paper [3]. On the other hand, as both nudging and few-shot prompting work by formatting the model’s answer, how to effectively combine these two paradigms at inference time is a deep question and requires future exploration.

Finally, repeating the limitations question I asked originally. What about cases where instruction finetuning is done to instill capabilities like longer context, add more reasoning capabilities to base model, coding skills, etc.

We agree that extensive post-training can indeed instill new capabilities, such as longer context or more reasoning capabilities. We find instructions that are beyond the inherent abilities of the base model are usually also challenging for nudging. For example, consider the instruction from the just-eval-instruct dataset: "Write a sentence with all words starting with the letter 'Y' to praise me." We find this instruction to be challenging for nudging because the base models lack the capability to follow such a specific and challenging instruction (see more in our newly added discussion in Appendix E.1). Notably, none of the large aligned models can generate a correct answer for this instruction either. To conclude, we acknowledge that nudging, while simple and training-free, is not a comprehensive solution yet and is constrained by the underlying capabilities of the base model (we have added this discussion to the limitation section). Nevertheless, we believe it represents a step in the broader exploration of techniques to adjust model behavior at the token level during inference. We are excited to investigate ways to enhance nudging in the future to address such limitations.

[1] https://github.com/xjdr-alt/entropix/tree/main/entropix

[2] Wei J, Wang X, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[J]. Advances in neural information processing systems, 2022, 35: 24824-24837.

[3] Mosbach M, Pimentel T, Ravfogel S, et al. Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation[J]. arXiv preprint arXiv:2305.16938, 2023.

Nudging brought up an interesting insight that reasoning abilities can be elicited simply by adding a few tokens during decoding, and we have shown that nudging (zero-shot) can match the performance on reasoning tasks with manually and carefully constructed few-shot examples [1]. Despite this, we’d like to clarify that comparing with few-shot prompting on reasoning tasks is not the main focus of our work. As indicated by the title of our paper, our primary focus is on alignment, where the model needs to respond to user queries that are highly diverse in topics and language [2, 3]. In such scenarios, providing carefully crafted few-shot examples for every possible user query is impractical or even impossible. As a result, most recent works on alignment/RLHF/safety evaluate models in a zero-shot setting [3, 4, 5, 6, 7]. Following these works, we focus on the zero-shot evaluation setup.

In general, while few-shot learning has demonstrated powerful capabilities, zero-shot abilities are equally important [8, 9, 10, 11, 12]. Zero-shot abilities enable models to generalize to new tasks without relying on task-specific examples, which may not be available or practical in real-world applications, reducing potential computational overheads and avoiding issues like prompt variability [13, 14, 15].

Regarding your point about prompting the model with phrases like "Think step by step", we acknowledge that such prompts can enhance reasoning to some extent [8]. However, our method focuses on a more general approach to alignment that does not rely on specific prompt engineering, which may not generalize across all tasks and user inputs.

We hope we have clarified the motivation of our work, and are willing to discuss further if the reviewer has any more questions.

[1] Wei J, Wang X, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[J]. Advances in neural information processing systems, 2022, 35: 24824-24837.

[2] Zhao W, Ren X, Hessel J, et al. Wildchat: 1m chatGPT interaction logs in the wild[J]. arXiv preprint arXiv:2405.01470, 2024.

[3] Cui G, Yuan L, Ding N, et al. ULTRAFEEDBACK: Boosting Language Models with Scaled AI Feedback[C]//Forty-first International Conference on Machine Learning. 2024.

[4] Zheng L, Chiang W L, Sheng Y, et al. Judging llm-as-a-judge with mt-bench and chatbot arena[J]. Advances in Neural Information Processing Systems, 2023, 36: 46595-46623.

[5] Zhou C, Liu P, Xu P, et al. Lima: Less is more for alignment[J]. Advances in Neural Information Processing Systems, 2024, 36.

[6] Dubois Y, Galambosi B, Liang P, et al. Length-controlled alpacaeval: A simple way to debias automatic evaluators[J]. arXiv preprint arXiv:2404.04475, 2024.

[7] Ghosh S, Evuru C K R, Kumar S, et al. A Closer Look at the Limitations of Instruction Tuning[J]. arXiv preprint arXiv:2402.05119, 2024.

[8] Kojima T, Gu S S, Reid M, et al. Large language models are zero-shot reasoners[J]. Advances in neural information processing systems, 2022, 35: 22199-22213.

[9] Wei J, Bosma M, Zhao V Y, et al. Finetuned language models are zero-shot learners[J]. arXiv preprint arXiv:2109.01652, 2021. [10] Sanh V, Webson A, Raffel C, et al. Multitask prompted training enables zero-shot task generalization[J]. arXiv preprint arXiv:2110.08207, 2021.

[11] Chung H W, Hou L, Longpre S, et al. Scaling instruction-finetuned language models[J]. Journal of Machine Learning Research, 2024, 25(70): 1-53.

[12] Achiam J, Adler S, Agarwal S, et al. Gpt-4 technical report[J]. arXiv preprint arXiv:2303.08774, 2023.

[13] Zhao Z, Wallace E, Feng S, et al. Calibrate before use: Improving few-shot performance of language models[C]//International conference on machine learning. PMLR, 2021: 12697-12706.

[14] Liu J, Shen D, Zhang Y, et al. What Makes Good In-Context Examples for GPT-$3$?[J]. arXiv preprint arXiv:2101.06804, 2021.

[15] Lu Y, Bartolo M, Moore A, et al. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity[J]. arXiv preprint arXiv:2104.08786, 2021.

We thank the reviewer for highlighting the importance and potential impact of our work.

[Q1] The benefits of using nudging “words” instead of nudging “tokens”.

We observed that the performance drop on several datasets was significant (over 10% absolute for LLama-2 on GSM8K for example) in our early experiments, and we did not explore this more in-depth. The intuition is that the model thinks it's at token boundaries more often than it should, leading to deviations from the right answer. As an example, for LLama-2 models on GSM8K, the nudging model mostly starts the answer with “Sure”, and the base model completes the word with “ly”, ending up with “Surely”, which usually leads to worse answers (See Appendix 1.1 for more discussion). Also, we demonstrated in Section 4.4 how using nudging “words” enables completely different model families (with different tokenizations of words) to collaborate effectively.

[S1] Suggestions and typos.

We thank the reviewer for the suggestions. We have modified the paper accordingly.

We thank the reviewer for the positive and constructive feedback. We appreciate the reviewer for recognizing that our results are easy to follow and providing great insight. We address the reviewer’s concerns below.

We kindly request the reviewers to consider whether our clarifications and additional experiments adequately address their concerns, and if so, we would greatly appreciate reconsideration of the scores to better reflect the revised assessment.

We highlighted the changes and the added new content in our latest submission to make it easier for the reviewers to track them. We will remove the highlights at the end of the discussion session.

[W1&Q1] Analysis of computational efficiency

We thank the reviewer for bringing up the discussion of computational efficiency. We conduct additional experiments and provide a detailed analysis addressing the reviewer’s concerns, and we have added this discussion to the paper (Appendix C):

1. Computational efficiency comparison with the baselines. Table 2 only gives a rough comparison of the inference time of different methods. To make a more direct comparison to the baselines, we compare the wall clock running time of nudging and the two baselines: ensemble and proxy-tuning on 100 samples on GSM8K (with Gemma-2 models). As shown in the following table, nudging is nearly 10x faster than the ensemble and 18x faster than the proxy-tuning, both of which require calling the base model for every generated token. Although nudging discards some generated tokens, the wall clock running time results suggest that the number of API calls is the most important factor for computational efficiency, since for the later tokens in the answer, every API call needs to reprocess the full context. By making significantly fewer API calls to the base model, nudging achieves a much faster inference speed than the baselines.

   	Nudging	Ensemble	Proxy tuning
   wall clock running time	286s	3026s	5330s
   Acceleration	x1	x10.6	x18.6

2. Analysis of the discarded base token ratios. The ratio of the number of tokens generated by the base models that are discarded is indeed an important aspect of efficiency. In the following analysis, we focus on the discarded token ratio of the base model, as the nudging model is much smaller and has a minor effect on the inference speed. In the paper, we reported the nudging token ratio in the appendix (Fig. 11). Here we first show that it is strongly connected to the discarded base token ratio, which is defined as the number of discarded base model tokens due to the nudging model divided by the total number of tokens generated by the base model. Assuming in a nudging answer there are $N$ nudging tokens, $B$ base tokens, $T=N+B$ total tokens. The nudging token ratio is therefore defined as $R_N=N/T$. In each nudging round, the nudging model generates 1 nudging token and then the base model continues by generating $L$ completion tokens each time. As a result, there can be at most $L$ base tokens that are discarded in each round. So an upper bound of the discarded token ratio $R_D$ can be derived as $R_D \leq \frac{N * L} {B + N*L} = \frac{L}{R_N^{-1} + L - 1} := R’_D$ We calculate the nudging ratios $R_N$, actual discarded base token ratio $R_D$, and our derived upper bound $R'_D$ for 3 model families on the just-eval-instruct dataset. As shown in the following table, we find that this simple upper bound gives a fairly accurate estimate of $R_D$. The $R_D$ with $L=16$ in practice is usually around 50% to 80%, which can be further optimized for efficiency by choosing an $L$ more carefully or using an adaptive $L$. However, we note that for the API-based implementation, the inference time is dominated by the number of API calls. We leave the improvement of efficiency as future work.

   	Lama-2 ($\gamma=0.4$)	Lama-2 ($\gamma=0.3$)	Gemma ($\gamma=0.4$)	Gemma ($\gamma=0.3$)	OLMo ($\gamma=0.4$)	OLMo ($\gamma=0.3$)
   $R_N$	15.7	11.4	12.7	5.5	23.3	17.9
   $R'_D$	74.9	67.3	70.0	48.2	82.9	77.7
   $R_D$	73.3	62.4	69.2	47.5	82.0	76.0

[W2] Nudging underperforms the large aligned model on most datasets.

We want to point out that nudging is a simple and training-free inference-time method, and large aligned models are not one of our baselines to beat (see more discussion in the general response). Instead, we want to demonstrate that without explicit alignment training, nudging a large base model with a much smaller aligned model can achieve performance close to a large aligned model. The strong performance of nudging on math and symbolic datasets shows a particular type of task that nudging would potentially benefit from.

[W3&Q3] Improving the clarity of Section 4.4

We appreciate the review for pointing this out. It is correct that the results in Section 4.4 are conducted on smaller subsets (200 samples for each dataset). We apologize for the confusion and have already updated the full dataset results in the latest submission. In the following, we address the reviewer’s concerns individually:

• The motivation. The main goal of this section is to demonstrate that different from previous model ensembling methods, e.g., proxy-tuning, nudging allows to combine models from different families. This can be potentially useful in many scenarios. For example, when a powerful series of base models comes out. One can easily improve the nudging system’s capabilities simply by switching to a better base model. We aim to simulate such a use case using Table 6, where Gemma-2 is the “new” series of base models. Nudging Gemma-2-27b with “existing” small chat models like Llama-2-7b-chat can largely outperform the best “existing” aligned model Llama-2-70b-chat.
• Comparison with Gemma-2-2b-it nudging results, and Gemma-2-27b-it results. As discussed in the previous point, comparing with Gemma-2-2b-it nudging results or the Gemma-2-27b-it results is not the main point of the section. Importantly, despite being a smaller model, Gemma-2-2b-it largely outperforms OLMo-7b-it and LLama-2-7b-chat on most datasets, especially on math datasets. This may explain the observation that nudging with Gemma-2-2b-it significantly outperformed nudging with OLMo-7b-it and LLama-2-7b-chat on GSM8K. Still, on MMLU, nudging with Gemma-2-2b-it (66.8) performs similarly to nudging with OLMo-7b-it (64.4) and LLama-2-7b-chat (67.0). Considering the different pretraining and post-training data distributions and training settings for different model families, using nudging models from the same model families might lead to better performance in general. However, answering this question is beyond the scope of this paper, and we would love to understand this problem in future works.

[Q2] Nudging benefits certain tasks.

The fact that nudging benefits particularly math and symbolic reasoning tasks is intriguing. As discussed in Section 4.2, there are existing works [1] showing that instruction-tuned models can underperform their base versions in factual and reasoning tasks. As an example, we find on the Coin Flip dataset, that both the LLama-2 and Gemma-2 instruction-tuned models tend to answer that the coin has an equal chance of being head or tail up. The base models explore less of such a behavior. By disentangling alignment and pre-training, nudging can potentially mitigate some countereffects introduced by the alignment process on certain tasks. However, we don’t have the model training data to accurately attribute this phenomenon.

[Q4&Q6] Typos in section 4.3 and Algorithm 1.

We are sorry for the confusion and have fixed the typos in our latest submission.

[Q5] The interoperability of nudging with non-greedy sampling.

Our code supports different sampling methods for both base and nudging models. For reproducibility and simplicity, we use greedy sampling for all our experiments following previous work [2, 3].

[1] Wang Y, Ivison H, Dasigi P, et al. How far can camels go? exploring the state of instruction tuning on open resources[J]. Advances in Neural Information Processing Systems, 2023, 36: 74764-74786.

[2] Liu A, Han X, Wang Y, et al. Tuning language models by proxy[J]. arXiv preprint arXiv:2401.08565, 2024.

[3] Kojima T, Gu S S, Reid M, et al. Large language models are zero-shot reasoners[J]. Advances in neural information processing systems, 2022, 35: 22199-22213.

[W1&Q1] Analysis of computational efficiency (continued)

3. How is L=16 chosen? Having a large $L$ means fewer potential API calls but more discarded tokens, while a small $L$ makes more calls and reduces discarded tokens. A proper $L$ should balance these two, we choose a slightly larger number based on our analysis in Section 2: choosing a threshold that captures 10% - 15% tokens would capture most alignment-related tokens, we choose an $L$ that leads to a nudging token ratio $1/L$ that approximately matches this number and slightly higher to reduce the potential API calls. We didn’t optimize heavily on this.
4. Choosing L for caching-based implementation. A speculative decoding style implementation that caches the question, as well as the generated token, should largely improve the inference time of nudging, and we will release this with the final version. In this case, indeed we can generate only 1 token at a time to avoid discarding tokens. We thank the reviewer for the constructive feedback and will take this into consideration in our implementation while balancing other potential computational overheads.

We thank the reviewer for the helpful feedback. We are grateful that the reviewer recognizes our method as simple and novel. We address the reviewer’s questions below.

We kindly ask the reviewers to consider whether our responses address their concerns and, if so, to reconsider their scores accordingly.

[W1] The effectiveness of nudging would decrease if the base models themselves are strong

Using a strong base model would increase the performance of nudging, and we view it as an advantage of the modularity of nudging. The motivation for nudging is to disentangle the general and alignment-related capabilities of LLMs. We rely on a small nudging model for formatting the answer and the large base model for general capabilities like factual knowledge and math reasoning. This modularity of nudging allows for improving both sides separately. As Section 4.5 shows, using a stronger base model consistently improves nudging performance across model families and tasks. Also, we show in Section 4.4 that using a stronger base model (Llama-2-70b -> Gemma-2-27b), even if it is from a different model family, can significantly improve nudging performance.

[W2] The practical scenarios for using nudging

We thank the reviewer for bringing up the discussion of the potential applications of our method. We have addressed this concern in the general response.

[W3] The benchmarks selected are not very challenging

We thank the reviewer for the suggestion on evaluating nudging on a wider range of tasks. Nevertheless, in this work, we focus on understanding alignment and proposing an inference-time algorithm for aligning the base LLMs. The benchmarks we used in our work, like GSM8K, MMLU, and Arc-challenge, are widely used in assessing LLMs (both the base and instruction tuned). On these well-studied benchmarks, nudging still gives significant improvements over the large base and small aligned models, and sometimes surpasses the large aligned models’ performance without any training.

For the somewhat synthetic Coin Flip task, the results are also intriguing. We find that in a zero-shot setting, both Llama-2-chat models and Gemma-2-it models tend to answer the coin can either be head or tail up, which aligns with existing works [1] showing that instruction-tuned models can underperform their base versions in factual and reasoning tasks, while nudging addresses the problem (we also discussed the Coin Flip task in Section 4.2, the third paragraph).

[Q1] Does this method conflict with other methods that enhance performance during the inference stage, such as Chain of Thought and Speculative Decoding?

No, chain of thought (CoT) is a prompting method and we can use it in addition to nudging. In some of our experiments, we prompt the model to use CoT by asking it to answer the question by walking through the reasoning steps (see our prompt for reasoning tasks in Figure 10).

For speculative decoding, as we discussed in the discussion section (Section 7), we can implement nudging in a similar way that caches generated tokens during inference to largely reduce inference latency.

[Q2] Evaluation nudging on other tasks involving long contexts, such as essay writing and story generation.

We thank the reviewer for the suggestion, and we are also excited about investigating how nudging works in other tasks in future works. For essay writing and story generation specifically, the Just-Eval-Instruct dataset, which we used to evaluate nudging, already contains relevant questions, such as 'Write an essay discussing the importance of communication in a relationship.” For this example, we checked that the nudging answers are rated high scores of all 5 dimensions by GPT-4o, and nudging performs on par with the chat models on the whole dataset (Section 4.3).

[1] Wang Y, Ivison H, Dasigi P, et al. How far can camels go? exploring the state of instruction tuning on open resources[J]. Advances in Neural Information Processing Systems, 2023, 36: 74764-74786.

Thank you for your feedback so far. As the deadline for modifying the PDF has passed, we can no longer make updates to the submission. However, we are happy to continue discussing any remaining issues or concerns you may have.

We thank the reviewer for the positive and helpful comments! We address their questions below.

We highlighted the changes and the added new content in our latest submission to make it easier for the reviewers to track them. We will remove the highlights at the end of the discussion session.

[W1] The effect on Llama-3 models

We tested Llama 3 on the standard benchmarks. We found that, as in the other three model families, nudging shows a significant improvement over the large base model on most datasets. However, nudging Llama-3-70b with Llama-3-8b-instruct underperforms the small nudging model alone (Llama-3-8b-instruct) on many datasets. As shown in the Llama-3 report [1], the llama-3 model family has a specific post-training process aiming to strengthen various capabilities of the model (math, coding, reasoning, etc). As a result, the small instruction-tuned model has better task-relevant abilities than the base models, explaining why including the base model did not lead to further benefits. We have added the discussion and the results to the paper.

[W2] Adaptive threshold

We thank the reviewer for the suggestion. Using adaptive thresholds is a promising direction to improve the effectiveness and efficiency of nudging. We are excited to explore this in future works.

[W3] Extra latency to the inference process

We agree with the reviewer that extra inference latency is a potential concern for the API-based implementation of nudging. Fortunately, as discussed in Section 7, nudging can be implemented in a speculative decoding [2] manner: we cache the current output for both the base and the nudging model and use it for decoding future tokens instead of making separate API calls. Since the nudging models are usually much smaller than the base models, the inference speed of nudging would be similar to the base model alone. We will include an implementation with our code release.

[1] Dubey A, Jauhri A, Pandey A, et al. The llama 3 herd of models[J]. arXiv preprint arXiv:2407.21783, 2024.

[2] Leviathan Y, Kalman M, Matias Y. Fast inference from transformers via speculative decoding[C]//International Conference on Machine Learning. PMLR, 2023: 19274-19286.

Thank you for your feedback so far. As the deadline for modifying the PDF has passed, we can no longer make updates to the submission. However, we are happy to continue discussing any remaining issues or concerns you may have.

Since it is almost the end of the final day we were able to modify the PDF, we have removed the highlights that were initially included to indicate changes in our paper. The reviewers can still refer to our summary of modifications in the general response for the changes we made.