Selecting Influential Samples for Long Context Alignment via Homologous Models’ Guidance and Contextual Awareness Measurement

W2 and Q2:

W2: The paper does not discuss the computational resources required to implement the GATEAU framework. A selection method that is slow or computationally prohibitive is less feasible in practical scenarios, especially for extremely long instruction-following examples.

Q2: Would this approach remain feasible for datasets of larger scale, and what are the expected costs in terms of time and resources?

Response:

Thanks for your advice, we believe adding such details will help our paper be more comprehensive.

<1> Experimental Device: As shown in the Appendix A, experiments are conducted on 8xA800 80G GPUs (experiments are not limited to this type of GPU).

<2> GPU Execution Time: Based on the principle of making full use of GPUs (e.g., multi-processing and selecting large batch sizes, etc), we list the execution time in following table:

As shown in this table, we can find our method (HMG + CAM) introduces acceptable offline time overhead compared to the supervised fine-tuning stage and improves the overall performance of long-context LLMs. Perplexity Guidance applies a single LLM to compute the score, thus it achieves less execution time but worse performance in our experiments. Meanwhile, another strong baseline Cherry Selection introduces an additional training stage and computes the proposed Instruction-Following Difficulty (IFD) by applying the forward propagation twice on a single long SFT data, thus necessitating more execution time compared to our proposed HMG. Meanwhile, our CAM and HMG can process the data in parallel to further decrease the execution time, e.g., only 8 hours with 16xA800 80G GPUs. Overall, compared to other baselines, the experimental results of our proposed GATEAU (consists of HMG and CAM) demonstrate that the additional execution time is worthwhile.

<3> GPU Memory Burden: As our method is designed to score the long SFT data, and then select the influential samples used for the SFT stage, thus our method does not introduce additional memory burden during the supervised fine-tuning and inference stage of the long-context model $\theta_{B}$. For your concerns about HMG, we compute perplexity scores generated from two models $\theta_{A}$ and $\theta_{B}$ for a given SFT data in parallel, and use the computed perplexity scores (cached in JSON files) to get the homologous models’ perplexity score HMP as shown in Eq. (2). Thus HMG does not introduce additional GPU memory burden, only introducing acceptable additional execution time as shown in Execution Time Table. The GPU memory requirements of CAM tem from the calculation of the attention scores for lengthy inputs, as well as the perplexity score computation. This process is equivalent to performing two forward passes over the dataset without updating gradients, thus it does not add an extra GPU memory burden.

\\

W3:

W3: Although the paper includes ablation studies, it lacks an experiment testing the effectiveness of using a single model's perplexity (e.g., LLaMA-2-7B-base-64k) as the sole criterion for data selection. Adding this experiment would provide valuable insight into the authors' claim on page 4, line 174, that high perplexity alone does not adequately reflect response difficulty in long-context scenarios.

Response:

Thank you for your suggestion! Actually, we have already compared this important baseline, namely Perplexity Guidance in our paper, in our comprehensive experiments (e.g., Table 1, Table 2, Table 3, Table 4, and Table 8 in the Appendix). We find that while using a single model's perplexity can be used as weak but positive guidance to select SFT samples compared to using the full long SFT dataset, our proposed GATEAU achieved consistently better performance, showing that high perplexity alone does not adequately reflect response difficulty in long-context scenarios.

\\

Q4:

Q4: Is there some display problem in Figure 2?

Response:

Figure 2 illustrates the "Needle in the Haystack" test, which evaluates the model’s ability to utilize information from 10 different positions within long contexts of varying lengths (1k–60k). The results are displayed using different colors. The green color across Figure 2 indicates that the model with GATEAU successfully passes all settings, resulting in a uniform green display. This comparison highlights the effectiveness of our method.

\\

We hope this detailed explanation successfully addresses your concern. Your advice has significantly contributed to the quality of our paper.

Thanks for your valuable review and suggestions! It is encouraging to see you find our methodology well-motivated and effective.

We sincerely thank you for your time and constructive comments. Below, we provide detailed replies to your comments to resolve your concerns.

W1 and Q1:

W1: The proposed method may assign similar high scores to duplicate or highly similar samples, assuming they contribute independently to model improvement. However, repeated exposure to similar samples may not add incremental value and could even undermine the alignment process of LLMs.

Q1: How does the proposed framework handle potentially redundant or highly similar samples? Could repeated high-scoring samples lead to an overrepresentation of certain types of data, thus limiting the diversity and richness of long-context dependencies in the selected dataset?

Response:

This is an interesting question. In our long SFT data selection process, we have partially considered such sample redundancy. In particular, HMG and CAM separately measure the difficulty of generating corresponding responses and understanding long input contexts due to the long-range dependencies, thus the final score derived from two different perspectives inherently reduces the influence of redundant samples. As shown in Table 7 in the Appendix, our method achieves better overall performance and more balanced performance in 8 different tasks, showing the effectiveness and diversity of selected samples by GATEAU.

Meanwhile, to further explore whether our method inherently reduces the influence of redundant samples, we employ the cluster model as [1] to cluster all candidate instruction pairs into k clusters. Specifically, we employ the k-Means algorithm and a sentence transformers model which is used to map sentences to a 384-dimensional dense vector space. Subsequently, semantic features are PCA-reduced to retain 95% of dimensions. Finally, by setting the number of clusters as $k = \sqrt{n/2}$ for $n$ long SFT samples, all 10k long SFT samples are clustered into 70 clusters. Finally, all samples are sorted based on their scores according to Eq. (6), and the top $n_1$ samples are selected. Within each cluster, samples are sorted by score, and the top $n_2$ pairs are chosen. We set $n_2$ to 1, which is the same as [1]. Finally, we can get $n_1 + k * n_2$ (i.e., $4300 + 70 * 1$) samples and use these selected data to train the model, namely -w Diversity-preserved Selection. We report the results of GATEAU-LLaMA - 50% on LongBench-Chat and MT-Bench in two settings.

In this table, we find that using the Diversity-preserved Selection does not consistently improve the final performance, showing our proposed GATEAU has partially addressed the sample redundancy and implicitly ensured the diversity of selected long SFT data.

[1] Clustering and ranking: Diversity-preserved instruction selection through expert-aligned quality estimation. EMNLP 2024

Dear Reviewer 93rU,

We would like to thank you again for your detailed reviews. We have updated our draft and added replies to your concerns with our latest experimental results.

Since the rebuttal deadline is approaching soon. Given that your current score is 5, we would appreciate it if you could let us know if our responses have addressed your concerns satisfactorily. If your concerns have not been resolved, could you please let us know about it so that we have the opportunity to respond before the deadline?

We would be happy to have any follow-up discussions or address any additional concerns.

Thanks very much! Looking forward to your reply.

Best,

Authors

Dear Reviewer,

Since it is the last day of the discussion, we hope that you can take a look at our response. Thanks.

Best regards,

The Authors

Thanks for your valuable review and suggestions! It is encouraging to see you find our methodology well-motivated and interesting.

We sincerely thank you for your time and constructive comments. Below, we provide detailed replies to your comments to resolve your concerns.

W1:

W1: The study only tests on the LLaMA2 family, which may restrict the generalizability of findings. Additionally, HMG relies on perplexity differences between homologous models with different context windows, leaving unclear guidance on applying the technique to other models lacking such variants.

Response:

Thanks for your interest! Our HMG method indeed requires two homologous models with different context windows, thus limiting the range of models we can use to further conduct the experiments. However, in practical scenarios, training a powerful long-context LLM always involves homologous models with different context windows (though these models may not be open-sourced). This is because existing LLMs are often initially pre-trained on a large-scale corpus with smaller context windows due to device limitations, they then conduct continual pre-training to extend the window size. Therefore, our method still remains effective in real-world scenarios.

\\

W2:

W2: There is no analysis of the time efficiency of the data selection method. Since CAM requires estimating importance scores for each segment, which may be computationally expensive, time efficiency is a critical factor.

Response:

Thanks for your advice, we believe adding such details will help our paper be more comprehensive.

<1> Our Experimental Device: Firstly, we want to explain the experimental device. As shown in Appendix A, all experiments are conducted on 8xA800 80G GPUs (the experiments are not limited to this type of GPU).

<2> GPU Execution Time: Based on the principle of making full use of GPU devices (e.g., using a multi-processing strategy and choosing large batch sizes, etc, we list the execution time in the following table:

As shown in this table, we can find our method (HMG + CAM) introduces acceptable offline time overhead compared to the supervised fine-tuning stage and improves the overall performance of long-context LLMs. Perplexity Guidance applies a single LLM to compute the score, thus it achieves less execution time but worse performance in our experiments. Meanwhile, another strong baseline Cherry Selection introduces an additional training stage and computes the proposed Instruction-Following Difficulty (IFD) by applying the forward propagation twice on a single long SFT data, thus necessitating more execution time compared to our proposed HMG. Meanwhile, our CAM and HMG can process the data in parallel to further decrease the execution time, e.g., only 8 hours with 16xA800 80G GPUs. Overall, compared to other baselines, the experimental results of our proposed GATEAU (consists of HMG and CAM) demonstrate that the additional execution time is worthwhile.

\\

W3:

W3: The CAM module’s process for calculating attention weights is vague, particularly in obtaining the attention weight across tokens in response y to token tj. Including pseudo-code or further clarification would make the implementation more accessible.

Response:

Sorry for the misunderstanding, we first compute attention weights for each token in response $y$ to the token $t_j$ in the segment $s_i$. Then we average the attention weights from each token in response $y$ to get the score $Attn_{\theta}(t_{j}|y;\theta)$ in Eq. (4), then compute the $Attn_{\theta}(s_i)$ according to Eq. (4). Meanwhile, we harness the attention weights averaged across different decoder layers and attention heads to thoroughly model how the LLM utilizes the long input contexts according to [1].

[1] Found in the middle: Calibrating positional attention bias improves long context utilization. ACL Findings 2024

W4 and W6:

W4: The method of dividing long inputs into equal segments may overlook the fact that important information could span multiple segments. Additionally, segment importance can depend conditionally on other segments, an aspect not accounted for in the current approach.

W6: Key hyperparameters, such as the number of segments, remain unexplored, missing potential for optimization in specific contexts.

Response:

Our CAM aims to evaluate whether LLMs’ attention is appropriately focused on important segments within the long input contexts. We separately compute the designed important score for each segment to the given response and calculate the LLM’s attention weights on each segment. In this way, even if the important information could span multiple segments, our method still calculates scores for several different segments to get the final CAS. As shown in our experiments, our method achieves better performance in Multi-doc QA and long-context Summarization, showing that our method can effectively handle the scenario that important information could span multiple segments. Meanwhile, Therefore, we further explore the impact of the important hyperparameter segment length. Intuitively, an excessively large segment length tends to prevent the model from focusing on the fine-grained information within the segment, whereas an excessively small segment length can lead to more semantically incoherent segments. We report the results of GATEAU-LLaMA - 50% on LongBench-Chat in Real-world Settings.

As shown in this table, different segment lengths affect the model's performance; however, as long as a reasonable length value is chosen, the fluctuations in model performance are not significant. Meanwhile, the performance will always be improved over using the whole long SFT dataset (namely Full-100%) and only using the HMG method (namely -w/o CAM), showing the effectiveness of our proposed CAM.

\\

W5:

W5: The method averages attention across heads and layers without distinguishing them, potentially introducing noise, as different heads or layers may capture varied aspects of the input. Calibrating the attention aggregation could improve focus on relevant input sections.

Response:

This is an interesting point. Inspired by previous work[2], we harness the attention weights averaged across different decoder layers and attention heads to thoroughly model how the LLM utilizes long input contexts during response generation. In this way, as shown in our experiments, our method achieves consistently better performance. Meanwhile, choosing the optimal head and layer for long context alignment may require additional validation data and computational resources.

[2] Found in the middle: Calibrating positional attention bias improves long context utilization. ACL Findings 2024

\\

We trust that this additional information addresses your concerns, and we welcome any further inquiries or feedback you may have. We also hope that you can kindly increase your score if our response has helped address your concerns.

Dear Reviewer KGEd,

We would like to thank you again for your detailed reviews. We have updated our draft and added replies to your concerns with our latest experimental results.

Since the rebuttal deadline is approaching soon. Given that your current score is 5, we would appreciate it if you could let us know if our responses have addressed your concerns satisfactorily. If your concerns have not been resolved, could you please let us know about it so that we have the opportunity to respond before the deadline?

We would be happy to have any follow-up discussions or address any additional concerns.

Thanks very much! Looking forward to your reply.

Best,

Authors

Dear Reviewer,

Since it is the last day of the discussion, we hope that you can take a look at our response. Thanks.

Best regards,

The Authors

W2:

W2: LESS is an optimizer-based method that select a subset of instruction-following dataset by estimating data influence (selecting data points that minimizes the validation loss) and this can be as one of the baselines.

Response:

LESS is a wonderful and interesting work in SFT data selection. The reason why we do not choose it as the baseline is that LESS needs to extract a subset of the evaluation benchmark to construct a validation set and estimate data influence on this validation set. However, the evaluation benchmarks MT-Bench and LongBench-Chat only contain 80 and 50 test data points respectively. Therefore, extracting a validation set from MT-Bench and LongBench-Chat and using the remaining data as a new test set would severely compromise the final results. This contract the LESS to make a fair comparison with our method and other baselines. We have cited this wonderful work LESS in our related work section.

\\

Q1:

Q1: For perplexity guidance (line 377), which $\theta$ do you use ($\theta_{A}$ or $\theta_{B}$ -- I am assuming $\theta_{B}$ here)?

Response:

Yes, we use the long-context model $\theta_{B}$ for baseline Perplexity Guidance. We add a more detailed description in line 377 of our paper (highlighted in red in the uploaded revision pdf). Thanks for your advice.

\\

Q2:

Q2: What is the context length of MT-Bench? The paper mentions that MT-Bench is for short-context instruction following. Since the proposed method, GATEAU, is designed for long-context, do you have any hypothesis on why it also improves on the short-context instruction following tasks (Table 4)?

Response:

In MT-Bench, the length of the instruction does not exceed 300 words. We discuss this interesting phenomenon in line 448 of our paper, we conjecture that handling complex tasks (i.e., long input contexts) contributes to handling the easy ones (i.e., short input contexts).

\\

Q3:

Q3: For Table 5, current bold numbers are a bit misleading. Usually bold numbers indicate the highest numbers across some category but here it seems the proposed method is bold. Also, for 13B models, it seem the w/o HMG and w/o CAM settings are not reported. Is there a particular reason of not doing so (e.g., computational constraint)? The ablation study on 7B model does show the effectiveness.

Response:

Sorry for the misunderstanding due to our typos in the submission version. We actually want to use bold numbers to indicate the highest numbers across methods that use the same ratio of long SFT data. Now we have modified our paper to correctly show the experimental results. Meanwhile, as shown in line 477 in our paper, we want to explore whether our method GATEAU can fit in larger LLMs in Table 5. The 13B model (GATEAU-LLaMA-13B) shows consistent improvements on three benchmarks. This indicates that GATEAU scales effectively to larger-scale models. For your concerns, we further conducted the additional ablation study for 13B models in our revised paper (highlighted in red in the uploaded revision pdf). This indicates the effectiveness of GATEAU and using both two methods can further improve the overall performance as they separately measure the difficulty of generating corresponding responses and understanding long input contexts due to the long-range dependencies.

We sincerely appreciate your careful review and valuable feedback, which have significantly contributed to the improvement of our paper.

Thanks for your valuable review and constructive comments. Below, we provide detailed replies to your comments to resolve your concerns.

W1:

W1: The method proposed heavily depends on the perplexity that measures the similarity between the base model's answer and the desired answer. The model $\theta_{A}$ (e.g., LLaMA-2-7B-base-64k as mentioned in the paper) in the method that calculates the perplexity is a base model (model that only does completion). It does not make sense to measure the perplexity of a base model on an instruction-following dataset because the base model wasn't trained to follow instruction. In this case, the perplexity can be high and it does not accurately measures the difficulty of a document. The author also acknowledges this and in the first metric (HMG) uses a homologous short context model $\theta_{B}$ as a reference model and calculates the difference between PPL of $\theta_{A}$ and $\theta_{B}$. The author claims that this "mitigate the influence brought by lacking other capabilities" (e.g., instruction-following capability, long-context capability). However, there is no guarantee that lacking other capabilities contribute equally in increasing PPL of two models. Essentially the data where the models measure the PPL on is out-of-distribution data (both models were not trained on the instruction-following data and $\theta_{B}$, which is extended to a longer-context window using zero-shot method, was not trained on long-context data) and model's behavior is unpredictable. Plus, $\theta_{A}$ was obtained by continual pretraining on long-context data. While HMG assumes $\theta_{A}$ and $\theta_{B}$ are similar models, it also depend on what the continual pretraining dataset is (here the method implicitly assumes that the additional dataset is small). If the additional dataset is very large, $\theta_{A}$ and $\theta_{B}$ can perform very differently.

Response:

Thanks for your valuable questions. We want to clarify the following to make our paper sound more sound.

For your concerns about HMG and CAM:

<1> Is the perplexity score from the base model really so high that it cannot accurately measure the difficulty?

Intuitively, since the base model performs well on conditional generation tasks (e.g., continuation), it should also be able to generate accurate perplexity scores on the response of instruction-following data, even though the model might not be able to produce high-quality responses correctly, because these two capabilities are not the same. Therefore, we explore whether our long-text LLM $\theta_{B}$ would produce incorrect perplexity values. We calculate the average perplexity value generated by the long-text LLM $\theta_{B}$ for the entire long SFT dataset during the whole HMG process, which is 2.61. Therefore, there is no issue of the perplexity from the base model being too high to accurately measure the difficulty.

<2> Can the perplexity score generated from the base model be used as guidance to select influential samples?

The perplexity (PPL) of the responses computed with the base model is an intuitive metric, as it measures the difficulty of the data sample during the generation. In our experiments, we find simply using high perplexity (namely Perplexity Guidance in our paper) can also improve the performance compared with using the whole long SFT dataset, indicating that the effectiveness of the perplexity score from the base model in selecting long SFT samples. Previous work [1] also finds using the Instruction-Following Difficulty (a variant of perplexity score) computed with the base model also works in selecting SFT samples. According to these experiments, we believe that the perplexity generated from a base model can be used as positive guidance to select SFT samples. Therefore, the use of the perplexity score generated from the base model in our method makes sense when selecting long SFT data. Meanwhile, our method HMG is designed to minimize the influence of other factors (e.g., the limited instruction-following ability of a base model) and model the difficulty in modeling the long-range dependencies to construct the more effective guidance of long SFT data selection, and further improve overall performance. For CAM, utilizing perplexity scores to compute importance scores is also reasonable, and the experiments show improvement even when only using CAM.

[1] From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning. NAACL 2024

Dear Reviewer pvW2,

We would like to thank you again for your detailed reviews. We have updated our draft and added replies to your concerns with our latest experimental results.

Since the rebuttal deadline is approaching soon. Given that your current score is 5, we would appreciate it if you could let us know if our responses have addressed your concerns satisfactorily. If your concerns have not been resolved, could you please let us know about it so that we have the opportunity to respond before the deadline?

We would be happy to have any follow-up discussions or address any additional concerns.

Looking forward to your reply.

Best,

Authors

I thank the authors for providing the response. However, the main concern I have is on the rationale of choosing the base model as a reference model because. I agree that on instruction-following data this can be less of a problem, but in long-context data this can be more of an issue as the base model was not trained on long-context data and the perplexity value is less meaningful. I will keep the current score.

Dear Reviewer,

Since it is the last day of the discussion, we hope that you can take a look at our response. Thanks.

Best regards,

The Authors

Thanks for your valuable review and suggestions! It is encouraging to see you find our paper truly makes sense, and that the proposed method is simple, motivated, and effective.

We sincerely thank you for your time and constructive comments. Below, we provide detailed replies to your comments to resolve your concerns.

W1, W2 and W3:

W1: In Eq.(2), the explanations for the new notation \theta _{A} and $\theta _{B}$ should follow immediately after their appearance in Eq.(2).

W2: In Tables 1-4, the captions should be put at the top (instead of the bottom) of the table.

W3: In section 4.2 Impact of GATEAU, the paper first analyzes the experimental results in Tables 2 and 4 (line 413), and then analyzed the results in Tables 1 and 3 (line 429). It will be better to analyze the experimental results according to the order of Tables.

Response:

Thanks again for your detailed suggestions in our presentation. We have modified our paper in the uploaded revision pdf to make the organization clearer (highlighted in red in the uploaded revision pdf).

\\

W4 and Q1:

W4: The memory storage and running time of the proposed algorithm is missing, which in my opinion can help readers understand the proposed algorithms more comprehensively. For example, in Eq.(2), HMP model uses short context model $\theta _{A}$ and long context model $\theta _{B}$ to compute the perplexity distance, so does the proposed algorithm have more model parameters (e.g. of $\theta _{A}$ and $\theta _{B}$ ) than the existing algorithm (e.g. only has $\theta _{B}$)? Therefore, I would suggest to add a table to compare the memory burden as well as the execution time with the existing methods on long context alignment.

Q1: Could you give some comparisons of memory burden between the proposed algorithms and other exiting methods?

Response:

Thanks for your advice, we believe adding such details will help our paper be more comprehensive.

<1> Our Experimental Device: Firstly, we want to explain the experimental device. As shown in Appendix A, all experiments are conducted on 8xA800 80G GPUs (the experiments are not limited to this type of GPU).

<2> GPU Execution Time: Based on the principle of making full use of GPU devices (e.g., using a multi-processing strategy and choosing large batch sizes, etc, we list the execution time in the following table:

As shown in this table, we can find our method (HMG + CAM) introduces acceptable offline time overhead compared to the supervised fine-tuning stage and improves the overall performance of long-context LLMs. Perplexity Guidance applies a single LLM to compute the score, thus it achieves less execution time but worse performance in our experiments. Meanwhile, another strong baseline Cherry Selection introduces an additional training stage and computes the proposed Instruction-Following Difficulty (IFD) by applying the forward propagation twice on a single long SFT data, thus necessitating more execution time compared to our proposed HMG. Meanwhile, our CAM and HMG can process the data in parallel to further decrease the execution time, e.g., only 8 hours with 16xA800 80G GPUs. Overall, compared to other baselines, the experimental results of our proposed GATEAU (consists of HMG and CAM) demonstrate that the additional execution time is worthwhile.

<3> GPU Memory Burden: As our method is designed to score the long SFT data, and then select the influential samples used for the SFT stage, thus our method does not introduce additional memory burden during the supervised fine-tuning and inference stage of the long-context model $\theta_{B}$. For your concerns about HMG, we compute perplexity scores generated from two models $\theta_{A}$ and $\theta_{B}$ for a given SFT data in parallel, and use the computed perplexity scores (cached in JSON files) to get the homologous models’ perplexity score HMP as shown in Eq. (2). Thus HMG does not introduce additional GPU memory burden, only introducing acceptable additional execution time as shown in Execution Time Table. The GPU memory requirements of CAM tem from the calculation of the attention scores for lengthy inputs, as well as the perplexity score computation. This process is equivalent to performing two forward passes over the dataset without updating gradients, thus it does not add an extra GPU memory burden.

W5 and Q2:

W5: It is a surprise for me that in Table 1 and 3, the proposed algorithm with less dataset can achieve SOTA results in all settings. So, can you give some example (whether simulation or real-world dataset, or theoretical analysis) to explain in what kind of situation the proposed algorithm may fail to achieve SOTA results?

Q2: Could you give some examples to show the failure of the proposed algorithm? It is a little unbelievable that one algorithm can achieve the best performance on all datasets, especially with less training data than existing ones.

Response:

Thanks for your interest in the performance of our method. Previous work [1] suggests that data quality is more important than data quantity and shows that instruction tuning 1% selected high-quality SFT data can outperform the method employing the entire dataset. As mentioned in our paper, previous works attempt to scale up the available data volume by synthesizing long instruction-following samples. However, the absence of a clear strategy for ensuring data quality may lead to the inclusion of low-quality samples. Thus, it is predictable that better performance can be achieved by using fewer but high-quality long SFT data.

Regarding your question, one possible limitation of our method is that our method is designed to improve overall performance in instruction-following and long-context understanding tasks. However, it is not suitable for improving performance in a targeted capability **or task, e.g., only improving the performance of mathematical questions. As shown in Table 7 in the Appendix, our proposed method does not consistently improve the performance across all the different capabilities, e.g., our method achieves unsatisfactory performance in the role-playing task.

[1] One-Shot Learning as Instruction Data Prospector for Large Language Models. ACL 2024

\\

W6 and Q3:

W6: If I understand correctly, in the proposed algorithm, we need to choose some part of the whole dataset (e.g. 10% or 30%) to train the model. How does this paper choose this percentage? And in real-life applications, how do we choose the part (whether 10% or 30%) from the whole dataset to align LLM?

Q3: Could you give some insights into the setting of the utilization percentage of the whole dataset of the proposed algorithm?

Response:

Thanks for your interesting and practical question. The ratio of selected long SFT samples is an important hyperparameter in our method. According to the results of experiments, we find our method is robust in the different ratios of used long SFT samples, including 10%, 30%, and 50%. Regarding your question, based on our comprehensive experiments in real-world settings, it is advisable to select 30% of the total data for real-life applications. When computational resources are abundant, we also recommend selecting the optimal ratio by evaluating it in your own real-life scenarios.

\\

We hope this detailed explanation clarifies your concerns and underscores the significance of our contributions. If you have any further questions or require additional clarification, please feel free to reach out.

Dear Reviewer,

Since it is the last day of the discussion, we hope that you can take a look at our response. Thanks.

Best regards,

The Authors