TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees

References

[1] "Learning to rank: from pairwise approach to listwise approach." ICML (2007).

[2] "Learning to rank using gradient descent." ICML (2005).

[3] "Listwise approach to learning to rank: theory and algorithm." ICML (2008).

[4] "The lambdaloss framework for ranking metric optimization." CIKM (2018).

[5] "On optimizing top-k metrics for neural ranking models." SIGIR (2022).

We have added the additional case study analysis and the comparison of different ranking losses to the revised manuscript. To help you quickly locate our modifications, we have highlighted the changes using stone blue and bright red fonts. (Please refer to the Guideline for Reviewers on the last page of the revised manuscript.)

We want to express our sincere gratitude for your review. If you have any further questions or concerns, please feel free to contact us at any time. We are always available and look forward to further discussions with you.

Best regards,

All Authors

Response to W2 and W3 about the analysis of Adaptive Step Reward. Thank you for your suggestions. We are pleased to discuss our further experiments and insights on the Adaptive Step Reward with you.

• T-test on Adaptive Step Reward. To assess the statistical significance of our proposed Adaptive Step Reward, we performed 20 independent experiments for both TPO and TPO w/o Adaptive Step Reward under identical experimental conditions. The performance metrics from these experiments were then analyzed using an independent t-test to compare the means and the standard error of the two groups. The experimental results are shown in the table below, where $\^{*}$ denotes $p<0.01$, and $\^{+}$ denotes $p<0.05$.

The null hypothesis assumed no significant difference in performance between TPO and TPO w/o Adaptive Step Reward. A p-value less than 0.01/0.05 was considered statistically significant, indicating that the observed differences were unlikely to occur by chance. The results showed that TPO consistently outperformed the TPO w/o Adaptive Step Reward, with statistically significant improvements in pass@1 accuracy, validating the effectiveness of Adaptive Step Reward.

• Case study on Adaptive Step Reward. We further clarify our inspiration for Adaptive Step Reward through a case study. The following case is provided:

**Prompt:**
Let $A=2+i$, $B=3-2i$, $O=-4$, $P=-i$, $Q=5+3i$, $S=5-3i$, and $T=-3-6i$. Calculate $(A+B-O+P+Q*S)+T$.
Let's think step by step. 
Step 1: 
First, we need to find the sum of all the complex numbers in the problem. 
We are given $A=2+i$, $B=3-2i$, $O=-4$, $P=-i$, $Q=5+3i$, $S=5-3i$, and $T=-3-6i$.
We need to find $(A+B-O+P+Q*S)+T$.
Step 2:

**Response_1**
Let's add all the complex numbers together.
First, add $A$ and $B$:
$A+B = (2+i) + (3-2i) = 2+3+i-2i = 5-i$
Then, subtract $O$:
$5-i - (-4) = 5-i+4 = 9-i$
Next, add $P$:
$9-i + (-i) = 9-2i$
Then, calculate $Q*S$:
$(5+3i) * (5-3i) = 25-15i+15i-9 = 16$
Next, add $Q*S$:
$9-2i + 16 = 25-2i$
Finally, add $T$:
$25-2i + (-3-6i) = 25-2i-3-6i = 22-8i$
So, $(A+B-O+P+Q+S)+T = 22-8i$.

**Response_2**
First, let's add all the given complex numbers:
$A+B = (2+i) + (3-2i) = 5-i$,
$-O = -(-4) = 4$,
$P = -i$,
$Q*S = (5+3i) * (5-3i) = 25+15i-15i-9 = 16$,
$T = -3-6i$.
Now, let's add these results together:
$5 - i + 4 - i + 16 -3 -6i = 22 - 8i$.

**Response_3**
Let's add all the complex numbers together.
First, add $A$ and $B$:
$A+B = (2+i) + (3-2i) = 2+3+i-2i = 5-i$
Then, subtract $O$:
$5-i - (-4) = 5-i+4 = 9-i$
Next, add $P$:
$9-i + (-i) = 9-2i$
Then, add $Q$:
$9-2i + (5+3i) = 9-2i+5+3i = 14+i$
Next, multiply $S$:
$(14+i) * (5-3i) = 70+5i-42i-3 = 67-37i$
Finally, add $T$:
$67-37i + (-3-6i) = 67-37i-3-6i = 64-43i$
So, $(A+B-O+P+Q*S)+T = 64-43i$.

It is noteworthy that $Responses_1$ and $Responses_2$, despite having significantly different expressions, both follow the correct steps, whereas $Response_3$, though similar to $Response_1$ in expression, fails to prioritize multiplication and thus yields an incorrect result. In the ground truth ranking, the preference ranking is $Response_1 \succ Response_2 \succ Response_3$. However, during LLMs’ preference learning, since implicit reward are computed token by token, the significant overlap of tokens between $Response_3$ and $Response_1$ leads to an incorrect ranking of $Response_1 \succ Response_3 \succ Response_2$.

Based on this observation, we propose the Adaptive Step Reward mechanism leveraging step-level semantic similarity to adjust the reward margin between preferences. We emphasize the importance of semantics in preference ranking and believe this is particularly critical in tasks like mathematical reasoning, which emphasize semantic similarity rather than token overlap.

We sincerely invite you to review the analysis provided in Line 1026 of the revised manuscript, where we present additional case studies and demonstrate the success of the Adaptive Step Reward in these cases. These examples help to further elucidate the insights and effectiveness of our Adaptive Step Reward approach.

Thanks a lot for your acknowledgment of the writing, presentation, effectiveness and novelty of our submission!

Response to W1 about the reasons for choosing LambdaLoss.

Thank you for your valuable suggestion in emphasizing the advantages of LambdaLoss. As stated in Line 230 of the original manuscript, we chose LambdaLoss because it emphasizes the absolute position of preferences in the list, which helps further guide LLMs in aligning preferences.

• We have also introduced additional ranking losses, including Pointwise SoftMax [1], Pairwise Logistic [2], and Listwise MLE [3], to further confirm the validity of choosing LambdaLoss for TPO. The experimental results are shown in the table below:

We observe the following:

• Pointwise SoftMax demonstrates the poorest performance, indicating that learning only the reward values of preferences is insufficient. The relative relationships between preferences are particularly important.

• Pairwise Logistic and Listwise MLE perform worse than LambdaLoss. This is because both approaches only consider the relative relationships between preferences while neglecting their absolute positions in the ranked list. This limitation has been shown in existing Learn-to-Rank literature [4,5] to be detrimental to ranking optimization. It also underscores the motivation for introducing lambda weights in this work (see Line 230 of the original manuscript).

Response to W2 about the clarification on Equation 10.

• In Line 298 of the manuscript, the reason for $\mathcal{RM}=0$ is that $\beta\log\frac{\pi _ \theta(s^{i} _ k\mid x)}{\pi _ {\mathrm{ref}}(s^{i} _ k\mid x)}-\beta\log\frac{\pi _ \theta(s^{j} _ k\mid x)}{\pi _ {\mathrm{ref}}(s^{j} _ k\mid x)}=0$, not $1+\frac{emb(s^i _ k)\cdot emb(s^j _ k)}{\|emb(s^i _ k)\|\|emb(s^j _ k)\|}=0$. We have revised the manuscript to clarify this description.

Dear Reviewer Svzf,

We understand that you may be busy, and we sincerely appreciate your efforts in reviewing our paper. We have made significant efforts to address the concerns you raised during the initial review, but we have not yet received any feedback.

As the rebuttal phase is nearing its end, we would like to kindly follow up. We would greatly appreciate it if you could share your thoughts on our responses. If there are still any issues or areas that require further improvement, we are more than willing to continue refining our work.

Thank you again for your time and valuable feedback!

Thanks a lot for your acknowledgment of the writing, presentation and novelty of our submission!

Response to W1 about the data construction method. We acknowledge that the data construction method provided in the manuscript has certain limitations, which are discussed in detail in Line 517 of the original manuscript. Nevertheless, we still believe that our data construction method is reliable, for the following reasons:

• In the experimental setup of this work, our goal is to align open-source models with GPT-4, the current strongest long-chain reasoning models. The performance of GPT-4 in long-chain reasoning has been validated by our experiments (Tables 1 and 2 of manuscript). In this alignment process, we propose a more effective method, TPO.

• Existing works [1,2,3] have shown that using GPT to construct data and iteratively enhance open-source models is a common practice in the academic community, which validates the reliability of our approach.

Response to W3 about the fairness of the comparison between DPO and TPO.

Thank you very much for your valuable suggestions. We acknowledge that this was an oversight in our experimental setup, and we believe this is a very crucial experiment.

We introduce two new baselines, which extract paired data from the preference list and applies the DPO algorithm. Specifically, given data $[x, y_1, y_2, ..., y_n]$ with the preference ranking $y_1 \succ y_2 \succ, ... , \succ y_n$, we extract two types of data: $[x, (y_1, y_2), (y_1, y_3), ..., (y_1, y_n)]$ (denoted as $DPO^{*}$) and $[x, (y_1, y_2), (y_2, y_3), ..., (y_{n-1}, y_n)]$ (denoted as $DPO^{+}$). We conducted experiments on Qwen2-7B-Instruct and DeepSeekMath-7B-Instruct, and the experimental results are shown in the table below. The experimental results indicate that TPO outperforms all the baseline algorithm.

Our perspective is as follows:

• We argue that all variants of DPO focus solely on the relative likelihood between chosen and rejected preferences. However, this approach causes the likelihood of the chosen preferences to decrease during the optimization process [4,5]. In contrast, TPO introduces lambda weights into the ranking algorithm to provide absolute positional information for preferences within the list, mitigating data likelihood decline issues.

[1] "Visual Instruction Tuning." NeurIPS (2023).

[2] "GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text." WWW (2024).

[3] "Emovit: Revolutionizing emotion insights with visual instruction tuning." CVPR (2024).

[4] "Noise contrastive alignment of language models with explicit rewards." arXiv preprint arXiv:2402.05369 (2024).

[5] "Smaug: Fixing failure modes of preference optimisation with dpo-positive." arXiv preprint arXiv:2402.13228 (2024).

We have added the additional case study analysis and the extra baseline algorithms ($DPO^{*}$ and $DPO^{+}$) to the revised manuscript. To assist you in quickly locating our modifications, we have highlighted the changes using brown and stone blue fonts. (Please refer to the Guideline for Reviewers on the last page of the revised manuscript.)

We want to express our sincere gratitude for your review. If you have any further questions or concerns, please feel free to contact us at any time. We are always available and look forward to further discussions with you.

Best regards,

All Authors

Response to W2 about the discussion of Adaptive Step Reward. Thank you for your suggestions. We are pleased to discuss our further experiments and insights on the Adaptive Step Reward with you.

• T-test on Adaptive Step Reward. To assess the statistical significance of our proposed Adaptive Step Reward, we performed 20 independent experiments for both TPO and TPO w/o Adaptive Step Reward under identical experimental conditions. The performance metrics from these experiments were then analyzed using an independent t-test to compare the means and the standard error of the two groups. The experimental results are shown in the table below, where $\^{*}$ denotes $p<0.01$, and $\^{+}$ denotes $p<0.05$.

The null hypothesis assumed no significant difference in performance between TPO and TPO w/o Adaptive Step Reward. A p-value less than 0.01/0.05 was considered statistically significant, indicating that the observed differences were unlikely to occur by chance. The results showed that TPO consistently outperformed the TPO w/o Adaptive Step Reward, with statistically significant improvements in pass@1 accuracy, validating the effectiveness of Adaptive Step Reward.

• Case study on Adaptive Step Reward. We further clarify our inspiration for Adaptive Step Reward through a case study. The following case is provided:

**Prompt:**
Let $A=2+i$, $B=3-2i$, $O=-4$, $P=-i$, $Q=5+3i$, $S=5-3i$, and $T=-3-6i$. Calculate $(A+B-O+P+Q*S)+T$.
Let's think step by step. 
Step 1: 
First, we need to find the sum of all the complex numbers in the problem. 
We are given $A=2+i$, $B=3-2i$, $O=-4$, $P=-i$, $Q=5+3i$, $S=5-3i$, and $T=-3-6i$.
We need to find $(A+B-O+P+Q*S)+T$.
Step 2:

**Response_1**
Let's add all the complex numbers together.
First, add $A$ and $B$:
$A+B = (2+i) + (3-2i) = 2+3+i-2i = 5-i$
Then, subtract $O$:
$5-i - (-4) = 5-i+4 = 9-i$
Next, add $P$:
$9-i + (-i) = 9-2i$
Then, calculate $Q*S$:
$(5+3i) * (5-3i) = 25-15i+15i-9 = 16$
Next, add $Q*S$:
$9-2i + 16 = 25-2i$
Finally, add $T$:
$25-2i + (-3-6i) = 25-2i-3-6i = 22-8i$
So, $(A+B-O+P+Q+S)+T = 22-8i$.

**Response_2**
First, let's add all the given complex numbers:
$A+B = (2+i) + (3-2i) = 5-i$,
$-O = -(-4) = 4$,
$P = -i$,
$Q*S = (5+3i) * (5-3i) = 25+15i-15i-9 = 16$,
$T = -3-6i$.
Now, let's add these results together:
$5 - i + 4 - i + 16 -3 -6i = 22 - 8i$.

**Response_3**
Let's add all the complex numbers together.
First, add $A$ and $B$:
$A+B = (2+i) + (3-2i) = 2+3+i-2i = 5-i$
Then, subtract $O$:
$5-i - (-4) = 5-i+4 = 9-i$
Next, add $P$:
$9-i + (-i) = 9-2i$
Then, add $Q$:
$9-2i + (5+3i) = 9-2i+5+3i = 14+i$
Next, multiply $S$:
$(14+i) * (5-3i) = 70+5i-42i-3 = 67-37i$
Finally, add $T$:
$67-37i + (-3-6i) = 67-37i-3-6i = 64-43i$
So, $(A+B-O+P+Q*S)+T = 64-43i$.

It is noteworthy that $Responses_1$ and $Responses_2$, despite having significantly different expressions, both follow the correct steps, whereas $Response_3$, though similar to $Response_1$ in expression, fails to prioritize multiplication and thus yields an incorrect result. In the ground truth ranking, the preference ranking is $Response_1 \succ Response_2 \succ Response_3$. However, during LLMs’ preference learning, since implicit reward are computed token by token, the significant overlap of tokens between $Response_3$ and $Response_1$ leads to an incorrect ranking of $Response_1 \succ Response_3 \succ Response_2$.

Based on this observation, we propose the Adaptive Step Reward mechanism leveraging step-level semantic similarity to adjust the reward margin between preferences. We emphasize the importance of semantics in preference ranking and believe this is particularly critical in tasks like mathematical reasoning, which emphasize semantic similarity rather than token overlap.

We sincerely invite you to review the analysis provided in Line 1026 of the revised manuscript, where we present additional case studies and demonstrate the success of the Adaptive Step Reward in these cases. These examples help to further elucidate the insights and effectiveness of our Adaptive Step Reward approach.

Dear Reviewer UFQG,

We understand that you may be busy, and we sincerely appreciate your efforts in reviewing our paper. We have made significant efforts to address the concerns you raised during the initial review, but we have not yet received any feedback.

As the rebuttal phase is nearing its end, we would like to kindly follow up. We would greatly appreciate it if you could share your thoughts on our responses. If there are still any issues or areas that require further improvement, we are more than willing to continue refining our work.

Thank you again for your time and valuable feedback!

Thanks a lot for your acknowledgment of the motivation, presentation and novelty of our submission!

Response to W1 and Q3 about the related literature on preference list ranking.

• We present the optimization objectives of TPO and related literature [1,2,3] you provided as follows:

S-DPO [1]:

$$-\log\sigma\left(-\log\sum_{y_{l}\in\mathcal{Y}_{l}}\exp\left(\beta\log\frac{\pi _ {\theta}(y _ {w}|x)}{\pi _ {\mathrm{ref}}(y _ {w}|x)}-\beta\log\frac{\pi _ {\theta}(y _ {l}|x)}{\pi _ {\mathrm{ref}}(y _ {l}|x)}\right)\right)$$

RoseDPO [2]:

$$-(1-\epsilon_{\phi})\left(\beta\log\frac{\pi_{\theta}(y_{w}|x)}{\pi_{\mathrm{ref}}(y_{\mathrm{w}}|x)}-\beta\log\frac{\pi_{\theta}(y_{l}|x)}{\pi_{\mathrm{ref}}(y_{l}|x)}\right) -\epsilon_{\phi}\left(\beta\log\frac{\pi_{\theta}(y_{l}|x)}{\pi_{\mathrm{ref}}(y_{l}|x)}-\beta\log\frac{\pi_{\theta}(y_{\mathrm{w}}|x)}{\pi_{\mathrm{ref}}(y_{\mathrm{w}}|x)}\right)$$

Where $\epsilon_{\phi}$ represents the uncertainty assessment of preferences.

ODPO [3]:

$$-\log\prod_{i=1}^{N}\frac{\exp(\beta\operatorname{log}\frac{\pi_{\theta}(y_{i}|x)}{\pi_{\mathrm{ref}}(y_{i}|x)})}{\sum_{j=i}^{N}\exp(\beta\operatorname{log}\frac{\pi_{\theta}(y_{j}|x)}{\pi_{\mathrm{ref}}(y_{j}|x)})}$$

TPO (Ours):

$$-\lambda_{i,j}\sum_{v_{i}>v_{j}}\log\sigma(\beta\operatorname{log}\frac{\pi_{\theta}(y_{i}|x)}{\pi_{\mathrm{ref}}(y_{i}|x)}-\beta\operatorname{log}\frac{\pi_{\theta}(y_{j}|x)}{\pi_{\mathrm{ref}}(y_{j}|x)})$$

TPO vs. S-DPO: S-DPO [1] employs SoftMax to maximize the reward margin between chosen preference and all rejected preferences. However, S-DPO does not account for the contrasts between rejected preferences, nor does it consider the absolute positional information of preferences within the list.

TPO vs. Rose-DPO: On the one hand, Rose-DPO [2] does not consider the ranking of the preference list but instead incorporates the uncertainty of preferences. On the other hand, similar to TPO, both Rose-DPO and TPO use additional weights to adjust the reward margin. However, while Rose-DPO is based on uncertainty, TPO adjusts the margin based on the semantic similarity between steps.

TPO vs. ODPO: ODPO [3] utilizes Maximum Likelihood Estimation (MLE) to optimize list-wise preferences. However, existing Learn-to-Rank literature [4,5] suggests that list-MLE is not an ideal ranking optimization objective, as it enforces strict list ordering without considering the actual label values.

• It is noteworthy that references [2,3] were posted on arXiv on October 16, 2024, and October 22, 2024, respectively. The submission deadline for ICLR 2025 was October 1, 2024.

Response to W2 and W3 about more comprehensive comparison. We provide more extensive experimental results, as shown in the table below.

• More general LLMs. Following your suggestion, we further introduced Meta-Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3 as general-purpose LLM backbones. Experimental results demonstrate that TPO consistently outperforms the DPO algorithm, similar to the results observed on Qwen2-7B-Instruct.

• Baseline of supervised fine-tuning (SFT). We introduced SFT as a new baseline algorithm. The experimental results indicate that the performance of the SFT is, in most cases, even inferior to the DPO algorithm. This is because SFT only provides positive feedback to LLMs without suppressing incorrect reasoning paths. This limitation has been demonstrated in related work and is also illustrated in Line 40 of our original manuscript.

Response to W4, W5 and Q4 about results on general benchmark. We introduced two new evaluation datasets, BBH [6] and MMLU [7], to assess the model's performance on commonsense knowledge. Notably, BBH requires multi-step reasoning, whereas MMLU does not. The experimental results are shown in the table below.

• The experimental results indicate that TPO achieves optimal performance on the BBH task, which requires multi-step reasoning, outperforming all baseline algorithms. However, on the MMLU dataset, which does not require multi-step reasoning, the performance of all algorithms is nearly identical.

• Our perspective is as follows: during downstream fine-tuning (SFT, DPO, TPO), LLMs acquire two types of knowledge, including mathematical deduction and long-chain reasoning. The long-chain reasoning knowledge enables the LLMs to generalize its capabilities beyond the mathematical domain, leading to performance improvements in Coding and Reasoning tasks. For the MMLU dataset, however, since it does not require long-chain reasoning and TPO fine-tuning does not introduce new commonsense knowledge, the performance of LLMs fine-tuned with TPO does not improve further. Notably, TPO does not lead to performance degradation of LLMs on MMLU tasks, indicating that commonsense knowledge is preserved during the TPO fine-tuning process.

Response to Q1 about the link between DPO/TPO and CoT reasoning.

• The motivation for applying DPO to chain-of-thought reasoning is that we aim to fine-tune LLMs using downstream task data. However, conventional supervised fine-tuning (SFT) only provides positive feedback and does not suppress erroneous outputs from LLMs, which is particularly critical in long-chain reasoning. An error at any step can lead to incorrect results for all subsequent steps. In contrast to DPO, TPO provides preference list ranking optimization, which can accommodate a broader range of preferences. By leveraging the preference relationships provided in the ranking, TPO aligns LLMs more effectively.

• We introduce two new baselines, which extract paired data from the preference list and applies the DPO algorithm. Specifically, given data $[x, y_1, y_2, ..., y_n]$ with the preference ranking $y_1 \succ y_2 \succ, ... , \succ y_n$, we extract two types of data: $[x, (y_1, y_2), (y_1, y_3), ..., (y_1, y_n)]$ (denoted as $DPO^{*}$) and $[x, (y_1, y_2), (y_2, y_3), ..., (y_{n-1}, y_n)]$ (denoted as $DPO^{+}$). We conducted experiments on Qwen2-7B-Instruct and DeepSeekMath-7B-Instruct, and the experimental results are shown in the table below. The experimental results indicate that TPO outperforms all the baseline algorithm.

Our perspective is as follows:

• We argue that all variants of DPO focus solely on the relative likelihood between chosen and rejected preferences. However, this approach causes the likelihood of the chosen preferences to decrease during the optimization process [8,9]. In contrast, TPO introduces lambda weights into the ranking algorithm to provide absolute positional information for preferences within the list, mitigating data likelihood decline issues.

Response to Q2 about the data construction method.

• We provide the detailed construction method for the training data in Line 311 of the original manuscript. Specifically, we use pure GPT-4 to construct the preference rankings without any manual annotations. It is worth noting that the performance of GPT-4, as verified by our experiments (Table 1 and Table 2 of manuscript), is the strongest for long-chain reasoning tasks. In addition, existing works [10,11,12] have shown that using GPT to construct data and iteratively enhance open-source models is a common practice in the academic community, which validates the reliability of our approach.

[1] "On Softmax Direct Preference Optimization for Recommendation." arXiv preprint arXiv:2406.09215 (2024).

[2] "RosePO: Aligning LLM-based Recommenders with Human Values." arXiv preprint arXiv:2410.12519 (2024).

[3] "Optimal Design for Reward Modeling in RLHF." arXiv preprint arXiv:2410.17055 (2024).

[4] "The lambdaloss framework for ranking metric optimization." CIKM (2018).

[5] "On optimizing top-k metrics for neural ranking models." SIGIR (2022).

[6] "Challenging big-bench tasks and whether chain-of-thought can solve them." ACL (2023).

[7] "Measuring coding challenge competence with apps." NeurIPS (2021).

[8] "Noise contrastive alignment of language models with explicit rewards." arXiv preprint arXiv:2402.05369 (2024).

[9] "Smaug: Fixing failure modes of preference optimisation with dpo-positive." arXiv preprint arXiv:2402.13228 (2024).

[10] "Visual Instruction Tuning." NeurIPS (2023).

[11] "GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text." WWW (2024).

[12] "Emovit: Revolutionizing emotion insights with visual instruction tuning." CVPR (2024).

We have incorporated the discussion of related works (S-DPO, Rose-DPO and ODPO), additional baseline algorithms (SFT, $DPO^{*}$ and $DPO^{+}$), more general backbone LLMs (Meta-Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3), and additional benchmarks (BBH and MMLU) into the revised manuscript. To facilitate quick identification of our modifications, we have highlighted the changes using orange and brown fonts. (Please refer to the Guideline for Reviewers on the last page of the revised manuscript.)

We want to express our sincere gratitude for your review. If you have any further questions or concerns, please feel free to contact us at any time. We are always available and look forward to further discussions with you.

Best regards,

All Authors

Dear Reviewer 2RYw,

We understand that you may be busy, and we sincerely appreciate your efforts in reviewing our paper. We have made significant efforts to address the concerns you raised during the initial review, but we have not yet received any feedback.

As the rebuttal phase is nearing its end, we would like to kindly follow up. We would greatly appreciate it if you could share your thoughts on our responses. If there are still any issues or areas that require further improvement, we are more than willing to continue refining our work.

Thank you again for your time and valuable feedback!