Dear Reviewer BKzM,

We want to express our gratitude for the thorough review and the constructive feedback on our paper.

Weaknesses:

1.Causality does not guarantee correctness; ATE constraints ensure causal consistency between OSR and the next state, while cross-entropy control path choose correctness.

2.We explained this in lines 191-192. For your convenience, we explain further: we evaluate the causal relationship between OSR and state transitions with the help of a similar idea in the CRE section, where we assess the causal relationship between tree unfolding hierarchy and distance between object blocks between states, where distance refers to the sum of the distances between the positions of each block of the object in the current state and the positions of each block of the object in the target state The distances are computed using Euclidean distances. This is what is meant by calculating the ATE between A and B. We clarify that the way ATE(A) is written may have caused misunderstandings, and we revise it to ATE(A, B).

3.DES is unnecessary within the advantage interval (e.g., below 6 steps). DES rounds up long problems into the advantage interval. CRE is an internal oversight mechanism for the model’s reasoning process, while DES compensates for the model's shortcomings in long process tasks. When task difficulty exceeds the model's inherent performance, DES ensures reasoning within the advantage interval.

Questions:

CRE:

1.We share your view and do not believe that only the transition from the correct OSR to the proper state is causal, while the other three incorrect scenarios are non-causal. However, we would like to clarify that the example indicated by the absolute drop in Fig. 2 ATE is causal. We are completing data observations based on multiple independent repetitive experiments with the same question asked repeatedly to observe the model's results.

2.It's not a contradiction. The idea is that for multiple independent repetitions of the experiment, the causal hallucinations in the model output are in the minority. Taking the hallucination scenario listed in Fig. 2 as an example, when the distribution of experimental results keeps changing (i.e., when hallucinations are still produced erratically), the absolute value of ATE is increasing, and when the distribution of experimental results tends to be stable (i.e., when the results tend to be stable and unchanged under different inference paths), the absolute value of ATE is decreasing. That is, after the cross-entropy selection path, the constraint by the decreasing absolute value of ATE makes the model state transition output results tend to be stable.

3.Not the three error scenarios, we clarify that the illustration figure may cause a misunderstanding; we want to express to reduce the occurrence of all causal error scenarios under a single path choice, where the causal error scenarios depend on the choice of path. The correctness of the categories has been proven by previous work that cross-entropy is effective enough, so the synergistic optimization of both, with simultaneous reduction, has the effect of converging to causally correct scenarios under the proper categories.

4.You can refer to our previous explanation.

5.You can refer to our previous explanation.

DES:

1.The cross-entropy determines the OSR, but the calculation of the distance needs to ensure that the next state is accurate, so the same logic as in the CRE section, what we need to ensure is that the entire distribution of observations is stable, i.e., that the phantom scenarios between the OSR and the output of the following state are eliminated as much as possible, and that the more complex the distribution is (and the more scenarios there are), the larger the ATE is.

2.For example, the unfolding direction is to move the red block from the left to the right, and the counterfactual is the observation when this is not done or when the move is to another color block. Since the training phase produces more hallucinatory output, these observations can help us somewhat with the counterfactual assessment.

3.We want the tree to unfold in a helpful direction for the solution, i.e., the head tree unfolds in the Goal State. In contrast, the tail tree unfolds in the Initial State, and we want the tree unfolding and the distance reduction to be positively correlated or even strongly causal. As shown in the second half of Fig. 1, we will compute the Euclidean distance between the current position and the goal position for each block's coordinates, which in turn makes all object blocks move towards the goal.

4.Each layer unfolded represents one step out of every 4 steps done. The model is a one-time output of 4-steps; only when determining the target state for the model to achieve in the first 4-steps will we first search; that is, the model generates several alternative 4-step states based on the Initial State and then selects the optimal solution from them. 4-steps can be done, and so can 2 or 3 steps, only that the 4-step efficiency will be higher. The time consumption is less under the condition of guaranteeing accuracy. If the modeling ability is further improved, excluding extended 6-step reasoning may be possible.

5.Determining states at steps 4 and 8, then reasoning from step 4 to 8. Experimental accuracy drop is due to the need to improve the success rate in this process.

Experiment

1.Training and test sets are mutually exclusive, randomly sampled from the original dataset. Default hyperparameters of the pre-trained model were used, with inputs and outputs detailed in the Appendix and discussed in the response to Reviewer yRix.

2.The shortest step, because there can be many ineffective repetitive steps, such as repeatedly moving the red block left and right. Determine the steps needed as given by the data set.

Thank you for your time and consideration.

Dear Reviewer BKzM,

We have submitted our rebuttal several days ago, and the discussion process is now more than halfway through. However, we have not yet received your response. We greatly value your insights and are eager to hear your further feedback on our paper. Your comments will be instrumental in helping us improve the manuscript.

Thank you for your time and consideration.

Sincerely, Authors

Thanks for the detailed responses. The submission's statement suggests that the goal is to reduce the causal influence of OSR on state transitions, thereby making the results stable and unchanged under different inference paths. However, several issues require further clarification:

1. The Concept of Causality. The concept of "causality" is repeatedly mentioned in the paper, like "ensure the causality between OSR and state transition in LLMs," including the causality measure between OSR and state transition," enhancing the causality of state transitions," etc. Combined with the feedback in the rebuttal, does it mean "distribution of experimental results tends to be stable" is equivalent to "enhancing the causality"? If these two concepts are equivalent, please provide relevant references to support this equivalence.

2. The relationship between "Enhancing causality" and "Lower |ATE|". ATE is an estimand for causal effect estimation. "Low |ATE|" typically indicates a non-significant causal effect, what is the relationship between "lower |ATE|" and "enhancing causality"? The statement in the rebuttal, "we would like to clarify that the example indicated by the absolute drop in Fig. 2 ATE is causal" also raises this issue. If there is additional clarification regarding this, please illustrate with equations and references.

3. The Benefits of Stability of Results. Setting aside the causal effect estimand ATE, in the rebuttal, does "the results tend to be stable and unchanged under different inference paths" mean that the correctness or incorrectness of OSR has a minimal causal impact on state transition outcomes? What are the advantages of this approach? Is the stability of results inherently beneficial? If so, please provide references to support this.

Detailed Explanation

When $\sigma^2$ (variance) is small, the treatment effects for all individuals are very close to the mean $\mu$. This suggests that the treatment effect is highly consistent and stable across different individuals:

Causal Certainty: Due to the high consistency of individual effects, the treatment’s impact can be considered specific rather than incidental or person-dependent. This consistency is usually regarded as a hallmark of a strong causal relationship.

Robustness of Causal Effects: Even if the average effect (ATE) is small, this consistency indicates that the treatment has a similar impact on all individuals, making the causal relationship robust and strong.

Conversely, when $\sigma^2$ is large, the treatment effects among different individuals may vary significantly, leading to a high level of inconsistency:

Causal Uncertainty: The treatment effect may manifest as a strong positive effect in some individuals and a negative effect in others. This high variability implies that the treatment's impact is uncertain, leading to a weaker causal relationship.

Inconsistency of Causal Effects: Even with a large average effect (ATE), the significant differences in individual responses make it difficult to assert that the treatment effect is consistent across all individuals. Consequently, such a causal relationship is generally considered inconsistent or weak.

$\sigma^2 = \frac{1}{N} \sum_{i=1}^{N} (\tau_i - \text{ATE})^2$

Practical Example and Application to Our Work

To better understand this extended perspective, let's consider a specific example:

Take the Blocksworld scenario as an example. At the beginning of training with a fixed OSR, various state transition results such as A, B, C, D, and E might indicate low consistency. By using methods like fine-tuning that rely on the model's cross-entropy loss, the model can be constrained to produce fewer state transition results, such as only A, B, and C. However, more is needed, as the ultimate goal of using ATE is to constrain the state transition results to only one outcome, specifically A.

Towards the end of a training process, the following two situations may occur: Assume we plot A, B, C, D, and E on the X-axis and the frequency of the corresponding output samples on the Y-axis and observe the distribution. Our objective is to make the distribution tend toward one with low variability.

Scenario A: High Consistency, Low ATE: In this case, the variability of the distribution is small, meaning fewer types of situations occur.

Scenario B: Low Consistency, High ATE: In this case, the variability of the distribution is large, meaning more types of situations occur.

From here, we begin to consider the issue of consistency. These two scenarios have commonalities with the model training process. The combination of OSR and state transitions is not unique for a single sample under 2-class- and 3-class samples. Therefore, we aim to achieve a one-to-one correspondence between OSR and state transitions. Note that a one-to-one correspondence does not necessarily mean the combination of OSR and state transitions is correct. However, during the training process of a large model, the model's cross-entropy loss function also plays a role. Just as with methods like RAP and CoT that freeze the model, the model's cross-entropy loss function alone can somewhat control consistency. Additionally, because our model is not frozen, the output consistency of the model improves with each epoch during training. Therefore, our training process is an optimization process from Scenario B to Scenario A. ATE does not work in isolation but cooperates with the cross-entropy loss function.

The dimensions of cross-entropy loss and ATE are different. In the early stages of model training, the changes in cross-entropy loss (or PPL) are dramatic, far exceeding the changes in ATE. As a result, ATE plays a minor role at this stage. Only in the later stages of training when the cross-entropy loss decreases to a certain extent, and its rate of change approaches that of ATE, ATE begins to play a role, assisting the model in further optimization.

The above discussion focuses on a single-sample perspective. However, to evaluate the overall model training process across epochs, we assess the model by its accuracy in answering validation set questions, not by ATE.

Conclusion

We believe that this reviewer's comments convey the same concerns that other reviewers may have, so we hope everyone can take a look.

In summary, the method presented in our paper involves fine-tuning during the training process, and the reviewer's perspective might have been overly broad.

Meanwhile, we have attached a relevant reference for your further understanding:

[1] Bao G, Zhang H, Yang L, et al. Llms with chain-of-thought are non-causal reasoners[J]. arXiv preprint arXiv:2402.16048, 2024.

Please review the 4 and 5 sections.

Thank you once again for your efforts and consideration.

Dear Reviewer BKzM,

Thank you very much for your thoughtful feedback and for taking the time to carefully review our responses. We are pleased to hear that our explanations have resolved your major concerns, and we sincerely appreciate your willingness to raise your rating accordingly.

We also value your insightful suggestions regarding the differentiation and elucidation of the concepts of ATE and variance for ITE, as well as the distinction between the significance and the value of ATE. We will certainly incorporate these points into our revised submission to further improve the clarity and impact of our work.

Once again, thank you for your constructive feedback and support. Your comments have been extremely helpful, and we are committed to making the necessary revisions to enhance our paper.

Best regards,

Authors

Dear Reviewer yRix,

We want to express our gratitude for the thorough review and the constructive feedback on our paper.

Weaknesses:

Major concerns:

1.(1)In terms of scalability, due to the faster reasoning, it is expected for tasks with high FPS rate requirements, such as unstructured autopilot tasks (turn left, turn right, etc.), and for actions with contextual causal logic reasoning relationships, such as gathering wood, digging up rocks, lighting torches, etc., as in the Minecraft example, it is expected in our current preliminary experiments to be realized. We initially plan to leverage the open-source Minecraft simulator from our previous work as a calibration tool for the long task decomposition steps we have planned, to assess the accuracy of our decomposition steps, and then to try to generalize our approach to open-world tasks, moving closer to the goal of embodied intelligence. We believe that simulation-based game simulations such as Minecraft have an expected better migration adaptation to the real world, but of course, there is more to it than that, and we also have a vision of developing simulators based on platforms such as Unreal to visualize decision-making process simulations.

1.(2)We recognize that this is the main limitation of our current work, partly because the middle gap portion formed after the DES bipartite search is insufficient for similar examples in the training dataset. The decrease in effectiveness is mainly due to the failure of the double-ended search (i.e., the inability to approach from both ends to the middle or the failure of inference in the middle gap part). We believe that further expansion of the dataset, such as additional generation of more training data that conforms to the rules, may be helpful for the overall performance improvement.

2.(1) Yes. Instruction Format: Our format differs from CoT's, designed per the pre-trained model publisher's manual and to differentiate input data parts accurately.

2.(2)Correlation and causality are different concepts; put, causality further enhances correlation. For example, if the sample regression equation is y = x, y and x are positively correlated. Still, it isn't very objective to say that there is strong causality between y and x because we cannot confirm whether a counterfactual sample that is not observed would still be able to be fitted to this equation.

2.(3) CRE increases training time compared to frozen models like RAP and CoT. However, RAP's complex Monte Carlo search tree and node inferences require more time. Our 7B model's inference speed is faster than 70B's. DES search expands smaller than Monte Carlo trees, as shown in our time statistics.

2.(4) We need to clarify that we did not perform hyperparameter optimization. The hyperparameters we used are the default options officially given by the open-source pre-trained models on the Huggingface website.

2.(5) The first paper on prompt optimization uses the GSM8K dataset but differs in research focus. The second paper on RAP is a benchmark for comparison, referenced in our work.

Minor concerns:

1.We acknowledge that our experiments were primarily conducted on 7B parameter models. The choice was made due to computational constraints and availability. The Mixtral-8x7B model is also larger than the 7B model so that it can be used as a reference for the performance of larger sizes. We conducted some additional experiments at the model size of 13B, but due to time constraints, only the following experimental results were obtained.

Blocksworld

GSM8K

Hanoi Tower

From the results, there is not much difference between the experimental results under 13B and 7B, and we believe that the difference can be regarded as a random error generated by different training. From the performance comparison between the 70B model and the 7B model under the RAP method, the performance of the 70B model will be relatively improved. However, considering inference speed, the 70B model is much slower than the 7B, and it needs to be loaded with a certain amount of quantization, and the performance loss is equally present.

2.Hanoi Tower tasks are harder than Blocksworld's due to stricter constraints. The cause may be the 7B model's limited inference capacity and room for method optimization to control strict order.

3.Blocksworld generally does not involve random steps..

4.In Blocksworld, only top squares can move; middle or bottom squares cannot. Similarly, in the Tower of Hanoi, size discrimination prohibits larger blocks on smaller ones. Such states are considered solving failures.

5.We will adjust these issues in the revised version.

We hope these clarifications address your concerns and improve the overall understanding of our work. We are committed to enhancing the paper based on your valuable feedback. Thank you for your time and consideration.

Dear Reviewer yRix,

We have submitted our rebuttal several days ago, and the discussion process is now more than halfway through. However, we have not yet received your response. We greatly value your insights and are eager to hear your further feedback on our paper. Your comments will be instrumental in helping us improve the manuscript.

Thank you for your time and consideration.

Sincerely, Authors

Dear Reviewer yRix,

Thank you for taking the time to review our paper and for your thoughtful comments throughout the process. We greatly appreciate your recognition of our detailed responses and your continued support of our work.

We respect your decision to maintain your score and are grateful that you consider our paper to be above the acceptance bar. Your feedback has been invaluable in refining our research, and we are hopeful that the contributions we have made will positively impact the field.

Thank you once again for your efforts and consideration.

Best regards,

Authors

Dear Reviewer MmUh,

We want to express our gratitude for the thorough review and the constructive feedback on our paper.

Weaknesses:

ATE (Average Treatment Effect) is a metric used in causal inference to measure the average impact of a treatment or intervention on an outcome across a population. Specifically, ATE represents the difference in the average outcome between individuals who receive the treatment and those who do not.

ATE is a reasonable metric capable of reflecting causality in causal inference. While real-world causal relationships are complex, and no single metric can capture all aspects perfectly, ATE is widely recognized as one of this domain's most reasonable and practical metrics. Although every metric has limitations, ATE has been validated and accepted by the research community for its robustness in estimating causal effects. We acknowledge the complexity of real-world causality and will consider incorporating other metrics in future work to enhance our causal analysis further.

The current work focuses on highly structured tasks to demonstrate the effectiveness of the CreDes framework. We agree that exploring its applicability to less structured, dynamic tasks is essential.

We are already working on designing related experiments based on open-world scenarios such as Minecraft and expect to argue further in a subsequent paper. We are currently actively designing experiments concerning the following related work: https://github.com/CraftJarvis

[1]. Wang Z, Cai S, Liu A, et al. Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models[J]. arXiv preprint arXiv:2311.05997, 2023.

[2]. Wang Z, Cai S, Chen G, et al. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents[J]. arXiv preprint arXiv:2302.01560, 2023.

We initially plan to leverage the open-source Minecraft simulator from our previous work as a calibration tool for the long task decomposition steps we have planned, to assess the accuracy of our decomposition steps, and then to try to generalize our approach to open-world tasks, moving closer to the goal of embodied intelligence.

We believe that simulation-based game simulations such as Minecraft have an expected better migration adaptation to the real world, but of course, there is more to it than that, and we also have a vision of developing simulators based on platforms such as Unreal to visualize decision-making process simulations.

The purpose of this paper is to eliminate the reasoning illusion of LLM. At the same time, ATE is a measure of causal significance, and the causality of LLM's reasoning at each step can be enhanced by combining ATE and LLM. Although this paper does not change LLM on the infrastructure, based on ATE and bidirectional search, it does improve the reasoning accuracy of LLM on some representative tasks. Thus, our experiments can support the motivation. Of course, we will consider improving LLM's infrastructure to enhance its fundamental reasoning ability. Due to our arithmetic limitations, we are currently tepid about conducting experiments such as LLM architecture tuning based on something other than open-source pre-trained models, but we will remain active.

Questions:

1.There is no guarantee, and our experiments confirm that DES searches are more efficient and have improved accuracy. The accuracy improvement is because we cut off long problems that are difficult for the model to solve into short issues that the model excels at, adjusting the problem to the advantageous range of the model's capabilities while ensuring that the search is relatively reliable with the appropriate search method.

2.This depends on the complexity of the actual application; for example, in the Minecraft example, for collecting wood, digging stones, lighting torches, and other actions that have contextual causal logic reasoning relationships, it is expected to be realized in our current preliminary experiments. We initially plan to leverage the open-source Minecraft simulator from our previous work as a calibration tool for the long task decomposition steps we have planned, to assess the accuracy of our decomposition steps, and then to try to generalize our approach to open-world tasks, moving closer to the goal of embodied intelligence. We believe that simulation-based game simulations such as Minecraft have an expected better migration adaptation to the real world, but of course, there is more to it than that, and we also have a vision of developing simulators based on platforms such as Unreal to visualize decision-making process simulations.

We hope these clarifications address your concerns and improve the overall understanding of our work. We are committed to enhancing the paper based on your valuable feedback. Thank you for your time and consideration.

Dear Reviewer MmUh,

We have submitted our rebuttal several days ago, and the discussion process is now more than halfway through. However, we have not yet received your response. We greatly value your insights and are eager to hear your further feedback on our paper. Your comments will be instrumental in helping us improve the manuscript.

Thank you for your time and consideration.

Sincerely,

Authors

Dear Reviewer Y4qM,

We want to express our gratitude for the thorough review and the constructive feedback on our paper.

Weaknesses:

1.(a):

The concept is that, for multiple independent repetitions of the experiment, causal hallucinations in the model output are infrequent. Using the hallucination scenario depicted in Fig. 2 as an example, when the distribution of experimental results continues to fluctuate (i.e. when hallucinations are still produced sporadically), the absolute value of ATE increases. Conversely, when the distribution of experimental results stabilizes (i.e. when results become consistent and unchanging across different inference paths), the absolute value of ATE decreases. This means that after the cross-entropy selection path, the decreasing absolute value of ATE constrains the model's unexpected state transition, leading to more stable output results.

1.(b):

Assumption 1: The causal illusions in the model output are few, i.e., the cases where causality between OSR and the next state is lost are few, as demonstrated in our experiments with practice when the pre-trained model can do so.

Assumption 2: Equation (2) uses McNemar to test the significance of ATE to enhance the causal link between OSR and the next state, i.e., ATE decreases when OSR is the only cause of the next state.

Assumption 3: The dataset given OSRs, i.e., the OSR of the desired reach and the OSR of the alternative paths, are mutually biased; for example, i.e., the OSR of the alternative paths is used as an intervention for the OSR of the desired reach path during the ATE evaluation process, and the OSR of the alternative paths is used as an intervention for the OSR of the alternative paths during the ATE evaluation process, of course, the paths' OSRs can be of many entries, for example The RAP method unfolded with Monte Carlo search will produce many alternative paths (the model is frozen), and the output of an untrained model is equally diverse.

Assumption 4: During the ATE evaluation process, no attention is paid to the correctness of the OSR path selection, only to the strong causal between the OSR and the next state.

1.(c):

Here are some references:

[1] ANGRIST J, IMBENS G. Identification and Estimation of Local Average Treatment Effects[R/OL]//Econometrica,Econometrica. (2016-03). http://dx.doi.org/10.3386/t0118. DOI:10.3386/t0118.

[2] Donald B Rubin. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of educational Psychology, 66(5):688.

[3] Eric Nichols, Leo Gao, and Randy Gomez. 2020. Collaborative storytelling with large-scale neural language models. In Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games, pages 1–10.

According to the previous reference, Equation (2) is correct in using McNemar's test for the significance of the ATE. Equation (5) is designed to evaluate the causal link between the distance reduction and the number of tree unfolding layers, with the aim of better compressing the search space, and we clarify that this writing of ATE (A) may have caused a misunderstanding, and we will subsequently revise it to ATE (A, B)

2.(a) We revise paragraphs 78-80 to read: while COT makes sequence inference possible, its main contribution is interpretability, and the inference power is relatively limited. We will replace it in the revision.

2.(b) We will address this in our revision.

2.(c) We will address the issue of graph plotting in our revision, clarifying that the choice of paths can be controlled by cross-entropy, proven in the baseline method RAP. Our improvement is also not in proposing that cross-entropy controls the selection of paths but instead that it reduces the illusions in the paths after the paths have been chosen.

2.(d) All our do(.) refer to do-calculus.

2.(e) We clarify that due to our writing error, you misunderstood that the do(.) operation on X results in a change in Y and that X is the cause of Y. In this paper, the interventions we impose on OSRs are, in fact, cross-references between desired and alternative OSRs, e.g., in computing the ATE of a desired OSR, several alternative OSRs will be interventions, as we state in the weakness. Thus, we minimize the ATE by stabilizing its distribution, not by simply enhancing the causal link between a particular OSR and the next state, but by enhancing the causal link between all OSRs and the next state, which is a holistic assessment process.

Questions:

1. As a further explanation, we refer you to the statement of weakness 1(a)(b); in our experimental data, when the model reaches a late stage of training, and the state transitions tend to stabilize (i.e., the number of state samples generating hallucinations decreases), the causal relationship between the OSR and the next state is strengthened, i.e., the OSR and the next state tend to be in one-to-one correspondence, which is different from what will happen in the RAP.
2. In this context, "robust" refers to the stability and reliability of the causal relationship between OSR and state transition under various conditions. You can refer to our related statements in the Weaknesses section.
3. The answer to this question is mentioned in our elaboration on weaknesses, which you can refer to in the weaknesses section.

We hope these clarifications address your concerns and improve the overall understanding of our work. We are committed to enhancing the paper based on your valuable feedback. Thank you for your time and consideration.

Dear Reviewer Y4qM,

We have submitted our rebuttal several days ago, and the discussion process is now more than halfway through. However, we have not yet received your response. We greatly value your insights and are eager to hear your further feedback on our paper. Your comments will be instrumental in helping us improve the manuscript.

Thank you for your time and consideration.

Sincerely,

Authors

Dear Reviewer Y4qM,

Thank you very much for your thorough review and for taking the time to carefully consider our responses. We are pleased to hear that our clarifications have increased your confidence in the soundness of our paper.

We sincerely appreciate your willingness to re-evaluate our work and for raising your score. Your feedback has been instrumental in helping us improve our research, and we value the opportunity to learn from your insights.

Thank you once again for your thoughtful review and consideration.

Best regards,

Authors

Dear Reviewer nWAC,

We want to express our gratitude for the thorough review and the constructive feedback on our paper.

Weaknesses:

1.Limited Model Sizes:

We acknowledge that our experiments were primarily conducted on 7B parameter models. The choice was made due to computational constraints and availability. The Mixtral-8x7B model is also larger than the 7B model so that it can be used as a reference for the performance of larger sizes. We conducted some additional experiments at the model size of 13B, but due to time constraints, only the following experimental results were obtained.

Blocksworld

GSM8K

Hanoi Tower

From the results, there is not much difference between the experimental results under 13B and 7B, and we believe that the difference can be regarded as a random error generated by different training. From the performance comparison between the 70B model and the 7B model under the RAP method, the performance of the 70B model will be relatively improved. However, considering inference speed, the 70B model is much slower than the 7B, and it needs to be loaded with a certain amount of quantization, and the performance loss is equally present.

2. Lack of Error Analysis:

We agree on the need for detailed error analysis. Errors vary with different reasoning structures. (Using a Blocksworld 4-steps question as an example):

Considering the Characters Limitation, please see the Official Comment.

Our method is closest to the expected output, with RAP errors including correct paths with wrong outcomes and vice versa, while RoT and CoT errors often occur when attention at the reasoning chain's end is ineffective.

3.Dataset Validity and Construction:

The Hanoi Tower dataset is more complex than Blocksworld, involving a judgment on stacking order. Errors arise if the stacking order is violated, making the task harder. The dataset's size matches Blocksworld, with all steps being odd numbers based on the minimum steps required.

4.Computational Efficiency and Scalability:

The 7B models fit within a single A100 GPU which is mentioned in paper. The 13B models have similar time requirements, as quantization isn't needed. However, 70B models experience significant speed drops, likely due to quantization and their size. Despite these challenges, our work shows potential for expansion due to its time efficiency.

5.Generalization to Less Structured Tasks:

Our work focuses on structured tasks to validate the CreDes framework. We are designing experiments for open-world scenarios like Minecraft, referencing works such as Jarvis-1 and interactive planning with large language models. These experiments will help generalize our approach to open-world tasks and advance towards embodied intelligence.

[1]. Wang Z, Cai S, Liu A, et al. Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models[J]. arXiv preprint arXiv:2311.05997, 2023.

[2]. Wang Z, Cai S, Chen G, et al. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents[J]. arXiv preprint arXiv:2302.01560, 2023.

6.Reporting Error Bars and Statistical Significance:

We apologize for not including error bars and statistical significance. The revised paper will address this to validate our results. The current data suggests the error is insignificant.

Questions:

1.Please refer to Weakness 1 for the description.

2.Validation method: Similar to Blocksworld. Outputs are correct if they match expectations; otherwise, tested using PDDL. Violations of stacking order rules mark outputs as wrong, Hanoi Tower is stricter than Blocksworld.

3.Training speed difference is minimal between our model and the baseline, with the main difference in inference speed. RAP uses Monte Carlo search requiring multiple inferences, while CoT involves chained inferences. Our approach outputs answers directly, reducing search expansions compared to RAP and CoT.

4.GSM8K experiments address less structured tasks, with open-world reasoning tasks planned for future work. Refer to the Weaknesses section for details.

5.The inference process has less memory overhead, and our DES search method unfolds bidirectionally, allowing simultaneous head and tail tree searches, as shown in Fig.1. Our approach outputs short step sets (under 6-7 steps for 7B models) directly from the model which is different from RAP.

We hope these clarifications address your concerns and improve the overall understanding of our work. We are committed to enhancing the paper based on your valuable feedback. Thank you for your time and consideration.

We respect the rules of rebuttal, but there is a part of the question you want to know that we can't write in 6,000 characters, so we have created a separate official comments section for that, and would appreciate it if you would take it into consideration.

To Reviewer nWAC:

Due to the input limit of 6000 characters in the Rebuttal input box, we have no choice but to adopt this approach to improve our statement. This message may be released before the start of the discussion, and we apologize for any inconvenience caused. You should be able to see the complete statement after the debate begins.

Error analysis provided to Reviewer nWAC:

Initial State:

The blue block is clear, the orange block is clear, the hand is empty, the blue block is on top of the yellow block, the orange block is on top of the red block, the red block is on the table and the yellow block is on the table. 

Goal State:

The orange block is on top of the yellow block.

Expected output:

<unstack the blue block from on top of the yellow block> 
<put down the blue block>
<pick up the orange block>
<stack the orange block on top of the yellow block>

Actual output:

RAP:

A structure that unfolds in a tree shape:

First Layer:

<unstack the blue block from on top of the yellow block>

Second Layer:

<put down the blue block>  
or  <put down the yellow block> (hallucination)

Third Layer:

<unstack the orange block from on top of the red block> (hallucination)
or  <unstack the orange block from on top of the yellow block> (hallucination) 
or  <unstack the orange block from on top of the blue block> (hallucination)

Fourth Layer:

<stack the orange block on top of the yellow block> 
or  <stack the orange block on top of the blue block> (pruned)

CoT:

The logic of CoT reasoning output is to solve complex problems by step-by-step reasoning and refining intermediate steps, ensuring the accuracy and reliability of the final answer. First Input: Initial State

First Output:

<unstack the blue block from on top of the yellow block>
<put down the blue block>

Second Input: Initial State + First Output:

Second Output:

<pick up the blue block> (hallucination)
<stack the orange block on top of the blue block>

CRE(Ours):

Model one-time output of the whole process:

<unstack the blue block>
<put down the blue block>
<pick up the orange block>
<stack the orange block>

It should be clarified that CRE's mistake lies in the possibility of incomplete answers as mentioned above.

The above appendix will be included in our revised version of the paper.

Dear Reviewer nWAC,

Thank you for your feedback and your willingness to reconsider your evaluation of our work. We appreciate your concerns regarding the generalization of our proposed method, particularly in relation to addressing long-range causal reasoning problems.

1. Regarding Experimental Comparison:

Many reasoning tasks or long-range sequence decomposition tasks in real life fundamentally follow the same paradigm as our research work. We understand that you may still have concerns about the real-world applicability of our approach, so we would like to provide a few scenarios to help clarify it.

For instance, in the scheduling of port container stacking or the arrangement of goods in warehouse areas, there are already mature algorithms in the logistics field. However, our work attempts to leverage Large Language Models (LLMs) to enhance reasoning in these contexts. The goal is to bridge the communication gap between human operators and algorithm engineers by using LLMs to facilitate clearer and more effective interactions. We hypothesize that LLMs can receive and understand human instructions, adjusting their actions accordingly, which would improve collaboration between humans and algorithms.

While we haven't yet validated our approach with real-world data, the essence of many real-world reasoning or long-range sorting tasks closely aligns with the experimental paradigm we employed in our paper.

To further illustrate this, consider the example of warehouse item arrangement. This task might involve organizing items based on criteria like size, weight, or frequency of access. Although it may initially seem like a complex, monolithic task, it can actually be decomposed into a series of smaller, more manageable sub-tasks. For instance, the first sub-task could involve categorizing items by size, followed by arranging them within sections based on weight, and finally, placing them in specific locations depending on access frequency. Each sub-task is interdependent, with the completion of one informing the next, thereby creating a continuous sequence of actions that leads to the overall goal.

We have observed that many related works do not establish a strong connection with real-world scenarios, and our experimental scope is similar to theirs. Specifically, other works have adopted similar test scenario and dataset, which strengthens our confidence that our experiments are robust enough to validate our ideas. For example, in addition to the baseline papers we cited, the following three papers also used the blocks world scenario to validate the performance of task decomposition. This demonstrates that the examples or experiments provided in our paper are highly effective for validating our reasoning capabilities and are indeed verifiable.

In fact, Blocksworld is recognized, and we have omitted three papers from our baseline. The following are relevant papers based on Blocksworld validation for effectiveness:

In subsequent work:

[1] Yu F, Jiang L, Kang H, et al. Flow of Reasoning: Efficient Training of LLM Policy with Divergent Thinking.

The experimental results (based on LLaMA3 8B, which was released after the submission of our paper) are as follows:

[2] Liu Z, Hu H, Zhang S, et al. Reason for Future, Act for Now: A Principled Architecture for Autonomous LLM Agents.

The experimental results are presented in Fig.8 of their paper. Their baseline model, LLaMA-33B, shows performance similar to ours, but it struggles with 8-step, 10-step, and 12-step scenarios.

Concurrent work with ours:

[1] Zhang S, Zheng S, Ke S, et al. How Can LLM Guide RL? A Value-Based Approach.

We noticed this work during the preparation of our paper. Due to their reliance on the OpenAI API for open-source code, we did not include it as one of our baseline comparison methods. However, they also compared our baseline method, RAP, as shown in Fig.5. Our Step-6 accuracy is higher compared to theirs.

Additionally, if you have any real-world datasets or validation scenarios, we would greatly appreciate it if you could share them with us. Access to such datasets or scenarios would allow us to further validate our work and ensure its practical applicability in real-world contexts.

2. Regarding Theoretical Analysis:

We would like to clarify which aspects of the theoretical analysis you would like us to elaborate on. We acknowledge that due to word limitations, the theoretical sections addressed to other reviewers were brief. If you have specific concerns, and if time permits, we will do our best to provide a complete explanation. In the revised version of the paper, we plan to enhance the theoretical analysis in the appendix, incorporating feedback from all reviewers, and we welcome any further suggestions you may have.

Thank you for your time and consideration.

Authors

Dear Reviewer nWAC,

Thank you for your valuable feedback on our manuscript. The theoretical analysis you inquired about has been addressed in detail in our response to Reviewer BKzM's comments. We recommend that you refer to our reply to Reviewer BKzM for the relevant information.

https://openreview.net/forum?id=azkuhJBZXi&noteId=cbDKQGUyra

https://openreview.net/forum?id=azkuhJBZXi&noteId=RTQb3tqD03

We appreciate your support of our work, and please feel free to reach out if you have any further questions.

Best regards,

Authors

Dear Reviewer nWAC,

Thank you for your continued engagement and for your constructive feedback on our paper. We appreciate your recognition of the promise in our work and your thoughtful suggestions for improvement.

We will certainly take your recommendations to heart, especially regarding the need to refine the writing style and presentation. We also acknowledge the importance of including the detailed theoretical analysis and more comprehensive implementation details, particularly in light of the high efficiency and performance scores achieved on the datasets with CREDES performance. We will make these aspects a priority in our final draft to enhance the clarity and impact of our paper.

Your insights have been invaluable in guiding us toward a stronger submission, and we are committed to making the necessary revisions.

Thank you once again for your time and consideration.

Best regards,

Authors