Thank you for reviewing our paper and for your valuable feedback. Below, we address your concerns point by point, and we will revise our paper according to your suggestions. We would appreciate it if you could let us know whether your concerns are addressed by our response. We hope that you might consider raising your score in light of these clarifications.

---

Q1: How to define the parallelism? Could you provide some parallelism metric? The speedup is unmatched with the number of parallel branches.

A1: In this paper, we define the degree of parallelism as

$\text{parallelism} = \frac{\text{total tokens generated}}{\text{total tokens generated **sequentailly**}}$.

Because Multiverse can generate multiple tokens per step, its denominator, “sequential tokens”, can be smaller than the total number of generated tokens. This value remains tied to the maximum generation length, which in turn is defined by the highest position id in the generation.

Crucially, the number of parallel branches alone only illustrates the frequency of implicit parallel generation, but does not determine this ratio. What matters is each branch’s coverage—the portion of the overall task (generated tokens) that it handles in parallel. In practice, major parallel structures arise within specific subtasks rather than spanning the entire reasoning trace.

To further understand this question, we also compute the parallelism in our Multiverse-1K , which is 16.7 % on average. This value matches the parallelism and speedup seen at inference time. We will further include this value and clarify their difference in our later version for better understanding.

---

Q2: Whether LLM can generate in parallel or not? The key difference between section 3.2 and section 5.1

A2: The key difference lies in whether the model is creating a parallel structure from scratch or identifying parallelism within an existing structure.

• In section 3.2, we verified that LLM struggles to generate a parallel thinking trace directly from the instruction prompt. We call this explicit parallelism.
• In section 5.1, when provided with a pre-existing, sequentially-generated thinking trace, LLM can effectively identify the inherent parallel logic within that trace. We call this implicit parallelism.

---

Q3: The difference between Multiverse 32B-zero vs Multiverse 32B

A3: Thank you for the question. Both Multiverse-32B-zero and Multiverse-32B use the same fine-tuned checkpoint; the only distinction is the inference prompt:

• Multiverse 32B-zero is prompted with "Let's think step by step."
• Multiverse 32B is prompted with "Let's think step by step and in parallel."

We included both variants to understand the prompt’s impact on parallelism and overall performance. As shown in Table 2, adding the phrase “and in parallel” preserves accuracy while markedly increasing the degree of parallelism. Therefore, we view this technique as an effective way of modifying model behavior. Moreover, this prompt design also mirrors our training pipeline: autoregressive data were tagged with “Let’s think step by step,” while Multiverse data were tagged with “Let’s think step by step and in parallel.” The correspondence between these training prompts and the observed inference patterns underscores the critical role of prompt engineering in shaping model behaviour with prompt fine-tuning.

---

Q4: What is the baseline in table 2 exactly? How was the autoregressive model trained?

A4: To train the autoregressive baseline, we keep the data mixture, learning rate, number of epochs, and all other hyperparameters identical to those used for Multiverse-32B. The only changes are:

• Attention mechanism: Multiverse attention is replaced with standard causal attention.
• Corpus processing: All special tokens are stripped out, and we delete the content in Map phases (<Goal> … </Goal>) and Reduce phases (<Conclusion> … </Conclusion>) so that the remaining text reflects a purely sequential chain-of-thought (CoT) process. This is the main baseline we hope to compare with in Tab 2.

---

Q5: How to measure the implicit parallelism in Table 1 and Figure 3? Is it by hand or prompt?

A5: Thank you for the question. We measure implicit parallelism with a two-step process that combines LLM automation and human verification:

• First, we prompt Gemini-2.5-Pro to identify and list every instance of implicit parallelism.
• Next, a human annotator verifies each instance and categorizes it as either a selective or collective branch.

---

Q6: What is P in figure 8?

A6: The variable P represents the degree of parallelism, with the parallel metric clarified in A1. For readability in the current version, we scale it by a factor of 10; thus, P = 11 corresponds to an actual parallelism of 1.1. In the camera-ready version, we will remove this scaling factor, allowing the degree of parallelism to be visualized directly.

---

Q7: Line 83: What is the varying batch size?

A7: The varying batch size corresponds to the experiment detailed in Figure 8b where we tested the speedup under different batch size. The results show that the speedup from multiverse generation is scalable with batch size increase and maintained stable under different settings.

---

Q8: Line 132: For the hidden states probing experiment, use the internal layer instead of the last layer.

A8: Here we provide the probing test of different models in the internal layer. Table R1: Probing Test for QWQ-32B

Table R2: Probing Test for R1-Distilled 32B

Table R3: Probing Test for R1-Distilled 32B

The result verifies the conclusion that LLM does not truly understand parallelism.

---

Q9: Line 154: Why are there square brackets in x[1:3,s] not in x[1,1:6]

A9: Thank you for your question! Here, we use x[1:3, s] to combine the variables of $x_1$, $x_2$ and $x_s$. Therefore, the square brackets denotes a list with two separate part. For $x\_{1,1:6}$, we do not use square brackets as we view 1:6 as a whole. We will further add square brackets.

---

Q10: Line 204: How does the rewrite work? Is it implemented by hand?

A10: The entire data curation workflow, including all rewriting steps, was conducted via API calls to the Gemini 2.5 pro model. Here is the detailed prompt for rewrite:

# Rewriting Paths in the Structured Reasoning Trajectory

You are given a full structured reasoning trajectory inside a `<Parallel>` block, consisting of:

* one `<Goal>` block with multiple `<Outline>` elements
* multiple `<Path>` blocks
* one `<Conclusion>` block.

Some `<Path>` blocks may contain an entire nested `<Parallel>` structure (from `<Parallel>` to `</Parallel>`). These nested blocks should be rewritten using the same rules recursively.

***

## For `<Goal>`

* Rewrite each `<Outline>` into a **concise statement of what is being calculated or determined**.
* Remove any content describing **how** the problem is solved or intermediate reasoning steps.

***

## For each `<Path>`

* Keep the original numbering prefix (e.g., `1:`, `2:`).
* Rewrite the content as a **complete, fluent, and logically self-contained paragraph**.
* Do **not** use transitional phrases like “First," “Then," “Next," “On the other hand," etc.
* If the `<Path>` contains **five or fewer sentences**, rewrite them together as a single coherent paragraph, ensuring logical flow and fluency without using transitional phrases.
* If the `<Path>` contains **more than five sentences**: Rewrite the first five sentences together as a single unit, forming a fluent paragraph. For the remaining sentences, rewrite each one individually, based on its meaning, as clear and fluent standalone statements.
* If the `<Path>` contains a **nested `<Parallel>` block**, apply all these rules recursively to the nested block.

Each `<Path>` must make sense independently, even if it contains a nested reasoning chain.

***

## For `<Conclusion>`

* Rewrite the conclusion as the **most concise and synthesized summary** of the main outcomes from all `<Path>` blocks.
* You may combine or compare results from different paths, but keep it succinct and direct.

***

## Nested `<Parallel>`

* A nested `<Parallel>` may appear only **as a full block inside a `<Path>`**.
* If a `<Path>` contains a nested `<Parallel>...</Parallel>` block, process that inner block exactly as you would the top-level one:
    * Rewrite the inner `<Goal>`, `<Path>`, and `<Conclusion>` elements accordingly.
    * Maintain the XML structure — do not reindent or alter the tag hierarchy.

Thank you for your suggestions. We have responded to your questions below and would appreciate it if you could let us know if our response addresses your concerns. We hope that you might consider raising your score in light of these clarifications.

---

Q1: Instead of “Map-Reduce” why not use spawn-collect (scatter-gather, or even simpler, fork-join)?

A1: Thank you for this suggestion. After revisiting the core definitions of MapReduce and Fork-Join, we now recognize that Fork-Join more accurately captures the type of parallelism enabled by Multiverse on modern hardware. Unlike MapReduce's batch-oriented, fault-tolerant processing across large clusters, Fork-Join's work-stealing scheduler and fine-grained task decomposition better exemplify the low-overhead, multi-core parallel computation with shared KV cache that we leverage. Framing Multiverse within a Fork-Join paradigm is indeed more precise and should facilitate better understanding, especially for a broader audience. We will incorporate this revised terminology in our camera-ready version if the paper is accepted.

---

Q2: Explain during inference, why different paths can share the same position ids when “reduce”.

A2: First, although we refer to “shared position IDs,” we do not actually have any computation between different query and key tokens for the same position. This is because softmax attention only considers the distance (i.e., $QK^T$) between token pairs that are not masked, as defined in Equation 1. Thus, the only constraint is that the query token’s positional id must be greater than or equal to the key token’s positional id to ensure the causal relation. By assigning the “Conclusion” token a positional id equal to one more than the maximum id used by any path, we satisfy this causal requirement. (For a deeper discussion of why this works and KV states from different paths can be merged, please see APE [1].)

Second, we employ the same position id assignment rule during both training and inference, ensuring full consistency. This symmetry means the optimization process directly reflects inference behavior, because the model is trained to attend to tokens that share identical positional ids.

Finally, position sharing itself is not new; it has been widely adopted in prior work on LLM length extrapolation [2, 3]. Because a token’s representation combines its hidden state and its positional embedding, modest adjustments to positional ids (even only during inference) have little adverse effect on model performance, as evident by these work.

---

Q3: The argument that “current LLM can not parallelize” is rather weak

A3: We thank the reviewer for highlighting this point and agree that the statement “current LLMs cannot parallelize” is ambiguous. Modern LLMs do exploit parallelism in several aspects, such as prefilling input tokens or serving multiple requests concurrently. Our argument, however, centers on a more specific and fundamental bottleneck: the autoregressive decoding process. This focus is critical because autoregressive modeling remains the dominant architecture underlying today’s leading LLMs.

During generation for a single response, each token is conditioned on all preceding tokens—a dependency represented by $P(\text{token}_n \mid \text{token}_1, \ldots, \text{token}\_{n-1})$. This strict causal dependency prevents parallelization within a single decoding sequence, leading to inherently sequential generation. As a result, latency remains high, particularly in long-form generation tasks such as Chain-of-Thought (CoT) reasoning. Drawing on the concept of goodput [4], we emphasize that overcoming this fundamental bottleneck is critical for improving efficiency in latency-sensitive applications.

In the camera-ready version, we plan to include these details in our introduction for improved clarity. We will also expand the related work section with additional hardware details. Specifically, we will highlight that modern GPUs achieve speedups primarily through parallel computation, yet autoregressive decoding is largely I/O-bound, meaning that increased compute resources do not proportionally reduce latency. This reinforces the importance of rethinking decoding architectures to unlock genuine gains in responsiveness.

---

Q4: Suggestions about citation on works that aim to allow different paths to see each other during inference.

A4: Thank you for your suggestions. We will definitely include work in this direction in our next version, including [5] and [6]. Additionally, we have conducted an ablation study on Multiverse’s sharing mechanism. In this setting, each parallel path can read the tokens produced by the other paths after a fixed interval of k sequential tokens, both during training and inference. As shown in Table R1, this cross-path visibility yields no performance improvement; in fact, overly frequent sharing slightly degrades accuracy, likely because the model becomes confused by mixing KV-cache states from different paths during the generation of each path. (Here, k=$\infty$ means the strategy used in our paper.)

Table R1: Ablation Study between sharing frequency and downstream performance of Multiverse-32B.

---

Reference: [1]. Yang, Xinyu, Tianqi Chen, and Beidi Chen. "Ape: Faster and longer context-augmented generation via adaptive parallel encoding." arXiv preprint arXiv:2502.05431 (2025).

[2]. Jin, Hongye, et al. "Llm maybe longlm: Self-extend llm context window without tuning." arXiv preprint arXiv:2401.01325 (2024).

[3]. Rerope: https://github.com/bojone/rerope

[4]. Zhong, Yinmin, et al. "{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving." 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 2024.

[5]. Rodionov, Gleb, et al. "Hogwild! inference: Parallel llm generation via concurrent attention." arXiv preprint arXiv:2504.06261 (2025).

[6]. Hsu, Chan-Jan, et al. "Group Think: Multiple Concurrent Reasoning Agents Collaborating at Token Level Granularity." arXiv preprint arXiv:2505.11107 (2025).

Dear Reviewer,

Could you please check if the authors’ rebuttal adequately addresses your concerns? If so, kindly acknowledge the rebuttal and provide any additional comments. If not, it would be greatly appreciated it if you could engage in a discussion with the authors. Your input at this stage is essential to the review process. Thank you very much for your time and effort!

AC

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Thank you for your suggestions. We have responded to your questions below and would appreciate it if you could let us know if our response addresses your concerns. We hope that you might consider raising your score in light of these clarifications.

---

Q1: The empirical improvement of the proposed method appears limited.

A1: We appreciate your question and agree that the current empirical improvement is limited. This outcome is directly linked to the degree of parallelism in our training data, Multiverse-1K. Our statistics shows that Multiverse-1K has an average parallelism degree of 1.16 which matches the parallelism observed at inference. This design choice aligns with our core research objective: rather than artificially increasing parallelism, we focus on externalizing the implicit parallel generation capabilities in modern autoregressive LLMs. Therefore, the observed parallelism degree of 1.16 represents the natural baseline of parallel reasoning these models inherently possess, which our Multiverse seeks to make explicit and accessible.

---

Q2: Error bars are not provided thus it is unclear the gains are statistically significant.

A2: Thank you for your suggestions. Here we provide our results with error bars for Tab 2, Figure 7 and Figure 8b.

Table 1: Reasoning Performance

Table 2: GPQA-Diamond Performance with control budget

Table 3: Math 500 Performance with control budget

Table 4: Full Speedup Results Across Varying Degrees of Parallelism and Batch Sizes

---

Q3: Ablation study is missing. How important is each of the three components?

A3: In our multiverse framework, the three core components—Data, Algorithm, and Engine—are intricately combined. A critical issue arises when fine-tuning the model directly on structured outputs: this process creates a mismatch between the position IDs during fine-tuning and those during inference stage, leading to significant performance degradation.

---

Q4: Ablation study is missing. How important is each of the three components?

A4: Here we provide the full prompt we use during the prompt test.

**Core Directive**
You are an expert mathematician and reasoner. Your primary mode of operation is to solve problems efficiently. 
When you think, you can process information in parallel to solve multiple parts of a problem at once. This requires you to first analyze the structure of any given problem.
**Conditional Logic Mandate:**
During your Chain of Thought thinking progress, you should think in parallel. Specifically determine if the problem can be decomposed into independent, non-sequential sub-tasks.
* **If the problem is decomposable**, you should use the Parallel Processing Format defined below.
* **If the problem is inherently sequential** or too simple for decomposition, solve it using a standard, linear step-by-step explanation.
You are to make this decision yourself. Do not ask for permission.
---
**Parallel Processing Format**
When a problem is decomposable, you must structure your entire thought process as follows:
```xml
<Parallel>
  <Goal 1>
    Define the first independent objective.
  </Goal>
  <Goal 2>
    Define the second independent objective.
  </Goal>
  
  <Path 1>
    Execute the steps to achieve Goal 1. Show your work.
  </Path>
  <Path 2>
    Execute the steps to achieve Goal 2. Show your work.
  </Path>
  
  <Conclusion>
    Synthesize the results from all Paths to generate the final answer.
  </Conclusion>
</Parallel>

We observed the reasoning generation with this prompt and analyzed the parallelism within the chain of thought.

We would like to sincerely thank reviewer eCfu for the insightful and valuable comments! They are invaluable for further improving the quality of our paper. We will revise our manuscript in the next version to address all of your concerns. We hope that you might consider raising your score in light of these clarifications.

---

Q1: In the experimental section, are there more settings provided, such as the code? A1: Thank you for your suggestions. We first need to clarify that our experiments in the paper follow the setups outlined in the S1 paper, which cover both training and evaluation of math and scientific reasoning. This setup is standard and ensures coherence between our training and inference. To address your concern, we further conduct experiments on IFEval and LiveCodeBench as instruction-following and code benchmarks. As shown in Table R1, our method can perform on-par with autoregressive models on these benchmarks. However, they do not increase the performance over base model due to the inconsistency of training and test domains.

Table R1: Experimental Results of various models on instruction-

---

Q2: Aime score seems low. Is it difficult to interpret the effectiveness of the method? A2: We emphasize that this work focuses on developing novel model architectures for parallel generation instead of improving reasoning ability, with Multiverse being the only open-source non-autoregressive model to achieve an AIME score above 50—setting a new standard in this area. For a fair comparison with autoregressive models, we adopt the s1 experimental setting, training on the same corpus (with CoT rewriting), and demonstrate performance comparable to s1.1 while achieving significantly higher efficiency. Given that s1.1 was released only two months before our submission and is a strong reasoning model trained purely with supervised fine-tuning, we consider it a reasonable and fair baseline. Although larger datasets, model scales, or compute budgets could further enhance Multiverse, these resources exceed typical academic constraints and lie orthogonal to our architectural design. In this context, we believe that achieving an AIME score above 50 thus represents both a rigorous standard and clear differentiation of model quality.

---

Q3: At a parallelism ratio of 1.3, the speedup is 18.5% which seems not strong enough. A3: As discussed in lines 290-291, the 18.5% speedup is an average across data exhibiting parallelism from 1.0 to 1.3 instead of 1.3 only. Therefore, our speedup should have a strong correlation with the parallelism ratio, which can be further verified in our Figure 8(b).

Dear Reviewer,

Could you please check if the authors’ rebuttal adequately addresses your concerns? If so, kindly acknowledge the rebuttal and provide any additional comments. If not, it would be greatly appreciated it if you could engage in a discussion with the authors. Your input at this stage is essential to the review process. Thank you very much for your time and effort!

AC