Response to Reviewer Yrc6

We thank reviewer for thoughtful and constructive feedback. We address each concern and question in detail below.

Weakness

1. Lack of Architectural Innovation

The method relies solely on fine-tuning and input reformulation without architectural innovation, potentially limiting perceived novelty.

From past viewpoint with smaller models, framework innovation was essential; But for the current generation of large-scale VLMs, we argue that data-centric and unified multi-task fine-tuning itself constitutes a significant innovation.

Although each innovation individually offers distinct advantages, as discussed in the paper (line 41), prior architectural innovations are fragmented and difficult to integrate, which limits the development of HMER domain.

In contrast, our Uni-MuMER leverages VLM input-output to integrate multiple architectural strengths, achieving SOTA performance. This novel recognition paradigm can further generalize to related tasks and enhance robustness by integrating diverse data sources, providing new insights for applying VLMs in expression recognition tasks.

Looking ahead, our approach extends in two promising directions:

• enhancing general VLM capabilities through richer, domain-specific data resources;
• integrating reinforcement learning strategies, such as R1-like self-reflective error correction,

2. Auxiliary Tasks (AST Reasoning and Symbol Counting) Are Not New

"...the tasks are not new to the HMER. Prior works such as CAN and tree-based models have explored similar directions."

We appreciate the reviewer’s insightful comment.

1. Our contribution is not about proposing any single auxiliary task in isolation. Rather, we introduce a unified modeling paradigm capable of jointly addressing multiple HMER-related tasks (as detailed in our response to Weakness 1).
2. Specifically, the proposed Error Detection and Correction tasks represent a novel direction for HMER, laying the foundation for self-reflective capabilities in future HMER.
3. Previous works primarily experimented with smaller models. Their effectiveness when applied to modern large-scale pretrained VLMs has not been validated, and their improvements may not translate to current VLMs
4. Additionally, prior studies typically experimented on limited datasets. We found that the performance of these lightweight models often decreases significantly (as demonstrated in Appendix C.1 Table 7).

Table 7: Impact of incrementally training data scaling on CROHME evaluaed with ExpRate@CDMmailto:ExpRate@CDM %

3. Alternative explorations format

The Tree-CoT serialization and symbol counting formats are hand-designed and lack alternative explorations.

For Tree-CoT serialization formats, including S-expression, JSON AST and our proposed AST format.

• S-expression provides minimal structural detail.

  • mu_{pj}

• JSON AST is overly verbose and redundant.

  • [{'type': 'supsub', 'value': {'base': {'type': 'mathord', 'value': '\mu'}, 'sub': {'type': 'ordgroup', 'value': [{'type': 'mathord', 'value': 'p'}, {'type': 'mathord', 'value': 'j'}]}}}]

• our chosen AST format offers a clear and concise hierarchical representation, balancing minimal redundancy with structural clarity.

\mu
    p (sub)
    j (right)

We evaluated our method across those formats, observing improved performance in all cases, with the our AST format achieving the best results

We explored another counting format, “Map Counting” with higher redundancy. Experiments confirm our format are better.

For x + 1

Map Counting Format

{'x':1,'+':1,'1' 1}

Our Counting Format

x: 1,+: 1,1: 1

4. Applicability of Tree-CoT to Non-Tree Structures:

The Tree-CoT task ... which limits its applicability to more complex layouts like matrices, piecewise functions, or multi-line expressions that follow grid-like or block structures rather than tree-like ones.

We clarify that complex layouts can indeed be transformed into hierarchical tree structures, including these multi-line expressions. While earlier HMER datasets, such as CROHME, omitted these formats, recent datasets, such as Mathwriting include them. We address this gap by introducing relation tokens (e.g., nextline).

For instance, a piece wise function:

$$y = \begin{cases} 0 & x < 0 \\\\ 1 & x \ge 0 \end{cases}$$

y
= (right)
\begin (right)
    0 (begin_cases)
    & (right)
    x (right)
    < (right)
    0 (right)
          1 (nextline)
          & (right)
          x (right)
          \ge (right)
          0 (right)
    \end (end_cases)

Questions

1. CROHME-Only Fine-Tuning Results

Could you provide Uni-MuMER’s results when fine-tuned only on CROHME (without external data) for fair comparison with prior methods (Table 1)?

In fact, We stated in Section 4.1 (lines 240), and our model surpassing previous SOTA.

This setup applies consistently across Tables 2, 9–12. Both Vanilla and Uni-MuMER were trained solely on the respective training sets. We only employed external data in models explicitly marked with a dagger ($\dagger$), such as Uni-MuMER$^\dagger$.

2. Filtering out CJK in HME100K Evaluation

For HME100K, Could the authors clarify the rationale behind filtering CJK characters, and whether it significantly affects the reported performance or comparability with prior work?

Thank you for raising this point. We excluded samples containing CJK characters primarily for two reasons:

1. Our prompt specifically targets recognizing mathematical expressions, not general textual content such as CJK characters. This filtering ensures a clear evaluation of expression recognition capability.
2. Some CJK-containing samples introduced problematic annotations unrelated to pure expressions (e.g., textual descriptions or numeric lists)

We evaluated the impact of this filtering and found minimal effect on comparability.

To further address your concern, we retrained Uni-MuMER† with the CJK characters in training set and compared its performance with w/o CJK version, evaluate on ExpRate (%):

• For instance, the performance of CoMER and TAMER remained nearly identical before and after filtering.
• our model still outperforms previous methods, with the CJK characters same in training set

However, a decrease in expression recognition performance when add CJK into training set.

We will include the above details in Section 4.2 Main Results in the final version.

3. Clarification of “External Data” Definition

“External data” is mentioned in both Table 1 and Table 2. Could the authors clarify which datasets or sources are included in this category?

Although we mentioned in Section 4.2 (lines 247), we realize that it was unclear.

As detailed in Appendix B.1 (Paragraph Data Construction, lines 1007–1014), the “external data” collectively includes CROHME, CROHME 2023, HME100K, MathWriting, and Im2Latexv2, totaling 392K images. These datasets, in combination with three tasks (Tree-CoT, EDL, and SC), form the 1.6M samples.

We will clarify this in Section 4.2 to avoid confusion, and open-source our dataset.

Limitations

Hallucination and Repetition Issues

Do fine-tuned large VLMs like Uni-MuMER suffer from hallucination compared to specialized models? This limitation should be discussed.

Both fine-tuned large VLMs and specialized models are probabilistic model, making them possible to hallucination and repetition issues.

To quantitatively analyze this, we preliminary adopt two metrics: Hallucination Token Rate (HTR) (percentage of predicted tokens not in the ground truth) and Missing Token Rate (MTR) (percentage of ground truth tokens not in the pred). Additionally, we categorized hallucinations by token type (numerical(0-9), variable(a-zA-Z).

Experimental results (without external data) on the CROHME 2014, 2016, and 2019 datasets clearly demonstrate that Uni-MuMER (HTR=1.18%) significantly reduces hallucination compared to both baseline and specialized models.

We will include the above details in Section 5 Analysis in the final version.

We sincerely appreciate the reviewer’s constructive suggestions. The additional experiments conducted based on your feedback have strengthened our paper. We hope these improvements provide clear grounds for reconsidering your evaluation and potentially raising your score.

Response to Reviewer QkdQ

We thank the reviewer for the thoughtful and constructive feedback. We address each concern and question in detail below, aiming to clarify our design choices and highlight the contributions of our work.

Weakness1 Comparisons with UNIMer and ICAL in Table 1

However, the comparison section lacks comparisons with works like UNIMer and ICAL (Authors do mention the see works in the related work section, and, IIUC, the numbers for those will be lower for CROHME datasets than the ones reported by the authors, but potentially higher than other baselines reported by them, but it would nevertheless strengthen the paper to include them in the table 1).

We appreciate the reviewer highlighting these missing comparisons. Due to page constraints, we were initially unable to include these results. In the revised manuscript, we will incorporate comparisons with UnimerNet and ICAL in the updated Table 1 to strengthen our evaluation:

Weakness 2 Limited Dataset Evaluation Reporting in the Main Text

Furthermore, authors only report results on MNE, HME100k, and Mathwriting in the appendix, and on a limited subset of papers - it would be beneficial to see a wider comparison, whenever available, and in the main manuscript.

We appreciate the reviewer’s suggestion. Due to strict page limits and the fact that some test sets (e.g., MNE, MathWriting) are not yet widely tested in prior relative work, we included these results in the appendix. We agree that presenting a broader set of comparisons in the main text will strengthen the paper, and we will move key results from these datasets into the main paper in the camera-ready version.

Originality: Novelty is Limited

The core of the work is a multi-task training setup on a number of datasets, with the observation that multi-task training helps, and having the training dataset corresponding to the test is beneficial for performance (l. 341) - the novelty is quite limited, with the exception of the specific task setup.

We appreciate the reviewer’s comment regarding novelty.

(1) Novelty from Paradigm Shift:

From past viewpoint with smaller models, framework innovation was essential; But for the current generation of large-scale VLMs, we argue that data-centric and unified multi-task fine-tuning itself constitutes a significant innovation.

Although each innovation individually offers distinct advantages, as discussed in the paper (line 41), prior architectural innovations are fragmented and difficult to integrate, which limits the development of HMER domain.

Previous works primarily experimented with smaller models. Their effectiveness when applied to modern large-scale pretrained VLMs has not been validated, and their improvements may not translate to current VLMs

(2) Empirical Novelty (Data Utilization Efficiency):

Our method enhances data utilization efficiency. Such advantages are broadly applicable to other domains facing challenges with limited or diverse training resources.

(3) Generalization beyond Task-Specific Datasets:

Moreover, the reviewer correctly observes that more data generally improves performance. Yet, we stress that this outcome is non-trivial and unique in the context of HMER:

• traditional lightweight models (e.g., CoMER) have limited benefits or even performance degradation. As shown explicitly in our ablation (Appendix C.1 Effects of Data Diversity Compared with Specialized HMER Model, Table 7), incremental dataset scaling substantially benefits our model (Uni-MuMER), whereas the performance of traditional specialized models is unstable or declines:

  Model	CROHME Train	+ CROHME-23	+ HME100K	+ MathWriting	+ Im2Latexv2
  CoMER	58.14	60.08	63.10	52.17	52.29
  Uni-MuMER	75.36	76.27	78.01	83.20	82.86

• Additionally, prior studies typically experimented on limited datasets. We found that the performance of these lightweight models often decreases significantly (as demonstrated in Appendix C.1 Table 7).

Questions: Cross-Dataset Generalization – Odd-One-Out Experiment

You mention that the performance does rely heavily on having the target dataset in the training mixture, but that this does depend on the variety in the data. One thing that could strengthen the work could be showing the generalization ability, ex. by looking how does the model perform in an odd-one-out setting, where it is trained on N-1 datasets and is evaluated on the Nth - do you have some observations around this?

We thank the reviewer for this insightful suggestion. As suggested, we conducted an “odd-one-out” generalization experiment, training Uni-MuMER on N-1 datasets and evaluating it on the held-out N-th dataset.

The results below present the expression recognition accuracy (ExpRate@CDM%) in each scenario. For comparison, we include the baseline CoMER trained on all datasets:

Observations:

Our results clearly demonstrate the strong generalization capability of Uni-MuMER compared to the baseline model (CoMER):

• Even without training on the target domain, Uni-MuMER consistently achieves substantially higher accuracy than the baseline CoMER† trained on all datasets. For example, Uni-MuMER without MathWriting achieves 32.55% accuracy on MathWriting, significantly surpassing CoMER’s 26.34% accuracy (trained on MathWriting explicitly).

This finding confirms that Uni-MuMER effectively leverages shared knowledge across domains and generalizes well to unseen datasets, validating its strong multi-task learning capabilities.

However, we also notice that the degree of generalization depends on data distribution similarity:

• For datasets closely related to others (e.g., CROHME and CROHME2023), accuracy drops are minimal (generally within 1-3%), further indicating good cross-domain transfer.
• For datasets with distinct characteristics (e.g., HME100K for real-life, low-quality, MathWriting for densely structured expression, and Im2LaTeXv2 for printed expression), accuracy drops are more pronounced. Nevertheless, even in these challenging cases, Uni-MuMER outperforms baseline methods significantly.

In summary, our “odd-one-out” experiments demonstrate that Uni-MuMER exhibits remarkable generalization capability and robustness compared to previous methods, validating the benefits of the proposed multi-domain modeling approach.

We will clearly highlight these findings in the revised manuscript and further discuss the implications and limitations of our approach.

Response to Reviewer uB1H

We thank the reviewer for the positive assessment of our work. We are glad that the reviewer found our unified and efficient framework, as well as the strong performance improvements of Uni-MuMER. We address each weakness, question, and point raised by the reviewer in detail below.

Weakness: Inference Speed and Computational Cost

For the easy, clear, reasonable, and effective method proposed in the paper, there are not many weaknesses. The dominating weakness is the inference speed and the computational cost increment introduced by the general-purpose large language model when compared with the previous small and specific models.

We appreciate the reviewer’s valid concern about inference speed and computational cost. Indeed, using a general-purpose VLM incurs higher computational overhead compared to prior lightweight models.

Our approach, which leverages the optimized vLLM inference engine, significantly mitigates this issue. By using some GPU (like A800 80GB), our method achieves 8.29× faster inference speed compared to previous smaller, specialized models, as mention in Appendix C.2 Inference Efficiency Table 8.

By retaining a standard VLM architecture without structural modifications, we fully exploit community-optimized inference framework, which futher highlighting a key strength of our work.

Besides, our empirical results demonstrate that this cost brings significant performance and speed improvement.

Question1: Error Analysis

Have you done some error analysis?

Thank you for the suggestion. To deepen our analysis of error types in mathematical expression recognition, we have introduced definitions for hallucination and omission at the token level:

• Hallucination Token Rate (HTR): the rate of tokens predicted that do not appear in the ground truth.

  • HTR_num: hallucinated numeric digits (0–9)
  • HTR_var: hallucinated alphabetic variables (a–z, A–Z)
  • HTR_delimiter: hallucinated structural delimiters (e.g., [], {}, ())

• Missing Token Rate (MTR): the rate of tokens ground truth that do not appear in the predicted to quantify omitted tokens

We evaluated these metrics across the CROHME 2014, 2016, and 2019 test sets, ensuring that no external data was used during training. The results are summarized below:

Uni-MuMER significantly reduces hallucinated tokens and omissions compared to both the vanilla baseline and prior SOTA specialized models.

• Tree-CoT improves structural reasoning and reduces structural hallucinations
• EDL corrects frequent symbol-level mistakes,
• SC enforces token consistency, reducing omissions and hallucinated insertions.

We will incorporate this analysis in Section 5 of the final paper to highlight Uni-MuMER’s robustness.

Question2: Which types of Symbols are Most Challenging?

Which types of symbols remain most challenging for Uni-MuMER?

Thank you for raising this insightful question, as it pertains directly to inherent challenges within the HMER domain and future work (response Question 3).

We have conducted a detailed analysis of the error cases from the CROHME datasets. Our observations indicate that the majority of remaining errors can be attributed primarily to two factors:

1. Hallucination: As discussed previously (Response to Question 1), hallucination is a known issue inherent to VLMs. While Uni-MuMER substantially alleviates this issue—as demonstrated by the significantly reduced Hallucination Token Rate (HTR) shown previously—it is challenging to entirely eliminate due to inherent ambiguities in handwritten data.

2. Similar-character (Token-level) Errors: Another primary error source stems from visually similar or ambiguous characters. Without explicit context, distinguishing certain symbol pairs—such as:

   • Lowercase "s" and uppercase "S" (s_a vs. S_a),
   • Digit "0" and uppercase letter "O",
   • Lowercase letter "a" and Greek letter (\alpha),
   • Digit "1" and lowercase "l", is particularly challenging. These symbol pairs often share substantial visual similarity, making isolated recognition inherently ambiguous even when trained on extensive datasets containing diverse handwriting styles.

Addressing these challenges transcends the traditional exact-match metrics (e.g., ExpRate) and relies on more sophisticated context-aware evaluation methods beyond token-level accuracy.

Question3: Additional Training Tasks for Improvement

Are there more training tasks that can be used to further improve the model？

Looking ahead, we envision that improvements in the model’s integration of broader world knowledge could significantly alleviate issues mentioned in Response Question 2. As the model gains more extensive and nuanced reasoning capabilities, it can better infer plausible characters within ambiguous contexts. Moreover, as mentioned in Section 6 (Discussion), promising directions include:

• R1-like Reflective Self-Correction: Allowing the model to iteratively reflect on and correct its predictions, thereby reducing errors due to ambiguous or visually similar symbols.
• O3-like Functional Call Visual Reasoning: Enabling explicit visual reasoning steps, such as zooming into particular regions of an expression image to resolve fine-grained details and mitigate visual ambiguity.
• Hierarchical Recognition with Integrated Reasoning: Structuring the recognition task hierarchically—first counting symbol, then constructing the Abstract Syntax Tree (AST), and embedding reflective and functional-call reasoning (R1-like and O3-like) within the AST construction process to ensure accuracy and structural consistency.

We plan to explore these exciting avenues in future research, extending Uni-MuMER’s capabilities while leveraging the strength and flexibility of our existing framework.

Thank you very much for your detailed review and valuable feedback on our manuscript. We have noted that the discussion deadline is August 6th (AoE). If you have any further comments, questions, or suggestions, please don't hesitate to share them with us. We would be glad to address them promptly and make any necessary improvements. We greatly appreciate your insightful contributions and look forward to refining our paper further based on your valuable advice.

Please do not hesitate to reach out if you require any additional information or clarification. We remain committed to actively engaging in the revision process and responding to your feedback thoroughly.

Response to Reiewer DUPV

We sincerely thank the reviewer for the thoughtful feedback, which is helpful to improve the paper.

Weakness 1: Limited Novelty of Using a vLLM

The scientific novelty of using a vLLM for this task is limited, even though the experimental exploration is solid.

Due to space limitations, please refer to Reviewer QkdQ’s comments regarding Originality for further discussion on this issue.

Weakness 2: Dependence on In-Domain Data

The effectiveness of the approach seems heavily dependent on access to in-domain data, which limits its general applicability.

Thank you for highlighting this point. To investigate our model’s generalization, we conducted an additional “Odd-One-Out” experiment (training on N-1 datasets, testing on the held-out dataset), evaluted with ExpRate@CDMmailto:ExpRate@CDM %.

The odd-one-out evaluation confirms that our unified model can generalize to an unseen dataset reasonably well. When the unseen domain is very different, performance drops more markedly, which highlights an area for improvement. HME100K for real-life, low-quality images, MathWriting for densely structured expression, and Im2LaTeXv2 for printed expression images, each of which represents a markedly different and challenging domain.

Crucially, after introducing even modest amounts of domain-relevant data, our unified approach easily achieves top-tier performance. Thus, our results indicate that, compared to existing specialized models and general VLM baselines, our approach provides superior generalization with minimal domain-specific data requirements, highlighting a clear advantage and direction for future improvement.

We will include these findings in the paper, discuss the implications for generalization and acknowledge the limitations.

Weakness 3 : Computational Cost of Error-Driven Learning (EDL) with Multiple Models

Thank you for raising this important point. To balance effectiveness and computational efficiency, we chose a modest value (k=3) for the k-fold setup, considering both data quality and training costs. While this does increase computational cost, we believe it is justified because:

1. Error-Driven Learning provides a general, reusable strategy enabling continuous model improvement.
2. The resulting error corpus, which will be open-sourced, contains inherently valuable, challenging examples tailored specifically to our strongest current model (Uni-MuMER), offering lasting benefits for future HMER research.

Weekness 4 : Clarity of Future Work on Self-Correction

The future research direction on self-correction lacks clarity, especially regarding handling of context-dependent expressions.

Thank you for your interest in our proposed future direction on self-correction.

Uni-MuMER introduces a novel perspective on enabling self-reflective behavior for mathematical expression recognition:

1. Uni-MuMER brings a Tree-aware Chain-of-Thought mechanism and basic error-correction capabilities. Combining these aspects can naturally facilitate self-reflection behaviors, similar in spirit to Deepseek-R1.

2. Specifically, we can obtain data with error correction behavior through synthesis or manual annotation to train the model to acquire similar capabilities, and then enhance it through RL. (Similar to the process of Deepseek-R1)

3. To illustrate how our envisioned method handles context-dependent expressions, consider the example expression \lim _ { n \to \infty } ( \sum _ { k = 1 } ^ n \frac 1 { k ^ 2 } ): $lim _ { n \to \infty } ( \sum _ { k = 1 } ^ n \frac { 1 } { k ^ 2 } )$

• Without Tree-CoT and error-correction, the model might mistakenly decode the final token as “2”.

• With Tree-CoT and self-correction capabilities, upon initially decoding the token as “2”, the model can reflect and identify an unclosed left parenthesis earlier in the expression. By systematically reviewing the tree structure (as enabled by Tree-CoT), it can correct this token, determining that a right parenthesis is contextually more appropriate than “2”.

This example highlights the practical benefit of our envisioned approach for handling context-dependent expressions, and we will include a clearer description of this idea in the revised manuscript.

Question 1 Base Model Selection (Why Qwen-2.5-VL-3B?)

Why choosing Qwen-3B for fine-tuning, considering that the 7B version already achieves better than Uni-MuMER?

Thank you for raising this question. We chose Qwen2.5-VL-3B to strike a balance between performance and computational cost, ensuring our approach remains suitable for real-world deployment.

Importantly, we also trained a 7B version of Uni-MuMER for comparison and observed only marginal improvements:

Additionally, as noted in Table 1, even the larger zero-shot models (7B, 72B, Gemini2.5-flash) significantly underperform compared to specialized HMER methods, while our fine-tuned 3B model surpasses them by over 20%. This strongly indicates that effective domain-specific fine-tuning matters more for HMER than the base model size, further justifying our choice of the smaller, more efficient 3B version.

Question 2 Inference Time in Table 8

In Table 8, inference time comparisons suggest that smaller models are slower than Qwen. How is this possible?

The higher inference speed of Qwen-based Uni-MuMER is due to the vLLM inference engine, which provides highly optimized batched decoding and caching mechanisms, including optimized caching mechanisms such as PagedAttention for efficient KV caching. With access to large-memory GPUs (e.g., A800 80GB), vLLM can cache more key-value pairs, further accelerating inference. In contrast, smaller models are often compute-bound rather than memory-bound.

Additionally, previous specialized models typically rely on beam search, which is inherently slower than the nearly greedy decoding used in VLMs. vLLM performs an optimized prefill stage, amortizing the initial encoding cost across multiple subsequent decoding steps and boosting per-sample inference speed in batched processing.

Many specialized models employ custom architectural modifications (e.g., tree decoders), which prevent them from leveraging these general-purpose inference optimizations available in developer groups.

Our approach retains standard VLMs architecture without structural changes highlighting its key strength.

Question 3 Open-Sourcing

Will the source code and trained models be made publicly available?

Yes, we will make the code, trained models, training data and reproducible training scripts publicly available.

Question 4 k-Fold Value in Error-Driven Learning

Thank you for pointing out this oversight. We employed a 3-fold on the training set to generate corpus, which was chosen as a balance between generating error corpus and keeping computation feasible.

We will clarify this detail in Section 3.3 in the final version.

Question 5 Clarification on Table 3 (Ablation Results)

In Table 3, the dataset version is not specified. Moreover, I could not locate the reported result for CoT+EDL+SC (73.29) in other tables, particularly Table 1. Could you clarify this point?

The result of 73.29% ExpRate corresponds to the Uni-MuMER (w/o †) row in Table 1, under the “Average Exp” column (second-to-last row, second-to-last column).

This version of Uni-MuMER includes all three proposed tasks (Tree-CoT, EDL, and SC) and is aligned with the “Vanilla+CoT+EDL+SC” setting in Table 3. To ensure a fair comparison with previous work, this model was fine-tuned only on the CROHME training set, without any external data.

Question 6 Generalization vs. In-Domain Performance

The experiments in Section 5.2 suggest that the specialization of the Qwen model is only effective when fine-tuning is performed with in-domain data...

Thank you for your comment. We appreciate the opportunity to clarify.

Our intended message was that fine-tuning effectiveness indeed varies based on the closeness of the training and test datasets:

• In-domain fine-tuning consistently achieves the highest performance. For example, printed-expression data fine-tuning significantly outperforms using handwritten data alone (90% vs. 70% ExpRate@CDM).

• Partial out-of-domain data can still improve performance, as illustrated in Figure 6, where printed data enhanced some handwritten datasets.

• Mismatched datasets may degrade performance due to substantial domain gaps, such as training on clean expressions and testing on low-quality real-life images (e.g., HME100K), may degrade performance due to substantial domain gaps.

Additionally, we are actively exploring the integration of general SFT datasets (e.g., LLaVA’s One Vision data) with our HMER-specific training data to assess broader generalization performance. Due to computational constraints, this evaluation is ongoing, and we plan to discuss these findings further in future revisions.

Limitaions

Due to space limitations, detailed explanations of future work and limitations will be provided later.

Generalization vs in‑domain data

We discuss our method's generalization capabilities across datasets and domains.

Additional experiments showing that:

• Uni-MuMER effectively leverages shared knowledge across domains and generalizes well to unseen datasets
• Uni‑MuMER retains strong general capabilities. Uni‑MuMER-LLAVA benefits from general domain pre‑training.

Odd-one-out experiment: We conducted an experiment which trained on N-1 datasets and evaluated on N (suggested by QkdQ). The results show that Uni-MuMER effectively leverages shared knowledge across domains and generalizes well to unseen datasets.

Geneal domain Comparison: We ran an experiment to validate whether Uni‑MuMER benefits from general domain data beyond HMER‑only training (inspired by DUPV). Specifically, we compares:

• Uni‑MuMER: fine‑tuned only on HMER data;
• Uni‑MuMER‑LLAVA: fine‑tuned on a 1 : 1 mixture of HMER and LLaVA‑OneVision data.

The results show that Uni‑MuMER, despite being fine‑tuned exclusively on HMER, retains strong general capabilities.

For Uni-MuMER-LLAVA, it improves its overall capabilities while sacrificing only marginal HMER performance.

---

Efficiency and Cost

Regarding concerns about increased computational resources, we clarify below the primary resource-consuming stages and the efficiency of our approach in training, error-driven learning (EDL) corpus collection, and inference:

• Training: Our model can be effectively trained on 8 A100 GPUs within approximately 20 hours, ensuring manageable training cost.
• EDL: The EDL corpus collection requires computational resources similar to those for the original dataset. We use k=3-fold cross-validation to balance quality and cost. The open-sourced challenging error/correct pairs provide valuable, reusable resources.
• Inference: Under identical hardware, Uni-MuMER achieves significantly higher throughput compared to traditional specialized models (see Appendix C.2, Table 8). Without architectural modifications, our approach benefits directly from existing community optimizations such as vLLM.

New Experiments

We have updated the manuscript according to reviewers’ comments. Specifically:

• To clarify the generalization ability of our proposed approach, we conducted an “odd-one-out” experiment on training datasets and compared Uni-MuMER and Uni-MuMER-LLAVA using general multimodal reasoning benchmarks. The results clearly demonstrate Uni-MuMER’s strong generalization capability.
• Regarding concerns about comparability with prior work, we evaluated the impact of filtering and found minimal effects on comparability.
• To address the hallucination issue in fine-tuned large VLMs, we introduced two metrics—Hallucination-Token-Rate (HTR) and Missing-Token-Rate (MTR)—and compared Uni-MuMER with specialized models. Results highlight that Uni-MuMER significantly reduces hallucinations.
• We conducted ablation studies on alternative explorations such as Tree-CoT and Symbol Counting formats. Evaluation results confirm that our chosen formats are optimal.

Writing

We have edited our manuscript to improve the clarity of experiments, figures, and tables. Specifically:

• Explicit discussions on external data have been moved from Appendix B.1 to Section 4.
• Explicit discussions on Uni-MuMER’s generalization are now included in Sections 5 and 6.
• Explicit discussions on future directions have been added to Section 6.
• Key results from additional datasets have been moved into Section 4.
• Alternative explorations are explicitly discussed in Appendix C.3.
• Error analysis and discussions on the hallucination issue of VLMs are detailed in Appendices C.4 and E.