FreeMesh: Boosting Mesh Generation with Coordinates Merging

1. RMC Algorithm Clarification

Reviewer Concern: Handling groups with fewer than 9 coordinates (e.g., 7) was unclear.
Response:
For sequences with length < 9 (e.g., 7):

1. ​Truncation: Reduce to the largest multiple of 3 (e.g., 6).
2. ​Permutation: Reorder coordinates from xxyyzzz → xxyyzz → xyzxyz.

• ​Implementation: Full algorithm provided in serialization.py.

2. Per-Token Entropy Design Rationale

Reviewer Concern: Why not use standard entropy ($-\sum p\log p$)?
Response:
Standard entropy fails to capture compression efficiency. For example:

• String aaabbb with vocab {a,b}: $H = \log 2$
• Compressed as XY (vocab {X,Y,a,b}): $H = \log 2$ (unreasonably identical)
• ​PTME yields $\frac{1}{3}\log 2$, correctly reflecting reduced complexity.
• ​Example: Demonstrates PTME's superiority over naïve entropy in intuition.py.

3. PCME vs. PTME Differentiation

Reviewer Concern: Clarify the relationship between PCME and PTME.
Response:
Both PCME and PTME are derived from Equation 6 in Section 3.2.

• ​PCME: Measures entropy at the unmerged coordinate level, where each token represents one geometric coordinate (x/y/z). This granularity enables direct comparisons between geometric tokenizers like RAW/AMT/EDR.
• ​PTME: Aggregates entropy calculation over merged coordinates, with each token encoding multiple coordinates. PTME generalizes PCME by using abstract token representations instead of individual coordinates as the fundamental unit.

4. Expanded Evaluation Metrics

Reviewer Concern: Limited evaluation metrics for mesh quality.
Response: We now report:

• ​Boundary Edge Rate: This metric reflects the mesh hole rate by detecting boundary edges (edges that are used by only one face). The detection is carried out using the scripts detect_boundary.py.
• ​Topology Score: This metric evaluates the topological quality of a mesh by converting triangular faces into quadrilateral faces. A high-quality quadrilateral mesh should be regular, with its surrounding shapes being as close as possible to rectangles or trapezoids, and the ratio of opposite sides should not be excessively large. The evaluation is performed through the scripts tri2quad.py and topo_score.py.
• ​User Study: 10 participants rated meshes on a 5-point scale

5. Modality Focus Justification

Reviewer Concern: Why prioritize point-cloud conditioning? not other modality.

Response:

1. ​Technical Impact: Point-cloud conditioning is the de facto standard for industrial remeshing pipelines (e.g., MeshAnythingV2, Meshtron).
2. ​Task Complexity: Class-conditioned generation on ShapeNet is oversimplified compared to real-world application. Cross-modal systems (text/image → 3D) like EdgeRunner universally use point-cloud conditioning as their foundational step. These systems align image/text features to point-cloud latent spaces. Currently, this method is not very effective.
3. Scope Alignment: Our work focuses on coordinate merging, which are modality-agnostic. Modality-specific adaptations (e.g., encoders for text/images) would not affect our core contribution. Point-cloud conditioning suffices to validate our method.

1. Benchmarking with BPT Method

Reviewer Concern: Missing comparison with BPT for point-cloud conditioned generation.
Response:
We have added quantitative and qualitative comparisons with BPT under identical settings:

• RMC achieves better compression ratio (21.2%​ vs 26.0%) and topology quality.
• Visual comparisons of complex cases provided in bpt_compare.

2. Complex Mesh Generation

Reviewer Concern: Lack of high-fidelity complex mesh results.
Response:

• ​Current Limitation: Our method uses a plain Transformer with a 9k context window, which restricts high-face mesh generation capability. The comparison above with BPT shows our relatively complex cases.
• ​Future Direction: Integrating RMC into architectures like Meshtron (hourglass Transformer) or DeepMesh (BPT+Meshtron+DPO) could bridge this gap.

3. Data Selection Protocol

Reviewer Concern: Clarify data selection.
Response: Refer to my response (Reviewer NgVn/Q3).

4. Multi-Modality Evaluation

Reviewer Concern: Missing text/image-conditioned results.
Response:
As highlighted in responses (Reviewer Ri2A/Q5), we prioritize point-cloud conditioning. Currently, direct end-to-end training of text/image conditioning has poor results. Others are trained with point cloud conditioning in the first stage, and align the text or image with the point cloud in the second stage. The decoder is frozen in this process, so it is enough to only look at the point cloud condition.

5. Watertight Meshes Justification

Reviewer Concern: How closeness is guaranteed for watertight meshes?
Response:
Current explicit mesh representations (including ours) cannot inherently guarantee watertightness. To address this, we implement a post-processing pipeline (post_process.py) that ensures watertight meshes through three steps:

1. Hole filling using robust mesh completion algorithms,
2. Small component removal to eliminate floating artifacts,
3. Folded face repair via mesh regularization.

This pipeline leverages established mesh processing libraries to robustly mitigate non-watertight geometry in final outputs.

6. DI-PCG Discussion

We will add this paper in our related work.

1. PTME-CD Correlation Analysis

Reviewer Concern: Need rigorous correlation between PTME and generation metrics.
Response:
For EDR+RMC, we calculated the Pearson correlation between PTME and Chamfer Distance (CD) under varying vocabulary sizes:

• ​Pearson r = 0.965 , indicating a strong positive linear correlation.
• ​Visualization: EDR_RMC_PTME_CD.

2. PTME Increase with Naive Merging

Reviewer Concern: Why does Basic Merge Coordinates (MC) increase PTME?
Response:
PTME balances two factors:

1. ​Compression ratio: Merging increases average token length.
2. ​Entropy: Merging may flatten token distribution (higher entropy).

Key Insight:

• ​Example: Naive merging increases PTME. Rearranging coordinates before merging, lowering PTME. intuition.py

• ​Proof: High co-occurrence probability between adjacent pairs is critical for PTME reduction. proof

3. Training Set & Generalization

Reviewer Concern: Data selection and generalization analysis.
Response:

• Data Selection:you can Refer to my response (Reviewer NgVn/Q3).
• Generalization: While our method demonstrates robustness on simple geometries, architectural models with complex geometries often fail. failure_case

4. Training on Same number Face

Reviewer Concern: Exclusion of high-face meshes.
Response:
We conducted experiments on meshes with 500-1k faces selected based on maximum RAW sequence length from the original training set. Results demonstrate consistent improvements:

5. Vocab Size Saturation

Reviewer Concern: Does CD plateau with larger vocab?
Response:

• PTME and CD are linearly correlated (r=0.965).
• Saturation occurs when PTME stabilizes (vocab ~8k), aligning with CD trends.

6. Figure 5 Discussion

Reviewer Concern: Insufficient analysis of compression ratio trends.
Response:
We discuss it in Line 263: Larger vocab sizes reduce compression ratio.

7. GPU Specification Correction

Typo Fix:

• Original: "H120 GPU" → Revised: "NVIDIA H20 GPU".

8. Chinese Character Mapping

Reviewer Concern: I'm confused about "mapping integer coordinates (0-127) to atomic Chinese character units."

Response:
This mapping ensures compatibility with SentencePiece’s tokenization workflow while preserving coordinate integrity:

1. Purpose: SentencePiece requires string inputs, but naive coordinate-to-string conversion (e.g., (12, 34) → "12,34") splits digits into separate tokens (["1", "2", ",", "3", "4"]).
2. Solution: We create a bijective mapping where each integer (0-127) corresponds to a unique Chinese character (e.g., 124 → "亜"), ensuring each coordinate occupies one atomic token.

Implementation:

• Tokenizer training: train_vocab.py
• Bidirectional conversion: decimal_to_chinese.py

This prevents token fragmentation and maintains structural consistency for model training.

1. PTME vs Perplexity (PPL)

Reviewer Concern: Lack of theoretical validation for PTME compared to metrics like PPL.
Response:

1. ​Fundamental Difference:

   • PPL requires model training and correlates poorly with final generation quality in our task.
   • Empirical observation: Loss plateaus early (e.g., 0.2 for vocab=8k, 0.1 for vocab=256) while quality improves post 100k steps.
   • Weak Coorelation: calculated Pearson’s (r = -0.407, p = 0.423) between $PPL = e^{\text{Loss}}$ and CD

     Method	Loss (w/o RMC)	CD (w/o RMC)	Loss (w/ RMC)	CD (w/ RMC)
     RAW	0.103	0.326	0.202	0.282
     AMT	0.105	0.219	0.205	0.164
     EDR	0.099	0.198	0.198	0.123

2. ​PTME Advantage:

   • Training-free evaluation of tokenizers.
   • Strong correlation with downstream CD (r=0.965, p=0.0004) as shown in EDR_RMC_PTME_CD.

2. Evaluation on Low-Face Meshes (~200-500 faces)

Reviewer Concern: Missing analysis on meshes with fewer faces.
Response:

• ​Performance Consistency:

  Method	CD (w/o RMC)	CD (w/ RMC)
  RAW	0.171	0.163
  AMT	0.152	0.121
  EDR	​0.144	​0.106

  Most of simple cases (~200-500 faces) show lower Chamfer distance, but can still benefit from the RMC method.

3. Data Selection

Reviewer Concern: Unclear Data Selection.
Response:
We filter low-poly CAD meshes and retain human-crafted meshes with complex geometries. Training data predominantly contains meshes with <5k faces. During training, sequences exceeding 9k tokens after tokenization are discarded (see data_distribution), showing a long-tailed distribution where most meshes have moderate face counts.

Our RMC method enables training on more high-face meshes while reducing average sequence length. Verification scripts: get_data_distribution.py. For the usable mesh number of each method, you can refer to my Section 4.3.

The tokenizer is trained on the first 10k meshes (by UID) from 1M filtered Objaverse samples. Evaluation data strictly follows the same distribution as training.

4. Training Efficiency

Reviewer Concern: Does sequence compression reduce training time?
Response:

• ​Empirical Evidence:
  Initially, to ensure fairness in our experiments, all the methods were trained for the same duration, and we directly compared the performance based on the last checkpoint. From the perspective of training steps, a shorter sequence length undoubtedly leads to a shorter time per iteration. However, this isolated time metric doesn't provide a comprehensive picture of the training efficiency.

In DeepMesh's (DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning) observations (Figure 10), shorter sequences can accelerate the convergence process.

5. Baseline Performance on ~500 Faces

Reviewer Concern: Why RAW/AMT perform poorly compare to their paper?

Response:

• ​Key Limitation:
  • RAW Representation: Our decoder(512M) is smaller compared to the MeshXL(1.3B)architecture.
  • AMT Representation: To isolate merged token contributions, we omitted face embeddings used in the MeshanythingV2 baseline.
• ​Case Selection: While some easy examples were well-generated by all methods, we intentionally selected challenging examples with high surface detail from the 500-face dataset.