CReFT-CAD: Boosting Orthographic Projection Reasoning for CAD via Reinforcement Fine-Tuning

We sincerely thank Reviewer LRf1 for the thoughtful comments and for recognizing the clarity of presentation, the promising results, and the value of comparative analysis. Below we address your main concerns regarding (1) technical contribution beyond data, (2) dataset realism and diversity, and (3) generality and real-world applicability of our method.

W1 difference between the proposed method and other similar methods in CAD

SFT offers stable token-level supervision based on cross-entropy loss, optimizing the likelihood of the next token. While this is effective for mimicking labeled samples, it is insufficient for orthographic reasoning tasks, which require multi-step spatial inference and arithmetic computation—especially for counting parameters and composite calculation parameters.

In contrast, we adopt a curriculum-based RL framework with task-level semantic rewards. Instead of optimizing per-token accuracy, we define goal-directed reward functions for full-task correctness: This method encourages the model to learn actual reasoning chains, not just memorize patterns. More importantly, RL allows effective learning without requiring large-scale high-quality annotations—an essential benefit in the CAD domain, where ground-truth labeling is both costly and domain-expert dependent.

W2&Q1 Diversity

Thank you for raising this important point. First, our choice of bridge pier drawings is motivated by their non-trivial geometric constraints, which inherently offer rich parameter spaces and meaningful semantic structures.

Second, CAD drawings across domains exhibit substantial structural commonality. Tasks such as parameter extraction, component counting, and composite parameter calculations are ubiquitous subtasks in orthographic projection reasoning, regardless of the specific industry or object type. The three reinforcement learning tasks and their associated reward functions that we propose are directly transferable across different CAD domains sharing the orthographic paradigm. As such, although the current dataset may have limitations in geometric variability, it remains a solid foundation for developing robust and generalizable training strategies.

Third, our dataset is a high-quality benchmark with detailed parameter annotations and real-world CAD drawings. It is extremely difficult to obtain, as it typically requires access to proprietary engineering documents and expert-level annotation. Its richness and annotation depth provide a solid foundation for developing and evaluating CAD reasoning task in realistic scenarios.

Finally, since our approach does not rely on mimicking the original data distribution, but instead focuses on inducing true orthographic reasoning capability, we believe the model would maintain strong performance even if trained on other object types or visual styles. In fact, integrating multiple categories into the training process may introduce beneficial diversity, enhancing the model’s reasoning generality through exposure to broader spatial and semantic patterns.

W3&Q3 Limited real-world applicability

We fully acknowledge that achieving high accuracy in real-world orthographic projection reasoning, particularly under out-of-distribution (OOD) conditions, remains a significant and ongoing challenge.

While the current performance on real-world data does not yet meet the stringent requirements of industrial deployment, it is nonetheless encouraging that our model consistently outperforms existing vision–language baselines under such demanding conditions. We believe this offers valuable insights and direction for future research within the community.

As illustrated in Figure 3 of the main paper and Figure 1 of the supplementary material, real-world data in our benchmark presents three key challenges:

Multi-part CAD Assemblies: A single real-world CAD drawing often contains all orthographic views of multiple components within one large drawing. This makes it non-trivial for the model to localize and reason about individual parameters. For visualization clarity, we cropped the most relevant regions for display in the paper, but the model itself must operate on the entire, cluttered canvas.

Occlusions Across Components: Due to the multi-part layout, structural components often occlude each other (as shown in Fig. 3 top view). This creates visual interference that degrades spatial clarity and makes reasoning harder.

Semantic Noise from Irrelevant Annotations: Because the drawing includes annotations for many unrelated parts, the model must disentangle relevant cues from semantically unrelated or misleading labels.

These factors collectively make the real-world subset of TriView2CAD significantly harder than the synthetic benchmark. That said, we view our current gains on this data as a promising first step toward reliable CAD understanding, reflecting clear momentum rather than a shortcoming of the approach.

As presented in Section D of the supplementary material, we provide visualizations and detailed analyses of failure cases encountered during testing on real-world data. These examples highlight specific challenges that remain open. In future work, we plan to further enhance our framework to address these issues more effectively and improve robustness under complex real-world conditions.

Q2 applied to other data and tasks

Based on our extensive experience with CAD-related tasks, we have found the following: The parameter space exhibits strong consistency across different types of CAD objects. Tasks such as parameterization, counting, and composite parameter computation are commonly found across a wide range of engineering domains.

However, CAD drawings are used in many industries, and some sectors may indeed involve specialized domain requirements. In such vertical applications, customized parameter space design may further improve performance.

The three reinforcement learning tasks and their associated reward functions that we propose are directly transferable across different CAD domains sharing the orthographic paradigm.

Therefore, we believe our training paradigm is both robust and scalable, and can be effectively applied to new CAD fields—even those with greater geometric complexity or differing annotation standards.

We thank Reviewer jq8V for carefully reading our paper and for identifying important areas that warrant clarification and improvement. We appreciate your recognition of the value of combining reinforcement learning with post-tuning and the contribution of our TriView2CAD dataset to the CAD reasoning community. Below, we respond to each concern in detail and outline how we plan to address these points in the revised version.

W1&Q2 real-world accuracy

We fully acknowledge that achieving high accuracy in real-world orthographic projection reasoning, particularly under out-of-distribution (OOD) conditions, remains a significant and ongoing challenge.

While the current performance on real-world data does not yet meet the stringent requirements of industrial deployment, it is nonetheless encouraging that our model consistently outperforms existing vision–language baselines under such demanding conditions. We believe this offers valuable insights and direction for future research within the community.

As illustrated in Figure 3 of the main paper and Figure 1 of the supplementary material, real-world data in our benchmark presents three key challenges:

Multi-part CAD Assemblies: A single real-world CAD drawing often contains all orthographic views of multiple components within one large drawing. This makes it non-trivial for the model to localize and reason about individual parameters. For visualization clarity, we cropped the most relevant regions for display in the paper, but the model itself must operate on the entire, cluttered canvas.

Occlusions Across Components: Due to the multi-part layout, structural components often occlude each other (as shown in Fig. 3 top view). This creates visual interference that degrades spatial clarity and makes reasoning harder.

Semantic Noise from Irrelevant Annotations: Because the drawing includes annotations for many unrelated parts, the model must disentangle relevant cues from semantically unrelated or misleading labels.

These factors collectively make the real-world subset of TriView2CAD significantly harder than the synthetic benchmark. That said, we view our current gains on this data as a promising first step toward reliable CAD understanding, reflecting clear momentum rather than a shortcoming of the approach.

W2&Q3 Diversity

Thank you for raising this important point. First, our choice of bridge pier drawings is motivated by their non-trivial geometric constraints, which inherently offer rich parameter spaces and meaningful semantic structures.

Second, CAD drawings across domains exhibit substantial structural commonality. Tasks such as parameter extraction, component counting, and composite parameter calculations are ubiquitous subtasks in orthographic projection reasoning, regardless of the specific industry or object type. The three reinforcement learning tasks and their associated reward functions that we propose are directly transferable across different CAD domains sharing the orthographic paradigm. As such, although the current dataset may have limitations in geometric variability, it remains a solid foundation for developing robust and generalizable training strategies.

Third, our dataset is a high-quality benchmark with detailed parameter annotations and real-world CAD drawings. It is extremely difficult to obtain, as it typically requires access to proprietary engineering documents and expert-level annotation. Its richness and annotation depth provide a solid foundation for developing and evaluating CAD reasoning task in realistic scenarios.

Finally, since our approach does not rely on mimicking the original data distribution, but instead focuses on inducing true orthographic reasoning capability, we believe the model would maintain strong performance even if trained on other object types or visual styles. In fact, integrating multiple categories into the training process may introduce beneficial diversity, enhancing the model’s reasoning generality through exposure to broader spatial and semantic patterns.

It is also worth noting that domain specificity is a common design choice in this field. For example, in the recent work: CAD2Program[1], the entire training and evaluation pipeline is conducted on a single CAD domain—kitchen cabinet drawings. Their goal, which aligns with ours, is to test large-model reasoning capabilities under domain-specific constraints. In contrast, our contribution lies in introducing a reinforcement-learning-based, multi-task curriculum that strengthens generalization and structured reasoning.

[1]Wang et al., From 2d cad drawings to 3d parametric models: A vision-language approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 7961–7969.

Q1 How sensitive are results to the chosen reward weights?

Thank you for raising this insightful point. We are happy to clarify how reward functions are applied in our framework.

First, as noted in Equations (1)–(3) of the main paper, each sample in our dataset corresponds to a specific task type, and hence at training time, only one reward function is active per sample. Thus, there is no simultaneous interaction or weighting between different reward functions, and no reward balancing across task types is needed. Regarding reward value sensitivity (particularly in Eq. 2 and Eq. 3), we did not perform exhaustive ablations on individual reward magnitudes for the following reasons:

Relative reward differences—not absolute values—drive optimization. The design goal of our reward functions is to distinguish high-quality from suboptimal outputs. As long as there are meaningful gaps in reward assignment, the policy gradient update encourages exploration along paths of relatively greater return. This mostly influences convergence speed, rather than final policy quality, especially after large-scale sampling.

During RL updates, we compute advantage functions using normalized returns, which further attenuates the effect of raw reward magnitudes. This helps ensure training stability and makes performance less sensitive to specific numeric values. That said, we fully agree that studying reward value configurations is valuable. We are preparing an ablation experiment that tests representative reward schemes, including variations in the partial-credit structure for Eq. 2 and the difficulty-tiered scaling in Eq. 3. We will include these results in the revised version, and we appreciate your suggestion to further ground the method in principled analysis.

We are writing to kindly check if you have any remaining questions, concerns, or further suggestions. If there are any aspects that require additional clarification or discussion, we would be more than happy to engage in further communication to address them promptly.

Dear jq8V (the authors are included in this one),

Please read the authors' answer on the dataset diversity. Given that the majority of reviewers lean towards acceptance, you would need to provide a more detailed justification of your rating / assessment. Please also provide an answer to the rest of the authors' rebuttal points (real-world accuracy, sensitivity of the method to the reward weights). Are their answers satisfying these concerns or not?

Thank you AC

We sincerely thank Reviewer Kh3L for the thorough and constructive review. We are grateful for your positive assessment of our motivation, technical design, and empirical contributions. We especially appreciate your recognition of the industrial value of orthographic projection reasoning, the cognitive structure of our curriculum-based RL method, and the strong performance of our model on the TriView2CAD benchmark. Below, we respond in detail to the two main concerns and two insightful questions you raised.

W1&Q1 CAD category limited

Our selection of prefabricated bridge pier drawings is grounded in their geometric complexity. Despite belonging to a specific subdomain, these structures exhibit rich parametric relationships and semantic constraints that pose meaningful challenges for orthographic projection reasoning.

Importantly, many CAD tasks across diverse domains frequently involve similar cognitive subtasks, such as parameter extraction, component enumeration, and multi-view composite parameter inference. The curriculum tasks we have designed for reinforcement learning are not tailored to a specific object type, but rather capture generalizable reasoning patterns that are widely applicable to other CAD categories. Since these tasks are grounded in fundamental principles of orthographic projection reasoning, they can be readily transferred and reused across domains that follow similar design logic.

We also emphasize that, our dataset is a high-quality benchmark with detailed parameter annotations and real-world CAD drawings. It is extremely difficult to obtain, as it typically requires access to proprietary engineering documents and expert-level annotation. Its richness and annotation depth provide a solid foundation for developing and evaluating CAD reasoning task in realistic scenarios.

Lastly, the core strength of our method lies in learning the reasoning process itself, rather than fitting to any fixed data distribution. This property makes the approach generalizable to other CAD categories. We further posit that expanding the dataset to include multiple CAD domains will likely further enhance model robustness by encouraging the learning of abstract and transferable inference patterns.

W2 reasoning ability

From a quantitative analysis perspective, we have initially conducted experiments using the same training data but applied only a supervised fine-tuning (SFT) strategy. Under evaluation settings analogous to those in Table 1 of the main paper, this SFT-only baseline achieved an overall accuracy of 80.3%, with 85.4% on recognition parameters, 97.1% on counting parameters, and 66.8%on composite calculation parameters. In contrast, our reinforcement learning–based training method achieved 84.03% overall accuracy, with 87.1% on recognition parameters, 97.3% on counting parameters, and a significantly improved 74.3% on composite calculation parameters. This comparison clearly demonstrates that reinforcement learning brings substantial gains, particularly for tasks requiring multi-step inference and reasoning, such as composite parameter calculation. These results provide strong empirical evidence that our curriculum-based RL strategy effectively endows the model with enhanced reasoning capabilities beyond what is achievable through pattern imitation via SFT alone.

From a qualitative analysis perspective, traditional supervised fine-tuning (SFT) offers token-level supervision via cross-entropy loss, optimizing the next-token likelihood. While this approach enables effective pattern replication from annotated samples, it typically lacks the capacity to instill genuine reasoning. In contrast, our framework applies curriculum-based reinforcement learning, with task-level, semantically meaningful rewards. Rather than optimizing for local token-level correctness, we design goal-driven reward functions that evaluate whether the model completes the entire task correctly. This structure encourages the learning of full reasoning chains rather than superficial pattern memorization. Through this setup, the model acquires multi-step inference abilities necessary for solving complex CAD tasks like parameter counting, component relationships, and dimensional composition.

Q2 Failure cases

As shown in Appendix section D, we provide several representative failure cases along with analysis.

We are writing to kindly check if you have any remaining questions, concerns, or further suggestions. If there are any aspects that require additional clarification or discussion, we would be more than happy to engage in further communication to address them promptly.

We sincerely thank Reviewer M7s3 for the constructive feedback and for acknowledging the significance of both our CReFT-CAD framework and the TriView2CAD benchmark. We appreciate your recognition of the real-world motivation, the alignment to CAD workflow stages, and the model’s superior performance on complex and out-of-distribution (OOD) tasks. Below we address your thoughtful critiques regarding prompt engineering analysis, dataset diversity, and comparisons to industrial-grade models.

W1 Analysis of Prompt Engineering

As detailed in Lines 300–305 of the main paper and further elaborated in Appendix B.3, we conducted a systematic study of prompt engineering from two perspectives: (1) the presence or absence of reasoning guidance, and (2) the effect of different input formatting styles, including attribute explanations and structural text augmentation. While we observe that reasoning-guided prompts lead to performance improvements across all models, CReFT-CAD consistently outperforms all baseline VLMs under identical prompt templates. This strongly suggests that the gains stem primarily from the enhanced reasoning ability acquired during RL fine-tuning, not merely from the prompt structure.

That said, we fully acknowledge the critical role of prompt design in VLM adaptation. Our experiments are meant to shed light on this often-understudied factor by providing comparative insights into how prompting strategies affect both our method and existing VLM baselines. We hope these results offer meaningful guidance to the community.

Importantly, the prompt templates are not hard-coded or universal—in practice, reasoning guidance provided by domain experts varies substantially across CAD categories. Moreover, we found that adding attribute-level textual explanations, such as descriptions of component functions or constraints, significantly enhances performance on orthographic-view reasoning tasks. The chosen input modality (text, image, etc.) should be seen as a flexible reference point rather than a rigid constraint.

W2 &Q1 Insufficient Analysis of Dataset Diversity

Our selection of prefabricated bridge pier drawings is grounded in their geometric complexity. Despite belonging to a specific subdomain, these structures exhibit rich parametric relationships and semantic constraints that pose meaningful challenges for orthographic projection reasoning.

Importantly, many CAD tasks across diverse domains frequently involve similar cognitive subtasks, such as parameter extraction, component enumeration, and multi-view composite parameter inference.

The curriculum tasks we designed for reinforcement learning are not tailored to a specific object type, but rather capture generalizable reasoning patterns that are widely applicable to other CAD categories. Since these tasks are grounded in fundamental principles of orthographic projection reasoning, they can be readily transferred and reused across domains that follow similar design logic.

We also emphasize that, our dataset is a high-quality benchmark with detailed parameter annotations and real-world CAD drawings. It is extremely difficult to obtain, as it typically requires access to proprietary engineering documents and expert-level annotation. Its richness and annotation depth provide a solid foundation for developing and evaluating CAD reasoning task in realistic scenarios.

Lastly, the core strength of our method lies in learning the reasoning process itself, rather than fitting to any fixed data distribution. This property makes the approach generalizable to other CAD categories. We further posit that expanding the dataset to include multiple CAD domains will likely further enhance model robustness by encouraging the learning of abstract and transferable inference patterns.

W3&Q2Evaluation of Industrial-Level VLMs.

This is an excellent and important suggestion. We have extended our evaluation to include several industry-level VLMs, specifically Gemini 2.5 Pro, Claude 4 Vision, and GPT-4o. The results of these evaluations have been compiled and presented in the same format as Table 1 in the main paper, for consistency and comparison. We will integrate these results into the revised version of the paper, along with additional analysis of each model’s strengths and limitations on orthographic projection reasoning tasks.

| Gemini 2.5 Pro | 24.23 | 30.09 | 29.39 | 25.55 | 25.68 | 30.61 | 32.30 | 35.88 |

| Claude 4 Vision | 23.47 | 29.67 | 30.04 | 25.58 | 24.94 | 33.00 | 34.13 | 36.32 |

| GPT-4o               | 26.06 | 31.56 | 32.28 | 27.52 | 27.79 | 34.17 | 39.71 | 38.56 |

We are writing to kindly check if you have any remaining questions, concerns, or further suggestions. If there are any aspects that require additional clarification or discussion, we would be more than happy to engage in further communication to address them promptly.

Thanks for the authors' reply. Most of my concerns have been addressed. I decide to keep my rating.

We sincerely thank Reviewer 8yVX for the thoughtful evaluation and for recognizing the technical soundness, originality, and relevance of our work. We appreciate your positive remarks on the importance of orthographic reasoning in CAD workflows, the value of our large-scale dataset, and the strong quantitative results. Below, we address your comments in detail.

W1&Q1: Why reinforcement learning is required

Orthographic projection reasoning presents fundamentally different challenges compared to conventional CAD tasks such as primitive detection or parametric annotation. While the latter focus on identifying individual geometric entities and retrieving associated dimensions, orthographic reasoning requires a holistic semantic understanding of multi-view technical drawings. This includes tasks such as cross-view correspondence, and performing composite parameter calculations that span multiple projections and demand multi-step spatial and logical inference.

Such complexity cannot be effectively addressed through supervised learning alone, which primarily drives models to imitate training distributions rather than truly reason. Specifically, Supervised Fine-Tuning (SFT) provides stable token-level optimization via cross-entropy loss, which is useful for formatting and mimicry, but inherently limited in fostering the higher-order reasoning skills required for orthographic tasks.

To overcome this limitation, we introduce a curriculum-based Reinforcement Learning (RL) framework with task-level, semantically grounded reward functions. Rather than optimizing token-level accuracy, our method defines goal-directed rewards based on full-task correctness, covering binary decisions, multiple-choice selection, and chain-of-thought parameterization. This approach enables the model to move beyond surface-level pattern recognition by cultivating structured reasoning chains tailored to the logic of orthographic projection. Moreover, by optimizing directly for task-level correctness through semantically meaningful rewards, the method reduces reliance on large-scale, high-quality annotations, which are often expensive to obtain and require significant domain expertise in the CAD field.

By combining curriculum-driven RL with SFT post-tuning, our framework balances reasoning capability with instruction-following fluency, making it well-suited for complex, industrial-grade CAD understanding.

W1: Clarifying Data-Training-Loss Connections

As described in Section 3.1 of the main paper, we have detailed the construction of our dataset, including the synthetic data generation pipeline. As described in Lines 192–220 of the main paper, during the reinforcement learning phase, we have constructed 160,000 image–instruction pairs from the synthetic subset of the TriView2CAD dataset. Each prompt contains an orthographic projection view alongside its complete set of 15 parametric key–value pairs. Based on our three designed learning tasks, we apply corresponding reward functions to compute relative advantages for policy optimization. This structured curriculum enables the model to progressively activate and refine its orthographic reasoning capabilities, rather than relying on superficial feature extraction. In the Supervised Post-Tuning phase, we have further refined the model using 20,000 instruction–response pairs, aiming to strengthen its instruction-following abilities in a more controlled and aligned setting. Beyond generic response formatting, this stage is specifically designed to equip the model with capabilities tailored to real-world CAD tasks. Based on our deep understanding of practical industrial workflows, we have structured the supervision to support four essential task types: producing a complete parameter list from multi-view drawings, validating parameter correctness against reference values, matching corresponding elements across orthographic views, and counting components embedded within complex assemblies. These tasks closely reflect operational needs in engineering design, simulation, and quality assurance, thus ensuring the model’s outputs are not only syntactically correct but also functionally relevant and deployable in real CAD scenarios.

W2&Q2 Limited Task Domain

First, our choice of bridge pier drawings is motivated by their non-trivial geometric constraints, which inherently offer rich parameter spaces and meaningful semantic structures.

Second, CAD drawings across domains exhibit substantial structural commonality. Tasks such as parameter extraction, component counting, and composite parameter calculations are ubiquitous subtasks in orthographic projection reasoning, regardless of the specific industry or object type. The three reinforcement learning tasks and their associated reward functions that we propose are directly transferable across different CAD domains sharing the orthographic paradigm. As such, although the current dataset may have limitations in geometric variability, it remains a solid foundation for developing robust and generalizable training strategies.

Third, our dataset is a high-quality benchmark with detailed parameter annotations and real-world CAD drawings. It is extremely difficult to obtain, as it typically requires access to proprietary engineering documents and expert-level annotation. Its richness and annotation depth provide a solid foundation for developing and evaluating CAD reasoning task in realistic scenarios.

Finally, since our approach does not rely on mimicking the original data distribution, but instead focuses on inducing true orthographic reasoning capability, we believe the model would maintain strong performance even if trained on other object types or visual styles. In fact, integrating multiple categories into the training process may introduce beneficial diversity, enhancing the model’s reasoning generality through exposure to broader spatial and semantic patterns.

W3&Q3:Downstream applications

As shown in Lines 527–537 of the main paper, we have conducted evaluations on real-world data under out-of-distribution (OOD) settings. Even in these challenging scenarios, our method demonstrate substantial performance gains, highlighting its robustness and generalization capability beyond the synthetic training distribution.

As presented in Section D of the supplementary material, we have provided visualizations and detailed analyses of failure cases encountered during testing on real-world data. These examples highlight specific challenges that remain open. In future work, we plan to further enhance our framework to address these issues more effectively and improve robustness under complex real-world conditions.

We are writing to kindly check if you have any remaining questions, concerns, or further suggestions. If there are any aspects that require additional clarification or discussion, we would be more than happy to engage in further communication to address them promptly.