CPO: Condition Preference Optimization for Controllable Image Generation

Before our response, we would like to explain a little about our intuition so that the reviewer can have a clearer understanding. Simply speaking, we construct a preference pair $c^w$ and $c^l$ and let the model learn that $c^w$ aligns better to the input image than $c^l$, where the input image is from the real-world dataset.

Question: Despite the novelty of the approach, it raises questions regarding the theoretical equivalence between the proposed CPO and the original DPO. As stated in the introduction, while the relationship between the control and the generated images may be bidirectional, directly switching to CPO changes the data distribution compared to DPO. If CPO is not strictly equivalent, some clarification or refinement in the introduction would be helpful to better explain the motivation and implications.

Answer:

1. DPO and CPO are similar. They cannot be strictly equivalent due to their difference in formulation and intuition. They are similar in training loss formulation. In plain words, DPO does not want to sample worse images (i.e., keep the sampling trajectory far from the trajectory of worse images). This raises concerns mentioned in our intro, such as the inability to discern the condition from the image to perform meaningful preference. CPO deals with the condition instead. It tries to keep the sampling trajectory far from the less controllable trajectory. Therefore, it functions similarly to DPO. However, they are different in the way the CPO deals with the condition part (C) of a probability distribution $P(X|C)$, but DPO deals with the variable part (X).
2. We did not mention the data distribution change in CPO. We did mention that the distribution mismatch in the DDPM forward and sampling process causes some limitations for ControlNet++.
3. If the reviewer has concerns about the generation data distribution, then our method aligns better to real-world images than DPO. DPO requires high-quality model-generated data for training, and therefore, its training distribution diverges from the real-world image. However, we still keep the real-world images, but we contrast conditions. This may raise a concern about conditions distribution (the reviewer’s next question). But the condition distribution still aligns. For Canny, Lineart, Depth, and HED, conditions are extracted with the same mode for both winning and losing conditions, and therefore they are in the same distribution. For pose and segmentation, conditions are drawn with a pre-defined visualization method. For example, in segmentation, we map labels (argmax of output distribution) to predefined colors. In pose, we draw lines with predefined colors between keypoints. This process can also be understood as coloring models that map labels/keypoints, and the coloring models are identical for the winning (ground truth) condition and the losing condition. Therefore, the winning and losing conditions are in the same distribution because they are mapped to the image (pixel) space by the same model. Importantly, in the same distribution does not mean that the losing conditions extracted from generated images are correct. They will be incorrect due to the non-perfect controllability, which is what we desire to construct the losing condition $c^l$.

Question: In practice, when applying perturbations, how can we ensure that the perturbed control signals extracted from generated images share a similar distribution with real control signals? The paper mentions that the perturbations are derived from the generated images—but are they of comparable quality to the original control signals? Why not directly perturb the original control inputs instead?

Answer: We include the response for the distribution in our previous answer. For the quality of the control signal, please see our previous response. We intentionally make the extracted control signals from the generated images worse to perform contrastive learning so that the model knows the winning signal is better than the losing one. For perturbing the original control inputs, we did think of doing such a processing at the beginning. However, the issue is that we need to carefully consider the richness of perturbation. Here are some problems that we encountered and found too hard to solve. Take the pose as an example, the direction and distance that we should move each keypoint is hard to determine. The perturbed keypoints look reasonable, so that the model can learn meaningfully from telling winning conditions are better? For edges, how can we perturb edges? Direct shifting to left/right does not change the shape of edges. Given these challenges, we came up with our method. The perturbation process will be automatic, and the perturbed conditions still seem reasonable.

Question: The authors should consider conducting a user study to better evaluate the results. Human judgment can provide a more reliable assessment of how well the generated images align with the control inputs, compared to simply comparing the extracted control signals from generated images.

Answer: We conduct a user study on hed and depth with controllability. We randomly select 25 groups of images for each task and ask 15 annotators to annotate.

Here are the results in win-rate(%):

We see that we outperform ControlNet++ by 10 percentage points in HED and 6 percentage points in Depth.

Question: While the method shows good controllability scores, it sometimes yields lower FID and CLIP scores. It would be helpful if the authors could discuss this trade-off and provide insights into how controllability affects image quality and alignment with the text.

Answers: The only time the FID and CLIP are apparently lower is on HumanArt. We mentioned it in lines 270-273 in our main paper. The drop in FID on this dataset is not due to lower quality but to a data distribution shift. Our model was fine-tuned on the realistic COCO-Pose dataset, whereas HumanArt contains numerous cartoons and artistic paintings. Fine-tuning shifted our model's output distribution closer to real-world photos, naturally increasing the FID distance to the artistic style of HumanArt. This demonstrates that our model successfully learned the target data distribution. A similar pattern was observed in GLIGEN's results, as it was also trained on COCO-Pose only. CLIP score is impacted in a similar way. We will add examples to showcase it for a better explanation in our final version.

For any other datasets, the changes in FID/CLIP are ignorable due to the inherent randomness of generation models. If you evaluate the dataset twice with different environments, machines, random seeds, etc, results will be slightly different. Therefore, it is inconclusive for the FID/CLIP on other datasets.

However, we do observe that, if we do not use regularization, the controllability will increase further with worse FID and CLIP, and we do not visually find a strong difference in image quality (we kindly refer the reviewer to Figure 7 in our appendix). One reason is that the model tries to overfit on controllability, and some unobservable features might be captured by FID/CLIP, which is the reason we use original training to regularize.

Dear Reviewer YcYa,

We appreciate your constructive comments and are trying our best to improve our finalized version. To ensure we have successfully addressed your concern, we would like to know if the explanation on the difference between CPO and DPO sounds reasonable to you. If you have any other concerns, we are more than happy to address them promptly.

Best, 17633 Authors

We appreciate the reviewer for the efforts and comments. The reviewer's concerns are addressed. After reading through other reviewers' comments, the reviewer also believes that our responses to other reviewers' comments address their concerns.

The quantitative comparisons between CPO and DPO are shown in Figure 6. We plot the mAP and FID values under different training steps of CPO and DPO. Our method is plotted on the upper left side of DPO, showing superior performance. For example, when the FID value of the evaluated model is around 39.5, our method achieves mAP 87.98% but DPO gets 86.90%.

We are still actively working on improving our final version and addressing existing concerns.

Best,

17633 Authors

Question: For the condition pair, how to control the differences? If the difference is very large, it might be invalid. The authors should detail the process of constructing the pair, such as pose, canny, depth, segmentation ,etc.

Answer: We currently do not have a process to control the difference. The reviewer’s comments are helpful and may possibly provide a way to improve our work. Our pair construction process is described in lines 195-200.

1. We first obtain the ground truth (segmentation/pose) or oracle (canny, lineart, hed, depth) as $c^w$.
2. Then, we generate an image $I$ with ControlNet++ controlled by $c^w$.
3. Finally, we use the condition detector to extract $c^l$ from $I$.

For example, in the canny edge task, the condition detector is just a canny edge detector.

Since ControlNet++ achieves strong controllability, we assumed the difference would not be large. However, this is more about curating a better dataset, and we will follow this direction and try to improve our dataset by adding some difference-control mechanisms.

Question: The backbone is still SD1.5. To better validate its effectiveness, please use SDXL, even SD3, and FLUX.

Answer: We are actively working on FLUX training. However, since FLUX requires $1024 \times 1024$ resolution images as inputs, we collected a high-resolution dataset first. We filtered out 3M images from the LIAON-Aesthetic-5+ dataset with an aesthetic score higher than 5.6, then we captioned all images with qwen-2.5-vl-32b for a high-quality text description. We are currently training FLUX-ControlNet, but FLUX takes weeks to train with 64 A100 80G gpus, and we only finished the first 5K steps of training FLUX-ControlNet in lineart task, but we do not have results now. It will take one more week to finish FLUX-ControlNet, if everything works as expected. Then we can proceed to CPO. We will include available results in our final version.

Dear Reviewer yv4M,

We appreciate your constructive comments and are trying our best to improve our finalized version. To ensure we have successfully addressed your concern, we would like to know if the details provided address your concerns. We are still training the FLUX ControlNet for Lineart, which is currently 6000+ steps. We inspect the generation quality at the 5000th step visually and find it on the right track (i.e., quality is good). However, it still needs more steps to converge. We should be able to provide more results in the final version. If you have any other concerns, we are more than happy to address them promptly.

Best, 17633 Authors

Dear Reviewer yv4M,

We have finished the experiments on selecting the best of five $c^l$ on ADE20K. Our original method achieves MIOU 44.81. With the new dataset that we control the difference from being too large, the MIOU is 44.83. Since the generation results are subject to randomness, such a difference is inconclusive. Therefore, our algorithm successfully overcomes such a problem by itself (see previous response on Equation (10)) without needing to control the difference between $c^w$ and $c^l$. We thank the reviewer for comments and follow-ups, and are trying our best to improve our final version. We hope that additional experiments can address the reviewer's concern.

We thank the reviewer for the constructive comments. The reviewer does agree with our novelty.

Question 1: On detector sensitivity and error propagation. The reviewer noted that our method uses a condition detector that is not perfect, which could lead to error propagation and the rejection of good images.

Answer: Thank you for this question. Our method fundamentally avoids this issue for the following reasons:

1. We Contrast "Conditions," Not "Images": Our method (CPO) contrasts a "winning condition" with a "losing condition," not the images themselves. Both conditions are applied to the same input image. The winning condition is always the ground truth or an oracle. Therefore, the losing condition is, by definition, less aligned with the input image than the winning one. This mechanism prevents good conditions from being mislabeled as losers, thus eliminating the risk of error propagation.
2. Robustness to Different Detectors: To further validate this, we experimented with different detectors for segmentation (ADE20K), pose (COCOPose), and depth (MultiGen20M) tasks. As shown in the table below, the performance shows almost no difference, demonstrating our method's strong robustness to the choice of detector.
3. Yes, we only sample one image to obtain the losing condition.

Question 2: Is the improvement solely due to longer training, not CPO itself?

Answer: The performance improvement stems entirely from CPO, not from longer training. We demonstrate this with the following experiments:

1. Extended Training of the Baseline: We trained the baseline model, ControlNet++, for the same training step as CPO. As shown below, it began to overfit on ADE20K (performance degraded) and showed negligible improvement on other tasks. In contrast, our CPO method consistently improves controllability without sacrificing FID. | Task | ControlNet++| ControlNet++ more training | ours | |----------|------------|------------|------------| | Seg (ADE20K) | 43.64/30.24 | 40.91/32.57 | 44.81/30.30 | | Canny| 38.03/20.16 | 39.01/22.50 | 39.19/19.68 | | HED | 80.97/15.01 | 80.81/14.92 | 82.01/13.35 | | Lineart | 83.99/13.88 | 84.04/13.80 | 84.31/13.84 | | Depth | 28.32/16.66 | 28.15/16.78 | 27.49/16.64 |

2. Applying CPO on a Fully Converged Model: On the COCOStuff dataset, we train the baseline model (ControlNet++) until full convergence (5000 more steps) because we observe the under-convergence of ControlNet++. Further training beyond 5000 steps does not improve controllability without sacrificing FID a lot. The performance is approximately 90% of Oracle, which is extremely challenging to further improve. Even so, our CPO still achieved a noticeable improvement, proving its effectiveness beyond simple training extension. | Task | Fully converged ControlNet++| ours | | --- | --- | --- | | Seg(COCOStuff) | 38.02/19.24 | 38.44/19.20 |

Question 3: Insignificant performance improvement and the effect of CFG-scale.

Answer:

1. On the Significance of Improvement: The improvement might seem modest for two main reasons:

• The baseline is already very strong: ControlNet++ achieves 80-84% of oracle performance on most tasks, leaving limited room for improvement.
• Balancing Controllability and FID: Our goal is to enhance controllability while maintaining or improving image quality (FID). As shown in the previous answer, longer training of ConrolNet++ either fails to maintain FID (canny) or improve further on controllability (other tasks), whereas CPO succeeds in achieving better controllability.

2. On CFG-scale: It is important to clarify that the CFG-scale is an inference-time technique, unrelated to model training or its parameters. It adjusts the strength of conditional guidance using the formula $pred = (c+1) \times pred_{cond} - c \times pred_{uncond}$. $pred_{cond}$ means the model output with full text input, and $pred_{uncond}$ means the model output with empty text input. Different CFG scales will thus naturally alter the output and the resulting evaluation scores. This is a standard practice for diffusion models and not a unique aspect of our method.

Question 4: Effect on background regions.

Answer:

1. Most Tasks Lack a Distinct "Background": In tasks like edge detection (Canny, HED, Lineart) and depth mapping, there is no strict "background" region. Every part of the image contains relevant conditional information. For example, the black region of edge detection tasks means no edges instead of no requirement.

2. Background Quality is Not Degraded: For tasks with backgrounds, such as segmentation and pose estimation, Figures 8 and 9 in our appendix clearly show that the quality of background regions is not degraded.

3. On the FID Score for HumanArt: The drop in FID on this dataset is not due to lower quality but to a data distribution shift. Our model was fine-tuned on the realistic COCO-Pose dataset, whereas HumanArt contains numerous cartoons and artistic paintings. Fine-tuning shifted our model's output distribution closer to real-world photos, naturally increasing the FID distance to the artistic style of HumanArt. This demonstrates that our model successfully learned the target data distribution. A similar pattern was observed in GLIGEN's results, as it was also trained on COCO-Pose only. We will add examples to showcase it for a better explanation in our final version.

Question 5: Potential interference when combining CPO and DPO.

Answer: This is an interesting direction, but combining CPO with DPO presents practical challenges.

1. Challenges in Data Curation: CPO leverages the model's inherent imperfection in controllability, requiring only one generated image to obtain an effective "losing condition." In contrast, to ensure a "winning image" is superior in both quality and controllability, DPO would require generating and filtering a large number of samples, which is computationally prohibitive. Importantly, DPO cannot be performed in quality only since the alignment to control signals cannot be identical. However, CPO is performed for controllability only.

2. Negative Interference: Our experiments confirm this. We split our training steps equally for CPO and DPO. As shown in the table below, combining DPO and CPO, regardless of the order, results in performance inferior to using CPO alone, especially in terms of FID. The reason is that we apply a worse method on top of a better one. Figure 6 in our main paper shows that DPO performs worse than our method.

We do think this is an interesting idea for future improvement and research, and appreciate reviewers’ comments.

Dear Reviewer Q7TW,

We appreciate your constructive comments and are trying our best to improve our finalized version. To ensure we have successfully addressed your concern, we would like to know if the experiments on DPO followed by CPO (or vice versa) were conducted reasonably to address your concerns. If you have any other concerns, we are more than happy to address them promptly.

Best, 17633 Authors

We thank the reviewer for the additional comments. Here are additional responses to the reviewer's comments.

A bad image might result in a good condition or vice-versa.

Please correct if we understand incorrectly. The reviewer may mean that the real ground truth condition $c^{gt}$ is not available. The method that we employ (detecting conditions from the ground truth image as the oracle $c^w$) to get $c^w$ and $c^l$ may result in a problem where $c^l$ is closer to the real ground truth than $c^w$. The reviewer points to an existing concern in DPO: noisy preference. Simply speaking, there may be some low-quality images that are labeled to be better than high-quality ones in the DPO dataset, and it is reasonable to have similar concern in CPO. We are more than willing to improve our method following the concern of noisy preference. However, we would like to let the reviewer know that this is not a unique concern for our method but also for DPO. Our goal is not to address such a problem (this is a different research problem). There are existing approaches that we can apply on top of our method to see if there is any improvement [1,2]. If we find these methods are effective, we will update them in our final version. However, we would like to claim that even if there might be a noisy preference problem, our method is still effective:

First, our quantitative results (Table 1 in the main paper) and qualitative comparisons (Figures 4 and 5 in the main paper) show that it is effective.

Second, our user study shows our method outperforms ControlNet++ in a subjective assessment. We include a user study in HED and Depth in the following table. Therefore, even if there is such a problem, our method still outperforms our base model.

The performance improvement is a concern because CPO is expensive.

We kindly refer the reviewer to our previous response. When ControlNet++ cannot be further improved by increasing training steps, our method can further improve it. Moreover, when ControlNet++ achieves 80-90% of oracle results, any further improvement is challenging, but our improvement is still consistent.

ControlNet++ might also provide better results for a different CFG scale.

We include the evaluation results under the CFG scale 4.0 here. Under CFG scale 4.0, our method still outperforms ControlNet++. We will include more thorough evaluations with the CFG ranges, similar to our ablation studies, in our final version and visualize graphically. Depth metric (RMSE) is the lower the better, and remaining tasks are the higher the better

We are actively evaluating with CFG scales 3 and 1.5 for ControlNet++ and will update accordingly. Since applying techniques regarding noisy preferences requires re-training, we will try to include them in the final version if they are effective.

If the reviewer still has unresolved concerns, we are more than willing to address them.

[1] Chowdhury, S. R., Kini, A., & Natarajan, N. (2024). Provably robust dpo: Aligning language models with noisy feedback. arXiv preprint arXiv:2403.00409.

[2] Zhang, L., Liu, C., Xu, C., Hu, K., Luo, D., Wang, C., ... & Yao, Y. (2025). When Preferences Diverge: Aligning Diffusion Models with Minority-Aware Adaptive DPO. arXiv preprint arXiv:2503.16921.

Dear Reviewer Q7TW,

We have finished the evaluation of ControlNet++ on CFG scale 3 and 1.5, shown in the tables below. We can find that under all CFG scales, our method outperforms ControlNet++.

CFG = 3

CFG = 1.5

Moreover, we also apply the Robust DPO method (an existing effective solution for the noisy preference problem, which is the reviewer's concern about bad images being labeled as good) on the depth task to deal with the noisy preferences issue. The reason we choose a depth task is that this is the task where noisy preference can mostly occur. The reason is that there is a ground truth depth by definition, but we cannot obtain the depth. For edges, there is no well-defined ground truth.

Our method achieves RMSE 27.49, and after we apply Robust DPO to deal with noisy preferences, the result becomes 27.70. This implies that there might not be a noisy preference problem because the solution does not improve the performance.

We thank the reviewer for comments and follow-ups, and are trying our best to improve our final version. We hope that additional experiments can address the reviewer's concern.

We thank the reviewer for providing constructive comments. The reviewer appreciates our motivation, writing, and results.

Questions: The number of images for qualitative evaluation is too limited, without comparative assessment through a user study.

Answer: We’d like to let the reviewer know that we follow the practice of ControlNet++ and ControlAR to provide a qualitative Comparison (one group of visual comparison for each task (in the main paper), and several visual examples without comparison (supplementary Figure 8 - 13)). We are, of course, willing to provide additional qualitative results. However, this year Neurips does not allow us to upload a PDF with figures, nor the links. We will include more in the final version.

For the user study, we conducted a user study on controllability on hed and depth. We randomly select 25 groups of images for each task and ask 15 annotators to annotate.

Here are the results in win rate (%):

In hed, we outperform ControlNet++ by 10 percentage points and 6 percentage points in Depth.

Questions: Is the improvement related to the correlation between the detector model for data generation and evaluation? If a different detector model is used during evaluation, will the same improvement still be achieved? Could it be possible that, visually, the pose of a person in the image appears incorrect, but YOLO-11x-Pose considers the pose more consistent? For the Pose mAP evaluation in Table 1, could another pose estimation model be used for assessment?

Answer: We do understand the reviewer’s concern that the improvement might be because we overfit to the pose detector. We provide two types of experiments to resolve the concern. First, our experiments show that changing the detector during dataset curation (i.e., the detector to construct $c^w$ and $c^l$) almost does not change the results. See the table below

The differences are primarily due to the randomness of the generative model, and we can conclude that there is almost no difference.

We also include mAP results where the dataset is curated with YOLO11x-pose but evaluated with a different model, mmpose (HRNet), below.

We can see that with HRNet, CPO still outperforms base models. HRNet tends to result in better evaluation results than YOLO11x-Pose. We will change the evaluation model to HRNet and update the entire table in our final version to resolve the reviewer’s concern.

Question: The comparison with DPO is not sufficiently thorough. Could you provide more qualitative results? Could you also provide more qualitative comparative results?

Answer: This year, Neurips does not allow us to provide any PDF or links. Therefore, we are not able to provide during the rebuttal. However, our structure follows previous SOTA ControlNet++ and ControlAR to provide one set of qualitative comparisons (in the main paper) and more visualizations with our models only (in the appendix). We will, of course, provide more qualitative comparisons in our final version, including CPO vs DPO and ours vs others.

Dear Reviewer HcdE,

We appreciate your constructive comments and are trying our best to improve our finalized version. To ensure we have successfully addressed your concern, we would like to know if the user study and changing detection models were conducted reasonably to address your concerns. If you have any other concerns, we are more than happy to address them promptly.

Best, 17633 Authors