Foundation Cures Personalization: Improving Personalized Models’ Prompt Consistency via Hidden Foundation Knowledge

Thanks for your constructive suggestions and positive comments on our insightful finding, strong generalization ability and promising results. We will raise the following points to address all your concerns.

1. More Diverse Segmentation Models and Attributes

Thank you for raising this point. First, our method can leverage any attribute segmentation model to handle different types of features. In Section A.2.4 of the supplementary materials, we discuss the performance of general semantic segmentation models (e.g., SAM) within our framework and address attributes that fall outside facial features. It is important to note that since most current encoder-based facial personalization methods extract and use the facial region as a conditional input, the copy-paste issue is most pronounced in facial attributes—which is the primary focus of our paper. In contrast, non-facial attributes, which are not subject to additional constraints during training, exhibit a less significant copy-paste effect.

However, we fully understand your concern regarding the completeness of this discussion and have conducted supplementary experiments to address it. Following PuLID’s prompt set, we evaluated prompts involving common objects, clothing, and other typical non-facial attributes, using Segment Anything for mask generation. The experimental results across several baselines are as follows:

Table 1: Additional Prompt Set with Non-facial Attributes | | new prompts | |---|------------------------------------| | 1 | a person wearing a black headphone | | 2 | a person wearing a doctoral hat | | 3 | a person holding a beer cup | | 4 | a person wearing a spacesuit | | 5 | a person wearing a yellow scarf |

Table 2: Perfomance on Additional Prompt Set with Non-facial Attributes | | PC | IF | Face Div. | IF x PC (hMean) | |---------------------------|-------|-------|-----------|-----------------| | Face2Diffusion | 22.87 | 41.08 | 41.34 | 29.38 | | Face2Diffusion + FreeCure | 23.34 (+2.06%) | 40.75 (-0.80%)| 41.91 (+1.38%) | 29.68(+1.01%)| | InstantID | 22.43 | 65.55 | 47.73 | 33.42 | | InstantID + FreeCure | 23.51(+4.81%) | 65.34(-0.32%) | 48.06(+0.69%) | 34.58(+3.46%)| | PhotoMaker | 24.67 | 52.14 | 46.02 | 33.49 | | PhotoMaker + FreeCure | 25.37(+2.84%) | 51.77(-0.71%) | 46.58(+1.21%)| 34.05(+1.67%) | | PuLID (SDXL) | 27.43 | 59.41 | 41.97 | 37.53 | | PuLID (SDXL) + FreeCure | 28.13(+2.55%) | 59.03(-0.64%) | 42.05(+0.19%)| 38.10(+1.52%)|

Experimental results indicate that the facial personalization model exhibits a weaker copy-paste effect in non-facial regions, leading to less pronounced improvements in prompt consistency compared to the facial attributes specifically addressed in our submission. However, we observed that FreeCure still demonstrates significant restoration effects for attributes such as headphones and hats, which are in close proximity to the face (additional comparative images will be included in the updated version). Moreover, since the restoration occurs outside the facial region, FreeCure has a more negligible negative impact on identity fidelity. In summary, while our method remains applicable to various non-facial attributes due to its generalizability, the optimization effects are less pronounced but also noticeable compared to facial features.

Analysis of Failure Cases. Although FreeCure shows consistent improvements in wide range of baselines and attributes. We observe that when enhancing transparent objects (e.g., cups), the restored images may exhibit optical distortions in transparent regions. We will include a discussion of this failure case in the Limitations section and explore potential optimizations for such cases in future work.

2. Comparison with Classifier-free Guidance

We agree that this detail should be clarified. First, all our experiments do not modify the classifier-free guidance configuration in each baseline's official implementation, matching with our method's plug-and-play manner. That is to say, PD and FD process works when classifier-free guidance is included. For original inference with classifier-free guidance, the batch-wise information can be shown in the following table:

Table 3: CFG Condition before Applying Dual Inference | Batch Index| 0| 1| |--|--|--| | Condition Type | {${\emptyset}\_{id}, {\emptyset}\_{txt}$} | {${C}\_{id}, {C}\_{txt}$} |

Table 4: CFG Condition after Applying Dual Inference | Batch Index | 0| 1| 2| 3| |---|--|--|-----|--| | Condition Type | {${\emptyset}\_{id}, {\emptyset}\_{txt}$} | {${\emptyset}\_{id}, {\emptyset}\_{txt}$} | {${C}\_{id}, {C}\_{txt}$} | {${\emptyset}\_{id}, {C}\_{txt}$} |

where ${C}\_{id}$ & ${C}\_{txt}$ denote original identity and prompt conditions and ${\emptyset}\_{id}$ & ${\emptyset}\_{txt}$ denote the null or zero-value conditions. This is similar to the situation that dual inference in running an inference with a batch size equal to 2 (the function of chunk() will operate correctly). Without the techniques, attention manipulation techniques like FreeCure, FD, and PD are independent. What should be emphasized is that such independence helps us to identify the foundational knowledge where high prompt consistency still exists in these personalization models.

The core difference between CFG and FreeCure's dual inference lies in how they use zero face embeddings in their corresponding paradigm. First, during the training process of encoder-based personalization models with CFG, their basic target is still to reconstruct the reference faces. That is to say, even when the model receives a zero face embedding during some training step, their learnable modules (e.g., adapters, learnable embeddings) do not explicitly receive knowledge or constraints from the zero face embedding's denoising process. This also explains why nearly all encoder-based personalization models still face the copy-paste problem, even though the CFG strategy is introduced. In contrast, FreeCure uses zero face embedding to explicitly trigger information from the denosing process with zero face embedding (e.g., via FASA's self-attention map manipulation) and proposes a mechanism to inject such information into the original personalization process to improve its prompt consistency. We hope the explanation above can highlight the core difference and address your concerns.

3. Implementation details on Full-attention of DiTs

Thanks for pointing out this. The implementation of FASA on DiT's full-attention is almost identical to U-Net's self-attention but with some slight differences in mask strategy. The input of full-attention is a joint sequence of noisy latent and text condition: $[X; C], X\in \mathbf{R}^{b\times l\_1 \times h}, C \in \mathbf{R}^{b \times l\_2 \times h}$. Since PD and FD share the identity text condition, we extract noisy latent relevant $K\_f^X, V\_f^X$ from the FD process and concatenate it to $K\_p^X, V\_p^X$ of the PD process, while $Q$ keeps consistent:

$$\hat{Q\_p} = [Q\_p^X; Q\_p^C]; \hat{K\_p} = [K\_p^X;K\_p^C;K\_f^X]; \hat{V\_p} = [V\_p^X;V\_p^C;V\_f^X]$$

The main difference lies in the mask strategy. Aligning with U-Net's implementation, we apply the all-one matrix to PD's original attention part. Specially, we assign scaling masks $\omega \mathcal{M}$ to attention map of $Q\_p^X and K\_f^X$ and assign a all-zero mask to attention map of $Q\_p^C and K\_f^X$ to preserve PD's original full-attention patterns:

$$\mbox{FASA}(\hat{Q\_p}, \hat{K\_p}, \hat{V\_p}) = \mbox{Softmax}(\frac{\mathcal{M}(\omega)\_{flux}\odot \hat{Q\_{p}}\hat{K\_p}^{T}}{\sqrt{d}})\hat{V\_p}$$ $$\mathcal{M}(\omega)\_{flux} = \begin{pmatrix} \mathbf{1}\_{l\_1\times l\_1} & \mathbf{1}\_{l\_1\times l\_2} & \omega \mathcal{M}\_{l\_1\times l\_1} \\\\ \mathbf{1}\_{l\_2\times l\_1} & \mathbf{1}\_{l\_2\times l\_2} & \mathbf{0}\_{l\_2\times l\_1} \end{pmatrix}$$

This mask pattern highly resembles FASA's implementation in U-Net, which injects attribute-aware features from FD to PD while keeping the original full-attention's pattern. We will update this formula for FLUX in the updated version of the submission.

1. Inference Cost

We conduct experiments on a single H800 GPU with 80 GB VRAM to measure each baseline’s inference time before and after applying FreeCure. The table below reports the results ("*" means 30-step denoising due to official guidance and 50-step denoising is applied for the rest baselines):

Table 1: Runtime Analysis | | Baseline (seconds) | Baseline + FreeCure (Seconds) | | | | |:----:|:--------:|:---------:|:------:|:-------:|:--------:| | | Total| Stage 1| Stage 2 FASA | APG | Total | | FastComposer |1.79| 1.96| 2.19| 1.83 | 5.98 | | Face-Diffuser |2.13| 2.63| 2.33 | 1.92 | 6.88| | Face2Diffusion* |1.13| 1.25| 1.47| 1.23 | 3.95 | | PhotoMaker |6.52| 7.04| 10.03| 6.1 | 23.17 | | InstantID* |7.12| 7.63| 10.65| 6.23 |24.51 | | PuLID-SDXL |7.43| 8.02| 10.96| 7.28 | 26.26 | | PuLID-FLUX* |12.84| 14.89| 16.86| 12.95 | 44.7 | | InfineteYou* | 8.34| 10.23| 13.53| 9.03 | 32.79 |

While we acknowledge that introducing extra calculation can lead to longer runtime, we also emphasize that most training-free methods introduce extra processes, including inversion operation, denoising processes, and extra attention calculation. Plus, comparing to the fact that most encoder-based personalization methods require a long range time of data collection, preprocessing and tuning (this may consume a large array of GPUs and thousands of GPU-hours), our training-free method provides an innovative perspective for prompt consistency improvement by the knowledge in themselves, without the need for designing new model architecture and dataset curation.

2. Robustness of Segmentation Model

Thanks for pointing this out. We would like to address your concern from two aspects:

• Robustness of FreeCure when dealing with inaccurate masks. FreeCure demonstrate consistent performance even when masks are inaccurate. To validate this, we design two-way validation experiments that introduce inaccuracy to attribute masks. First, we apply a dilation operation on masks, which might include some area which does not belong to the target attribute area. Second, we apply an erosion operation, which might cause loss of some useful information about attributes. To create significant distortion for masks, we set the receptive field of both dilation and erosion to 25 pixels. We conduct experiments on several baselines, and the table below reports the results for respective prompt consistency (PC) and ideneity fidelity (IF):

Table 2: PC and IF Performance for Inaccurate Masks | | PC with original masks | IF with original masks | PC with dilated masks | IF with dilated masks | PC with eroded masks | IF with eroded masks | |---|----|------|---|----|----|----| | InstantID | 23.62 | 62.01 | 23.45 | 62.23 | 23.21 | 62.23 | | PhotoMaker | 24.91 | 50.15 | 24.87 | 50.01 | 24.34 | 50.44 | | PuLID-SDXL | 26.05 | 56.95 | 25.51 | 56.34 | 25.36 | 57.01 | | PuLID-FLUX | 24.78 | 72.61 | 24.52 | 72.39 | 24.32 | 72.97 |

The results demonstrate that identity fidelity (IF) remains largely consistent across both dilated and eroded mask conditions, with slight improvements observed in eroded masks. This occurs because mask erosion preserves certain attributes similar to the reference image. As for prompt consistency, both dilation and erosion cause minimal degradation, though erosion exhibits marginally greater impact. We consider that this stems from cases where fine-grained attribute restoration (e.g., iris color, earrings) may fail when the mask completely degenerates due to erosion.

Generally, the performance degradation of FreeCure due to inaccurate masks is limited. Moreover, in practical applications, specialized segmentation models (e.g., BiSeNet, Segment Anything) exhibit high robustness, making failure cases caused by mask inaccuracies even rarer than observed in these deliberately designed experiments. This further validates the inherent robustness of the FreeCure framework.

• Automatic Extraction for Attribute Masks. FreeCure’s mask extraction process is auto-driven without any case-specific design. Our system employs a lookup table encompassing a broad spectrum of facial attributes or other attributes to align with the input prompt. When target attributes are matched, the face parsing model (e.g., BiSeNet) or Segment Anything Model (SAM) automatically generates the corresponding masks in a fully automated manner. Specifically, BiSeNet inherently predicts masks for all facial attributes, so only the matched masks need to be extracted. In the Segment Anything configuration, the model takes the matched attributes as input to produce the final detection mask. A merging process will gather all masks together and generate a mask relevant to all attributes:

$$\mathcal{M}(j,k) = max(\mathcal{M}_{i}(j,k)).$$

Thanks to these models’ expertise and generalization capabilities in segmentation, the entire process operates without manual intervention, eliminating the need for specialized handling of particular attributes.

3. Engineering Implementation

First, as discussed in Section 3 of our submission, our architectural innovation on self-attention is grounded in solid theoretical foundations: the application of attention theory in diffusion models – where cross-attention controls identity injection while self-attention regulates spatial feature arrangements. This constitutes FreeCure’s core rationale for achieving attribute improvement without compromising identity fidelity. Thus, our method is driven by clear motivations and rigorous theoretical reasoning, rather than accidental parameter tuning or architectural trial-and-error. Second, our approach does not rely on ad hoc parameter design, demonstrating its generalizability as a performance-enhancing framework rather than case-specific engineering. As evidenced throughout our experiments, we maintained identical hyperparameter configurations across all baselines ($\omega=2.0$ and $\gamma=0.5$). These two aspects collectively distinguish our method from engineering-driven solutions.

4. Implementation details on FLUX

Thanks for pointing out this. The implementation of FASA on FLUX's full-attention is almost identical to U-Net's self-attention but with some slight differences in mask strategy. The input of full-attention is a joint sequence of noisy latent and text condition: $[X; C], X\in \mathbf{R}^{b\times l\_1 \times h}, C \in \mathbf{R}^{b \times l\_2 \times h}$. Since PD and FD share the identity text condition, we extract noisy latent relevant $K\_f^X, V\_f^X$ from the FD process and concatenate it to $K\_p^X, V\_p^X$ of the PD process, while $Q$ keeps consistent:

$$\hat{Q\_p} = [Q\_p^X; Q\_p^C]; \hat{K\_p} = [K\_p^X;K\_p^C;K\_f^X]; \hat{V\_p} = [V\_p^X;V\_p^C;V\_f^X]$$

The main difference lies in the mask strategy. Aligning with U-Net's implementation, we apply the all-one matrix to PD's original attention part. Specially, we assign scaling masks $\omega \mathcal{M}$ to attention map of $Q\_p^X and K\_f^X$ and assign a all-zero mask to attention map of $Q\_p^C and K\_f^X$ to preserve PD's original full-attention patterns:

$$\mbox{FASA}(\hat{Q\_p}, \hat{K\_p}, \hat{V\_p}) = \mbox{Softmax}(\frac{\mathcal{M}(\omega)\_{flux}\odot \hat{Q\_{p}}\hat{K\_p}^{T}}{\sqrt{d}})\hat{V\_p}$$ $$\mathcal{M}(\omega)\_{flux} = \begin{pmatrix} \mathbf{1}\_{l\_1\times l\_1} & \mathbf{1}\_{l\_1\times l\_2} & \omega \mathcal{M}\_{l\_1\times l\_1} \\\\ \mathbf{1}\_{l\_2\times l\_1} & \mathbf{1}\_{l\_2\times l\_2} & \mathbf{0}\_{l\_2\times l\_1} \end{pmatrix}$$

This mask pattern highly resembles FASA's implementation in U-Net, which injects attribute-aware features from FD to PD while keeping the original full-attention's pattern. We will update this formula for FLUX in the updated version of the submission.

Thanks for your valuable comments and positive evaluation on our empirical analysis, method simplicity and experimental results. We propose the following points to address all your concerns:

1. More Comprehensive Prompt Set

Thank you for raising this point. First, our method can leverage any attribute segmentation model to handle different types of features. In Section A.2.4 of the supplementary materials, we discuss the performance of general semantic segmentation models (e.g., SAM) within our framework and address attributes that fall outside facial features. It is important to note that since most current encoder-based facial personalization methods extract and use the facial region as a conditional input, the copy-paste issue is most pronounced in facial attributes—which is the primary focus of our paper. In contrast, non-facial attributes, which are not subject to additional constraints during training, exhibit a less significant copy-paste effect.

However, we fully understand your concern regarding the completeness of this discussion and have conducted supplementary experiments to address it. Following PuLID’s prompt set, we evaluated prompts involving common objects, clothing, and other typical non-facial attributes, using Segment Anything for mask generation. The experimental results across several baselines are as follows:

Table 1: Additional Prompt Set with Non-facial Attributes | | new prompts | |---|------------------------------------| | 1 | a person wearing a black headphone | | 2 | a person wearing a doctoral hat | | 3 | a person holding a beer cup | | 4 | a person wearing a spacesuit | | 5 | a person wearing a yellow scarf |

Table 2: Perfomance on Additional Prompt Set with Non-facial Attributes | | PC | IF | Face Div. | IF x PC (hMean) | |---------------------------|-------|-------|-----------|-----------------| | Face2Diffusion | 22.87 | 41.08 | 41.34 | 29.38 | | Face2Diffusion + FreeCure | 23.34 (+2.06%) | 40.75 (-0.80%)| 41.91 (+1.38%) | 29.68(+1.01%)| | InstantID | 22.43 | 65.55 | 47.73 | 33.42 | | InstantID + FreeCure | 23.51(+4.81%) | 65.34(-0.32%) | 48.06(+0.69%) | 34.58(+3.46%)| | PhotoMaker | 24.67 | 52.14 | 46.02 | 33.49 | | PhotoMaker + FreeCure | 25.37(+2.84%) | 51.77(-0.71%) | 46.58(+1.21%)| 34.05(+1.67%) | | PuLID (SDXL) | 27.43 | 59.41 | 41.97 | 37.53 | | PuLID (SDXL) + FreeCure | 28.13(+2.55%) | 59.03(-0.64%) | 42.05(+0.19%)| 38.10(+1.52%)|

Experimental results indicate that the facial personalization model exhibits a weaker copy-paste effect in non-facial regions, leading to less pronounced improvements in prompt consistency compared to the facial attributes specifically addressed in our submission. However, we observed that FreeCure still demonstrates significant restoration effects for attributes such as headphones and hats, which are in close proximity to the face (additional comparative images will be included in the updated version). Moreover, since the restoration occurs outside the facial region, FreeCure has a more negligible negative impact on identity fidelity. Regarding attributes such as body pose and background, we believe their relevance to our paper’s focus—resolving face copy-paste issues—is minimal, and thus they were not discussed. We appreciate your understanding in this regard.

In summary, while our method remains applicable to various non-facial attributes due to its generalizability, the optimization effects are less pronounced but also noticeable compared to facial features.

2. Detailed discussion for Section 3

Thanks for pointing out this detail. We have conducted a detailed analysis using finer-grained $\alpha$ values and their corresponding quantitative outcomes. Below are the results:

Table 3: Fined-grained Quantitative Metrics for Examples of Cross-attention Interpolation in Section 3 | | $\alpha$ | 0 | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | 1 | |------------|--------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------| | top row | IF | 50.23 | 37.43 | 24.53 | 16.42 | 10.34 | 7.32 | 6.42 | 4.22 | 3.43 | 2.34 | 2.1 | | | PC | 24.98 | 25.39 | 25.5 | 25.61 | 26.39 | 27.53 | 28.54 | 29.43 | 30.43 | 32.34 | 33.01 | | bottom row | IF | 52.34 | 50.23 | 49.52 | 40.01 | 29.12 | 23.89 | 15.75 | 10.34 | 6.32 | 4.11 | 3.2 | | | PC | 26.76 | 28.43 | 30.34 | 31.12 | 31.09 | 31.87 | 32.98 | 33.43 | 34.23 | 34.79 | 35.02 |

The results demonstrate that identity fidelity declines sharply even when only a small portion of the original cross-attention map is altered, suggesting that identifying a 'well-optimized sweet spot' through cross-attention manipulation alone is highly challenging. Plus, as you noted, the fine-grained performance varies considerably between different $\alpha$ ranges (e.g., $\alpha \in [0.2, 0.4]$ for the top row and $\alpha \in [0.4, 0.6]$), further highlighting a key limitation of cross-attention manipulation: its optimal control parameter can exhibit substantial variability. We will include a more detailed version of Figure 2 in the updated submission to provide further clarity.

3. Matrices Dimensionality Change of FASA

We reflect this question by providing the details of FASA. In the original self-attention mechanism with batch size 1, the dimensionality of $Q, K, V$ considering with $n$ heads is:

$$Q, K, V \in \mathbf{R}^{1\times n \times l \times (\frac{h}{n})}$$

The calculation of attention map $A = Softmax(\frac{QK^T}{\sqrt{d}}) \in \mathbf{R}^{1\times n \times l \times l}$ is straightforward. After we replace this self-attention layer with FASA module, typically we have batch size 2 due to dual inference design:

$$\hat{Q}, \hat{K}, \hat{V} \in \mathbf{R}^{2\times n \times l \times (\frac{h}{n})}$$

In FASA inference process, the PD process is in the first batch while FD is in the second. We copy the FD batch and concatenate it to PD batch along the sequence dimension $l$:

$$\hat{K}\_{[0, :, :, :]} \leftarrow \mbox{concat}(\hat{K}\_{[0, :, :, :]}, \hat{K}\_{[1, :, :, :]}, \mbox{dim=2})$$ $$\hat{V}\_{[0, :, :, :]} \leftarrow \mbox{concat}(\hat{V}\_{[0, :, :, :]}, \hat{V}\_{[1, :, :, :]}, \mbox{dim=2})$$

Now the PD process have an extended matrices of $\hat{K}\_{p}, \hat{V}\_{p} \in \mathbf{R}^{1\times n \times 2l \times (\frac{h}{n})}$. Therefore, FASA's attention map matrices $\hat{A\_p} \in \mathbf{R}^{1\times n \times l \times 2l}$ should also be extended. As for the scaling mask strategy, we resize the extracted mask $\mathcal{M}$ to the size $l\times l$ and replicate it to target size $1\times n \times l\times l$ after concatenating with a matrix with all elements equal to 1; this new mask can process $\hat{A\_p}$ accordingly. The entire process of PD process after applying FASA is as follows:

$$\mbox{FASA}(\hat{Q\_p}, \hat{K\_p}, \hat{V\_p}) = \mbox{Softmax}(\frac{[\mathbf{1},\omega\mathcal{M}]\odot \hat{Q\_{p}}\hat{K\_p}^{T}}{\sqrt{d}})\hat{V\_p}.$$

This process aligns with Eq. 4 in the submission. Afterwards, PD's output dimensionality aligns with the original FD process and can be gathered together for the inference process of subsequent layers (e.g., FFN). Please note that since the PD and FD's matrices of $K$ and $V$ have different dimensions, their attention scores are calculated separately. However, this is easy to implement because the communication happens between two batches, and the external mask $\mathcal{M}$ can be defined during FASA modules' initialization.

4. FASA's Implementation in FLUX

Thanks for your comments on this. We agree that this point should be clarified. The implementation of FASA on FLUX's full-attention is almost identical to U-Net's self-attention but with some slight differences in mask strategy. The input of full-attention is a joint sequence of noisy latent and text condition: $[X; C], X\in \mathbf{R}^{b\times l\_1 \times h}, C \in \mathbf{R}^{b \times l\_2 \times h}$. Since PD and FD share the identity text condition, we extract noisy latent relevant $K\_f^X, V\_f^X$ from the FD process and concatenate it to $K\_p^X, V\_p^X$ of the PD process, while $Q$ keeps consistent:

$$\hat{Q\_p} = [Q\_p^X; Q\_p^C]; \hat{K\_p} = [K\_p^X;K\_p^C;K\_f^X]; \hat{V\_p} = [V\_p^X;V\_p^C;V\_f^X]$$

The main difference lies in the mask strategy. Aligning with U-Net's implementation, we apply the all-one matrix to PD's original attention part. Specially, we assign scaling masks $\omega \mathcal{M}$ to attention map of $Q\_p^X and K\_f^X$ and assign a all-zero mask to attention map of $Q\_p^C and K\_f^X$ to preserve PD's original full-attention patterns:

$$\mbox{FASA}(\hat{Q\_p}, \hat{K\_p}, \hat{V\_p}) = \mbox{Softmax}(\frac{\mathcal{M}(\omega)\_{flux}\odot \hat{Q\_{p}}\hat{K\_p}^{T}}{\sqrt{d}})\hat{V\_p},$$ $$\mathcal{M}(\omega)\_{flux} = \begin{pmatrix} \mathbf{1}\_{l\_1\times l\_1} & \mathbf{1}\_{l\_1\times l\_2} & \omega \mathcal{M}\_{l\_1\times l\_1} \\\\ \mathbf{1}\_{l\_2\times l\_1} & \mathbf{1}\_{l\_2\times l\_2} & \mathbf{0}\_{l\_2\times l\_1} \end{pmatrix}$$

This mask pattern resembles FASA's implementation in U-Net which injects attribute-aware features from FD to PD while keeping the original full-attention's pattern. We will update this formula for FLUX in the updated version of the submission.

1. Novelty Clarification

While we agree that both FreeCure and MasaCtrl apply self-attention manipulation. We argue that our method is significantly different from MasaCtrl in several points that guarantee its novelty:

• Different Generation Paradigm. Masactrl converts an input image into a latent code through inversion and then copies it to its dual inference counterpart. In contrast, FreeCure exhibits less dependency on this. FreeCure's primary process generates images directly from noise, and the initial noise for its FD and PD processes can be sampled arbitrarily without requiring consistency. This fundamental difference in the generative paradigm enhances the diversity of restoration results in FreeCure (see Section 5.2 and A.3.2 of the submission for details). By comparison, MasaCtrl's dual inference mechanism strictly depends on identical latent codes, lacking exploration into the effects of different initial latents.
• Different Task Focus. While MasaCtrl primarily investigates non-rigid image editing tasks such as changing pose or viewpoint, our work focuses on enhancing prompt consistency in facial personalization models. Moreover, whereas MasaCtrl performs editing on the entire subject (e.g., pose adjustment), our focus lies in refining internal features (regardless of their granularity) of the subject and can seamlessly integrate with various mainstream facial personalization models.
• Better Adaptability. The adaptability of the FreeCure extends beyond U-Net's self-attention mechanism. Its FASA algorithm can also be applied to the full-attention of FLUX. However, this aspect remains unexplored in MasaCtrl’s discussions. MasaCtrl also lacks discussion on the simultaneous editing of multiple object attributes. In contrast, our proposed FASA framework can effectively handle multiple facial attributes concurrently while maintaining high identity fidelity.
• Masking Strategy. Building upon the second point, while MasaCtrl’s mutual self-attention mechanism leverages cross-attention maps as guidance, its subsequent thresholding operation introduces great noise and fails to accurately localize fine-grained features. To further validate this, we modify FreeCure's mask by replacing segmentation-model-extracted masks with those derived from cross-attention maps. We aggregated cross-attention maps for the corresponding tokens in a way identical to MasaCtrl and applied Otsu’s thresholding method for binarization:

Table 1: Performance with Cross-attention Mask || PC of Original FreeCure|IF of Original FreeCure|PC of FreeCure with cross-attention mask|IF of FreeCure with cross-attention mask| |-|-|-|-|-| |InstantID|23.62|62.01|19.23|59.23| |PhotoMaker|24.91|50.15|22.43|49.32| |PuLID-SDXL|26.05|56.95|24.52|53.43| |PuLID-FLUX|24.78|72.61|22.95|67.67|

The generated images will be updated in the new version. We can see that when using the cross-attention map as a mask, all baselines showed decreased performance in both prompt consistency and identity fidelity. This happens because compared to masks created by expert segmentation models, the cross-attention mask picks up too much irrelevant noise during the enhancement process. This noise undermines useful features and damages the original facial identity fidelity.

To sum up, we argue that although FASA of FreeCure and mutual self-attention of MasaCtrl share a similar concept of attention control, their difference lies in, but is not limited to, overall paradigm, problem focus, adaptability, as well as application complexity. Since other reviewers (uVNV,rzYz, and tTVJ) acknowledge the motivation, simplicity, and generalizability of FreeCure, its novelty and contribution should be guaranteed.

2. Supplementary Explanation of APG

• Formula and implementation of APG. During APG, FASA is excluded since this stage only involves single-batch inference of the baseline model. We employ the original baseline's FD process independently and leverage its inherent consistency in handling abstract attributes to further refine the results improved by FASA. The process can be described via the following formula:

$$\hat{z_{\gamma T}} = \mbox{Inversion}(I_r, c_0); I_{out} = \mbox{Denoise}(\hat{z_{\gamma T}}, c_1)$$

where $I\_r, I\_{out}, \hat{z\_{\gamma T}}$ matches the annotation in Figure 3 of submission, $c\_0$ denotes a template prompt without any attribute description (e.g., "a man") and $c\_1$ denotes prompt containing target attirbutes (e.g., "a man laughing").

• Reason for separating APG from the second step. Such separation allows distinct handling of spatially localized features and abstract attributes. Processing these features jointly within the same denoising stage would hinder their effective decoupling, ultimately compromising the enhancement of individual characteristics.

• Value selection for $\gamma$. All experimental data and visual results in our main submission were obtained consistently using $\gamma=0.5$. While we acknowledge that the optimal $\gamma$ may vary slightly across different baselines, its value consistently falls within the range of $[0.5, 0.6]$. For a plug-and-play method, we argue that this minor variation is acceptable due to the diversity of baselines.

• Usage of APG. During the generation phase, if no prompt related to abstract features (e.g., facial expression) is matched, the APG module can be safely omitted.

3. Attribute Entanglement

We argue that the comments of "trade-off in nearly all examples" do not have specific evidence. The problem of "...The prior identity information from the hidden states of the FD process will also be introduced..." can be directly answered by Figure 4 of our submission. Our mask guidance mechanism provides an effective solution to protect the identity fidelity of PD from FD. Our visual experiments in Section 5.2 also confirm that in non-target regions, the FD process exerts minimal influence on the PD denoising process. Plus, our method's superiority has been validated both qualitatively and quantitatively and acknowledged by other reviewers. A supplementary user study required by Reviewer uVNV also highly aligns with our consistent improvement of the submission. Therefore, we respectfully suggest that you reconsider your assessment.

4. Mask Dependency

You may have some misunderstandings which we would like to clarify: FreeCure’s mask extraction does not rely on any case-specific "tuning" or "manual processing". Instead, our system employs a lookup table encompassing various facial attributes to align with the input prompt. When target attributes are matched, the face parsing model (e.g., BiSeNet) or Segment Anything Model (SAM) automatically generates the corresponding masks. Specifically, BiSeNet inherently predicts masks for all facial attributes, so only the matched masks need to be extracted. In the Segment Anything, the model takes the matched attributes as input to produce the mask. Thanks to these models’ generalization capabilities in segmentation, the entire process operates without manual intervention.

5. Discussion of MoA

MoA's prior attention primarily addresses the generation of regions unrelated to the subject (e.g., background). This objective fundamentally differs from FreeCure, which is designed to enhance prompt consistency for features within the main subject. The only commonality between the two approaches may be their shared goal of integrating foundational model knowledge into personalized models. Additionally, FreeCure is a plug-and-play framework, compatible with diverse personalization methods. Its relationship with MoA is better characterized as complementary (i.e., upstream/downstream) rather than being a "trainable counterpart". Given FreeCure’s demonstrated adaptability across a wide range of personalization models, we anticipate its potential application to MoA as well. In the updated version of our submission, we will include this reference and discuss the feasibility of integrating FreeCure into MoA.

6. Missing Baseline Comparison

We argue that the pipeline proposed by the reviewer cannot be comparable to our method. The reviewer suggests using a 'reduced inference weight' to generate results, but this approach harms both identity fidelity and prompt consistency (see our discussion in Section 3). Instead, directly adopting our proposed FD process (equivalent to setting the 'inference weight' directly to 0) could be a more viable solution.

Plus, this pipeline critically depends on face swapping models to disentangle identity and attributes, particularly when dealing with occlusions (e.g., sunglasses, masks) or spatial misalignments in foundation model outputs. Current art face-swapping methods (e.g., DreamID, DiffSwap) rarely address these cases, and their failures mirror the copy-paste issues in personalized models. In contrast, FreeCure provides a simpler, more effective solution, as detailed in Section 5.2.

In summary, there exists a significant gap between the baseline mentioned by the reviewer and the problem setting addressed in our work. We contend that, in the context of facial personalization, this pipeline does not constitute a comparable paradigm to our proposed method.

7. Limitations

Although FreeCure shows consistent improvements in wide range of baselines and attributes. We observe that when enhancing transparent objects (e.g., cups), the restored images may exhibit optical distortions in transparent regions. We will include a discussion of this failure case in the Limitations section and explore potential optimizations for such cases in future work.

8. Ethical Concerns

Since our evaluation dataset comes from public resources (e.g., CelebA-HQ, public webpages). It is safe to say that concerns on data privacy is minimal.

Reference

[1] MoA, SIGGRAPH Asia 2024.

[2] DreamID, arXiv 2025.

[3] Diffswap, CVPR 2023.

We sincerely appreciate your valuable feedback and constructive suggestions, particularly the positive assessment of our paper's motivation, methodological simplicity, and comprehensive experimental analysis. In response to the raised concerns, we would like to address the following points for clarification.

1. Runtime Analysis

We conduct experiments on a single H800 GPU with 80 GB VRAM to measure each baseline’s inference time before and after applying FreeCure. The table below reports the results ("*" means 30-step denoising due to official guidance and 50-step denoising is applied for the rest baselines):

Table 1: Runtime Analysis | | Baseline (seconds) | Baseline + FreeCure (seconds) | | | | |:----:|:--------:|:---------:|:------:|:-------:|:--------:| | | Total| Stage 1| Stage 2 FASA | APG | Total | | FastComposer |1.79| 1.96| 2.19| 1.83 | 5.98 | | Face-Diffuser |2.13| 2.63| 2.33 | 1.92 | 6.88| | Face2Diffusion* |1.13| 1.25| 1.47| 1.23 | 3.95 | | PhotoMaker |6.52| 7.04| 10.03| 6.1 | 23.17 | | InstantID* |7.12| 7.63| 10.65| 6.23 |24.51 | | PuLID-SDXL |7.43| 8.02| 10.96| 7.28 | 26.26 | | PuLID-FLUX* |12.84| 14.89| 16.86| 12.95 | 44.7 | | InfineteYou* | 8.34| 10.23| 13.53| 9.03 | 32.79 |

While we acknowledge that introducing extra calculation can lead to longer runtime, we also emphasize that most training-free methods introduce extra processes, including inversion operation, denoising processes, and extra attention calculation. Plus, comparing to the fact that most encoder-based personalization methods require a long range time of data collection, preprocessing and tuning (this may consume a large array of GPUs and thousands of GPU-hours), our training-free method provides an innovative perspective for prompt consistency improvement by the knowledge in themselves, without the need for designing new model architecture and dataset curation.

2. Robustness under the Scenario of Inaccurate Masks

We reflect this concern by designing two-way validation experiments that introduce inaccuracy to attribute masks. First, we apply a dilation operation on masks, which might include area which does not belong to the target attribute area. Second, we apply an erosion operation, which might cause loss of some useful information about attributes. To create significant distortion of masks, we set the receptive field of both dilation and erosion to 25 pixels. We conduct experiments on several baselines, and the table below reports the results for respective prompt consistency (PC) and ideneity fidelity (IF):

Table 2: PC and IF Performance for Inaccurate Masks | | PC with original masks | IF with original masks | PC with dilated masks | IF with dilated masks | PC with eroded masks | IF with eroded masks | |---|----|------|---|----|----|----| | InstantID | 23.62 | 62.01 | 23.45 | 62.23 | 23.21 | 62.23 | | PhotoMaker | 24.91 | 50.15 | 24.87 | 50.01 | 24.34 | 50.44 | | PuLID-SDXL | 26.05 | 56.95 | 25.51 | 56.34 | 25.36 | 57.01 | | PuLID-FLUX | 24.78 | 72.61 | 24.52 | 72.39 | 24.32 | 72.97 |

The results demonstrate that identity fidelity (IF) remains largely consistent across both dilated and eroded mask conditions, with slight improvements observed in eroded masks. This occurs because mask erosion preserves certain attributes similar to the reference image. As for prompt consistency, both dilation and erosion cause minimal degradation, though erosion exhibits marginally greater impact. We consider that this stems from cases where fine-grained attribute restoration (e.g., iris color, earrings) may fail when the mask completely degenerates due to erosion.

Generally, the performance degradation of FreeCure due to inaccurate masks is limited. Moreover, in practical applications, specialized segmentation models (e.g., BiSeNet, Segment Anything) exhibit high robustness, making failure cases caused by mask inaccuracies even rarer than observed in these deliberately designed experiments. This further validates the inherent robustness of the FreeCure framework.

3. User Study

We appreciate your attention to this point and agree that a user study can provide constructive support for the evaluation of our method. We conduct an online questionnaire with 30 participants. Regarding the questionnaire design: For each baseline, we randomly select 10 samples. Each sample consists of a reference image, a prompt, the baseline-generated result, and the result after applying FreeCure. Participants are asked to evaluate the identity fidelity (IF) and prompt consistency (PC) of each result. Specifically, they rate their satisfaction on a scale of 1 to 10, considering identity fidelity and prompt consistency. The survey samples from different baselines are presented randomly, and the order of the baseline results and FreeCure-applied results is also randomized to ensure that participants' judgments for each sample remained relatively independent. Based on scoring results, we provide both the scoring average values and preference ratio.

Table 3: User Study Score | | Baseline | | Baseline + FreeCure | | |--|----|-----|-----|---| | | IF Score Avg. | PC Score Avg. | IF Score Avg. | PC Score Avg. | | FastComposer | 7.55 | 4.96 | 7.51 | 8.35 | | Face-Diffuser | 7.56 | 5.08 | 7.41 | 7.62 | | Face2Diffusion | 7.83 | 5.02 | 7.71 | 7.83 | | PhotoMaker | 6.95 | 6.41 | 7.49 | 8.04 | | InstantID | 7.78 | 5.28 | 7.79 | 8.20 | | PuLID-SDXL | 7.76 | 6.38 | 8.08 | 8.26 | | PuLID-FLUX | 7.78 | 6.56 | 8.08 | 8.02 | | InfineteYou | 7.83 | 6.52 | 7.79 | 8.18 |

Table 4: User Preference on Identity Fidelity | | Prefer Baseline | Prefer Equally | Prefer Baseline + FreeCure | |---|----|---|-----| | FastComposer | 28.88% | 60.54% | 10.58% | | Face-Diffuser | 25.33% | 63.83% | 10.83%| | Face2Diffusion | 18.88% | 68.00% | 13.13% | | PhotoMaker | 12.54% | 78.83% | 8.63% | | InstantID | 25.38% | 64.29% | 10.33% | | PuLID-SDXL | 10.67% | 81.38% | 7.96% | | PuLID-FLUX | 11.50% | 84.67% | 3.83% | | InfineteYou | 10.67% | 79.25% | 10.08% |

Table 5: User Preference on Prompt Consistency | | Prefer Baseline | Prefer Equally | Prefer Baseline + FreeCure | |---|--|---|---| | FastComposer | 4.29% | 14.38% | 81.33% | | Face-Diffuser | 5.17% | 16.71% | 78.13% | | Face2Diffusion | 4.04% | 16.04%| 79.92% | | PhotoMaker | 4.38% | 31.88% | 63.75% | | InstantID | 8.46% | 12.58% | 78.96% | | PuLID-SDXL | 4.29% | 16.92% | 78.79% | | PuLID-FLUX | 10.54% | 16.46% | 73.00% | | InfineteYou | 9.75% | 21.00% | 69.25% |

The results show that, based on human evaluation, FreeCure consistently improves every baseline's prompt consistency. Furthermore, we observe that there are many cases where participant tends to give the same score for identity fidelity. Such performance supports the fact that FreeCure's negative effect on identity fidelity is quite limited in a practical manner and promises its practical use in the future. In our updated version, we plan to provide the template example for the questionnaire and more straightforward visualization techniques (e.g., column chart) to demonstrate the results of the user study.

4. Hyper-parameter analysis

We acknowledge that the optimal configuration of FreeCure varies slightly for each baseline. However, this does not constitute a weakness: for each baseline, the fluctuation of their optimal hyperparameters is within an acceptable range:

$$\omega \in [1.8, 2.4], \gamma \in [0.5, 0.6]$$

Baselines built on Stable Diffusion v1.5 (Face2Diffusion, FastComposer, Face-Diffuser) require a larger $\omega$ value up to 2.4, and other baselines remain around 2.0. Within this range, the overall results do not differ much, which allows us to determine a unified hyperparameter setting without any further complicated strategy ($\omega = 2.0$ and $\gamma = 0.5$ throughout all experiments in the submission), and still achieved consistent performance improvement in prompt consistency with the maintenance of identity fidelity.

Secondly, we would like to point out that since the foundation models and architectures of each baseline are diverse, FreeCure—being a plug-and-play method—should be allowed for a reasonable adjustable range to better adapt to each baseline.

5. Code Release

Thanks for pointing this out. We will release the code of FreeCure later.

6. Limitations

Our method can seamlessly apply to most diffusion-based personalization models (including those based on SD-v1.5, SDXL, and DiT), which dominate the personalization task. Since currently there is no influential personalization method based on auto-regressive models, it may cost some extra effort to adapt our methods to these methods in the future. We will add the discussion of this matter in the limitations.