Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization

Q1: Computational Overhead for Inference: While the training pipeline is efficient for research (e.g., 2-3 GPUs for 1-3 hours), the inference time of "just over one minute" to generate a personalized image edit for a given user might be a practical limitation for applications requiring real-time interaction or high throughput.

We thank the reviewer for highlighting this practical consideration. Indeed, the reported inference time ("just over one minute") primarily reflects our choice of editing backend (Flux+ControlNet). Our personalization framework itself is editing-model agnostic, allowing easy integration with faster editing backends. For instance, we tested our pipeline with TurboEdit, which significantly reduced inference time to under 2 seconds per edit. We will include detailed results with TurboEdit in the final manuscript to further demonstrate our framework's flexibility and applicability in real-time scenarios.

Q2: Technique Novelty: Although the C-DPO looks interesting while solving this novel task, the technique novelty is pretty low since the DPO method has been widely studied in T2I generation and augmentation research direction. I would like to see what's the main technical novelty in this paper except the huge dataset contribution. /Novelty (same as weakness 2): The technique novelty referring to the C-DPO is low, which I would like the authors to claim more.

We thank the reviewer for highlighting this point regarding technical novelty. Indeed, while DPO has been previously studied in the context of T2I generation, our Collaborative Direct Preference Optimization (C-DPO) framework represents a novel and significant extension of the original DPO formulation. Our C-DPO introduces a graph-structured regularization term into the original DPO loss, explicitly modeling and leveraging collaborative relationships among user embeddings. This structured collaboration allows the model to capture nuanced preferences implicitly shared among like-minded users. Given its flexibility and general applicability, we expect that our novel collaborative regularization formulation can also be adapted beyond T2I generation scenarios. For instance, it has potential utility in broader personalization problems involving LLMs, such as personalized text generation, dialogue systems, or recommendation tasks, opening promising avenues for future research.

Moreover, to the best of our knowledge, while previous research has explored personalized image “generation”, ours is the first work to define and tackle the task of personalized image editing.

We believe these contributions represent substantial and novel technical advances beyond the original DPO formulation and existing personalized T2I generation methods. We will explicitly highlight and clarify these points in the revised manuscript.

Q3. Lack of qualitative comparisons: For qualitative comparisons, I would encourage the authors to include more in the supplementary material. I know there are some limits for the main paper. Putting more qualitative examples in the supplementary helps to know the quality of your method.

We thank the reviewer for this valuable suggestion. We fully agree that including more qualitative examples can significantly enhance readers' understanding of our method's capabilities and strengths. While the main manuscript was limited due to space constraints, we have provided additional qualitative comparisons extensively in the supplementary material (see Figures 1, 3, and 4). In our revised version, we will further expand these qualitative results in the supplementary to comprehensively illustrate our approach's performance and diversity across various user preferences and editing tasks.

Q4: More importantly, not only include human preferences, please consider common image editing evaluation metrics. The user study is an optional way to support the common evaluations, not the main evaluation metrics to make a fair comparison.

We thank the reviewer for this important point. In addition to the user studies, we have indeed evaluated our method using several standard quantitative image-editing metrics, including DINO (image-image similarity), CLIP-I (image-image similarity), CLIP-T (image-text similarity), and HPS scores. These metrics objectively quantify the alignment between edited images and user preferences, as well as the preservation of original image identity. These quantitative results are reported in Tables 1 and 2 of the supplementary material, demonstrating our method's significant improvements over baselines. In the revised manuscript, we will explicitly emphasize these common quantitative metrics alongside user studies, ensuring a balanced and comprehensive evaluation of our framework.

Q5: Whether it's allowed to put any anonymous links to some image collections during rebuttal/Better to compare with current state-of-the-art instruction-based image editing methods or unified models with user preference prompts such as Bagel, MmaDA, Blip-3o, Show-o, Janus-series, Nexus-Gen, UniVG, Step1X, GoT, VAR-GPT, SEED-LLAMA, AnyEdit, etc.

We thank the reviewer for raising this point. Due to NeurIPS rebuttal guidelines, unfortunately we are not allowed to share any visuals during this phase. However, as requested, to better position our approach relative to state-of-the-art instruction-based and unified image editing models, we conducted quantitative comparisons specifically with Bagel, SEED-LLAMA, and Nexus-Gen—strong recent baselines in this area. For these comparisons, we explicitly provided user preferences as textual prompt inputs to these models. As summarized in the table below, our approach clearly outperformed these methods across standard image editing evaluation metrics, highlighting the distinct advantage of our collaborative personalization framework:

We will explicitly incorporate these results into the revised manuscript to clearly demonstrate the superiority of our personalized editing approach over existing unified and instruction-based editing frameworks.

Q1: How would a new user that is not present in the training be handled? What does the input to the GNN look like? These details can be added to the manuscript to improve the understanding of the proposed method.

We thank the reviewer for highlighting this important point, and we agree that some details were briefly discussed in Section 4.1.1. due to space limitations. Specifically, we utilize a lightweight GNN to compute contextualized user embeddings by aggregating neighbor information in our heterogeneous graph of users and attributes. Attribute node features are dense text embeddings of editing instruction topics (such as watercolor, cats, etc). The model consists of a GNN encoder of 2 GraphSAGE convolutions using mean aggregation and an edge decoder of 2 linear layers. The GNN is pretrained on an auxiliary task of predicting user–attribute links (labeled positive or negative) to ensure semantically meaningful user representations. We train on 60% of edges in our graph, withholding 20% for validation and testing each. Our implementation closely follows standard approaches described in foundational works such as Hamilton et al. (NeurIPS 2017) and Kipf & Welling (ICLR 2016) as referenced in our paper. We will enhance clarity by adding more details as space permits, and explicitly directing readers to these key references in the manuscript.

At inference time, embedding a new user does not require retraining. Instead, a new user provides a small set of liked/disliked attributes; we then add a corresponding user node with a fixed-width, zero-initialized feature vector, link it to relevant attribute nodes, and apply our learned aggregation functions in the pretrained GNN encoder. The resulting embedding is computed inductively from the attributes and neighborhood structure alone. While periodic fine-tuning on the expanded graph may further enhance embedding quality, it is optional, not essential for inference, thus preserving the practicality and scalability of our method.

Q2: In line 190: "whereas the second term implements a graph-regularized collaborative filter that shares statistical strength among like-minded users—crucial for data-sparse personalization." What do the authors mean by this? Is the collaborative DPO term really essential for the training? I thought the collaboration comes from the GNN pretraining. What happens if you remove this term from your C-DPO (the second term which you term collaborative)?

We thank the reviewer for requesting clarification on the importance of the collaborative DPO term. Indeed, while the GNN pretraining provides an initial embedding structure that encodes user similarities, the collaborative DPO term (the second term in Eq. 7) is critical for effectively leveraging these similarities during the training process. Specifically, this term acts as a regularization mechanism during training, encouraging the model to leverage shared patterns among like-minded users.

We performed an ablation study confirming the importance of the collaborative term as shown in Fig. 2 in the Appendix. Setting the collaborative scale very low (e.g. λ=0.01) reduced our method's performance close to the user-only DPO baseline. This indicates that minimal collaborative signals lead to suboptimal personalization, highlighting the necessity of a balanced collaborative term.

Q3: The current setup in inference, generally embeds the user into a token and asks a generic question like "suggest a single cohesive edit". What happens if a user also asks for a particular edit? Would the LLM respond well? What if a user who is mostly interested in neon art asks for a water painting art? I am curious how the LLM would handle it.

We thank the reviewer for this insightful question. Indeed, our framework robustly handles scenarios in which users explicitly request edits outside their typical style preferences. Due to the flexible design of our personalized embeddings and the underlying LLM-based approach, the model naturally integrates explicit instructions provided at inference, even if they diverge from previously learned preferences.

We explicitly demonstrate this adaptability in the supplementary material (Fig. 4 and Table 3), where we repurposed our personalization framework—originally trained solely for image editing—to perform personalized image generation tasks without additional training. Our method successfully generalized to this new task and significantly outperformed state-of-the-art personalized generation approaches, such as VIPER. These results underscore the versatility of our framework and its effectiveness at integrating personalized embeddings with explicit user-provided instructions during inference.

Due to NeurIPS rebuttal guidelines, we are currently unable to share visuals in our rebuttal, but we will include additional examples showcasing our method's ability to adapt to such cases with the camera-ready version of the paper.

Q4: A naive baseline that comes to my mind is using a LLM with long context/RAG and ask for "bring instruction that are relevant to this user". Again, I am not someone who works with RAGs. So I would appreciate the authors thoughts on this?

We thank the reviewer for this interesting and relevant suggestion. Indeed, a RAG or long-context LLM baseline, where past instructions or user preferences are directly retrieved based on textual similarity and used as prompts, represents a plausible alternative approach.

However, such a method primarily relies on explicit textual similarity or keyword matching, potentially overlooking nuanced, implicit preferences that emerge from complex collaborative signals across users. In contrast, our GNN-based embedding explicitly models subtle user similarities via graph-structured relationships, capturing higher-order, implicit user preferences that may not be directly retrievable via textual matching alone.

We also consider GraphRAG where we could potentially combine the advantages of both explicit graph-based collaboration and retrieval-based personalization. Exploring this promising hybrid approach remains an exciting direction that we plan to pursue in future work.

Q5: For the user study, I think there is a gap to make it significantly more interesting and impactful. Is it possible to do user study by actually treating the users as new "candidates" in the graph and really collect their preferences and evaluate them directly? Rather than showing them a persona? I think showing the persona is a little unsatisfactory because the persona contains some keywords and users might end up looking for those words. But the work captures more nuances than simple keywords. And I think this is not tested.

We thank the reviewer for this insightful suggestion. We agree that directly collecting user preferences from participants could indeed provide a more realistic and nuanced assessment of our framework's effectiveness. We initially employed synthetic personas primarily due to constraints related to scalability and budget constraints. However, as suggested, performing a real-user evaluation—where each participant explicitly provides their own detailed preferences and is subsequently embedded into the graph—would significantly enhance our understanding of the framework's practical effectiveness and nuance-capturing capabilities. We recognize this as an important opportunity to further strengthen our evaluation.

Q6: How difficult would it be to crowdsource and collect preference data? I think such a data would be more natural and realistic. Adding this discussion to the paper would be beneficial to the future works

We thank the reviewer for this valuable suggestion. Crowdsourcing real-world user preferences indeed would yield more authentic and nuanced preference data, but it involves practical considerations such as scalability and budget constraints. Despite these challenges, collecting such data represents an exciting and beneficial direction for future research. We will briefly discuss this opportunity in the revised manuscript.

I appreciate the rebuttal response from the authors and addressing my concerns! I have some follow-up questions:

1. Regarding the large-scale real human data collection: thank you for acknowledging the discussion into the paper and touching upon the budget constraints. This is a valid response and I appreciate the authors' adding a note to the paper

While I agree with the data collection, I still do believe that a user study can be conducted where: First, collecting the users preferences from a pre-generated list of concepts. And then, conducting a user study by generating the images to evaluate the preference of the users. This should not be expensive in terms of time or compute; as it utilizes the same resources as the current user study. I agree that the ad-hoc generations might make the user study a bit different, but this will significantly improve the user study results and make a great impact for the paper. What are the authors' thoughts on this?

2. The c-dpo terms $\lambda$ seems to be a sensitive parameter. What is the resource budget used to find the optimal value of 0.15 ? How sensitive or stable is the algorithm? This seems to be the main challenge of the C-DPO algorithm - to find the optimal parameter where it doesn't overpower or underwhelm the user preferences

3. I am very pleasantly surprised that the baseline suggested by Reviewer F3Yh doesn't work! Can you please provide more details on this baseline? In my understanding you are passing the textual information about the user preference to a pretrained language model and asking it to generate prompts based on their preferences? Is this correct? Or are you using your fine-tuned llm to do this baseline? (For reference, I am leaving a comment on your rebuttal to Reviewer F3Yh)

4. Thank you for providing more information about the GNN inputs. I believe this is a very important detail which needs to be added to the main paper for better clarity. I am still unsure how the input is designed for the GNN. Can the users break down in more detail how the input of their GNN is designed during training and during inference when a new user is added to the graph?

I appreciate any further clarification provided by the authors!

Q1: The paper could benefit from a more thorough justification of its hyperparameters and architectural choices. For instance, the selection of the number of neighbors (K) for the GNN seems arbitrary. An ablation study exploring the impact of K and clarifying why not all nodes are considered (when looking up the nearest neighbors). Is there a study on the number of neighbors (K) used in the GNN and justify this design choice over a simpler approach that considers all nodes?

We appreciate the reviewer's request for a clearer justification of K. In our framework, K controls only the size of the collaborative neighborhood used in the C-DPO loss, specifically, how many of the most similar users contribute weighted preference signals. It does not limit the message-passing radius inside the GNN itself. The GNN still aggregates over all incident edges when computing the user embeddings.

We conducted preliminary experiments during development to empirically select an appropriate K. We tested multiple values observing that smaller to moderate values strike a good balance between computational efficiency, embedding quality, and effective collaborative personalization. To help the reader further understand the selection of K, we conduct an additional ablation which we will include in our revised manuscript. Increasing K beyond a certain threshold results in diminishing returns or even degraded performance, likely due to noise from less relevant neighbors.

Q2: Can you please expand on the GNN's training methodology, specifically detailing the input data format, the network architecture, and the optimization process to improve reproducibility?

We thank the reviewer for highlighting this important point, and we agree that some details were briefly discussed in Section 4.1.1. due to space limitations. We would like to clarify our architecture and training strategy. Specifically, we utilize a lightweight GNN to compute contextualized user embeddings by aggregating neighbor information in our heterogeneous graph of users and attributes. Attribute node features are dense text embeddings of editing instruction topics (such as watercolor, cats, etc). The model consists of a GNN encoder of 2 GraphSAGE convolutions using mean aggregation and an edge decoder of 2 linear layers. The GNN is pretrained on an auxiliary task of predicting user–attribute links (labeled positive or negative) to ensure semantically meaningful user representations. We train on 60% of edges in our graph, withholding 20% for validation and testing each. We will elaborate this further in our final manuscript.

Q3: Could you elaborate on the generation process for the synthetic dataset? Specifically, what model was used, and what steps were taken to ensure the preference data is of high quality?/The method used to generate the synthetic data should also be detailed. For e.g. how is preference data curated?

We thank the reviewer for raising this important point. Our synthetic dataset comprises 144K editing preferences across 3,000 user profiles, each designed to simulate diverse and realistic aesthetic tastes. To curate this data, we first define user profiles with distinct demographic attributes (e.g., age, region, taste clusters) which guide their style preferences. Each user is assigned four base image captions (sampled from 80 COCO categories), and for each, we generate two pairs of preferred and rejected editing instructions across six edit types (e.g., color, material, style, overlay), totaling 48 edits per user. These instructions are produced using GPT-4o, explicitly conditioned on the user’s profile. Preference labels are assigned via templated prompts ensuring stylistic alignment with the user's traits. To promote collaborative learning, we construct a user graph based on shared preference attributes and pretrain a GNN to embed these relationships. This process ensures that our preference data is semantically rich, structurally diverse, and aligned with realistic editing behavior, enabling effective training and evaluation of our collaborative personalization framework.

Q4: Have you considered comparing your method against a strong baseline that uses prompt engineering with the "user description" (including the neighbours) to steer the model, in order to justify the added complexity of the GNN framework?/A simple (but potentially strong) baseline is missing from the evaluation. It is unclear how C-DPO compares against a simpler approach where user preferences are injected into the model via a more sophisticated prompt engineering (e.g., using the user profile description and/or aggregate neighbour information directly in the prompt). This comparison is helpful to justify the added complexity of the GNN-based framework.

We thank the reviewer for highlighting this important comparison. Indeed, we conducted experiments comparing our GNN-based collaborative method (C-DPO) with a strong baseline employing sophisticated prompt engineering. Specifically, this baseline directly injected detailed textual descriptions of user preferences—including aggregated neighbor information—into the prompt itself, without using the GNN-based embedding.

The results clearly demonstrated that our method significantly outperformed the sophisticated prompt-engineering baseline as shown below:

This indicates that our proposed framework provides a meaningful benefit over simpler, prompt-based personalization methods, effectively justifying the additional complexity introduced by the GNN. We will explicitly include these results and discussions in the revised manuscript to clarify this important point.

Q5: To strengthen the claims of real-world applicability, have you considered evaluating your framework on a dataset of real user preferences, even if smaller in scale?/The experiments are conducted exclusively on synthetic user data. While the dataset is large, its validity as a proxy for real-world preferences is not established. The paper would be significantly strengthened by including experiments on a dataset of real user preferences to show the transferability of the approach.

We thank the reviewer for highlighting the importance of evaluating our framework on real-world data. Collecting real-world data at sufficient scale for training is challenging due to typical academic budget constraints. Indeed, validating our framework on a dataset of real user preferences (albeit smaller in scale) would significantly enhance the strength of our claims about real-world applicability. While the current experiments utilize a synthetic dataset primarily due to budget and scalability constraints, we acknowledge this limitation and we will strongly consider collecting a real-world user-preference dataset and performing additional validation experiments to include in the camera-ready version, further demonstrating our method's practical utility and generalization capabilities.

Q6: It is not fully clear if incorporating new users into the graph would require retraining the GNN.

Our approach leverages an inductive embedding framework (GraphSAGE) rather than a transductive one, explicitly designed to generalize efficiently to new users without retraining. Traditional transductive methods are inherently limited to fixed graphs, unable to generalize embeddings to unseen nodes or evolving structures. Rather than generating isolated embeddings for each user, GraphSAGE learns a set of aggregation functions which intake feature information from a node’s neighborhood and aggregates. Applying these aggregation functions efficiently generates embeddings for previously unseen users based solely on their attribute connections, provided the attribute schema matches the training data.

At inference time, embedding a new user does not require retraining. Instead, a new user provides a small set of liked/disliked attributes; we then add a corresponding user node with a fixed-width, zero-initialized feature vector, link it to relevant attribute nodes, and apply our learned aggregation functions in the pretrained GNN encoder. The resulting embedding is computed inductively from the attributes and neighborhood structure alone. While periodic fine-tuning on the expanded graph may further enhance embedding quality, it is optional, not essential for inference, thus preserving the practicality and scalability of our method.

Q7: The reference section is very messy and needs to be revised.

We thank the reviewer for pointing this out. We will thoroughly revise and reorganize the reference section in the final manuscript.

W1: The architecture and training strategy of the GNN.

We thank the reviewer for highlighting this important point, and we agree that some details were briefly discussed in Section 4.1.1. due to space limitations. We would like to clarify our architecture and training strategy. Specifically, we utilize a lightweight GNN to compute contextualized user embeddings by aggregating neighbor information in our heterogeneous graph of users and attributes. Attribute node features are dense text embeddings of editing instruction topics (such as watercolor, cats, etc). The model consists of a GNN encoder of 2 GraphSAGE convolutions using mean aggregation and an edge decoder of 2 linear layers. The GNN is pretrained on an auxiliary task of predicting user–attribute links (labeled positive or negative) to ensure semantically meaningful user representations. We train on 60% of edges in our graph, withholding 20% for validation and testing each. We will elaborate this further in our final manuscript.

W2: How preference annotations are formatted and how they are integrated with GNN-based C-DPO training.

During GNN pretraining, we leverage both liked and disliked user-attribute pairs in an auxiliary edge prediction task. For instance, edges may take the form of (“user1”, “likes”, “neon green”) or (“user2”, “dislikes”, “rainbows”). Once pretraining is complete, we discard the classification head and retain the resulting user embeddings. In the full C-DPO training stage, we use these user embeddings in combination with the caption of the target image (e.g., “White ceramic bowl”). Positive and negative pairs are constructed using editing instructions that the user liked or disliked (e.g., “Add a holographic pattern to the bowl” as a positive, and “Overlay a floral pattern on the bowl” as a negative).

W3 & Q7: When conditioning on a test-time user w who is not seen during training, how is the corresponding embedding h_w obtained?

Indeed, our approach leverages an inductive embedding framework (GraphSAGE) rather than a transductive one, explicitly designed to generalize efficiently to new users without retraining. Traditional transductive methods are inherently limited to fixed graphs, unable to generalize embeddings to unseen nodes or evolving structures. Rather than generating isolated embeddings for each user, GraphSAGE learns a set of aggregation functions which intake feature information from a node’s neighborhood and aggregates. Applying these aggregation functions efficiently generates embeddings for previously unseen users based solely on their attribute connections, provided the attribute schema matches the training data.

At inference time, embedding a new user does not require retraining. Instead, a new user provides a small set of liked/disliked attributes; we then add a corresponding user node with a fixed-width, zero-initialized feature vector, link it to relevant attribute nodes, and apply our learned aggregation functions in the pretrained GNN encoder. The resulting embedding is computed inductively from the attributes and neighborhood structure alone. While periodic fine-tuning on the expanded graph may further enhance embedding quality, it is optional, not essential for inference, thus preserving the practicality and scalability of our method.

W4: (..) more in-depth quantitative analysis is needed to verify whether correlated attributes, implicitly preferred by similar users, are actually reflected in generated images.

In response to the reviewer’s suggestion, we conducted additional quantitative experiments to assess whether correlated attributes implicitly preferred by similar users are reflected in the generated outputs. For each user, we identified their top 10 most similar users based on the user-preference graph and collected the attributes those users liked. We then measured the CLIP between these collected preferences and the images generated by our method (CLIP-Neighbor), using the same evaluation set as in Tables 1 and 2 of the Appendix:

We will include this experiment to camera ready.

Q1: How is the GNN constructed? (..) How is the computational cost?

The users and their respective liked/disliked edit instructions are represented in a heterogeneous bipartite graph, with users as one node type and attributes as another. We condense edit instructions to keyword attributes to capture broader topics. The graph is not fully connected as users are only connected to attributes they have history with. Users are only indirectly connected via shared keywords; there are no direct user–user edges. The GNN architecture (see W1 above) is pretrained on this graph and only takes ~2 minutes and is frozen for C-DPO training, which remains efficient, completing in about one hour (see Appendix Table 8 for details).

For similarity metric: we rank the neighbors by the # of shared preference attributes i.e. one-hot neighbors in the graph. Thus, the nearest neighbors are the ones who like the most attributes in common with a user.

Q2: In Eq. 7, it seems that the positive/negative pairs (y+, y-) are defined based on user u's preference. (...)

The positive/negative pair (y+,y−) is defined by the anchor user u. The collaborative term does not relabel pairs using a neighbor’s preferences; instead, it re‑evaluates the same pair under the policy conditioned on a neighbor v. This encourages the policy to prefer u’s chosen edits in contexts where like‑minded neighbors would also agree, providing a graph‑weighted regularizer on top of the primary per‑user DPO term.

Potential mismatch is controlled in two ways. (i) We choose only the K nearest neighbors, so the auxiliary signal comes from users with aligned tastes. (ii) The collaborative contribution is weighted by w_uv and scaled by λ ensuring it cannot overpower the individual term. This also ensures that each neighboring user gets a signal that is proportional to how similar those users are to the anchor user. Practically, this design augments u’s sparse supervision with corroborating signals from nearby users.

Q3-1: How is the dataset constructed?

Please see our answer at reviewer F3Yh-Q3.

Q3-2: Instruction generation in a user-agnostic manner vs user u's preferences. How is p_u generated?

Instruction Generation (user-agnostic): Initially, we fine-tune an LLM using LoRA adapters in a user-agnostic manner. This model generates general-purpose editing instructions conditioned solely on image captions and high-level text prompts, without user-specific embeddings.

Personalization via Preferences (user-specific): Personalized instructions p_u are generated by injecting user-specific embeddings derived from our trained GNN. Specifically, each user's embedding (obtained via GraphSAGE from their profile p_u) is converted into soft tokens, prepended to the textual input prompt, and passed through the fine-tuned LoRA-adapted LLM. This strategy yields personalized instructions that reflect the user's editing preferences.

The overall format of the dataset includes pairs of base-image captions and edit instructions, annotated with explicit user identifiers and associated binary preference labels (liked or disliked). Each user profile p_u contains structured sets of preferred attributes (positive) and explicitly avoided attributes (negative), spanning several dimensions such as stylistic themes (e.g., rustic, luxury), color palettes, lighting preferences, visual textures (e.g., distressed, polished), object-level attributes (e.g., wooden beams, marble finishes), and decorative overlays (e.g., floral, cosmic effects).

Q4: In comparison experiments, were the prompts used for baseline methods augmented with both positive and negative attributes from P(u)?

In our comparisons, we indeed augmented the input prompts for all baseline T2I editing methods with both positive and negative attributes explicitly derived from the user's profile P(u). Specifically, we standardized prompt inputs across all tested methods by explicitly providing both liked (positive) and disliked (negative) preferences from each user’s profile. This ensured that all methods received identical, detailed conditioning information, enabling a fair and direct comparison of their personalization capabilities.

Q5: In col. 1 (Q1) of Table 1, DPO-User performs worse than DPO-Vanilla, which seems inconsistent with the authors' claims. (...)

While it is surprising that DPO-User performed slightly worse than DPO-Vanilla on Q1 in our user study, we confirm that our user study was conducted rigorously and carefully controlled. Annotators were recruited via a reputable platform (Prolific) with screening to ensure participants responded reliably. When considering quantitative metrics (CLIP similarity scores reported in Table 2), the DPO-User model outperformed DPO-Vanilla, confirming that user-specific embeddings do, in fact, better capture personalized intent.

Q6: In Table 2, is the reported score based on CLIP similarity between the instruction and the generated image?

In Table 2, the reported scores represent CLIP similarity between the generated textual instructions and the ground-truth textual user preferences. We agree that image-based metrics are also critical for evaluating our method's effectiveness. Therefore, in Tables 1, 2, and 3 of the supplementary material, we provide extensive image-based metrics. Specifically, in Table 1 (supplementary) "CLIP-T-Pref" explicitly reports CLIP similarity computed between the edited images and the user's textual preferences, directly assessing how faithfully the images align with user intent.

Thank you for the detailed response. My concerns regarding the experimental details have been resolved, and the practicality of the proposed method has been well demonstrated, particularly through its ability to handle new users at inference time.

As Reviewer bMep pointed out, conducting experiments using real human preference data, rather than synthetic data, could further strengthen the empirical validity and usefulness of the approach.

I appreciate the authors' detailed rebuttal. The clarification helped improve my understanding of how the dataset is used to construct the GNN and how the model handles unseen users at inference time without additional training. This capability significantly contributes to the practical applicability of the proposed method. I believe that incorporating these explanations into the manuscript would enhance its clarity.

I would like to ask a follow-up question. As far as I understand, the current experiments in the draft seem to evaluate the users that were also seen during training. Could the authors clarify whether this is the case? If so, the validity of the proposed method would be further strengthened by including an evaluation on unseen users through a proper train/test split of the dataset.

Additionally, my concerns regarding the fairness of the comparisons with baselines have been addressed through the authors' response. It would be beneficial to include the detailed prompting conditions used for evaluating those baselines in the manuscript to support reproducibility and transparency.

Lastly, the proposed method appears to follow a two-phase pipeline: i) GNN construction and training, and ii) subsequent diffusion model fine-tuning with DPO. I wonder whether it is possible to extend this framework toward an end-to-end training setup. If such an integration is feasible, I believe it would further enhance the novelty and originality of the work.

Q: As far as I understand, the current experiments in the draft seem to evaluate the users that were also seen during training. Could the authors clarify whether this is the case? If so, the validity of the proposed method would be further strengthened by including an evaluation on unseen users through a proper train/test split of the dataset.

We clarify that our current evaluation is indeed conducted on a held-out test set of unseen users (i.e., users who were not included during training); otherwise, the evaluation would not constitute a fair assessment of our method's generalization capability. After generating the data, we explicitly divided it into training and test splits, where approximately 2900 users were included in the training set and 100 users were reserved exclusively for testing. The test users were selected to exhibit diverse preferences to ensure broad coverage of the dataset. A clear example of our test split is provided under Supplementary Material/Code/dataset/test. We will explicitly emphasize this important detail regarding the train/test split in the revised manuscript to ensure maximum clarity and reproducibility.

Q: Additionally, my concerns regarding the fairness of the comparisons with baselines have been addressed through the authors' response. It would be beneficial to include the detailed prompting conditions used for evaluating those baselines in the manuscript to support reproducibility and transparency.

We thank the reviewer for highlighting this aspect. We agree that explicitly documenting these conditions is important for reproducibility and transparency. As an illustrative example, when evaluating baselines with combined liked and disliked preferences, we used this structured prompt:

System Prompt: 
"You are an expert at providing user personalized image-editing instructions."

User Prompt:
"Likes: [e.g. colorful and playful styles, unicorn motifs, rainbows]
Dislikes: [e.g. minimalistic styles, muted tones, overly realistic imagery]

Original Image Caption: [e.g. "A white ceramic bowl"] 

Suggest a single, cohesive image-editing instruction."

We use the same prompt template across all baselines to ensure fairness during evaluation. In our revised manuscript, we will explicitly include this detailed prompting template, clearly specifying how user preferences (liked, disliked, or both) were integrated.

Q: Lastly, the proposed method appears to follow a two-phase pipeline: i) GNN construction and training, and ii) subsequent diffusion model fine-tuning with DPO. I wonder whether it is possible to extend this framework toward an end-to-end training setup. If such an integration is feasible, I believe it would further enhance the novelty and originality of the work.

We share the reviewer’s enthusiasm for an end-to-end alternative and thank them for highlighting this direction. Our current two-phase procedure is motivated by practical stability and we would like to clarify the role of each phase. Phase 1 serves two critical purposes: (i) a lightweight link-prediction pre-training that prevents cold-start collapse of user embeddings, and (ii) supervised fine tuning of a lightweight LoRA adapter that becomes the frozen reference, an essential step that supplies a well-formed editing-instruction providing as stable anchor for the KL term during phase 2.

Phase 2 then updates the GNN, the projection MLP (soft-prompt tokens), and a trainable LoRA policy adapter jointly under the collaborative DPO loss; thus every parameter that learns from user preferences is trained end-to-end once phase 2 begins.

Nonetheless, integrating these components into a single phase unified, end-to-end training framework represents an exciting direction for future exploration and we will clearly discuss this opportunity in the revised manuscript.