JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent

We are grateful for your positive feedback and your recognition of our motivation, professional retouching results, and the contribution of the MMArt dataset. Please find our detailed responses to the review comments below.

Q1: Participant demographics in the user study.

To ensure the comprehensiveness of our user study, we recruited 80 participants with diverse professional backgrounds, including art, computer science, photography, and design. The cohort comprises both male (50%) and female (50%) participants aged 18-35, including amateur/professional photographers (35%), digital artists (25%), and general users (40%). This multi-dimensional sampling strategy helps evaluate system performance across different user groups and application scenarios. This will be clarified in the manuscript.

Q2: Evaluation criteria in the user study.

The evaluation criteria focus on two key dimensions: (1) image consistency, which measures how closely the content of the retouched image aligns with the source image; and (2) aesthetic quality, which evaluates the artistic merit of retouched images based on professional photography standards, including color harmony, composition balance, and visual impact. To maintain consistency with prior user studies [1][2], we adopt the five-level quality scale due to its simplicity, efficiency, and effectiveness in distinguishing significant differences in retouching quality. Notably, during the aesthetic assessment, participants are required to evaluate the overall aesthetic quality based on three key aspects—color harmony, lighting and shadow, and visual appeal—before scoring. This will be clarified in the manuscript.

Q3: Discussion of refined aesthetic evaluation.

Pairwise comparison is a more refined evaluation method where participants are shown two images side-by-side and asked to choose their preference or identify the higher-quality one. This approach excels at capturing subtle differences in user preference and can mitigate biases like the anchoring effect, but its efficiency significantly decreases with a larger number of images due to the exponential increase in comparisons needed, potentially leading to participant fatigue and requiring more complex statistical analysis.

---

References:

[1] Step1x-edit: A practical framework for general image editing. Arxiv 2025, Github star 1.5k.

[2] MagicQuill: An Intelligent Interactive Image Editing System. CVPR 2025.

We are grateful for your positive feedback and your recognition of our writing quality, the promise of MLLM-driven artistic agents, and the value of the MMArt dataset. Please find our detailed responses to the review comments below.

Q1: Open-source plan for the MMArt dataset.

The dataset will be released once the paper is accepted.

Q2: Technical novelty of this work.

As acknowledged by Reviewer NfhL, one of our primary contributions is the development of a robust and scalable infrastructure, comprising a data-synthesis pipeline and a fine-tuning framework that effectively transform MLLMs into artistic agents. While our training framework relies on widely used techniques like SFT and RL, we propose GRPO-R, an RL method tailored for tool-integrated retouching tasks. In contrast to standard GRPO—which targets problems with single, verifiable answers—retouching tasks involve two key challenges: (1) predicting multiple tools and their parameters, and (2) managing the inherent ambiguity in parameter combinations that yield visually similar results. This complexity makes reward design particularly difficult. GRPO-R addresses this by incorporating two reward components: retouching operation accuracy for parameter prediction and perceptual quality for visual fidelity. This specific reward design enables scalable reinforcement learning, thereby enhancing the agent's reasoning ability, tool proficiency, and generalization.

Q3: Accuracy of the intermediate steps of MLLM.

During the data curation process, to ensure the accuracy of intermediate reasoning steps generated by the MLLM, we adopt a two-stage verification approach: (1) the Qwen2.5-VL-72B model evaluates the alignment between instructions and reasoning via multimodal input prompts, and (2) manual sampling checks are conducted to further validate quality. This verification procedure will be clarified in the manuscript.

During JarvisArt's inference phase, artistic reasoning may introduce error accumulation that degrades overall performance. For example, if the model predicts an incorrect adjustment such as "exposure +50", it may produce suboptimal visual results. Since the model lacks immediate visual feedback during reasoning, such errors are not corrected and may compound in subsequent steps. Introducing stepwise visual rewards during the RL phase may allow the model to validate intermediate predicted parameters, thereby improving the reliability of the model during inference and reducing errors. Ongoing work aims to further optimize this implementation.

Q4: Impact of reward scores on performance outcomes.

To analyze the correlation between reward scores and performance metrics, we conducted progressive assessments of checkpoints at four training milestones (25%, 50%, 75%, and 100% completion) during RL training. As shown in Table 1, results on MMArt-Bench demonstrate a statistically positive correlation between reward scores and performance metrics. This confirms that the GRPO-R can effectively enhance the model's reasoning, tool proficiency, and generalization during reward optimization.

Table 1: Analysis of correlation between rewards and performance

Q5: Failure cases and future work.

In this work, we focus on pushing the boundaries of photo retouching and offer new insights into how artificial intelligence can empower and elevate artistic creation. While promising, the current system has several limitations. First, it lacks multilingual capabilities, which we aim to address by adding linguistically diverse training data. Second, it struggles to capture fine-grained user intent, particularly in localized facial edits (e.g., eyes and lips). To improve performance, we propose two strategies: (1) augmenting the training dataset with examples reflecting subtle and specific user editing preferences, and (2) designing fine-grained reward mechanisms in the RL phase to help the model better capture nuanced user intent and invoke appropriate retouching operations.

Q6: Citations of related works.

We will cite all listed works.

We appreciate your positive feedback and recognition of our clearly articulated motivation, comprehensive data generation and training workflow, and the contribution of the MMArt dataset. Please find our detailed responses to the review comments below.

Q1: Full hyper-parameter setting for SFT and GRPO-R.

To ensure reproducibility, we provide the complete hyperparameter settings for both the SFT and GRPO-R phases in Table 1. These details are included in the revised manuscript.

Table 1: Hyper-parameter settings for SFT and GRPO-R

Q2: Rationale for 5K samples in GRPO-R training.

We use 5K training samples for the GRPO-R phase for two main considerations.

• First, as shown in Table 2 of the manuscript, even this modest data yields substantial improvements over the SFT baseline: L1 and L2 errors drop notably (14.42 → 12.44, 44.38 → 30.56), while SC, PQ, and O all increase (7.32 → 7.53, 8.67 → 9.82, 7.94 → 8.52).
• Second, scaling beyond 5K samples significantly increases computational costs, especially due to the overhead of parallel Lightroom invocations. Given our current resource constraints, large-scale RL training is not yet feasible, though we anticipate further gains with increased data in future work.

Q3: Quantitative comparison with commercial photo retouching methods.

We provide both scene- and region-level quantitative comparison, including commercial retouching tools and instruction-driven generative editing models. As shown in Table 2, JarvisArt combines superior pixel-level content preservation—comparable to commercial tools like Google Photos and Lightroom Auto—with advanced instruction-following capabilities on par with GPT-4o and Gemini-2-Flash. This integration positions JarvisArt as a versatile system that unifies the strengths of both paradigms. Notably, evaluating instruction-driven retouching tasks is inherently subjective: reference images represent only one of many acceptable outputs, and traditional image metrics often fail to reflect aesthetic quality or user preferences. As discussed in Section 4.2.2 of the manuscript, user studies may provide a more reliable and human-centric evaluation.

Table 2: Quantitative comparison on both scene-level and region-level

Q4: Inference efficiency.

Table 3 summarizes the average wall-clock time for processing 200 images at 24MP resolution across various deployment frameworks and precision settings. The inference process comprises two main components: (1) The VLM generates Lightroom retouching parameters; (2) these parameters are applied locally within the user's Lightroom environment. Notably, we set VLM to support input images up to 1,048,576 pixels for efficiency. For larger images, automatic resizing with aspect ratio preservation ensures no distortion. Since the VLM is solely responsible for parameter generation, the actual retouching is performed by Lightroom, which supports arbitrary resolutions and image formats. Results show that vLLM with bfloat16 precision yields the optimal wall-clock performance (18.21s total, with 12.51s for VLM and 5.69s for Lightroom). Future work will explore further acceleration strategies, such as model quantization and speculative sampling. As a pioneering effort, this study introduces the concept of an artist agent and establishes an effective fine-tuning framework. We will continue to enhance this paradigm.

Table 3: Comparison of inference time across frameworks and precision settings

System configurations:

• VLM inference time is measured on an NVIDIA A100 GPU (80 GB).
• * denotes VLM inference time measured on an NVIDIA RTX 4090 GPU (24 GB).
• Lightroom processing is conducted on a Windows 11 system equipped with an AMD Ryzen 9 5900X CPU and an NVIDIA RTX 3090 GPU.

Q5: Privacy considerations.

To ensure user privacy, the Agent-to-Lightroom (A2L) protocol adopts a strictly sandboxed execution architecture and a fully localized processing workflow. As detailed in Supplementary Section A, the protocol runs entirely within the user's local Lightroom environment: (1) File verification is performed via OS-level checks without any external transmission; (2) all operations are confined to Lightroom's Lua environment through official APIs; and (3) retouching tasks are processed asynchronously in the local background. This design guarantees that user photos never leave the local Lightroom instance—no data is transmitted or stored outside the user's device.

Q6: Inference cost on mobile or low-compute devices.

In this work, we focus on pushing the boundaries of photo retouching and offer new insights into how artificial intelligence can empower and elevate artistic creation. Deploying models to mobile devices or edge devices is beyond the scope of this paper. For such scenarios, we believe that model compression techniques—such as quantization, pruning, and knowledge distillation—can effectively reduce model size and inference costs, thus supporting such applications.

Q7: Failure cases and future work.

While JarvisArt is promising, the current system has several limitations. First, it lacks multilingual capabilities, which we aim to address by adding linguistically diverse training data. Second, it struggles to capture fine-grained user intent, particularly in localized facial edits (e.g., eyes and lips). To improve performance, we propose two strategies: (1) augmenting the training dataset with examples reflecting subtle and specific user editing preferences, and (2) designing fine-grained reward mechanisms in the RL phase to help the model better capture nuanced user intent and invoke appropriate retouching operations.

Q8: Support for Lightroom’s AI features.

To facilitate the model's adaptation to AI functionalities in Lightroom—such as background removal—we will employ the proposed data pipeline to generate paired artist-edited datasets aligned with specific AI operations. Fine-tuning on this targeted data is expected to enhance the model’s proficiency with these tools.

Q9: Paper presentation and citations of related works.

We have made the necessary revisions and will cite all listed works.

We sincerely appreciate your feedback and recognition of our comprehensive evaluation, professional retouching results, the contribution of the MMArt dataset, and the potential of MLLM-based artistic agent applications. Please find our detailed responses to the review comments below.

Q1: Contributions of this work (infrastructure and GRPO-R).

As you mentioned, one of our key contributions is the development of a robust and scalable infrastructure, comprising a data-synthesis pipeline, a fine-tuning framework, and an Agent-to-Lightroom protocol that effectively transform MLLMs into artistic agents. Additionally, we introduce GRPO-R, a tailored RL method for tool-integrated retouching tasks. In contrast to standard GRPO—which applies to mathematical tasks with a single, easily verifiable correct answer—retouching involves multi-dimensional complexity: (1) the model must predict multiple tools and their parameters, and (2) diverse parameter combinations may yield visually similar results. Designing effective reward signals in such a setting remains an open challenge. GRPO-R addresses this by incorporating two reward components: retouching operation accuracy for parameter prediction and perceptual quality for visual fidelity. This specific reward design enables scalable reinforcement learning, thereby enhancing the agent's reasoning ability, tool proficiency, and generalization. We believe this system design and tailored RL optimization can provide inspiration for the development of domain-specific intelligent agents, beyond the demonstrated success in Lightroom.

Q2: Scope limitations.

This work focuses on a natural language-driven retouching task, which differs from generative editing. While generative editing involves creating new content or modifying image structures for creative purposes, photo retouching aims to enhance the original image by adjusting visual attributes—such as color, tone, and sharpness—without introducing external elements. Additionally, our method can be extended to support generative editing operations through the proposed data pipeline, which enables generating paired artist-edited data aligned with specific AI operations (e.g., addition, deletion), allowing the model to learn these capabilities.

Q3: Misleading pixel-level comparisons.

To ensure a fair and comprehensive comparison, we evaluate JarvisArt against both traditional photo retouching methods and instruction-driven generative editing models. Following prior studies [1][2][3], we employ L1 and L2 metrics to assess pixel-level content preservation, and SC, PQ, and O metrics to measure instruction adherence. As shown in Tables 1 and 4 of the manuscript, our method achieves pixel-level fidelity comparable to traditional retouching while significantly outperforming generative models. Moreover, the SC and PQ results indicate competitive instruction-following performance relative to generative editing approaches. Through these multidimensional evaluations, we aim to highlight that JarvisArt integrates both the pixel fidelity of traditional retouching and the instruction-following capabilities of generative models, effectively achieving the strengths of both approaches.

Q4: Lack of sufficient robustness of dataset curation.

To guarantee data quality, we employ a two-step verification process: (1) The Qwen2.5-VL-72B model evaluates the alignment between instructions and reasoning via multimodal input prompts. (2) Manual sampling is conducted for additional validation. This process will be clarified in the manuscript. Aggregating outputs from MLLMs and applying consensus-based filtering can enhance data quality, but it may incur higher computational and resource costs. To further improve sample quality, we currently regenerate instruction and chain-of-thought (CoT) annotations using Gemini-2.5 Pro. We plan to release this version as open-source.

Q5: Explanation of ROC file generation.

As described in Section 3.2 of the manuscript, the retouching operation configuration (ROC) file and corresponding before/after-retouch images are prepared in advance during the instruction and chain-of-thought (CoT) reasoning generation process. After generating the instruction and reasoning text, the ROC file is inserted within the <answer> tag to produce an R1-style QA sample. The MLLM is not required to regenerate the ROC file, as its parameters are summarized manually by professional photographers.

Q6: Versatility of the MMArt dataset across research fields.

The MMArt dataset may provide significant value for research in three key areas: (1) unified multimodal understanding and generation, (2) text-to-image generation, and (3) image-to-image generation. Specifically, the dataset includes diverse user instructions, high-quality photographic images, and chain-of-thought (CoT) annotations, all of which support the development and evaluation of models across these tasks.

Q7: Detail of technical terms.

In Lightroom, a preset refers to a pre-saved retouching operation configuration (ROC) file, typically created by photographers or artists. It encapsulates stylistic parameters tailored for specific visual aesthetics—for example, a Japanese-style fresh look. Applying a preset enables users to efficiently reproduce a desired effect across multiple images. However, presets are often context-specific and may not yield satisfactory results for all image types. As outlined in Section 3.2, Stage I of the manuscript, we enhanced the accuracy of preset recommendations for the Qwen2.5-VL-72B model by summarizing each preset’s functional description. For example, the preset {"id": "PERSON-G20", "name": "Japanese Fresh Style"} is described as follows: “This preset reduces overall saturation (-5) and warms tones (yellow/orange saturation -43/-42), emphasizes green brightness (+50), and enhances blue saturation (+31) to create a refreshing visual effect. It is suitable for outdoor portraits and Japanese-style compositions under natural lighting.” These terms will be detailed in the manuscript.

Q8: Typo errors and paper presentation.

All typo errors have been corrected in the revised manuscript. Additionally, we have simplified the diagram, clarified the transitions, defined all relevant terms, and explicitly distinguished the stages of dataset generation, training, reasoning, and output. PQ and SC metrics will be explained in the main text, with Lightroom protocol details included in the Appendix.

---

References:

[1] MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing. NeurIPS 2023 Datasets and Benchmarks.

[2] Emu Edit: Precise Image Editing via Recognition and Generation Tasks. CVPR 2024.

[3] In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer. Arxiv 2025, Github star 1.9k.