To Reviewer WirW

We sincerely thank you for reviewing our work. After reading your comments carefully, we summarize the following questions.

Q1: Cross-Dataset Generalization

---

Please refer to Q1 in the To All section for a detailed answer to this question. To summarize, we conducted additional evaluations using the HDR+ dataset to demonstrate our RL-based framework's ability to generalize effectively to unseen datasets. Our method achieved a PSNR of 31.54 on the HDR+ photo-finishing tuning task, which is comparable to the 32.47 achieved on the FiveK dataset, and it outperforms all baselines.

This highlights our method's strong generalization capabilities across different datasets, outperforming other methods such as CMAES, Greedy Search, Cascaded Proxy, and Monolithic Proxy. Our approach leverages a robust state representation that captures invariant features for photo finishing, enabling it to adapt to diverse inputs and target outputs beyond the training distributions. These results affirm that our RL-based framework is not only effective in the scenarios it was trained on but also exhibits superior performance on datasets it has not previously encountered.

Q2: Efficiency Comparison of CMAEAS

---

Our implementation of CMAES takes full advantage of parallel computing capabilities on a high-performance 48-core CPU, ensuring a fair efficiency comparison for efficiency.

Parallelization Enabled by pymoo Library: For the CMAES method, we implemented the baseline using the widely-used Python library pymoo, which supports parallelization. In our experiment, we enable the parallelization options provided by pymoo. This is done by setting classpymoo.algorithms.soo.nonconvex.cmaes.CMAES(... , parallelize=True, ...) and enabling elementwise_renner when calling the pymoo library.

Hardware Configuration: Our speed testing experiments were conducted on a high-performance server-level system equipped with AMD EPYC 7402 (48 Cores) @ 2.8 GHz CPU, 8 NVIDIA RTX 4090 GPUs, 512 GB of RAM, and running CentOS 7.9. With this configuration, we utilized the full capabilities of the 48-core CPU to explore potential speed-ups for CMAES through multi-processing. This setup ensures that the CMAES method was not constrained by computational resources and accurately reflects the method's performance potential.

Multi-Core Execution: To clarify, the reported inference times for the CMAES method were obtained by running the algorithm on all available 48 CPU cores, rather than a single core. By doing so, we provided a realistic comparison with other methods.

Dear Reviewer WirW,

In our rebuttal, we test our model on an unseen dataset, HDR+. Results demonstrate that our method generalizes well to unseen datasets, outperforming all baselines by a large margin. We also explained that the CMAES baseline is implemented with a multi-core CPU with parallel acceleration.

We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed.

Thank you again for your time and effort.

To Reviewer kK87

Q1 addresses the implementation details of three baselines. Q2 Q3 explain certain behavior (hue shift, green image) of baselines. Q4 extends extra details on baseline implementation and proves our baselines were implemented correctly.

Q1: Full Details on Baseline Implementations, Terms "Cascaded & Monolithic"

---

Please refer to Q3 in the To All section for how we implement each baseline, including code used and full details. Please also refer to Q2 in the To All section for clarification of the terms "cascaded & monolithic proxy".

Reviewer pointed out an incorrect citation of CMAES in the appendix. On line 487, we make a typo on the citation of CMAES; the correct citation should be [18], not [26]. We apologize for this misunderstanding and will correct it in our paper. Since [18] does not provide code, we reproduce it using pymoo.

Q2: Similar Hue Shift for Proxy-Based Methods [26,27]

---

Hue shift is a common artifact. As shown in Figure 2 of the PDF file, snapshots from the result figures in the [27] paper also exhibit severe hue shift. In our paper, [26] and [27] have similar hue shifts in some cases, and we believe it was a result of inherent challenges in proxy-based methods, and is due to similar errors in proxy networks.

The similar inaccuracies of proxy networks are due to the similar data bias introduced during proxy training data generation. [27] utilize a uniform strategy to generate proxy training data:

"We uniformly sample 100 points for each slider when generating the data for each intermediate." --Section 5,[27]

This uniform sampling strategy introduces bias because sub-ranges of ISP parameters may not distribute evenly and often have non-linear effects. And ISP operations are black-box so the parameter range cannot be considered during data sampling. For example, the white balance parameter (RGB color channel coefficients) ranges from [0.6, 1.65], where 1 represents no gain. Uniform sampling results in more data points between [1, 1.65] than [0.6, 1], potentially leading to biased training data. Since the proxy network is purely data-driven, such data bias can lead to similar inaccuracies. As a result, both proxies tend to behave similarly, especially in complex ISP operations like color adjustments.

Then, both methods use the same regression optimization approach with the Adam optimizer and identical hyperparameters, leading to similar optimization dynamics. It results in similar parameter shifts, particularly in white balance adjustments, where even slight inaccuracies can cause visible hue shifts. Importantly, these shifts do not occur in all cases, as shown in Figure 3 of our paper and Figure 1 of the rebuttal PDF. In conclusion, similar hue shifts are expected due to shared conditions and challenges in proxy-based ISP tuning.

Q3: Green Image in CMAES Figure 4

---

The color shift in Figure 4 is an occasional failure case specific to the stylization task, not the photo finishing tuning task. It occurs because style loss is too complex for the CMAES to explore the search space, causing the optimization stuck in local minimal. In contrast, our RL-based method has superior exploration capabilities under such cases.

Q4: Extra Details on Each Baseline Implementation

---

We extend extra details on the performance and implementation of each baseline, to prove our baselines' correctness.

Cascaded Proxy [27]

Baseline [27] underperforms due to (1) our pipeline having more image processing operations, (2) the neural proxy cannot generalize well to unseen images, and (3) the unavailability of the dataset used in [27].

Accuracy Drop with More ISP Hyperparams: As shown in the table below, proxy approximation accuracy and finishing quality of [27] decreases with more ISP operations. This is due to error accumulation in the proxy pipeline with more ISP operations, which leads to inaccurate gradient and suboptimal parameter recovery, causing the finishing quality to decrease.

Since [27] recovers only 4 parameters while our experiment recovers 9, the finishing quality of is naturally lower in our paper. Additionally, ISP operations like sharpening are difficult for neural networks to approximate, as mentioned in [27], which further impacts performance.

Neural Proxy's Generalization: The neural proxy learned in [27] struggles to generalize to unseen images, as shown in proxy accuracy drops on validation data:

This explains the poorer performance in our additional HDR+ experiments.

Dataset used in [27] not released. Therefore, the numbers reported by [27] are not reproducible. We trained this baseline on the FiveK dataset, following the details in [27].

Monolithic Proxy [26]

Our implementation reported a PSNR result above 20 on the FiveK dataset, comparable to the result reported by [27] when they reproduced this baseline. Like [27], [26] struggles with complex ISP operations and generalization. The vanishing gradient issue, analyzed in [27], further affects its performance.

CMAES [18]

Reported PSNR Comparable to [27] Paper: Our paper shows a PSNR above 28 for CMAES [18] on ISP tuning, comparable to the result reported by [27].

CMAES Performance with More Iterations: In the table below, the performance of CMAES improves with more iterations, eventually reaching a PSNR of 32.12 on the FiveK dataset, comparable to RL's quality. While this indicates that our reproduction of the CMAES baseline is correct, the increased iterations lead to slower performance due to more queries to the ISP.

Dear Reviewer kK87,

In our rebuttal, we address the implementation details of three baselines, explain the code used, and clarify terms in the paper, along with citations of CMAES (Q1). We also point out that certain behaviors of the baselines (such as the similar hue shift and green image in Figure 4) are due to the inherent flaws of the baselines (Q2 Q3). Additionally, we provide more detailed experiments of the baselines to prove the correctness of our reproduction (Q4).

We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed.

Thank you again for your time and effort.

To reviewer syDC

We sincerely thank you for reviewing our work. After reading your comments carefully, we summarize the following questions.

Q1: Choice of RL Algorithm

---

In our task, the choice of RL algorithm is not the primary factor driving performance; instead, our proposed state representation plays the most crucial role. We selected the TD3 algorithm due to its common usage and robustness to hyperparameters, but alternative algorithms like SAC or PPO would yield similar results.

To support this, we performed an additional ablation study comparing different RL algorithms. We trained our photo-finishing policy using SAC and PPO algorithms, keeping the state representation, policy network, reward design, and termination conditions identical to our original TD3 implementation. All implementations were based on the stable-baselines3 library in PyTorch. We evaluated each algorithm's performance on the photo-finishing tuning task using the FiveK-Random dataset.

Table 1: RL Algorithm Comparison

As shown above, different RL algorithms achieve comparable results on this task. However, removing the proposed state representation from the TD3 algorithm results in a significant performance drop, highlighting its importance. The results demonstrate that the state representation, not the specific choice of the RL algorithm, is the key factor behind our method's superior performance.

Q2: Additional Ablation over Greedy Algorithm

---

In our paper, we have shown that our RL-based method achieves higher photo-finishing quality and efficiency than optimization-based methods. We compared our method with baselines using zeroth-order search and first-order optimization with differential proxies but did not previously compare it with a simple greedy algorithm. Here, we provide an additional ablation study using a greedy algorithm.

For the greedy algorithm, we followed Algorithm 1 from the [CRISP] paper, implementing a greedy search algorithm to find the optimal ISP parameters. This greedy algorithm mimics human behavior by incrementally adjusting each element towards the desired photo-finishing result until a stopping condition is met.

[CRISP] Learning Controllable ISP for Image Enhancement. TIP. 2023.

As shown in Algorithm 1 in the Pseudo Code for Greedy Algorithm section below, the greedy algorithm mimics user behavior by incrementally improving image quality to achieve the desired results. The algorithm starts with the ISP parameters at $s_{init}$ and iteratively adjusts them with a step size $t$ until the stopping condition $K$ is reached. We set each element in $s_{init}$ to be 0, step size $t=0.1$, and iterations $K=200$ to be consistent with other optimization-based methods. We evaluated the greedy algorithm on the FiveK-Target and HDR+ datasets, with the results shown in Table 1 of the rebuttal PDF. We also present the results in the table below.

From the results, it can be observed that while the intuitive greedy algorithm achieves reasonable performance, it performs worse than the CMAES baseline under the same iterations. This indicates that the greedy strategy is not as effective as the evolutionary strategy in the CMAES algorithm. However, the greedy algorithm performs better than proxy-based methods [26, 27] because it requires no proxy training and searches directly in the parameter space. Note that like CMAES, the greedy tuning algorithm requires at least hundreds of queries to the ISP pipeline to converge, which are very time-consuming.

Both greedy and CMAES are outperformed by our RL-based approach. This is because their searching processes are blind and brute-force, not conditioned on the input image, target image, or any prior on image processing operations. Our RL-based approach considers all these factors, requires no proxy training, and thus achieves better photo-finishing quality and efficiency.

Q3: More Evaluation Dataset

---

Please refer to Q1 in the To All section for the answer to this question. To summarize, we conducted further evaluations using the HDR+ dataset, demonstrating our RL-based framework's ability to generalize effectively to unseen datasets. Our method achieved a PSNR of 31.54 on the HDR+ photo-finishing tuning task, comparable to the 32.47 achieved on the FiveK dataset, and outperformed all baselines. This highlights our method's generalization capability across different datasets and its superior performance compared to other baselines, including CMAES, Greedy Search, Cascaded Proxy, and Monolithic Proxy.

Pseudo Code for Greedy Algorithm

---

$**Algorithm 1: Greedy Tuning Algorithm**$

---

$**Inputs: ** s_{\text{init}} \in \mathbb{R}^{D}, \text{ Step size } t, \text{ Stop condition } K, \text{ Input image } x, \text{ Tuning target image } \hat{x}$
$**Initialization: ** e \gets \infty, s \gets s_{\text{init}}, d \gets 1, i \gets 0, k \gets 0$
$**While ** k \leq K ** do**$
$\quad **if ** e < \text{MSE}(f_{\text{PIPE}}(x, s), \hat{x}) ** then**$
$\quad \quad s_d \gets s_d - t, d \gets (d \bmod D) + 1, i \gets i + 1, k \gets k + 1$
$\quad \quad **if ** i = D ** then**$
$\quad \quad \quad s \gets s + t, d \gets 1, i \gets 0$
$\quad \quad **end if**$
$\quad **else**$
$\quad \quad e \gets \text{MSE}(f_{\text{PIPE}}(x, s), \hat{x}), i \gets 0, k \gets 0, s_d \gets s_d + t$
$\quad **end if**$
$**end while**$

---

Dear Reviewer syDC,

In our rebuttal, we explain the reason for choosing TD3 and include further ablation studies to show that the choice of RL algorithm does not significantly impact performance. We also conduct more ablation over the greedy algorithm as suggested by the reviewer. Additionally, we test our method on the HDR+ dataset. Results demonstrate that our method generalizes well to unseen datasets.

We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed.

Thank you again for your time and effort.

Dear Reviewer syDC,

Thank you for your reply. We would like to further discuss the questions with you below:

Q1. why TD3 performs better

In our task, the RL algorithm is only a tool for optimizing the policy network. We analyze why TD3 performs better as follows:

TD3 is an off-policy RL algorithm with a deterministic policy. The deterministic policy allows it to explore the entire action space, including actions near the boundary, which is crucial for recovering boundary ISP parameters in our photo finishing task. TD3 also employs tricks like target policy smoothing and delayed policy updates to enhance robustness. In our task, actions like significantly brightening an image can cause large changes in the reward function, so these robustness tricks are crucial to the stability of the RL algorithm.

In contrast, SAC uses a stochastic policy, sampling actions from a Gaussian distribution, which reduces the likelihood of exploring boundary actions, impacting performance. PPO also faces this issue with its stochastic policy. Both SAC and PPO also lack robustness tricks like target policy smoothing. Moreover, PPO is an on-policy algorithm, making it less sample-efficient in our complex photo finishing environment, potentially leading to suboptimal results.

These differences explain the variation in performance, but the effect of the RL algorithm is still minor compared to state representation. As shown in the first table of the rebuttal, TD3, SAC, and PPO all yield a PSNR of around 37 with state representation, whereas TD3 without state representation achieves only 32.17.

Q2. multiple seeds

We set the random seed to 1 in all our experiments in the paper and the rebuttal, ensuring fair comparison. In our main comparisons with all baselines, our method outperforms them by a large margin (over 4 dB in PSNR). Even in the ablation study over different RL algorithms, TD3 still outperforms SAC by 1.16 dB in PSNR. As discussed earlier, the RL algorithm is just a tool within our framework, and we are confident that we have selected the RL algorithm that best fits our task. Additionally, we plan to include the average scores over multiple seeds in the camera-ready version of this paper.

Q3. greedy tuning performs well & stronger baseline with conditioning

There are no stronger baselines available. We followed the setup from [27] for baselines in the photo finishing task. We compared our method to existing work in the field of photo finishing tuning, including CMAES [18] and proxy-based methods [26, 27]. Currently, there are no stronger baselines for the photo finishing tuning task.

Search-based methods (Greedy & CMAES) perform well because they optimize over more iterations. Both the greedy tuning algorithm and CMAES are search-based methods. The search-based methods, including the greedy algorithm added in the rebuttal, only performs well when using 200 searching iterations, while our method can achieve better results with only 10 iterations. The main advantage of our RL method is its fast convergence speed, as shown in Table 2 in our paper.

Conditioning is not possible for greedy algorithm and is part of our contribution. Traditional search-based methods, including greedy algorithms, are driven purely by the loss function's output relative to the parameters being adjusted. These methods lack the ability to condition on complex features like the characteristics of the input image, the target image, or prior processing operations. Such conditioning requires an understanding capability and the ability to generalize from past experiences, which is beyond the capabilities of traditional search-based algorithms.

Developing a novel search algorithm conditioned on these complex characteristics is one of our contributions. Our proposed state representation encodes information about the input image, target image, and prior image processing operations. The policy network can then leverage this information to make decisions that are conditioned on these inputs, which leads to more efficient and effective optimization.

Q4. why the proposed method generalizes well

We have analyzed why our method generalizes better in Q1 (cross-dataset generalization) of the To All section in our rebuttal.

To Reviewer ASBs

We sincerely thank you for reviewing our work. After reading your comments carefully, we summarize the following questions.

Q1: Details on Baselines

---

In our main paper, we compare three baselines: (1) CMAES: a zeroth order optimization method, that does not need training, (2) Monolithic Proxy: first-order optimization through a single monolithic neural network proxy, that requires training a proxy network, and (3) Cascade proxy: first-order optimization through multiple cascaded neural network proxies, also requires training proxy networks.

Each of the baseline's implementation details is available in Q3 of To All section. Here, We briefly summarize the principal of each of these baselines:

1. CMAES [18]: CMAES is a gradient-free search (zeroth order optimization) method using an evolution strategy. For our photo finishing tuning task, it directly optimizes the parameters of the image processing pipeline to match the target rendering style. Therefore, it does not undergo a training stage and optimizes directly on the validation set for multiple iterations instead. Such method faces no generalization challenges but converges very slowly as the searching processing is blind and brute-force. It also lacks exploration capability and is prone to stuck in local minimal with complex objectives.
2. Monolithic Proxy [26]: is a gradient-based optimization method (e.g. gradient descent). Since our task involves tuning an image processing pipeline (ISP) that is a non-differentiable black box, the gradient cannot flow through the ISP pipeline directly to drive the optimization of ISP parameters. Therefore, it requires training a neural network proxy to approximate the behavior of the ISP pipeline. The differential nature of the neural network proxy allows gradient flow to directly optimize ISP parameters during inference. However, for a complex image processing pipeline, this proxy is hard to train and may not fully reproduce original pipelines.
3. Cascaded Proxy [27]: is also a gradient-based optimization like [26], but the proxy neural network has a different architecture. The difference is that: the Monolithic Proxy uses a single UNet for the proxy, but the Cascaded Proxy improves the UNet with multiple small neural networks that are cascaded to address the vanishing gradient problem. Yet it still faces challenges with error accumulation, limited generalization capability of neural proxy, and cannot be applied to black-box pipeline.

Please refer to Q3 in the To All section for further implementation details for these baselines.

Q2: More Evaluation

---

Please refer to Q1 in the To All section for the answer to this question. To summarize, we conducted further evaluations using the HDR+ dataset, demonstrating our RL-based framework's ability to generalize effectively to unseen datasets. Our method achieved a PSNR of 31.54 on the HDR+ photo-finishing tuning task, comparable to the 32.47 achieved on the FiveK dataset, and outperformed all baselines. This highlights our method's generalization capability across different datasets and its superior performance compared to other baselines, including CMAES, Greedy Search, Cascaded Proxy, and Monolithic Proxy.

Dear Reviewer ASBs,

In our rebuttal, we explain whether each baseline requires training. We also provide more evaluation on the HDR+ dataset. Results demonstrate that our method generalizes well to unseen datasets, outperforming all baselines by a large margin.

We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed.

Thank you again for your time and effort.