Event-Guided Consistent Video Enhancement with Modality-Adaptive Diffusion Pipeline

We sincerely thank the reviewer for the thorough evaluation and constructive feedback. Below, we provide detailed responses to each point.

Q1: Unfair Training Conditions

Sorry for the confusion. There may be a misunderstanding regarding the training pipeline of EvDiffuser and the comparison methods. We would like to clarify the following points:

1. Overall Framework: First of all, we clarify that our method addresses a novel and realistic problem: modality-adaptive video enhancement under various lighting conditions. Both the model architecture and the dataset are designed synergistically to tackle this problem.

2. Dataset Integration: To enable the model to handle different modality availabilities, our dataset integrates RGB, event, and RGB-event paired data. Based on a modality-agnostic training pipeline, our approach can effectively learn from all these data types. In contrast, the comparison methods are modality-specific and can only utilize a subset of the dataset, making direct training under integrated dataset infeasible. For example, training EvLowlight (event+RGB) on an RGB-only dataset is meaningless because it relies on valid inputs from both modalities for interaction. Similarly, Retinexformer (RGB) cannot be trained on the RGB-event dataset, as it is incapable of processing event data.

3. RGB Augmentation: To further handle diverse illumination conditions, our model applies random degradations to RGB images during training (as illustrated in Line 179). To validate this, we performed an additional comparison by applying the same augmentation strategy to the training sets of RGB-based baseline methods. As shown in the table below, although the baselines show modest improvements, they still struggle to adapt to varying illumination because their modality-interaction mechanisms are not designed for such conditions.

In conclusion, our research focuses on modality-adaptive video enhancement under diverse lighting conditions. The model design, training pipeline, and dataset are all developed in synergy to address this task. The additional comparison using similar RGB augmentation for baseline methods further validates the effectiveness of our approach. We hope this clarifies the misunderstanding and addresses your concerns.

Q2: Computational Efficiency Analysis

We clarify that our framework does not utilize two entire DiT models. Instead, building on the pre-trained CogVideoX DiT backbone (42 blocks), we employ only the first 18 blocks as the conditioning branch, as described in Line 169. This approach significantly reduces computational overhead while preserving effectiveness.

The table below summarizes the efficiency profile of our method. Compared to traditional frameworks, our diffusion-based approach requires more parameters but achieves significantly better generalization across diverse scenarios.

Q3: Single-Modality Performance Results

Fig. 7 have shown the qualitative results of EVDiffuser under single-modality configurations. Quantitatively, we provide results for EVDiffuser on the V-SDE-out dataset. These results demonstrate comparable performance in both event-only and RGB-only configurations compared to existing methods, validating our framework's robustness and flexibility. We will incorporate these results into Table 1 in the revision.

Q4: The smoothness effects introduced by the proposed method

We respectfully disagree with the assessment of smoothness in the “Indoor Scene 3” example. Our method successfully reconstructs fine details, such as the electric wire and its shadow, whereas existing methods exhibit significant artifacts and loss of detail. Quantitative evaluations in Table 1 further demonstrate that our approach achieves higher PSNR and SSIM scores, indicating superior detail preservation compared to existing methods. Additionally, we conducted a user study with 16 participants to evaluate the performance of different methods. We employed preference ranking to assess user satisfaction based on four key aspects: lighting continuity, video quality, detail preservation, and similarity to the ground truth (GT similarity). Across all participants and evaluation criteria, our method consistently achieved the best preference score, showing that it is perceived by users as significantly superior to current baseline methods. We will include these results in the revised manuscript.

We sincerely thank the reviewer for the thorough evaluation and constructive feedback. Below, we provide detailed responses to each point.

Q1: The experimental data are purely synthetic, which may limit the persuasiveness of the results.

Sorry for the confusion. We have included real-world self-captured sequence for qualitative evaluation in the supplementary material (Fig. 8). When transitioning from outdoor to dark indoor environments, our method successfully reconstructs videos with consistent lighting, even under extremely low-light conditions, thanks to the modality-agnostic framework. In contrast, existing methods either overexpose outdoor scenes or fail to reconstruct extremely low-light regions, performing well only within a narrow illumination range. This further demonstrates our method’s strong generalization capability in realistic scenarios. Following your suggestion, we will collect additional real-world data in the future and include it in the revised version.

Q2: Employing two DiT models could make the overall framework computationally expensive during both training and inference.

We acknowledge this concern and would like to clarify our architectural design. Our framework does not utilize two entire DiT models. Instead, building on the pre-trained CogVideoX DiT backbone (42 blocks), we employ only the first 18 blocks as the conditioning branch, as described in Line 169. This approach significantly reduces computational overhead while preserving effectiveness.

The computational requirements are detailed below:

Training Requirements:

• GPU Memory: 76GB for batch size 4
• Training Time: ~2 days on 8×H100 GPUs

Inference Requirements:

• GPU Memory: 31GB
• Inference Time: 5.63 seconds per frame on a single H100 GPU

Additionally, the table below summarizes the efficiency profile of our method. Compared to traditional frameworks, our diffusion-based approach requires more parameters but achieves significantly better generalization across diverse scenarios.

We sincerely thank the reviewer for the thorough evaluation and constructive feedback. Below, we provide detailed responses to each point.

Q1: Event data encoding and VAE usage without fine-tuning

We appreciate this concern regarding our event data processing. First, the three-channel conversion technique is widely adopted in existing methods. Regarding the potential domain gap and the use of a pre-trained VAE for encoding event frames, we evaluated the reconstruction performance and observed a sufficiently low reconstruction error (MSE: 0.0076), demonstrating that the pre-trained VAE can effectively encode event frames. Technically, the VAE is trained on large-scale image data and primarily serves as an encoder to project event frames into latent space. The unique characteristics of event data are captured and learned within the conditional DiT branch. Also, fine-tuning the VAE on event data remains a promising direction for future work and could further enhance performance.

Q2: Different injection approaches for RGB (LoRA) and event data (ControlNet-style)

Table 4 presents the results without LoRA for the image branch. Technically, our architectural design follows modern practices for conditioning diffusion models.

• RGB conditioning: We condition the diffusion backbone by concatenating each input RGB frame with the noisy latent. Since the image-to-video CogVideoX is pre-trained with similar conditioning strategy to extract RGB features, it naturally facilitates extending the diffusion prior to our low-light images. Leveraging this property, we repurpose the generic video-generation model for low-light video enhancement via a lightweight LoRA adaptation, thereby retaining the original diffusion priors while introducing only a minimal parameter overhead.

• Event conditioning: We employ a ControlNet-style conditioning approach, which is a widely adopted technique in recent conditional diffusion models. The ControlNet branch is essential for handling a completely different domain not learned by the pre-trained CogVideoX, similar to recent ControlNet applications (e.g., sketch-based and depth-based generation).

These design choices enable us to effectively leverage the prior knowledge of the pre-trained video diffusion model while adapting it to the specific requirements of our modality-agnostic low-light video enhancement. We will clarify this rationale in the revised version.

Q3: Auto-regressive long video reconstruction limitations

We have demonstrated long-video results (~300 frames) in our supplementary video demo. Technically, unlike existing image-to-video frameworks that rely solely on the first frame and lack intermediate control, our approach is constrained by sequential frames and event data. To further prevent over-reliance on the initial frame, we randomly add noise and mask to the frames during training (Line 191), encouraging the model to leverage both event streams and RGB information more effectively. Overall, these design choices substantially reduce error propagation compared with conventional first-frame-based auto-regressive methods.

Q4: "Modality-agnostic" terminology clarification

We appreciate this concern. By “modality-agnostic,” we mean that our framework can operate effectively and adaptively under varying modality availability, including cases where certain modalities are partially corrupted or entirely absent. This differs from modality-invariant approaches, which assume equal contributions from all modalities. We will make this clearer in the final version. In this work, we focus exclusively on RGB and event data for consistent low-light video enhancement. For modalities beyond RGB and event data, we believe that our modality-agnostic training pipeline is flexible enough to be extended to incorporate additional modalities for diverse downstream tasks, as different modalities can be treated as different conditions to the diffusion backbone. We will discuss this in the final version.

Q5: Limited methodological innovation in diffusion framework

We would like to re-iterate the innovation of our framework. While we build upon modern diffusion techniques, our contributions lie in the novel integration and adaptation of existing techniques for the challenging consistent LLVE task. We facilitate a modality-adaptive diffusion framework that enables consistent LLVE across varying illumination and modality availability for the first time. Technically, we propose a modality-adaptive training pipeline that injects both modalities and fine-tunes on integrated and augmented datasets. Additionally, we present a well-designed modality-adaptive guidance rescaling mechanism that dynamically adjusts guidance based on illumination conditions and modality reliability. Overall, the model design, training pipeline, and dataset are all developed in synergy to address this task.

Q6: Counterintuitive multi-modal performance in Table 1

Thank you for this question. This is indeed a surprising observation that many methods using both event and image modalities perform worse than single-modality approaches. For event-only methods, since these methods input only event data, they are not sensitive to image intensity variations across different scenarios. The intensity of reconstructed frames remains stable within a certain range, resulting in relatively higher PSNR compared to image-involved methods, which are severely affected by varying frame intensities. Additionally, we compare PSNR* with intensity calibration to evaluate the structural fidelity of reconstructed videos, where image-involved methods achieve better performance than event-only methods. When comparing image-only and multi-modality methods, we observe that EvLowlight and EvLight++ achieve better performance, while other methods show comparable performance. This is because these methods typically overfit to specific low-light conditions and mostly fail to adapt to varying illumination scenarios.

Q7: Computational efficiency concerns

The table below summarizes the efficiency profile of our method. Compared to traditional frameworks, our diffusion-based approach requires more parameters but achieves significantly better generalization across diverse scenarios. For efficiency improvements, we have tried one-step diffusion and flow matching, which can reduce sampling to 20 DDIM steps. However, we recognize this represents a trade-off between computational efficiency and performance, which may be a promising future direction.

Q8: Generalization to real-world low-light scenarios

Sorry for the confusion. We have included real-world self-captured sequence for qualitative evaluation in the supplementary material (Fig. 8). When transitioning from outdoor to dark indoor environments, our method successfully reconstructs videos with consistent lighting, even under extremely low-light conditions, thanks to the modality-agnostic framework. In contrast, existing methods either overexpose outdoor scenes or fail to reconstruct extremely low-light regions, performing well only within a narrow illumination range. This further demonstrates our method’s strong generalization capability in realistic scenarios. Due to the time limit, we will collect additional real-world data in the future and include it in the revised version.

Q9: Comparison with contemporary diffusion models

Direct comparison is not practical because EVDiffuser is designed for a different task than general video generation models (e.g., Wan). These large-scale models serve as powerful foundation models and can provide stronger backbones for EVDiffuser compared to CogVideoX. We attempted to adapt the Wan model for our task, and although it achieved better video quality, it incurred an excessively high computational cost, making it impractical for us.

Q10: Performance without both modalities

The model can still generate reasonable results when both RGB and event data are missing. This is because our model is based on the pre-trained CogVideoX and fine-tuned by randomly masking both modalities for CFG inference. However, this scenario is impractical for our LLVE task.

We sincerely thank the reviewer for the thorough evaluation and constructive feedback.

Q1: V-SDE/V-SDSD rely on synthetic illumination curves and may not cover real-world corner cases like sudden flashes.

We have conducted a detailed analysis of illumination frequency variations in our V-SDE dataset (see Table 7 in the supplementary material), which demonstrates the robustness of our method under extreme illumination changes. In addition to synthetic data, we included real-world, self-captured sequence with varying illumination for qualitative evaluation (Fig. 8 in the supplementary material), showcasing our method’s ability to generalize to realistic scenarios.

For even more challenging cases - such as sudden flashes, where real-world data are difficult to obtain - we performed additional quantitative experiments on V-SDE-in by randomly masking frame segments to simulate abrupt illumination changes. The table below shows that our method maintains robustness under such conditions and significantly outperforms existing approaches.

Q2: Performance trade-off between generalization and optimality in fixed conditions.

We appreciate the question. This trade-off is well recognized in the deep learning field (e.g., general-purpose models like GPT versus specialized translation models), where generalization is often prioritized over task-specific optimization due to its practical benefits. Our method follows this principle by focusing on generalization across diverse lighting conditions rather than optimizing for fixed low-light scenarios. While specialized models such as EvLight++ may achieve slightly higher PSNR in specific cases, they tend to overfit and lack robustness in broader contexts.

Q3: Please consider to add more human studies to validate "high-quality" reconstruction claims.

Thank you for this valuable suggestion. We conducted a user study with 16 participants evaluating the performance of different methods. We employ preference ranking to assess user satisfaction across different methods. The evaluation focuses on four key aspects: lighting continuity, video quality, detail preservation, and similarity to the ground truth (GT similarity).

Across all subjects and in every aspect, our method stood out with a consistently superior preference score, highlighting that it is perceived by humans as significantly superior to current baseline methods. We will include these results in the revised manuscript.

Q4: Auto-regressive inference error propagation analysis for long sequences.

We have demonstrated long-video results (~300 frames) in our supplementary video demo. Technically, unlike existing image-to-video frameworks that rely solely on the first frame and lack intermediate control, our approach is constrained by sequential frames and event data. To further prevent over-reliance on the initial frame, we randomly add noise and mask to the frames during training (Line 191), encouraging the model to leverage both event streams and RGB information more effectively. Overall, these design choices substantially reduce error propagation compared with conventional first-frame-based auto-regressive methods.

Q5: Runtime profile and parameter count not provided.

Thank you for the suggestion. The table below summarizes the efficiency profile of our method. Compared to traditional frameworks, our diffusion-based approach requires more parameters but achieves significantly better generalization across diverse scenarios.

Q6: Limited baselines and missing recent methods in Table 1.

We have expanded our comparison to include more recent methods: MambaLLIE (NeurIPS 2024 [1]) for image-based enhancement, BRVE (CVPR 2024 [2] )and Zero-TIG (arxiv 2025 [3]) for video-based enhancement. Since these methods being optimized for specific low-light scenarios, our method maintains a significant performance advantage across all comparisons, as shown below. This expansion will be included in the revision.

[1] Weng J, Yan Z, et al. Mamballie: Implicit retinex-aware low light enhancement with global-then-local state space[J]. NeurIPS, 2024, 37: 27440-27462.

[2] Zhang G, Zhang Y, et al. Binarized low-light raw video enhancement[C]. CVPR. 2024: 25753-25762.

[3] Li Y, Anantrasirichai N. Zero-TIG: Temporal Consistency-Aware Zero-Shot Illumination-Guided Low-light Video Enhancement[J]. arXiv preprint, 2025.

Q7: Inconsistent evaluation across modalities in Table 2.

There may be a misunderstanding regarding Table 2's setting. This table specifically evaluates the RGB-only performance on the V-SDSD dataset under varying illumination conditions, demonstrating our framework's generalization capability in single-modality settings. This design choice allows us to isolate and validate the robustness of our modality-agnostic approache.

Q8: Insufficient ablation study details.

Sorry for the confusion. The ablation study of LoRA on the image branch is presented in Table 4 and Lines 314–318. Regarding the LoRA insertion positions, since we incorporate LoRA only in the image branch, we ablate it by comparing the performance with and without LoRA. The CFG ablation results are shown in Figure 7 and Lines 319–342. For the dropout ratio, we followed the commonly used setting in CogVideoX and set it to 0.2. Due to time constraints, we will include more detailed sensitive analysis on the dropout ratios in the revised version.

Q9: Color prior learning from event-only input without RGB supervision.

We appreciate this question. First, our model is built on a pre-trained video diffusion model (CogVideoX) trained on large-scale RGB datasets, which provides rich color priors. Secondly, during fine-tuning, we introduce a modality-agnostic training pipeline by applying random RGB masking, forcing the model to learn semantic content from event data and learn to associate it with the appropriate color information. This combination of diffusion priors and the novel training strategy enables the model to generate plausible colors even from event-only inputs.

Q10: Color distortions in event-only outputs (Figure 5).

We respectfully disagree with the assessment regarding color distortions. Our method is able to recover plausible colors despite their absence in the event data, demonstrating the model’s ability to capture semantic information. In contrast, previous methods always lose color information entirely. For quantitative evaluation, event-to-video reconstruction is typically assessed in grayscale, focusing on structural fidelity rather than color accuracy. The table below shows quantitative results on the ECD dataset. Our method delivers performance comparable to specialized event-to-video approaches while providing superior generalization across varying illumination conditions and modality availability.

Q11: Training only on synthetic event data without real-world datasets.

There may be a misunderstanding regarding our training datasets. Our model is trained on a combination of synthetic event data and real-world RGB datasets (Line 175-185). Specifically:

• Event-RGB paired: The SDE dataset contains real-world RGB-event paired data captured using DVIS346C cameras.
• RGB-only: The SDSD dataset is a real-world RGB dataset.
• Event-only: The V2E2V dataset is Synthetic event data. Employing synthetic event data for event-to-video reconstruction is standard practice, as event cameras capture high-speed motion and extreme lighting conditions that are difficult for RGB cameras to record simultaneously, making paired real-world data collection extremely challenging.

Based on these training datasets, we conduct extensive quantitative evaluations on our synthetic datasets, as collecting ground-truth data in real-world illumination-varying scenarios is highly challenging. In addition, we include real-world self-captured sequence for qualitative evaluation in the supplementary material (Fig. 8), which demonstrates our method’s ability to generalize to realistic scenarios.

Dear reviewer 37iR,

Thanks again for your valuable suggestions! Given the discussion phase is quickly passing, we want to know if our response resolves your concerns. If you have any further questions, we are more than happy to discuss them. We are looking forward to seeing your response!

Best, All authors.