Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection

We sincerely appreciate Reviewer 1w7F's valuable feedback. Our responses to the weaknesses, questions, and limitations are listed below.

R-W1 Data augmentation explored before for maritime detection We respectfully disagree with this comment. To our knowledge, maritime scene generation for data augmentation has not been explored in any prior work. While image generation has been applied in other domains such as autonomous driving (the suggested [A, B]) and aerial imagery (the suggested [C]), these approaches are fundamentally incompatible with the maritime setting.

The cited methods assume structured environments with stable, predictable spatial layouts and well-defined object categories. In contrast, maritime scenes are inherently unstructured, with highly variable sea states, ambiguous object boundaries, and complex object-water interactions that lack reliable contextual anchors. These unique challenges render existing approaches ineffective when applied to this domain.

Moreover, our empirical results in Table 2 and Figure 4 clearly demonstrate that general or cross-domain augmentation strategies fail to produce meaningful performance gains in maritime detection. This reinforces the need for a domain-specific generation framework, which our work is the first to provide. Our approach directly addresses the critical aspects of maritime imagery that existing methods overlook, establishing a new and necessary direction for data-driven learning in this field.

R-W2 Domain-specificity We respectfully disagree with this comment. The domain-specific nature of our work should be seen as a strength, not a limitation. Addressing complex real-world problems, such as maritime object detection, requires deep domain expertise, problem-specific modeling, and careful attention to context-specific challenges. These qualities reflect the type of thoughtful, impactful work that NeurIPS has a long history of supporting.

It is a misconception to view NeurIPS as a venue limited to general-purpose algorithms. In reality, a significant portion of NeurIPS papers are domain-specific, leveraging machine learning to advance understanding in specialized areas. For example, NeurIPS regularly publishes work on medical images (e.g., tumor segmentation, radiology interpretation), astronomical imaging (e.g., telescope-based object detection), biological microscopy, remote sensing, and robotic perception. These works often include components tailored to their domains, because the structure and semantics of the data vary substantially across application areas.

Our work is in this tradition. Maritime images differ drastically from more commonly studied domains. They are visually ambiguous, lack structured background cues, and involve dynamic interactions between objects and water under varied environmental conditions. Developing effective solutions in this space necessarily requires specialized design choices, and our method was built to address those specific challenges directly.

In this sense, domain-specificity should not be equated with limited relevance. Rather, it reflects a deliberate effort to make machine learning effective in contexts where naive application of general tools would fail. We believe this aligns with the inclusive and applied spirit of NeurIPS.

R-L Dual-use nature We appreciate the reviewer's concern about the dual-use potential of our technology. Our proposed model is specifically designed for intelligent navigation and smart maritime applications, similar to autonomous driving and smart city scenarios, and is not intended for military purposes.

Nonetheless, we recognize the importance of addressing responsible deployment. In the final version of the paper, we will include a statement clarifying the intended scope of use and reinforcing our commitment to ethical and non-military applications to help mitigate risks of unintended misuse.

We sincerely appreciate Reviewer tKB5's valuable feedback. Our responses to the weaknesses and questions are listed below.

R-W1 Effective but not strongly novel We sincerely thank the reviewer for recognizing the effectiveness of our approach. While we acknowledge that the BiOW-Attn module builds upon existing attention mechanisms, our contributions extend beyond this single component. Our work makes three major contributions: (1) the BiOW-Attn mechanism, which is tailored to capture bidirectional object-water interactions, a critical yet underexplored factor in maritime scenes; (2) the novel AAS strategy that integrates active learning with attribute-aware sampling for synthetic data selection; and (3) the construction of the first benchmark dataset (MGD) specifically designed for maritime scene generation.

More importantly, this represents the first attempt to systematically apply generative models to the maritime domain—an important but often overlooked application scenario. We hope this pioneering effort in addressing the unique challenges of maritime object detection through synthetic data generation will not be overlooked, rather than focusing solely on the novelty of individual module designs.

R-W2 Marginal gains of AAS Thank you for your suggestion, which is similar to the Reviewer S8Ax's W1.3 question. We conducted more extensive experiments to illustrate this saturation effect. As shown in the table below, there is a clear saturation effect when the data volume increases from 10k to 20k samples. This phenomenon primarily comes from our proposed AAS method, which pre-selects the most valuable generated samples. The samples ranked lower in the selection process contribute minimally to detection performance improvement since the model has already performed well on similar cases. This active selection strategy enables our method to achieve faster performance improvements with lower computational costs, reducing training overhead while maintaining effectiveness. To better illustrate this point, we will add this discussion in the final version and include 2D curve visualization to better demonstrate the relationship between sample quantity increase and detection performance improvement.

R-Q1 Individual ATDF contribution The table below demonstrates the individual contribution of each attribute dimension (ATDF) used in our AAS strategy for sample selection. As more attribute dimensions are incorporated, higher-value samples are prioritized, leading to progressive performance improvements. This validates the effectiveness of our AAS method in combining multi-dimensional attributes to select the most valuable synthetic samples for detection enhancement.

R-Q2 More visualization of generation diversity Due to the rebuttal policy restrictions, we are unable to provide additional anonymous visualizations during this phase. However, we will include comprehensive visual examples in the final version to illustrate the generation diversity of our model under the same layout with different seeds and text prompts.

While these specific cases are not shown in the current version, we kindly refer the reviewer to Figures 8–11 in the appendix. These figures demonstrate the model’s ability to generate visually diverse maritime scenes without noticeable repetition in ship appearance, water texture, or overall scene composition, highlighting the model’s capacity for variation.

R-Q3 Image editing ability Our model supports scene-level editing by modifying the input layout, which enables the generation of diverse maritime scenes. While direct editing of real images (e.g., inserting objects into real maritime scenes) is not the focus of our work, our generator, like other diffusion models, can be used in conjunction with existing image editing techniques to perform such tasks.

Due to rebuttal policy restrictions, we cannot include additional visualizations at this stage, but we will provide relevant examples and discussion in the final version.

We sincerely appreciate Reviewer S8Ax's valuable feedback. Our responses to the weaknesses and questions are listed below.

R-W1.1 Extensive comparison of generative models for detection improvement Thank you for your suggestion. To demonstrate the benefits of our X-to-Maritime image generator in object detection, we conducted comparisons with state-of-the-art methods. The performance improvements of these methods on YOLOv10 detector are shown in the table below. It is clear that our method achieves the best performance in improving maritime detector performance, primarily attributed to the design of the BiOW-Attn module, which significantly enhances the modeling and control capabilities between objects and water bodies, thereby generating more realistic images. We will add further discussion in the final version.

R-W1.2 Extensive comparison of different configurations for detection improvement Thank you for your suggestion. The benefits of each component to object detection improvement are shown in the table below. It is evident that using only simple object conditions can hardly effectively improve performance, mainly because it tends to produce poor generation results (e.g., ships separated from water surface). Simply adding water conditions can further enhance detection performance, but the improvement is still limited. However, introducing the bidirectional attention module for object-water modeling can significantly enhance the realism of synthetic data, thereby better improving downstream detector performance. We will add this experiment in the final version.

R-W1.3 Correlation between detection accuracy and number of synthetic samples Due to the rebuttal requirment constraints, we present the results in tabular format. We conducted data augmentation experiments on YOLOv10 using 5k, 10k, 20k, 30k, and 50k synthetic samples, respectively. The experimental results are shown in the table below.

It is worth noting that there is a clear saturation effect when the data volume increases from 10k to 20k samples. This phenomenon primarily stems from our proposed AAS method, which pre-selects the most valuable generated samples. The samples ranked lower in the selection process contribute minimally to detection performance improvement since the model has already performed well on similar cases. This active selection strategy enables our method to achieve faster performance improvements with lower computational costs, reducing additional training overhead while maintaining effectiveness. We will include the suggested 2D curve visualization in the final version to more intuitively demonstrate this.

R-W2 YOLO score We thank the reviewer for this concern, which may stem from a misunderstanding. In fact, the detector used for YOLO score computation is pre-trained on large-scale datasets and then fine-tuned on our constructed small-scale maritime dataset (which does not contain any synthetic data). The fine-tuning is necessary because our dataset includes objects that are rare in general scenarios but particularly important in maritime contexts, such as buoys. The detector has never used any data generated by generative models during the training, so there is no unfairness issue. We hope this clarification addresses your concern.

R-W3 Water condition embedding This concern may stem from a misunderstanding caused by Figure 2. Actually, we use a single embedder to encode both object and water surface conditions simultaneously, rather than employing separate auto-encoders. We will modify Figure 2 in the final version to avoid this unclarity.

We sincerely appreciate Reviewer EyhV's valuable feedback. Our responses to the weaknesses, questions, and limitations are listed below.

R-W1 Limited to YOLO variants Our proposed method is indeed applicable to various types of detectors, including open-vocabulary detectors such as Grounding DINO. To demonstrate this generalization, we conducted additional experiments using Grounding DINO as the object detector. As shown in the table below, our proposed data generation method significantly enhances Grounding DINO's detection performance, achieving substantial improvements in mAP and mAP@0.5. This demonstrates that the effectiveness of our generation approach extends beyond common detectors represented by the YOLO variants to open-vocabulary detectors.

R-W2 & Q2 Sampling strategy comparison Thank you for your insightful comment. To fully demonstrate the advantages of our proposed method, we conducted comprehensive comparisons with 5 different active learning approaches, including uncertainty-based sampling methods (Entropy, Variance, and Margin) and diversity-based sampling methods (Greedy K-Center and K-Means Corset). For diversity-based sampling, we constructed 7-dimensional features using detection box count, average box area, box area standard deviation, average confidence, confidence standard deviation, average box x-coordinate, average box y-coordinate, and category count.

The results show that while these traditional methods also improve performance, our proposed AAS method achieves superior results by more comprehensively considering maritime-specific attributes such as water conditions, weather, and viewpoint information, leading to better detection performance.

R-W3 Assumptions and generalization We thank the reviewer for the thoughtful comment. While our method is specifically designed for maritime scene generation and detection, certain components may offer insights or inspiration for other domains that face similar challenges. For instance, the generation module could be adapted to tasks requiring fine-grained control over multi-condition interactions, and the AAS sampling strategy may be relevant to object detection problems involving multi-dimensional attribute distributions. We will expand the discussion in the final version to clarify the potential for such cross-domain relevance and the associated limitations.

R-Q1 BiOW-Attn and water condition The purpose of BiOW-Attn is not merely to introduce water conditions, but to better capture the interaction relationships between water bodies and water surface objects, thereby improving generation quality. The qualitative visualization results (Figure 10) and quantitative metrics (Table 2) comparing scenarios without water conditions and with simple water condition addition are already provided in our paper. From the experimental analysis, approaches that ignore or simply introduce water conditions tend to produce issues such as objects floating in the air above water and unrealistic water-object interaction boundaries. Our designed BiOW-Attn effectively addresses these problems by modeling the bidirectional attention between objects and water surfaces, leading to improved generation quality.

R-Q3 ATDF's detector dependence The purpose of ATDF is to observe the detection performance differences of specified detector across different attribute dimensions, thereby enabling the targeted selection of generated samples that better improve corresponding detector performance. Therefore, a worse initial detector does not degrade the effectiveness of AAS selection. This has been experimentally validated in Table 3 of the main text, where three detectors with different performance levels were used. When fine-tuned with the samples selected based on their respective ATDFs, all detectors showed significant performance improvements, demonstrating the robustness of our AAS strategy across different detector qualities.

R-Q4 & R-L Broader impact discussion While our work is intended primarily for maritime safety, monitoring, and research applications, we acknowledge that synthetic image generation capabilities could potentially be misused for creating deceptive maritime surveillance content. We will add a broader impact discussion in the final version that addresses these concerns and emphasizes the importance of responsible deployment of our framework.