PIVNO: Particle Image Velocimetry Neural Operator

Thank you very much for your valuable feedback and suggestions! We greatly appreciate your comments and treat them carefully. For each issue raised, we will make modifications or clarifications as needed. Below, we provide a brief discussion of the concerns raised in the review:

1. On the Conv-GRU Structure

We fully understand your concerns regarding the Conv-GRU structure, and we would like to briefly explain the deeper motivation behind our choice of the Conv-GRU iterative architecture.

Before applying Conv-GRU, we obtained flow features through the RoPE-GA module (see Supplementary Materials Figure 1). However, the RoPE-GA module only maps function spaces along the channel dimension but does not aggregate or organize the spatial information. This results in flow features that are often coarse, discontinuous, and contain local estimation errors, whereas spatial continuity is critical for accurate flow estimation. Therefore, we need a mechanism that can progressively optimize flow features in the spatial domain, which is why we did not simply stack more RoPE-GA layers.

In motion estimation fields such as PIV, optical flow, and disparity estimation, the Conv-GRU module has become a mainstream structure for iterative flow refinement, with methods such as RAFT [2] and FoundationStereo [3] adopting similar structures. The effectiveness of this approach has been widely validated, and we did not elaborate on the structural motivation in the main text. Our ablation experiments also confirmed the key role of this module in the final performance. However, we fully recognize your rigorous concern about this issue and are happy to further explain our understanding of its mechanism.

Unlike conventional convolution operations, the hidden state update in Conv-GRU is determined by multiple gated channels (such as the update gate 𝑧, reset gate 𝑟, and candidate state). Convolution essentially projects the input features into a subspace spanned by a set of convolutional kernel functions, and the optimization of these kernels allows for the best possible hidden representation. Comparatively, the convolutions in Conv-GRU operate separately on the current input features and on the previous hidden state, enabling differential nonlinear mappings across multiple subspaces during the update process. This structure allows the model to understand local motion trends from multiple perspectives and scales, and to iteratively correct the flow states of local regions at each iteration. Additionally, the nonlinear activation functions (e.g., sigmoid, tanh) introduced by the gating mechanism enhance the model's expressiveness, enabling it to adaptively regulate the retention, update, and fusion of information based on the current estimation state, thus overcoming the limitations of single-layer convolutions in modeling capacity.

Throughout the iterative process, Conv-GRU acts as a differentiable spatial transformation module with memory. In each iteration, it fuses the current input features with the previous estimation result (i.e., the hidden state) and continuously adjusts the direction and magnitude of the flow field estimate through locally sensitive convolutional mappings and gating mechanisms. This structure endows the model with the ability to progressively approximate the true flow field within local spatial ranges. Therefore, Conv-GRU in PIVNO is no longer a recursive structure used for handling time sequences but rather an iterative optimization unit with local geometric modeling capabilities that supports differentiated state updates. Its synergistic effect of spatially varying linear combinations and nonlinear gating mechanisms enables the correction and refinement of the local flow field.

2. Self-Supervised Fine-Tuning

Thank you for your attention and inquiries regarding the ablation experiments on the self-supervised loss terms in Table 3. We provide a unified response addressing both the theoretical basis and experimental design.

First, regarding your observation that “using only the divergence loss term $L_{\text{div}}$ leads to significant performance degradation,” we explain that this behavior is closely related to the ill-posed inverse problem nature of PIV. PIV maps observed image pairs to fluid motion fields, with inputs containing uncertainties such as repeated patterns, occlusions, and varying lighting conditions, making the inverse problem highly ill-conditioned and solutions non-unique. Under such conditions, a single regularization term is insufficient to constrain the solution space, easily causing the model to converge to physically unreasonable spurious solutions, thus losing estimation effectiveness.

In contrast, the work you mentioned [1] is based on a PTV framework where inputs are explicitly detected particle spatial coordinates, providing strong, unique observational support that substantially reduces the ill-posedness of the inverse problem. It is important to note that [1] also employs a combined loss of data consistency, smoothness, and divergence constraints during training. The “divergence-only” setting you referred to occurs only during testing, where the backbone network parameters are frozen and local adjustments are made via a residual module. Under such conditions, the “divergence-driven” mechanism does not independently constrain the backbone network, and thus cannot be directly compared to our independent loss configuration during training.

Second, regarding your concern that the ablation experiments do not fully reveal the individual contributions of each loss term, we acknowledge that not all possible loss combinations were presented in the main paper, but with primary focus on the core experimental questions. Our experimental design emphasizes two aspects: (1) verifying whether a single loss term is sufficient to avoid invalid solutions in PIV scenarios; and (2) assessing whether adding the physics-based divergence constraint to the traditional variational optical flow framework (i.e., data term plus smoothness term) significantly improves model performance. Both of these two points have been verified by our experiments, and we would like to supplement more results in the revised version.

3. Zero-Shot Resolution Generalization

Thank you for your recognition of the robustness of our model under low-resolution input conditions. We also agree with your suggestion to test "zero-shot resolution generalization" as it would be valuable for further validating the operator properties and scale generalization capabilities of the model. However, we would like to clarify that PIVNO adopts a multi-scale input strategy during training. Specifically, the resolution of input image pairs is randomly sampled within a downsampling range from 1× to 4× (see Supplementary Materials B3). This training approach explicitly exposes the model to a continuous range of resolution distributions, enabling it to adapt to inputs of any scale. In other words, the scale generalization performance of PIVNO is learned progressively through this random scale training strategy. Therefore, the setting you suggested—“train the model on a single resolution (e.g., 128×128) and directly test on higher resolutions”—does not align with the core idea of our current training design. This setup would artificially limit the model's ability to model scale variations and would reduce its true generalization performance. Additionally, we believe that the "neural operator" property emphasized by PIVNO is more focused on its ability to map between image pairs (observation space) and fluid motion fields (function space), which differs from purely seeking scale invariance in images.

Nevertheless, your suggestion has been highly insightful for us. We are currently conducting a set of "zero-shot resolution generalization" experiments, where the model is trained on a single resolution (128×128) and then evaluated at an unseen resolution (256×256). We expect to present the results of this test in the rebuttal in the coming days and further analyze its impact on the model's operator generalization ability. Once again, thank you for raising this thought-provoking question.

4. Inference Speed Comparison

Thank you for pointing out the limitations of the inference speed evaluation. We would like to further clarify the statement emphasized in our paper, "avoiding the computational overhead of cost volume," which is intended to highlight the structural advantage in computational complexity of the RoPE-GA module compared to traditional methods that rely on explicit cost volume (CV) construction.

Regarding your suggestion to directly compare the inference time with PIV-RAFT, we believe that, although both methods are comparable in terms of the task, the overall architecture differences (including feature extraction modules and the GRU update mechanism) make a direct runtime comparison potentially unfair and inaccurate in reflecting the actual contribution of the RoPE-GA module to the computational efficiency. Furthermore, it may not fully showcase the performance benefits brought by RoPE-GA.

To more rigorously evaluate the efficiency advantage of our method's structural design, we have included a controlled ablation experiment in Supplementary Materials A3, where PIVNO-CV follows the RAFT-PIV architecture, employing traditional cost volume construction, while PIVNO represents our proposed method based on the RoPE-GA representation. The results show that the RoPE-GA method significantly reduces inference time while improving accuracy, validating the effectiveness and practical value of our "cost volume-free" design strategy.

[1] Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss, in Neurips 2024.

[2] Recurrent all-pairs field transforms for optical flow, in ECCV 2020.

[3] Foundationstereo: Zero-shot stereo matching, in CVPR 2025.

Thank you very much for your valuable feedback and suggestions! We greatly appreciate your comments and treat them carefully. Below, we provide a brief discussion of the concerns raised in the review:

1. GRU-Based Flow Field Refinement and RoPE Galerkin Attention Mechanism

We sincerely appreciate your specific attention to the RoPE-GA mechanism and the GRU-based flow field refinement module. Regarding the design motivation for the RoPE-GA mechanism, we have provided a detailed explanation of its principles in Section 3.1.2 of the main text and Supplementary Materials A1. The discussion of comparisons with established neural operator benchmarks has been addressed in the response to the next question. As for the design motivation behind the GRU, since several reviewers have shown interest in this aspect, we will provide a more detailed response here.

Before applying Conv-GRU, we obtained flow features through the RoPE-GA module (see Supplementary Materials Figure 1). However, the RoPE-GA module only maps function spaces along the channel dimension but does not aggregate or organize the spatial information. This results in flow features that are often coarse, discontinuous, and contain local estimation errors, whereas spatial continuity is critical for accurate flow estimation. Therefore, we need a mechanism that can progressively optimize flow features in the spatial domain, which is why we did not simply stack more RoPE-GA layers.

In motion estimation fields such as PIV, optical flow, and disparity estimation, the Conv-GRU module has become a mainstream structure for iterative flow refinement, with methods such as RAFT and FoundationStereo adopting similar structures. The effectiveness of this approach has been widely validated, and we did not elaborate on the structural motivation in the main text. Our ablation experiments also confirmed the key role of this module in the final performance. However, we fully recognize your rigorous concern about this issue and are happy to further explain our understanding of its mechanism.

Unlike conventional convolution operations, the hidden state update in Conv-GRU is determined by multiple gated channels (such as the update gate 𝑧, reset gate 𝑟, and candidate state). Convolution essentially projects the input features into a subspace spanned by a set of convolutional kernel functions, and the optimization of these kernels allows for the best possible hidden representation. Comparatively, the convolutions in Conv-GRU operate separately on the current input features and on the previous hidden state, enabling differential nonlinear mappings across multiple subspaces during the update process. This structure allows the model to understand local motion trends from multiple perspectives and scales, and to iteratively correct the flow states of local regions at each iteration. Additionally, the nonlinear activation functions (e.g., sigmoid, tanh) introduced by the gating mechanism enhance the model's expressiveness, enabling it to adaptively regulate the retention, update, and fusion of information based on the current estimation state, thus overcoming the limitations of single-layer convolutions in modeling capacity.

Throughout the iterative process, Conv-GRU acts as a differentiable spatial transformation module with memory. In each iteration, it fuses the current input features with the previous estimation result (i.e., the hidden state) and continuously adjusts the direction and magnitude of the flow field estimate through locally sensitive convolutional mappings and gating mechanisms. This structure endows the model with the ability to progressively approximate the true flow field within local spatial ranges. Therefore, Conv-GRU in PIVNO is no longer a recursive structure used for handling time sequences but rather an iterative optimization unit with local geometric modeling capabilities that supports differentiated state updates. Its synergistic effect of spatially varying linear combinations and nonlinear gating mechanisms enables the correction and refinement of the local flow field.

2. Inverse Problems and Neural Operators

We appreciate the reviewers' questions regarding the applicability of inverse problem categories and comparisons with other neural operator methods. We provide a unified response here and further clarify the fundamental differences between the problem type addressed in this work and existing neural operator approaches.

The problem we investigate belongs to a special class of PDE-constrained inverse problems: recovering the underlying fluid motion field from particle image pairs. This problem is not only constrained by physical laws but also influenced by the imaging mechanism, making it essentially a visual inverse problem that combines computer vision and physical modeling, rather than a traditional PDE numerical solving task based on function samples (such as field value pairs). Therefore, there are significant differences when compared to methods like FNO [1], CNO [2], or neural operator frameworks designed for general inverse problems [3].

Specifically, these neural operators typically map between function spaces based on structured grids or a small number of scattered sample points, assuming well-defined problems with finite bandwidth, often suppressing high-frequency information during modeling (such as directly filtering out high-frequency modes in FNO). In contrast, the input for PIV consists of actual image pairs, which involve complex issues such as repeated patterns, occlusions, and uneven particle distributions. These issues manifest in the image space as significant local ambiguities and non-uniqueness, making the problem highly ill-posed. As a result, the solution is not unique. Therefore, PIV requires not only modeling the global mapping structure but also recovering fine-grained local high-frequency motion features, which is difficult to achieve directly using traditional neural operator paradigms.

For this reason, in PIVNO, we combine an image encoder, a RoPE-GA-based neural operator representation, and a Conv-GRU-based spatial refinement module. Notably, the introduction of Conv-GRU, whose mechanism has been widely validated in computer vision inverse problems such as optical flow and disparity estimation, allows for local state memory and nonlinear optimization within the image space through gating mechanisms, a feature not applicable to traditional PDE numerical problems.

Thus, our decision not to directly compare with the above methods is not for dodge but stems from the fundamental differences in problem paradigms, input modalities, and modeling objectives.

Additionally, regarding the question, "What are the implications for the model's performance when the inverse problem is ill-posed or lacks a unique solution?" we believe that PIV itself is a typical ill-posed inverse problem. Our experimental results under real-world conditions, including significant occlusions, sparse textures, and high shear deformations, already demonstrate the robustness and generalization ability of our framework in dealing with such ill-posed problems.

3. Graph Convolution

The introduction of the graph convolution layer primarily stems from the cross-resolution training strategy used in the SR Module. This strategy randomly samples the same number of pixels for training from image pairs at different resolutions, enhancing the model's generalization ability under scale variations (see Supplementary Materials B3 for details). However, this sampling method disrupts the regular, evenly spaced arrangement of pixels, making it difficult for traditional convolutions (such as standard 2D convolutions) to directly operate on these non-structured sampling points. Since capturing local features is critical for continuous spatial mapping, graph convolution naturally handles non-uniformly sampled inputs and aggregates local information through adjacency structures, enabling effective continuous mapping and feature fusion in space. Therefore, we believe that graph convolution is a reasonable choice for the task scenario at hand.

To verify its necessity, we replaced the graph convolution with a 1×1 convolution in an ablation experiment (the "GCN×" setting in Table 4). The results showed a significant performance drop, further demonstrating the unique advantages of graph convolution in handling irregular sampling points and capturing local geometric features.

4.Smoothness Term

For the fluid motion field $u(\mathbf{x}) \in \mathbb{R}^{C \times H \times W}$, the smoothness term in the loss function is defined as follows:

$L_{\text{smooth}} = \sum_{c=1}^{C} \sum_{i=2}^{H-1} \sum_{j=2}^{W-1} \frac{1}{(H-2)(W-2)} \left( \nabla^2 u_{c,i,j} \right)^2$

where $\nabla^2 u_{c,i,j}$ denotes the Laplacian of $u$ at position $(i,j)$. Specifically, the Laplacian is defined as:

$\nabla^2 u_{c,i,j} = u_{c,i+1,j} + u_{c,i-1,j} + u_{c,i,j+1} + u_{c,i,j-1} - 4u_{c,i,j}$

The smoothing term is designed based on the spatial continuity assumption and regularization theory. The spatial continuity assumption posits that in the physical world, unless there are object boundaries or occlusions, the fluid velocity field is typically continuous and smoothly varying in space, with abrupt changes being rare. The smoothing term aims to explicitly express and reinforce this prior knowledge, encouraging the estimated flow field to remain smooth and consistent spatially. From a regularization perspective, the particle image velocimetry problem is inherently an ill-posed inverse problem, and relying solely on data terms (such as brightness consistency) often struggles to produce a stable and reasonable solution. Introducing the smoothing term as a regularization term effectively constrains the spatial fluctuations of the solution, prevents overfitting to noise, and improves the stability and robustness of the model's estimation.

Thank you very much for your valuable feedback and suggestions! We highly appreciate your comments and treat them carefully. For each issue raised, we will make modifications or clarifications as needed. Below, we briefly discuss the points raised:

1.Limitations of 2D Estimation

We fully agree with your view that "extending to 3D flow fields is a crucial next step to improve the model’s applicability to more complex and realistic fluid systems." Our focus on 2D in this work is based on a careful assessment of existing mainstream 3D methods. Current 3D flow estimation methods mainly adopt particle tracking velocimetry (PTV), which requires a prerequisite particle detection phase followed by the paticle matching and flow estimation process. This approach is highly sensitive to detection quality and prone to error propagation, and produces sparse Lagrangian motion fields that are difficult to use for dense Eulerian field reconstruction. In contrast, PIVNO follows the PIV paradigm, estimating dense Eulerian flow fields end-to-end directly from particle image pairs, thus avoiding the instability and information loss introduced by intermediate steps. Therefore, we first establish a solid and reproducible baseline in 2D scenarios by integrating physical priors. On this foundation, we have already started developing tomography-based 3D PIVNO to further extend applicability to complex real-world 3D flows.

2.Physical Assumption Limitations of the Self-Supervised Mechanism

As you pointed out, we indeed introduce a divergence-free constraint based on the incompressible flow assumption, which may not fully hold in some experimental setups or real applications. It is important to emphasize that the divergence-free constraint is neither inherent nor the only physical constraint usable in the PIVNO framework. It serves as a physical prior tailored to the incompressible flow characteristics of our current datasets. For other fluid types, the PIVNO framework can similarly replace or extend physical constraints to fit new assumptions or scenarios. This constraint is flexible and should be set by the researchers according to the physical context and prior knowledge of the problem.

3.Generalization and Performance Degradation

We fully recognize that the complexity of fluid dynamics makes it impossible for the model to maintain ideal performance under all conditions. Hence, we propose a self-supervised fine-tuning mechanism specifically to address potential performance degradation in real and complex conditions. In our experiments, the three selected real scenarios differ significantly from the synthetic training data in flow states, particle density, illumination conditions, etc. As shown in Table 3 of the main text, the model’s performance does degrade noticeably when encountering novel flow states significantly different from training. However, self-supervised fine-tuning effectively mitigates this degradation and substantially improves the final estimation accuracy, enabling the model to remain robust and adaptable in complex real-world scenarios.

Thank you very much for your valuable feedback and suggestions! We greatly appreciate your comments and treat them carefully. For each issue raised, we will make modifications or clarifications as needed. Below, we provide a brief discussion of the concerns raised in the review:

1.Unclear Experimental Structure

You pointed out that the structure of the experimental section is somewhat confusing, especially the redundancy between Section 4.1 and the separate "Ablation Study" section. We fully understand and acknowledge this. The reason for presenting Table 3 separately is that it not only includes ablation experiments on the self-supervised loss components but also summarizes the overall performance of the self-supervised strategy on both synthetic and real datasets. Therefore, we believe it serves as a key indicator of the self-supervised strategy's effectiveness and deserves a separate presentation. The "Ablation Study" in Section 4.3 focuses on ablation analysis under other experimental settings. However, we realize that the current structure may not effectively guide readers through the hierarchy of experiments. To address this, we will include additional clarifications in the revised version to explicitly outline the logical structure of the experimental section, thereby improving clarity and coherence.

2.Rationality of Evaluation Metrics

Currently, in the PIV field, the Average Endpoint Error (AEE) is the mainstream evaluation metric. Moreover, many related works have not released their source code, which makes it less possible to conduct comparisons using alternative metrics. We fully agree with your concern about the rationality of evaluation metrics. To address this, we have additionally reported the maximum error（Max）and standard deviation (Std) of the error distribution on Synthetic Dataset 1. The experimental results are as follows (unit: pixels):

It is particularly noteworthy that under the 256²×1 resolution setting, the model exhibits significantly higher maximum error and standard deviation compared to other settings. Upon further investigation, we found that this is caused by a few extreme error points near the boundary regions, which raise the overall statistics. In contrast, the ×4 and ×2 resolution settings inherently introduce spatial averaging (filtering) effects during the super-resolution reconstruction process, which smoothes out the local noise and thereby reduces both the maximum error and the error variance.

3.Evaluation of Fine-Tuned Models on Synthetic Datasets

Our method primarily focuses on training the model on synthetic datasets to learn a general mapping from input particle images to output motion fields, followed by self-supervised fine-tuning in specific real-world scenarios to enhance performance for the task at hand. As our main goal is to improve performance in these specific real-world tasks, the generalization of the fine-tuned model back to synthetic datasets was not our primary consideration. However, we understand the reviewers’ concern about the generalization ability after fine-tuning and the consistance on its importance. Therefore, we have conducted experiments to evaluate the generalization of the fine-tuned model on the synthetic dataset. The results are as follows (unit: pixels):

Experimental results show that after fine-tuning on real-world datasets, the performance of PIVNO on synthetic datasets exhibits a noticeable decline. This reflects the nature of fine-tuning as a process of adapting a general model to specific scenarios. Given that PIVNO contains only 2.4M parameters, expecting it to maintain strong generalization across both synthetic and real domains without any trade-offs is somewhat overly demanding. Afterall, the core value of our model lies in its adaptability—achieving effective performance in diverse real-world situations through lightweight fine-tuning—rather than aiming for a one-size-fits-all solution through a single training process.

4.Involvement of Domain Experts

We fully agree with your point that qualitative assessment of flow field estimations by domain experts (e.g., fluid mechanics researchers) would be highly valuable. The datasets and baselines used in our study are widely recognized and scientifically grounded in the field, effectively reflecting the accuracy and applicability of our method. To promote interdisciplinary collaboration and further validate the physical reliability of the model, we will open-source our code and sincerely welcome experts in fluid mechanics and related fields to participate in qualitative evaluations and assesments, jointly advancing the application and development of our method to real-world flow problems.