Scale-Invariant Continuous Implicit Neural Representations For Object Counting

3) The paper could benefit from additional experiments testing SI-INR’s robustness with images at extreme resolutions, especially low resolutions (e.g., <200 pixels), to provide a more complete understanding of its limitations:

Thank you for your constructive suggestions. We here add an experiment to evaluate SI-INR’s robustness with low resolutions. On the RSOC building dataset, We resize all test images into $100\times100$ and compare the counting performance of SI-INR and baseline models. PSGCNet achieves an MAE of 18.15 and an MSE of 22.78, APDNet achieves an MAE of 20.04 and an MSE of 24.32, EfreeNet achieves an MAE of 24.41 and an MSE of 27.06. In comparison, our SI-INR significantly outperforms these methods, achieving an MAE of 8.53 and an RMSE of 12.65.

We plan to extend this analysis by testing at additional resolutions and plotting a performance curve to further illustrate the robustness of SI-INR against resolution changes.

4 Missing some important previous work:

We truly appreciate the suggestions on comparing SI-INR to important previous works. During the rebuttal period, we have further evaluated SI-INR on the large-scale and challenging UCF-QNRF dataset.

UCF-QNRF comprises 1,535 images with over 1.2 million annotated individuals, capturing a wide range of crowd densities. SI-INR has achieved competitive performance, with an MAE of 80.89 and RMSE of 134.73. For comparison, while P2PNet [1](A point-to-point matching method for crowd counting) achieves an MAE of 85.32 and an RMSE of 154.5, GauNet [2](leveraging Gaussian-based density maps) achieves an MAE of 81.60 and an RMSE of 153.71, APGCC [3](A point-to-point matching method for crowd counting) achieves an MAE of 80.10 and an RMSE of 136.60. PSL-Net [4] achieves an MAE of 85.50 and an RMSE of 144.40. Furthermore, compared to our baseline model, PSGCNet (MAE 86.3, RMSE 149.5).

The initial results illustrate that our model can indeed achieve comparable performance to the State-of-the-art crowd counting methods on UCF-QNRF dataset. In the final version, we will provide a more comprehensive comparison on additional crowd counting datasets with more baseline methods. For the RSOC, CARPK and PUCPR+ datasets, we will add more result by additional SOTA methods in Table 1 of the main text.

[1] "Rethinking counting and localization in crowds: A purely point-based framework." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

[2] "Rethinking spatial invariance of convolutional networks for object counting." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[3] "Improving Point-based Crowd Counting and Localization Based on Auxiliary Point Guidance." European Conference on Computer Vision. Springer, Cham, 2025.

[4] "Crowd Counting and Individual Localization using Pseudo Square Label." IEEE Access (2024).

5) Would you consider evaluating SI-INR’s performance in sequential data or video-based counting tasks?

We appreciate the reviewer's suggestion. We believe that leveraging the scale-equivariant and continuous properties of SI-INR can improve the performance in general, including tracking and counting people across video frames, while also retaining fine spatial details. If given more time, we will evaluate SI-INR's performance in more diverse computer vision tasks in our future research.

6) Have you identified any potential biases in SI-INR when applied to diverse environmental conditions, such as different lighting or weather effects in remote sensing?

We focus on addressing challenges due to scale/resolution variations by introduce scale-invariance implicit neural representations. To further overcome the issues under different adversarial conditions as pointed out by the reviewer, different model formulations and solution strategies such as incorporating potential physics models for corresponding lighting or weather effects, which can be potential research for integrating additional such modules. Without explicitly modeling these adversarial effects, we expect that our SI-INR will achieve similarly reported results in the object detection literature. We are open to evaluate these if given more time.

1) The paper does not fully explain the operational details of the "scale-invariant continuous mapping," especially regarding how SI-INR preserves fine details for low-resolution inputs. A more comprehensive description would enhance reproducibility:

Thank you for your valuable feedback. We appreciate your observation regarding the need for more details about the "scale-invariant continuous mapping" and its role in preserving fine details for low-resolution inputs.

The "scale-invariant continuous mapping" in SI-INR relies on the scale-equivariant/invariant properties of each network component and the implicit neural representation (INR) model to learn a continuous function that maps coordinates to density values.

Specifically, a scale-equivariant Backbone B which satisfies $B(p_1(h) (**I**_a)) = p_B(h)(B)(**I**_a)$ is adopted to extract deterministic features so that scale changes in objects will only affect the scale of feature maps while keeping the appearance. Later, a scale-invariant encoder $E$ maps the features into a constant latent space and an INR decoder is applied to transform the latent into a continuous function. $\Psi(p_1(h) (**I**_a))(\mathbf{x}) = \mathcal{H}(E(B(p_1(h) (**I**_a)))(\mathbf{x}) = \mathcal{H}(E(B(**I**_a)))(\mathbf{x}) = \Psi(**I**_a)(\mathbf{x})$, where $\Psi(\cdot)$ denotes in total SI-INR model, $\mathcal{H}(\cdot)$ represent our INR decoder.

For low-resolution inputs, the scale-equivariant property of our backbone B enables SI-INR to generate outputs that are much closer to the results obtained from the same input at a larger scale, outperforming traditional methods.

Traditional approaches, which often use fixed downsampling ratios, produce low-resolution density maps for low-resolution images. The corresponding unclear, low-resolution discrete density ground truth is less effective for training. In contrast, our SI-INR is trained using high-resolution ground-truth density maps, even for low-resolution inputs, avoiding significant information loss. This design enables SI-INR to capture spatial relationships beyond the input's native resolution and preserve fine details effectively.

To enhance reproducibility, we will include a more detailed explanation of the "scale-invariant continuous mapping" in the revised manuscript. This will cover the training process, the role of the INR model, and the mechanisms that help retain fine details for low-resolution inputs. Additionally, we will provide visual illustrations to clarify these concepts.

2) While computational demands are briefly mentioned, there is no clear comparison of SI-INR's runtime performance against baselines in various resolutions. This omission makes it difficult to assess scalability for real-time applications:

We appreciate the reviewer’s suggestion of including the runtime comparison. To address this, we have conducted a speed test using a workstation with an NVIDIA V100 32GB GPU. For the RSOC small-vehicle dataset, we observed the following inference times: ASPD-Net requires approximately 15.13 seconds, PSGC-Net takes around 2.47 seconds, eFreeNet takes around 3.84 seconds, and our SI-INR model requires about 3.87 seconds.

SI-INR requires more time for inference compared to PSGC-Net, due to the integration of the scale-equivariant models and the stacks of linear layers in the INR. However, thanks to our lightweight scale-equivariant backbone and the compact design of the INR model, which consists of only 4 linear layers, the inference cost remains manageable and acceptable. We will include these findings in the revised manuscript for clearer scalability discussion. We further give a more detailed discussion in our Rebuttal for all reviewers.

1) After reading the paper, my understanding is that a VAE-like approach is used to obtain a fixed-size output, ensuring that inputs of different scales can be mapped to the same output, thus addressing the issues mentioned in the paper. However, the paper is written in a very complex manner and requires careful reading to understand. I suggest redrawing Figure 1. First, the new Figure 1 should allow readers to immediately grasp your network, especially the input and output. Additionally, it should include both the overall framework and detailed components of the network. The current Figure 1 does not provide a clear understanding of your work, making it difficult to reproduce:

Thank you for your insightful comments. We appreciate your suggestion to improve Figure 1 and to provide a clearer explanation of the model architecture design. To address your concerns, we will redesign Figure 1 to provide a more intuitive illustration of corresponding components in our SI-INR. The revised figure will:

a) clearly depict the input and output of the SI-INR model to help readers understand the scale-normalizing process;

b) present the overall framework alongside detailed components, and how it handles scale variability;

c) include annotations and visual aids to clarify the flow of data through the model.

2) Although your method is reasonable, it does not achieve the result mentioned in line 139 of the paper, i.e., obtaining the same output from inputs of different scales. In previous methods, multi-scale image augmentation was commonly used during training to address this problem. Although your approach is different, the goal remains the same:

We appreciate the reviewer's comments. In detail, we choose SESN to achieve the scale-equivariant backbone, SESN relies on group convolutions to approximate scale-equivariance. However, the finite set of sampled scales used during training and inference means that scale-equivariance is not exact but rather approximate within a certain range of scales. Incorporating exact scale-equivariance for all scales would require infinite representations, which is computationally infeasible. Our SI-INR balances computational efficiency with such an exact equivariance guarantee, leading to trade-offs in its ability to generalize across scales. For example, a scale-equivariant steerable convolution will generate an output in shape [$B,S,C_{out},W,H$] for an input in shape [$B,C_{in},W,H$] compared with [$B,C_{out},W,H$] of the traditional convolution. The extra axis $S$ denotes the number of scales sampled from the scaling group. It is clear that a larger $S$ results in a closer approximation to exact scale-equivariance. In SI-INR, we choose 7 different scales from 0.8 to 1.2 to balance the computational efficiency with the exact equivariance, we will add this information in our final version.

Our framework integrates the scale invariance property of object counting into the inherent inductive bias of the model with the SESN encoder, which will guarantee the output being invariant to any change of scale while previous work relies on the heuristic data augmentation method, which may not cover enough range of size variations and does not have any guarantee on size-invariance.

We additionally perform a comparative study between (1) PSGCNet with multi-scale image augmentation and (2) PSGCNet with SI-INR's backbone and INR decoder. On the RSOC building dataset, PSGCNet with multi-scale image augmentation achieves an MAE of 7.04 and an MSE of 10.65, while PSGCNet with our components achieves an MAE of 6.54 and an MSE of 9.80. On the CARPK dataset, PSGCNet with multi-scale image augmentation achieves an MAE of 6.53 and an MSE of 9.41, while PSGCNet with our components achieves an MAE of 5.54 and an MSE of 7.43. We will test on more datasets and methods and add this ablation study to our revised manuscript.

4) Why can it be used to extract scale-invariant features?

To extract scale-invariant features, a scale-invariant encoder is required. Specifically in SI-INR, we choose scale-equivariant steerable convolution (SESN) to achieve the scale-equivariant backbone, which integrates steerable filters exacting features of different scales given the input image. This allows SESN to handle objects of varying sizes without needing separate filters for each scale. Additionally, the group convolution method is applied in SESN to ensure that the network’s output changes in the same way when the input is scaled. At this point, features with different scales result in the same output with corresponding scales. To extract scale-invariance features for the desired arbitrary-resolution output, we further apply a scale-invariant encoder, where rescaling operators are introduced with a Hybrid Pyramidal Scale module. Such a model architecture in our SI-INR extracts scale-invariant features, and the INR model is applied then to transform these features as the implicit neural representation parameters to achieve the final implicit continuous functions. Therefore, for different scales of the given input, SI-INR can derive scale-invariant outputs. We will provide a more detailed description in our final version.

5) Since the multi-scale challenge has been investigated for a long time, how does the performance compare to other approaches?

For efficiency and adaptability, scale-equivariant methods adjust to different scales without needing separate filters for each scale, unlike traditional methods that may rely on resizing inputs or using multiple filters for different scales. Many traditional multi-scale methods, such as image pyramids or multi-resolution networks, may struggle with high computational costs because they process the same image at multiple resolutions leading to increased complexity, especially with large images or when handling many scales. Besides, traditional multi-scale approaches do not focus on deriving scale-invariant outputs compared with our SI-INR.

We have conducted additional performance comparison experiments on the UCF-QNRF crowd counting dataset. In PSGCNet, the authors applied a pyramidal network to handle multi-scale challenges. Our SI-INR outperforms PSGCNet as demonstrated in our experiments. For MMNet [5], a method that leverages multi-level density-based spatial information, the model achieves an MAE of 104 and RMSE of 178 on the UCF-QNRF dataset. Similarly, MFANet [6], which focuses on multi-scale and multi-level feature aggregation, achieves an MAE of 97.7 and RMSE of 166. MSFFA [7], which integrates multi-scale feature fusion and attention mechanisms, reports an MAE of 94.6 and RMSE of 170.6. In contrast, our SI-INR achieves a significantly better MAE of 80.89 and RMSE of 134.73, demonstrating its superior performance in crowd counting tasks.

[5] "Crowd counting by using multi-level density-based spatial information: A Multi-scale CNN framework." Information Sciences 528 (2020): 79-91.

[6] "A multi-scale and multi-level feature aggregation network for crowd counting." Neurocomputing 423 (2021): 46-56.

[7] "MSFFA: a multi-scale feature fusion and attention mechanism network for crowd counting." The Visual Computer 39.3 (2023): 1045-1056.

1) The comparison methods are very limited; many state-of-the-art methods are not included, such as those using optimal transport and point-to-point matching:

We truly appreciate your constructive suggestion to test SI-INR on more datasets and compare it with state-of-the-art methods. During the rebuttal period, we have further evaluated SI-INR on the large-scale and challenging UCF-QNRF dataset.

UCF-QNRF comprises 1,535 images with over 1.2 million annotated individuals, capturing a wide range of crowd densities. SI-INR has achieved competitive performance, with an MAE of 80.89 and RMSE of 134.73. For comparison, while P2PNet [1](A point-to-point matching method for crowd counting) achieves an MAE of 85.32 and an RMSE of 154.5, GauNet [2](leveraging Gaussian-based density maps) achieves an MAE of 81.60 and an RMSE of 153.71, APGCC [3](A point-to-point matching method for crowd counting) achieves an MAE of 80.10 and an RMSE of 136.60. PSL-Net [4] achieves an MAE of 85.50 and an RMSE of 144.40. Furthermore, compared to our baseline model, PSGCNet (MAE 86.3, RMSE 149.5).

The initial results illustrate that our model can indeed achieve comparable performance to the State-of-the-art crowd counting methods on UCF-QNRF dataset. In the final version, we will provide a more comprehensive comparison on additional crowd counting datasets with more baseline methods. For the RSOC, CARPK and PUCPR+ datasets, we will add more results by additional SOTA methods in Table 1 of the main text.

[1] "Rethinking counting and localization in crowds: A purely point-based framework." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

[2] "Rethinking spatial invariance of convolutional networks for object counting." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[3] "Improving Point-based Crowd Counting and Localization Based on Auxiliary Point Guidance." European Conference on Computer Vision. Springer, Cham, 2025.

[4] "Crowd Counting and Individual Localization using Pseudo Square Label." IEEE Access (2024).

2) As shown in Figure 2, the objects are of similar size, making it difficult to justify that the method effectively addresses the multi-scale challenge:

Thank you for raising this question. There are several points to clarify:

Inter-image scale variation: Since the images shown in Figure 2 are remote sensing images, the objects within the same image naturally appear of similar size. However, remote sensing datasets, including RSOC, encompass images with a wide range of resolutions. As a result, object scales vary significantly across different images, even if they appear uniform within a single image. The results in Section 4.2 demonstrate that SI-INR can better handle inter-image scale variation than existing methods with higher detection accuracy. We are preparing a summary of additional RSOC image examples to illustrate scale variation across the dataset and will include the findings in the revised manuscript.

Intra-image scale variation: To further validate the effectiveness of our method in handling multi-scale challenges, especially for intra-image scale variation with different object sizes in the same images, we have extended our evaluation to the UCF-QNRF dataset, a benchmark that features diverse crowd scenes where objects vary significantly in scale within the same image. As reported in our responses for question 1, our method achieves competitive results on this more challenging dataset, demonstrating its robustness in multi-scale scenarios. These results will be included in the final version to strengthen the justification of our approach.

3) The description of the scale-invariant model is unclear:

We appreciate your constructive critique and recognize that the description of our scale-invariant model could be clearer. Specifically, we will reorganize our presentation by moving back some of the model formulation descriptions in our Appendix to the method section, expand on how our SI-INR handles scale variations using a scale-equivariant backbone and a multi-scale feature extraction encoder, and provide a more detailed description of our scale-invariant implicit function in our revised manuscript. We are actively exploring ways to address any remaining ambiguities or limitations and will incorporate these improvements. If the reviewer has additional suggestions, we would be more than happier to further improve our presentation.

1) The presentation could be significantly improved to enhance clarity. Currently, the methodology section is challenging to follow. For example, the paragraph from lines 137 to 142 is particularly dense: "Here h denotes an element of the Scale-Translation group H and represents one scale-translation operator, p1(·) denotes the corresponding group actions of h acting on the image domain." - what does it means by "Scale-Translation group H" or "scale-translation operator"?

Thank you for highlighting this point. We did have more detailed introduction of our model formulation in the Appendix sections due to limited space when we prepared for the submission. As detailed in Appendix A.1, the Scale-Translation Group $H$ is defined as a combination of two subgroups:

1. The Scaling Group $G_S$, which accounts for transformations that scale an object or function.
2. The Translation Group $G_T$, which handles shifting the object or function within its domain.

The overall group $H$ is defined as: $H = \{ h = (s, t) \,|\, s \in G_S, \, t \in G_T \}$, where each element $h \in H$ represents a scale-translation operator. Specifically: $s$ is the scaling parameter, indicating how the input is stretched or compressed. $t$ is the translation parameter, specifying the shifting in the domain.

When we refer to the "group actions of $h$," we apply scaling and translation transformations to the image domain. These actions operate on an input $x$ (e.g., a pixel location) as: $p_1(h)(x) = s \cdot x + t$, where $p_1(h)(\cdot)$ represents the transformation resulting from applying $h$. This formulation provides a unified framework to model scale and translation symmetries effectively. In the revised manuscript, we will reorganize and simplify the descriptions in this section to enhance readability and provide a more intuitive explanation.

2) In the following paragraph, the authors bring up the "continuous function space" without any explanation and with many notations are not clearly defined including $I_a$ or $D^{gt}$ (unclear where does this continuous ground-truth coming from?)

We appreciate the constructive critique on the clarity of the "continuous function space" definition and the notations $\mathbf{I}_a$ and $\mathbf{D}^{gt}$. We will clarify these in the revised manuscript:

1. $\mathbf{I}_a$ refers to an input image, potentially transformed by a scale-translation group action $h$.
2. $\mathbf{D}^{gt}$ represents a continuous ground-truth, specifically, in counting tasks, $\mathbf{D}^{gt}$ denotes the continuous ground truth density map, and we defined it in Section 3.3 in the main text.

The continuous function space $\mathcal{F}$ is formally defined as the set of continuous functions $f: \mathbb{R}^2 \to \mathbb{R}$ that predict density values $f(\mathbf{x})$ for any spatial coordinate $\mathbf{x} \in [0, 1]^2$. This framework ensures that our predictions are scale-invariant, and $f(\mathbf{x})$can be evaluated at arbitrary resolutions without relying on a fixed grid. We will make these definitions more explicit in the revised text for clarity.

3) Overall, the writing creates several logical gaps that make it difficult to fully grasp the method. To the best of my understanding, the method is basically: 1) extract deep-features using a scale-equivariant backbone B, 2) using an encoder to map output of B into a latent space and 3) decode this latent representation into a fixed scale for counting. Could the authors confirm whether this understanding is accurate?

We thank the reviewer for summarizing the workflow in SI-INR. The first two steps are indeed as what the reviewer described:

1. Extract deep features using a scale-equivariant backbone $B$.
2. Use a scale-invariant encoder to map the output of $B$ into a latent space.

However, the third step focuses on the flexibility of the arbitrary resolution output. Decode this latent representation to continuous density maps using implicit neural representations $u: \mathbb{R}^2 \to \mathbb{R}$.

In the process, we do not constrain the output to a fixed scale. Instead, we can generate arbitrary resolutions of output in step 3 since our INR model is representing continuous functions in continuous spatial coordinates. The rescaling operator works in step 2 to make sure that the encoder is scale-invariant.

Dear reviewer Jvjv,

We highly appreciate your insightful and constructive feedback on our work. Below are point-by-point responses to your suggestions with some additional results. We have also updated the manuscript accordingly.

1. The clarification was helpful. However, the entire paper requires substantial editing to ensure that all details are logical and easy to follow. This includes narrowing the scope and problem setting, revising the abstract and introduction, and refining the presentation of experimental results. I believe the manuscript would greatly benefit from another round of major revisions.

We sincerely appreciate your valuable feedback and suggestions. Following your advice, we have narrowed the scope of the manuscript from general object counting to remote sensing object counting, and we have revised the corresponding sections (title, abstract, introduction and problem setup) to ensure a clearer and more focused problem setting.

Additionally, we have carefully reviewed and hopefully improved the presentation in the revised Section 4.2 to ensure clarity and accuracy.

To further improve the readability and understanding of our approach, we have also revised the methodology section to better explain the continuous property of SI-INR.

We hope that these changes address your concerns and improve the overall quality and readability of the paper.

2. The table for UCF-QNRF does not include the SoTA method by Liu et al. [1], which achieves an MAE of 79.53. Overall, I do not see clear evidence that the proposed method consistently outperforms existing methods across all datasets.

Thank you for pointing out the paper by Liu et al. We appreciate your suggestion and have included this method in our comparison in Table 6 of the revised manuscript.

We have already narrowed our research scope to remote sensing object counting and revised related the title, abstract, and problem setup in our updated manuscript. Since SI-INR, along with our baseline models, primarily focuses on remote sensing object counting tasks, it is reasonable that SI-INR does not show significant improvements over crowd-counting methods when directly applied to crowd-counting datasets like UCF-QNRF without careful hyperparameter tuning (particularly the sampling algorithm and the setup of the scale-equivariance backbone). While SI-INR does not outperform all existing SOTA methods due to the influence of various factors on final counting performance, its superior results compared to our baselines highlight its effectiveness in handling scale variance.

In addition, even without such fine-tuning, SI-INR achieves comparable results to Liu et al.'s work on the UCF-QNRF dataset, with better MSE performance. This demonstrates the robustness of SI-INR. We are committed to further exploring its potential and will provide more extensive results in the Final version. Thank you again for your constructive feedback.

Thank you for the questions. Thanks to the continuous nature of SI-INR output, SI-INR supports the use of arbitrary grid sizes during training. As we mentioned in the paper, we used uniform sampling in our experiments. Compared with the traditional grid-based models, SI-INR can be trained under arbitrary sizes of grids. To simplify the task while balancing the computational efficiency and implementation convenience, we chose specific grids for different datasets for fair comparison, for example, we sampled $128 \times 128$ grids on the RSOC datasets, ensuring acceptable training speed while preserving fine details in the density maps.

During inference, density maps can be generated at different resolutions, with the final count estimation obtained by summing the values of the generated density maps. To ensure consistency, we reweight the density maps of size $2W \times 2H$ by a factor of $1/4$ so that their summation matches that of the $W \times H$ density maps. For low-resolution inputs, increasing the grid size can enhance counting performance.

Feel free to let us know if there are more questions.

Thanks for your suggestion and advice. We understand your query about whether this model learns a continuous representation. As we mentioned in our response, we do learn continuous representations in SI-INR by transforming the latent features into continuous ones first before we put them into the decoder, as we mentioned in Section 3.2.2.

During training, we randomly sample from INR to generate different scale outputs, and the density maps are rescaled to $128 \times 128$ when we compute the loss function. In this way, we indeed use random uniform grids while making it easy to apply the Bayesian counting loss function.

We agree that your suggested implementation can also achieve continuous representations, but our current SI-INR implementation also learns a continuous function, even in specific situations, these two methods are equivalent.

Thanks again for your suggestion.