FRACTAL CALIBRATION FOR LONG-TAILED OBJECT DETECTION

1. Although the proposed approach is novel, some of the individual steps are not sufficiently motivated. For example:

1.1. "Eq. 9 downweighs the classes that appear most uniformly and it upweighs the classes that appear less uniformly. In practice, this scheme enforces a centre bias for frequent classes and no bias for rare classes, as shown in Fig. 2-bottom-right. Intuitively, removing the bias from the rare classes is better than rectifying it because it produces balanced detectors and aligns better with the target distribution as shown in our ablation and our qualitative results." => It is not clear why this is enforced. Can we not have frequent classes that have some spatial bias (other than the center?) or that are uniformly sampled?

1.2. Figure 4d-e: The implication of weak correlation between n(y) and Phi(y) should not be surprising since the fractal definition in Eq 8 does not depend on the number of occurrences in a cell (since an indicator function is used). However, it is not clear why this is preferred.

2. Fractal dimension measures the uniformity of spatial distributions. Is this the best choice for this purpose, considering its limitations (not being able to take the limit on G, thresholding it and fitting a line etc.)? One could use many different methods available for measuring dispersion.

3. The extension to the binary case in Eq 11 appears ad hoc and is not justified properly.

1. I feel confused about the description of Fig. 1. Why class prior ps(y) is used to align ps(y, u|x) with pt(y|x) rather than align ps(y|x) with pt(y|x)? Do you mean that space information has been considered by previous work just for training data?

2. Although the fractal dimension is introduced, the theoretical explanation of how the fractal dimension specifically affects the detection of rare categories can be more detailed. It is possible to further discuss how the fractal dimension improves the performance of category-position dependencies or provide some more intuitive explanations.

3. Although FRACAL is applied at the inference stage, calculating the fractal dimension may affect the inference speed. It is recommended to add relevant complexity analysis to evaluate the impact of fractal dimension calculation on inference efficiency and the performance of this method on resource-constrained devices.

4. More visualization examples of experimental results can be added to demonstrate the performance of FRACAL in rare category detection, such as the detection location distribution or confidence changes of certain categories, to enhance the intuitiveness of the method's effect.

5. Fig. 2 needs to be explained in more detail, especially for the bar chart located in the right part.

6. Why do different datasets and ablation experiments have different evaluation indicators? I suspect that other indicators are not listed because they are not as good as existing methods. If not, please explain the reason.

7. More minor issues should be more attention. I just list part of them: (1) Figure 1-left should be Fig. 1-left. (2) Eq 3 lost dot (Eq. 3). (3) The sub-tables in Table 4 should appear in the order of the ablation experiments. (4) In Table 4-e, please clearly show the G values ​​used in the final model.

1. While the fractal dimension is a core component of the proposed method, the paper could benefit from a more detailed analysis on how different fractal dimension values influence performance across categories. This would add insight into the method's adaptability to datasets with varying spatial distributions.

2. The paper briefly discusses the limitations of grid-based approaches but does not thoroughly compare FRACAL against alternative spatial calibration methods. Including this would help contextualize FRACAL's advantages and trade-offs relative to other spatial calibration techniques.

3. The choice of parameters, such as the fractal dimension coefficient (𝜆) and log base (𝛽), might make FRACAL difficult to tune for practitioners who may not have expertise in spatial calibration. More guidance on parameter selection based on dataset characteristics could improve usability.

There are three major weaknesses:

1. Clarity: The paper is not well-written. Important pieces of information are not clear.
2. Substantiation of the main claim: I am not sure whether the main claim of the paper, which is incorporating spatial location distributions, is substantiated.
3. Performance comparisons: Experimental comparisons do not provide a clear indication whether we should use this technique with modern object detectors.

I explain them in detail below.

Clarity:

• Equation 2, which actually presents the mathematical model for class-location dependence, is not explained well. There are certainly some independence assumptions among category, location and generic objectness here. Such a key expression should be explained in sufficient detail.
• L213: “generic object occurrence” -> what does this mean exactly?
• The rule to deal with rare classes (lines 321-330) is not clear to me at all. It is not clear how orange and blue dots are identified.
• Figure 1, at the point where it appears, is not understandable at all. It heavily relies on notation, which is not introduced yet. The right panel of Figure 1 is simply a fancy figure. It doesn’t convey useful/key information to the reader.
• Figure 2 is also not understandable. If the p(hat,u) on the left reflects ground-truth, how come the p(hat,u) on the right is a good prediction? Same with tiara.
• Figure 5 is not understandable. Reading the explanation in the caption and the paragraph in lines 522-528 several times, I still do not know what this figure proves.
• In general, the math notation is lousy. E.g. z_y is called a logit (L251) and z_y,u is called a “predicted proposal” (L296). It would help if the notation is made more rigorous. “Generic object occurrence” is ambiguous. Instead, defining concepts using object proposals (boxes, anchors, etc.) would be more useful.
• Usage of all \cite commands are wrong. Authors should look up how to use \citep and \citet.
• What is “pp”? Percentage points? Should have been explained in text.

Substantiation of the main claim:

• The main claim of the paper is incorporating location information to the prior. But is there a discrepancy between train and test sets regarding location distributions? In L257, authors themselves write that object location distribution is the same in train and test sets.
• Does the box counting method really reflect the spatial location distribution of instances? I am not sure. It seems like a complicated way of counting instances. For example, it is hinted in Figure 2 that “hat” and “tiara” are objects that occur at the top portion of images. How is this location information taken into account by the box-counting method? A convincing ablation would be as follows: let’s create two classes of objects. For the first one, let all its instances be in the top part of the images, and for the second one, let all its instances be in the bottom part. Would FRACAL perform better than baseline or other detectors?
• Claims made about performance may not hold true if more modern object detectors are used (see below).

Performance comparisons:

• SOTA comparisons are using old backbones (Resnet50, 101), therefore the performance is limited. On LVIS, the performance is around 50 AP (https://paperswithcode.com/sota/object-detection-on-lvis-v1-0-val). Methods from 2022 report 53.4 AP_b, 48.1 AP_m (Li, Mao, Girshick, He; ECCV2022). Since FRACAL is training free, it should be easy to apply it to more recent detectors and report results. This would give us a better picture on how effective the proposed method is.
• More detailed results on COCO would help, such as AP_small. Also, it should be easy to create AP_rare, AP_freq on COCO.