DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception

But I think this paper has significant flaws:

1. The author's experimental results are not convincing. The author claims that their results achieved SOTA, but did not compare with better models in the DocLayNet dataset, such as VGT (Da, Cheng, et al. "Vision grid transformer for document layout analysis." ICCV 2023.), GLAM (Wang, Jilin, et al. "A graphical approach to document layout analysis." ICDAR 2023.), etc. These models have achieved better accuracy on DocLayNet.
2. In addition, the author did not compare the results with other experiments on larger benchmarks (DocBank, PublayNet). These datasets are authoritative benchmarks, similar to COCO in general object detection. However, the author's results on the in-house dataset DocStructBench lack authority.
3. The dataset DocSynth-300K synthesized by the author is not large enough, and some publicly available datasets such as DocBank possesses 500K data. Since the author has proposed a method for synthesizing datasets, the amount of data should not be less than some open-source datasets, and may even be more than 3 times larger than the largest dataset. Because the pre-trained dataset needs to be much larger than the fine-tuned dataset to demonstrate the effectiveness.
4. The innovation of this paper is limited, and the proposed model is based on Yolo without significant structural innovation.

1. The main concern is the motivation of the paper. In document layout analysis the problem isn't getting better performance for the well-defined class with structured layout. In contrast, the goal is to improve the performance of small-scale diverse datasets like the following: PRIMA: https://www.primaresearch.org/datasets/Layout_Analysis Historical Japanese dataset: https://dell-research-harvard.github.io/HJDataset/ There are no particular contributions regarding the performance improvement on these small-scale diverse datasets.

2. In the related works, Some Unimodal works have not been cited, which are pretty effective and faster compared to the proposed one. SwinDocSegmenter: https://arxiv.org/abs/2305.04609 SemiDocSeg: https://link.springer.com/article/10.1007/s10032-024-00473-y GraphKD: https://arxiv.org/abs/2402.11401

3. The main drawback of Meshgrid construction is the layout will be always structured (i.e. it ensures all the document elements must have a non-overlapping rectangular region). How they tackle complex layout which has overlapping region and unstructured documents like magazines and comics.

4. How does the GL-CRM impact computational efficiency, especially for larger models or when scaling? As it is just a combination of GeLU activation over batch-normalization. Why do other activation fails? Does LayerNorm can also affect performance?

The author's contributions in terms of both data and model design seem acceptable. As I am not in this field, I still need to refer to the opinions of other reviewers and the chair. I have some concerns as follows.

1. When the author introduces the two types of methods in the introduction, references should be added. Furthermore, you can introduce how some classic methods are done.

2. The author is extremely reluctant to add references throughout the paper, which is very unfriendly to researchers who are not in this field. In addition, references should also be added in the tables. Otherwise, it is not clear whether these are the author's own reproductions or results from other papers.

3. The performance comparisons in Table 2, Table 3, and Table 4 are unfair as the pre-training data of each method is different.

4. In various other fields, the feature fusion of global-local seems to be a common practice and there is nothing new about it.

5. The author seems not to have mentioned Figure 4 in the manuscript.

• The authors state in line 153 on page 3 that "unimodal pre-training datasets are characterized by significant homogeneity, primarily comprising academic papers." This statement is inaccurate. In fact, there are already datasets from diverse domains, such as M6Doc, which is a large-scale, multi-domain dataset. Moreover, the authors’ own DocSynth-300k dataset is constructed based on M6Doc, further illustrating the availability of varied data sources.
• The description of layout generation is not very clear. How does the candidate sampling and meshgrid construction process ensure that elements are placed effectively on a page layout without overlapping?
• The experiments conducted on in-house data are not very convincing. It would be better to include a different backbone model pre-trained on DocSynth-300k and test it on public datasets.
• Lack of comparison with SOTA: "M2Doc: A Multi-Modal Fusion Approach for Document Layout Analysis,AAAI 2024"