PRUNING CNNS WITH GRAPH RANDOM WALK & RANDOM MATRIX THEORY

• Wrong expression. In addition to weight pruning and filter pruning, there is also layer pruning.
• The baselines used for comparison are relatively old, mostly from 2022 and before.
• The title of Table 1 is below the table, while the title of Table 2 is above the table.
• There are only two tables on page 9, leaving a lot of blank space, making the article look rough.
• The model structure used for verification is too simple. Experiments were only conducted on ResNet-56, ResNet-110 and VGG-16. Can this filter pruning method be applied to existing large language models?
• What is the time cost of graph-based filter pruning algorithm?
• Does the number of filters affect the efficiency of the algorithm? If the author can solve my above problems, I will be happy to increase the score.

1. Limited Evaluation Across Diverse Architectures: While the method is tested on VGG-16 and ResNet models, both of which have relatively regular structures, the paper lacks experiments on more diverse architectures. Further, I find the performance improvements not substantial.
2. Sensitivity to Hyperparameters: The paper uses fixed values for key hyperparameters, such as the random walk transition probabilities and the pruning strength parameter, without exploring their influence on model performance or robustness. Hyperparameters may play a significant role in its effectiveness, especially in controlling the extent and impact of pruning.Conducting a sensitivity analysis or ablation study on important hyperparameters, such as the graph edge weights or the random walk probabilities, could provide insights into how these settings impact performance across different tasks.
3. Scalability and Applicability to Larger Networks: The method is primarily evaluated on relatively small datasets like CIFAR-10 and moderately large networks like ResNet-50 on ImageNet. However, it remains unclear if the approach scales effectively to larger architectures (e.g., ResNet-152, EfficientNet) or more complex datasets. As model and dataset sizes increase, the computational overhead of constructing and analyzing a filter graph could become a bottleneck, potentially limiting scalability. To strengthen this analysis, I recommend that the authors provide a computational complexity analysis or empirical runtime comparisons with existing methods, especially for larger architectures. Additionally, evaluating the method on more datasets like CIFAR-100 or MS COCO, would provide a more comprehensive understanding of the approach’s scalability and applicability in real-world scenarios.
4. Limited Comparison with Other Graph-Based Pruning Methods: Although the paper briefly mentions recent graph-based pruning methods, such as the use of Von Neumann graph entropy (Shi et al., 2023), it does not offer a direct performance comparison with these methods. Since both approaches leverage graph representations to model filter importance, comparing them directly could help clarify the advantages or limitations of the proposed method.
5. Lack of Qualitative Insights on Pruned Model Structure: The paper lacks qualitative insights into the structural impact of pruning. For instance, it would be valuable to see if the graph-based pruning method preserves specific types of filters, or if it tends to prune filters that correspond to certain types of visual features.

1. Limited datasets of experiments. Although the paper performs experiments on the CIFAR-10 and ImageNet datasets, it lacks validation on other types of datasets. This may limit the generality of the approach.
2. Comparison algorithms are few and not new enough.
3. The paper lacks an analysis of the algorithmic time complexity of the pruning process

Cons:

1. The description of the experimental results lacks clarity. For instance, in Table 2, how to select certain rows in bold is not explained. It is unclear why, for example, CHIP achieves better pruned accuracy in row 5 compared to the proposed method shown in row 7.

2. The experimental results do not include comparisons with more recent publications or other graph-based pruning methods, such as VNGEP.

3. The proposed method lacks a comprehensive theoretical analysis.

1. Unclear motivation in introduction:

• The third paragraph of the introduction appears redundant. If the authors aim to provide background information, the detailed descriptions of other methods might be better placed in the related work section. Alternatively, if the intention is to highlight the limitations of existing graph-driven methods to underscore the importance of their work, a more in-depth analysis is needed rather than concluding with a brief statement like "however, they only focus on the approximated error and haven’t dived into the relationships among filters."

• The fourth paragraph of the introduction uses a single sentence to point out the work's motivation: "However, most of these approaches predominantly rely on similarity metrics like Euclidean distance or cosine similarity to measure the similarity between filters, which only considers the pairwise relationships between filters, overlooking the interactions among them within the whole layer." This is insufficient to convey the depth of the motivation. The authors should expand on why pairwise operations overlook the underlying global information and how this impacts the effectiveness of pruning methods.

2. Lack of discussion on transformers: The paper lacks discussion on transformers, which are inherently dense deep neural network modules with significant pruning potential. Considering the conceptual connections between transformers and graph neural networks, it would be valuable to explore whether the proposed graph-driven filter pruning method could be even more effective on transformer architectures.

3. Insufficient theoretical justification: There is a lack of theoretical justification to demonstrate the advantages of the proposed method.

4. Outdated related works and missing references: The related work section is somewhat outdated and misses important recent studies [1, 2, 3].

5. Lack of discussion regarding the potential limitations of the proposed method.

[1] Interpretable Task-inspired Adaptive Filter Pruning for Neural Networks Under Multiple Constraints

[2] Pruning from Scratch via Shared Pruning Module and Nuclear norm-based Regularization

[3] Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition

1. This paper only conducts experiments on the CIFAR10 and ImageNet datasets, lacking testing on larger-scale datasets. Additionally, the comparison methods are relatively outdated, missing comparisons with the latest SOTA methods.
2. The study only explores the effectiveness on CNNs. Would this method also be effective on Transformer models?
3. The paper lacks ablation studies and does not include sensitivity analysis on key parameters.
4. There is no analysis of the algorithm's time complexity. At the same time, the results indicate that the performance improvement is modest, raising the question of whether the additional time cost is justified.
5. Why does Table 3 only show reduction rates for Params and FLOPs, instead of specific values as shown in Table 2?
6. The proposed method lacks theoretical support.
7. The paper contains some writing errors, such as a repeated period in line 516. Additionally, the text in the images is too small, making it difficult to read.

Methodological Gaps:

1. Edge weight construction process is not clearly defined.

2.Unclear choice between potential similarity metrics (Euclidean, cosine, inner product).

3.Lack of justification for specific graph construction choices.

Limited Evaluation:

1. Comparisons mainly with older methods.

2. Limited benchmark datasets.

3. Missing comparisons with recent state-of-the-art pruning methods.

Theoretical Integration:

1. Random matrix theory and graph-based approach treated separately despite potential unification.

2. Missing theoretical connection between these two perspectives.

3. Incomplete justification of random matrix assumptions.