Coupling Fairness and Pruning in a Single Run: a Bi-level Optimization Perspective

We thank Reviewer 5ZBL for the time and effort in reviewing our manuscript. We appreciate these valuable comments and will integrate them further to improve the manuscript. Below, we address each of the points raised in the reviews.

• W1: In our experiments, we utilized two methods for initialization. The first method sets the initial values to the pre-trained weights. The second method employs Kaiming Initialization [1]. We observed that both methods yield similar performance, indicating that different initialization approaches do not significantly impact the final results. We plan to include a more detailed discussion regarding this observation in the experimental section of our paper.
• W2: This work highlights a critical social issue in pruning and offers an efficient and effective solution. The theoretical proof will be the focus of our future research efforts.
• W3: We argue that the improvement is not marginal. The experimental results show substantial improvement. For instance, as illustrated in Fig. 4, on the Celeba dataset with 'Blond_Hair' as the target attribute and 'Male' as the sensitive attribute, our approach surpasses baseline models by achieving 6% higher accuracy and reducing bias by 300% at 80% sparsity. This indicates that our coupling method not only enhances accuracy but also significantly improves fairness. Moreover, our proposed method exceeds the performance of the best baseline methods while also substantially reducing training costs. Specifically, it saves 94.22% and 94.05% of training iterations on the LFW and Celeba datasets, respectively.
• W4: In our study, we deliberately selected smaller models to distinctly showcase the effectiveness and equity of the proposed pruning method. This approach results in a pruned model with reduced parameters while maintaining performance levels that are comparable to larger models. Regarding datasets, we adhered to the standard configurations commonly employed in related research, e.g., [1].
• W5: While it is indeed true that deep neural networks are inherently non-convex, the application of convex relaxation in our context serves a specific and strategic purpose. The convex relaxation simplifies these constraints, making them more tractable within the optimization process. In addition, convex relaxation effectively reduces estimation errors between the actual non-differentiable functions and their relaxed counterparts [2]. This approach simplifies the tuning process and facilitates a balanced trade-off between performance and fairness. We will add experiments with non-convex surrogates in the next version.

[1] Tang, Pengwei, et al. "Fair Scratch Tickets: Finding Fair Sparse Networks Without Weight Training." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

[2] Yao, Wei, et al. "Understanding Fairness Surrogate Functions in Algorithmic Fairness." arXiv preprint arXiv:2310.11211 (2023).

We thank Reviewer Wsp2 for the time and effort in reviewing our manuscript. We appreciate these valuable comments and will integrate them further to improve the manuscript. Below, we address each of the points raised in the reviews.

• W1: This work primarily concentrates on fairness in network pruning, and our key contribution is an efficient and equitable pruning strategy. While study [a] utilizes first-order derivatives to calculate the Implicit Gradient (IG) for bi-level optimization, thus laying a solid foundation for network pruning, our research explores a less examined societal dimension of this domain. We introduce a constrained bi-level optimization problem and propose an innovative approach to network pruning that integrates fairness considerations, distinguishing our work from existing methodologies.

• W2: We use Definition 2 in the experiment 5.3. Although we do not use Definition 3 in the evaluation section, it is useful to examine bias in terms of model pruning in other cases. We will integrate more evaluation perspectives in the next version.

• W3: In our study, we deliberately selected smaller models to distinctly showcase the effectiveness and equity of the proposed pruning method. This approach results in a pruned model with reduced parameters while maintaining performance levels that are comparable to larger models. Regarding datasets, we adhered to the standard configurations commonly employed in related research, e.g., [1]. We will continuously seek larger datasets for fairness and computer vision.

• W4: In Section 5.3, we outline the design of our baseline methods, integrating a two-stage fair training and pruning strategy. This approach sequentially combines model pruning and fair training, with a distinct contrast to our proposed single-run method. Whereas FairGRAPE is focused on minimizing the variance in performance disparities, our method is specifically aimed at reducing the relative disparities among various subgroups. Thus, we didn't include FairGRAPE in the comparison.

• Q1: We follow the surrogate functions in [1], and in the experiment, we use the logistic surrogate function.

• Q2: We follow the common practice of selecting m. In particular, in each iteration, we sort $m$, and assign top-$k$ elements to 1, and the rest of the mask to 0, where $k$ is a predefined density level.

[1] Bartlett, Peter L., Michael I. Jordan, and Jon D. McAuliffe. "Convexity, classification, and risk bounds." Journal of the American Statistical Association 101.473 (2006): 138-156.

We thank Reviewer SMS5 for the time and effort in reviewing our manuscript. We appreciate these valuable comments and will integrate them further to improve the manuscript. Below, we address each of the points raised in the reviews.

• W1: In our study, we deliberately selected smaller models to distinctly showcase the effectiveness and equity of the proposed pruning method. This approach results in a pruned model with reduced parameters while maintaining performance levels that are comparable to larger models. Regarding datasets, we adhered to the standard configurations commonly employed in related research, e.g., [1]
• W2: As highlighted in section 4.2, the complexity of bi-level optimization problems increases significantly with additional constraints. This complexity is particularly evident in our methods, where constraints encompass both upper and lower-level variables, making the problem more challenging to solve. Addressing the theoretical proof of our approach remains an area for future investigation. We plan to concentrate our future research efforts on this theoretical aspect. In this paper, we have brought attention to an important ethical issue in pruning, and have proposed a potential solution for it.

[1] Tang, Pengwei, et al. "Fair Scratch Tickets: Finding Fair Sparse Networks Without Weight Training." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.