Evolutionary Multi-View Classification via Eliminating Individual Fitness Bias

We sincerely thank you for your thorough reading and positive evaluation of our work. We are grateful for your recognition of our contribution in identifying and addressing the novel issue of fitness evaluation bias (FEB) in evolutionary multi-view classification (EMVC). We also appreciate your affirmation of the clear structure, well-designed illustrations, and comprehensive experimental setup of our paper. Your encouragement and suggestions are of great significance for the improvement of our future work.

Q1: How do the crossover and mutation operators work? More details are necessary to the readers that do not very familiar with this topic.

A1: For the crossover operation, a non-root node is first randomly selected from each of two randomly chosen binary trees to serve as the crossover point. The selected node along with its subtree is then pruned from the original tree and exchanged with the counterpart from the other tree. These subtrees are subsequently grafted onto the parent node of the original crossover point in the opposite tree, thereby generating two new individual structures. For the mutation operation, a node is randomly selected from the individual's tree structure. If the selected node is a fusion operation node, a new fusion operation is randomly sampled from a predefined list to replace the original operation. If the selected node is a view node, a new view is randomly chosen from a set of views to replace the original one.

Q2: How are the hyper-parameters of compared baselines set?

A2: Given that our proposed EFB-EMVC method essentially falls under the category of EMVC approaches, we have only listed the hyperparameter settings of representative methods within this category in the table. Considering the wide variety of EMVC methods, we selected two of the most representative works—namely, the pioneering method EDF and the latest state-of-the-art method TEF. Their detailed hyperparameter configurations are provided below.

In both EDF and TEF, all deep neural network models are trained using the Adam optimization algorithm. The learning rate is set to 0.001, with exponential decay rates of 0.9 and 0.999 for the first and second moment estimates, respectively. Each network is trained for 100 epochs. To prevent overfitting, early stopping is applied if the model performance does not improve for 10 consecutive epochs. To efficiently utilize GPU resources, the population size is set as a multiple of the number of GPUs, and is configured as 28. The number of generations is set to 20, with crossover and mutation probabilities of 0.9 and 0.2, respectively. It is worth noting that the primary goal of TEF in the first stage is to obtain a fusion architecture capable of generating high-quality pseudo views. The detailed hyperparameter configurations for the second stage of TEF are presented in Table (1).

Table(1): The detailed hyperparameter configurations for the second stage of TEF

In response to your comments, we will include detailed explanations of the "crossover mutation operation" (line 202) and the "baseline method settings for comparison" (line 233) in future revisions, in order to enhance readability and understanding for non-expert readers. Thank you once again for your valuable feedback and suggestions. We will carefully consider them and make corresponding improvements to the paper.

We sincerely appreciate your positive evaluation of our work. We are delighted that you recognize our introduction of the “fitness evaluation bias (FEB)” problem for the first time in the context of evolutionary multi-view classification (EMVC), and that you acknowledge the effectiveness of the proposed evolutionary navigators (ENs) in enhancing multi-view classification performance. We are also deeply grateful for your recognition of the soundness of our experimental design and the depth of our analysis, particularly the dissection of each module’s individual contribution through the ablation studies.

Q1: In the hyperparameter analysis section of the supplementary materials, the paper analyzes the impact of different view fusion dimensions on performance. However, hyperparameters such as batch size also affect model performance, and I hope to see detailed information on how such hyperparameters influence performance.

A1: To more comprehensively analyze the impact of hyperparameters (such as batch size) on the performance of EFB-EMVC, we conducted additional experiments using the NUS dataset as an example. We first randomly initialized a fixed population consisting of 15 individuals and examined the influence of hyperparameters on model performance within this fixed population. The experimental results are quantitatively presented using the population’s average performance (Avg) and the best individual performance (Max) as metrics. Besides batch size, we further analyzed the effects of epoch and patience on model performance to gain deeper insights into the model behavior.

① Batch Size:

The related results are summarized in Table (1). The results show that both Avg and Max performance exhibit a rising-then-falling trend, with both reaching their peak at a batch size of 256. Our analysis is as follows:

1. A small batch size (e.g., 32) allows MVM to see only very few samples at a time, resulting in unstable training and difficulty fitting the true distribution;

2. A moderate batch size (e.g., 256) provides enough samples to represent the overall distribution while avoiding local optima, leading to more stable training;

3. An excessively large batch size (e.g., 1024), although representative of the true distribution, tends to fall into sharp minima, which harms generalization ability.

In our main experiments, a unified batch size of 512 was used across all datasets.

Table(1) : Batch Size Impact on EFB-EMVC Performance (Avg and Max Accuracy)

② Epoch:

The relevant results are shown in Table (2). We observe that the Avg performance remains relatively stable, while the Max performance shows slight fluctuations, which can be attributed to the randomness in network initialization. Given the limited potential for further performance improvement, we hypothesize that most individuals have already converged before epoch 50. To verify this assumption, we further examined the convergence behavior of 15 individuals at epoch=50. The results, presented in Table (3), indicate that all individuals indeed completed convergence before 50 epochs, thereby confirming our hypothesis. In the main experiments, we uniformly set the number of training epochs to 200. Thanks to the adoption of an early stopping strategy with a patience of 10, this setting not only prevents excessive resource consumption but also effectively mitigates overfitting.

Table(2) : Epoch Impact on EFB-EMVC Performance (Avg and Max Accuracy)

Table(3) : Distribution of Convergence Epochs for 15 Individuals

③ Patience:

The related results are summarized in Table (4). Both Avg and Max improved as patience increased, with more noticeable gains when increasing patience from 10 to 20, but limited improvement from 40 to 50. We believe that moderately increasing patience helps the model achieve better solutions, but excessive training may lead to overfitting in MVM. Balancing efficiency and performance, we adopted a uniform patience of 10 across all datasets.

Table(4) : Patience Impact on EFB-EMVC Performance (Avg and Max Accuracy)

Q2:There are some missing details. For example, in line 133, the novel distillation loss function proposed by the author is named "WDK", and the ablation experiment on the right side of Figure 3 also ablates "WDK". However, in Table 5 of the appendix, the distillation loss function is "WDT". If this is a typo, it is recommended to rewrite it. If they are two different loss functions, it is suggested to provide more explanations.

A2: Thank you for pointing out the inconsistency in naming. We confirm that this was a typographical error: it should be “WDK,” but it was mistakenly written as “WDT” in Appendix Table 5. We will correct and unify the term to “WDK” in the revised version to eliminate any ambiguity.

Q3: Some sentences in the paper have incomplete semantics or unclear expressions. For example, the part about feature matrix construction in lines 117-120. The term "feature vector" is somewhat ambiguous. In this context, it is unclear which part of the output result the "feature vector" specifically refers to.

A3: In lines 117–120, our approach is to input the data of the $i$-th view into the pretrained teacher model for that view and extract the logits before the softmax layer as the feature vector. Therefore, the term “feature vector” here specifically refers to the logits output by the teacher model.

We sincerely appreciate your valuable comments regarding hyperparameter settings, consistency in terminology, and clarity of expression. In the revised manuscript, we will include additional experiments to analyze the impact of such hyperparameters on model performance, standardize relevant terminology, and refine the corresponding descriptions to enhance accuracy and clarity. Once again, thank you for your thorough review and constructive suggestions, which are of great significance in improving the quality of our paper.

We sincerely appreciate your thorough review and positive evaluation of our work. We are glad that you recognize our novel introduction of the knowledge distillation (KD) mechanism into the evolutionary multi-view classification (EMVC) task, as well as the effectiveness and practical value of our proposed evolutionary navigators (EN) mechanism in complex multi-view scenarios. At the same time, we thank you for acknowledging the soundness of our methodology, the rigor of our experimental design, and the thoroughness of our performance evaluation.

Q1: In Figure 1 of the paper, the performance curves of View 4 under three different scenarios are presented. Among them, "V4 only" refers to the pre-trained teacher model of View 4, and its pre-training method can be clarified according to the content shown in Figure 2 (b). However, there are still questions regarding the determination methods of the performance curves of View 4 in "V4 in MVM" and "V4 with EN". It is recommended to supplement the relevant details.

A1: In Figure 1, we present the performance curves of $V_4$ under three different scenarios: “$V_4$ only”, “$V_4$ in MVM”, and “$V_4$ with EN”. Their specific meanings and determination process are as follows:

① “$V_4$ only” refers to the performance of the teacher model corresponding to this view during the pretraining stage;

② “$V_4$ in MVM” represents the performance of this view when used as a branch in a conventional EMVC framework;

③ “$V_4$ with EN” indicates the performance of this view as a branch in our proposed EFB-EMVC framework.

The performance curves were obtained through the following process:

① Model Transfer: We extract the trained parameters of the $V_4$ branch from the final multi-view model and transfer them into a single-view neural network with an identical encoder structure; ② Feature Extraction: We input the training and testing data of $V_4$ into this single-view network to obtain intermediate representations; ③ Construction of the classification head network: This network consists of a fully connected layer (with output dimension equal to the number of classes) followed by a softmax layer. It takes the intermediate feature representations extracted by the single-view network as input and is trained using the standard cross-entropy loss; ④ Performance Plotting: Classification accuracy is recorded at each training epoch to plot the performance curves of $V_4$.

Q2: Compared to Table 1, Table 2 show that the proposed method has the limited improvement, it is necessary to provide more deep discussion.

A2: The core of the EFB-EMVC method lies in addressing the issue of fitness evaluation bias (FEB), which arises from the unbalanced utilization of views in traditional MVM frameworks. The degree of variation in information content between views across different datasets directly affects the severity of the FEB problem, which in turn influences the performance gains achieved by our method.

Taking the MVoxCeleb dataset from Table 1 and the NTU RGB-D dataset from Table 2 as examples, we present the classification accuracy of the pretrained teacher models corresponding to each view. The experimental results are summarized in Table(1) below. Based on these results, we draw the following conclusions:

① In the MVoxCeleb dataset, the number of strong and weak views is relatively balanced. Specifically, $V_0$, $V_1$, and $V_3$ are classified as strong views, while $V_2$ and $V_4$ are weak views. During the construction of the MVM, weak views have a higher probability of being selected. However, the joint training strategy employed by MVM tends to cause the model to overly rely on strong views, neglecting the weak ones. Our proposed EN mechanism effectively alleviates this bias, leading to a significant improvement in performance;

② In the NTU RGB-D dataset, strong views dominate, with only one weak view ($V_4$). Due to the large number of strong views, the probability of selecting the weak view is relatively low. Even when the weak view is selected during MVM construction, the abundant information from the strong views largely compensates for the weak view's shortcomings. Therefore, the performance improvement brought by the EN mechanism on this dataset is comparatively smaller.

Table(1) : Performance of Pre-trained Teacher Models for Each View on the MVoxCeleb and NTU RGB-D Datasets

Nonetheless, across all datasets, our method consistently achieves the best performance, which further demonstrates its broad applicability and effectiveness.

Q3: Table 3 shows the rank relationship between with-En and without-EN, it is very key for showing the necessary of the individual fitness bias elimination. So, it is suggested to add the detailed formulations of PC, SPC and KT, and give the association criterion.

A3: Thank you for your valuable suggestions concerning the statistical metrics used in Table 3. We fully agree that PC (Pearson Correlation Coefficient), SPC (Spearman’s Rank Correlation Coefficient), and KT (Kendall’s Tau Coefficient) play a crucial role in demonstrating the existence of the Fitness Evaluation Bias (FEB) problem and the necessity of its mitigation. To enhance the clarity and reproducibility of our paper, we will include formal definitions of these metrics in the revised manuscript.

① PC (Pearson Correlation Coefficient): PC measures the linear correlation between two variables. Its formula is: $\text{PC} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^{n} (x_i - \bar{x})^2} \cdot \sqrt{\sum_{i=1}^{n} (y_i - \bar{y})^2}}$ where $x_i$: the value of the $i$-th observation of variable $X$; $y_i$: the value of the $i$-th observation of variable $Y$. Similarly, $\bar{x}$ and $\bar{y}$ denote the sample means of variables $X$ and $Y$, respectively.

② SPC (Spearman's Rank Correlation Coefficient): SPC measures the monotonic relationship between two ranked variables. Its formula is: $\text{SPC} = 1 - \frac{6 \sum_{i=1}^n d_i^2}{n(n^2 - 1)}$ where $d_i$ is the difference in ranks of the $i$-th sample between the two rankings.

③ KT (Kendall's Tau Rank Correlation Coefficient): KT is based on the number of concordant and discordant pairs among all sample pairs, calculated as: $\text{KT} = \frac{C - D}{\sqrt{(C + D + T_x)(C + D + T_y)}}$ where:

• $C$ is the number of concordant pairs;
• $D$ is the number of discordant pairs;
• $T_x$, $T_y$ are the numbers of ties (equal ranks) in $x$ and $y$, respectively.

Regarding the “correlation discrimination criterion,” within the context of this paper, we consider that a low correlation (correlation coefficient close to 0) between the rankings with and without EN indicates a significant change in ranking caused by the introduction of EN. This suggests that the fitness evaluation is strongly influenced by EN, thereby validating the existence of FEB and its interference with the final results. Conversely, a high correlation (close to 1) would imply minimal impact of EN on fitness ranking, which contradicts our observations.

In the revised version, we will explicitly add the mathematical definitions and corresponding decision criteria for PC, SPC, and KT, provide a more detailed description of the process for plotting the performance curve of $V_4$, and further analyze the reasons behind the limited performance improvement observed in Table 2. We sincerely thank you once again for your thorough and insightful review comments, which are invaluable for the further refinement of our paper.

Thank you very much for your careful review and high praise of our paper. We are glad to see that you recognize our work’s positive contribution in identifying and addressing key issues in the evolutionary multi-view classification (EMVC) field, and we also appreciate your acknowledgment of the comprehensiveness of our experimental design.

Q1: As mentioned in the paper, evolutionary navigators are introduced into the multi-view model (MVM) to fully utilize the information of each view and solve the fitness evaluation bias (FEB) problem. I have reviewed the experimental section and supplementary materials of the paper. Although the authors have conducted relatively comprehensive experiments with broad coverage, they have not reported the accuracy of the pretrained teacher models for each view on the respective datasets. I suggest the authors include this information in the revision to enhance the interpretability and credibility of the experimental results.

A1: For each of the nine multi-view datasets, we pretrained a separate teacher model for each individual view to ensure reliable knowledge transfer. The architecture of the teacher model is consistent with the single-view encoder used in the multi-view model, specifically consisting of two fully connected layers. The output layer employs a softmax function to produce a predicted probability distribution, and the model is trained using the cross-entropy loss function. For the MVoxCeleb, YouTubeFace, Reuters5, Reuters3, and NUS datasets, we adopted a five-fold cross-validation strategy and reported the final accuracy of the pretrained teacher models in the format of “mean ± standard deviation.” For the MM-IMDB, NTU RGB-D, EgoGesture, and CB datasets, we followed the experimental setup of DC-NAS, conducting five independent runs using predefined train-test splits for each view. The results are likewise reported in the format of “mean ± standard deviation.” The corresponding experimental results are summarized in Table 1. In the revised version of the paper, we will additionally include the accuracy of the pretrained teacher models for each dataset to enhance the transparency and interpretability of our results.

Table(1) : Accuracy of the pretrained teacher models for each view on the nine multi-view datasets.

Q2: The paper uses well known public datasets such as MVoxCeleb, YoutubeFace, NUS-WIDE-128, Reuters, CB, MM-IMDB, NTU RGBD, and EgoGesture. These datasets are suitable for the multi-view classification task, but it should be mentioned if any preprocessing was performed, as it may affect the results.

A2: None of the nine multi-view datasets used in our experiments underwent any additional preprocessing. All data were loaded strictly according to the original dataset formats used by the corresponding baseline methods, ensuring fairness and reproducibility of the experimental comparisons.

Thank you once again for your valuable suggestions regarding critical details such as the accuracy of the teacher models and dataset preprocessing. We will carefully consider and address these points in the manuscript revision to further enhance the credibility and persuasiveness of our experimental results. We sincerely appreciate your recognition and support of our research.

We sincerely appreciate your positive recognition and constructive feedback on our work. We are truly pleased to see that you acknowledge the fitness evaluation bias (FEB) problem we proposed and our solution strategy, as well as your high appreciation for our experimental design and analysis.

Q1: If the proposed method were also trained for 20 iterations, would its performance continue to improve significantly, or has it already converged and stabilized around 94.35% after 5 iterations?

A1: To more comprehensively verify the convergence behavior of our method on the MVoxCeleb dataset, we take the first fold of data as an example. We set the maximum number of iterations to $T$ = 20 and record the model's best performance at $T$ = 5, 10, 15, and 20, to illustrate the trend of performance improvement as the number of iterations increases.

As mentioned in our supplementary material, we also adopted a smaller number of iterations on the YouTubeFace and Reuters5 datasets. To further demonstrate the convergence trend of our method across different datasets, we additionally conducted experiments on these two datasets, again setting the maximum number of iterations to $T$ = 20 and recording the best performance at $T$ = 5, 10, 15, and 20. The corresponding experimental results are shown in Table (1).

The results indicate that our proposed EFB-EMVC method demonstrates a steady performance improvement on the MVoxCeleb dataset from $T$=5 to $T$=15, reaching its peak at $T$=15 and stabilizing thereafter. This suggests that the model has essentially converged by the 15th iteration on this dataset. In contrast, on the YouTubeFace and Reuters5 datasets, the model's performance continues to improve throughout the entire iteration process, showing no clear signs of convergence.

This phenomenon indicates that the EFB-EMVC method exhibits varying sensitivity to the number of iterations across different datasets. Nevertheless, the overall trend suggests that appropriately increasing the number of iterations allows the algorithm to more thoroughly explore the solution space, thereby improving the likelihood of obtaining a global or near-optimal solution and ultimately enhancing model performance.

Table(1) : Classification accuracy (%) of EFB-EMVC under different iteration rounds $T$.

We once again sincerely thank you for your professional review and insightful comments. Regarding the performance trend of EFB-EMVC under different numbers of iterations, we will include the relevant experiments in the revised version to further strengthen the validation of our method's effectiveness and provide a clearer direction for future research.