AF-UMC: An Alignment-Free Fusion Framework for Unaligned Multi-View Clustering

************ For Weakness #1: ************

We provide a symbol list in the following table and we will add it in the final version.

---

************ For Weakness #2: ************

We provide the detailed training process of our proposed AF-UMC in Algorithm 1.

---

Algorithm 1: Training process of AF-UMC

Input: Unaligned multi-view data \mathbf{X} = \left\\{\mathbf{X}^v\right\\}\_{v=1}^V, number of clusters $c$, number of training epochs $T$.

Output: Clustering results.

a. Initialize autoencoders $\\{E^{v}(\cdot), D^v(\cdot)\\}\_{v=1}^V$ and cross-view consistent basis space $\mathbf{B}$.

b. for epoch $t = 1$ to $T$:

       for view $v = 1$ to $V$:

           Extract consistent representation $\mathbf{H}^v$ by projecting $\mathbf{X}^v$ into $\mathbf{B}$

       end for

       Globally bring $\\{\mathbf{H}^v\\}\_{v=1}^V$ closer to fuse a cross-view consistent representation $\mathbf{H}^{core}$.

       Optimize model by $\mathcal{L}\_{r}$, $\mathcal{L}\_{s}$ and $\mathcal{L}\_{c}$.

   end for

c. Perform k-means clustering on $\mathbf{H}^{core}$.

---

************ For Weakness #3: ************

We analyze our proposed AF-UMC in terms of space/time complexity.

Space Complexity: In our method, the memory costs contain a basis space matrix $\mathbf{B} \in \mathbb{R}^{c\times d}$, $V$ autoencoders and $V$ representation matrices $\\{\mathbf{H}^v\\}\_{v=1}^V \in \mathbb{R}^{n\times c}$, where the space complexity of an autoencoder is $\mathcal{O}(lnd)$ and $l$ is the number of MLP layers. As a result, the total space complexity of our AF-UMC is $\mathcal{O}(cd+Vlnd+Vnc)$.

Time Complexity: The time cost of AF-UMC arises from three parts: (1) $\mathcal{O}(Vnd+c^3+nV^2d)$, the cost of computing three loss functions. (2) $\mathcal{O}(Vlnd)$, the cost of optimizing $V$ autoencoders. (3) $\mathcal{O}(cd)$, the cost of optimizing a cross-view consistent basis space $\mathbf{B}$. Therefore, the total time cost of AF-UMC is $\mathcal{O}(Vnd+c^3+nV^2d+Vlnd+cd)$.

---

************ For Question #1: ************

Since the basis space is shared across multiple views and optimized to minimize the overall reconstruction loss, components that are consistently present in all views contribute more effectively to reducing the overall loss. Therefore, when the basis space is designed to reconstruct samples from multiple views, it naturally prioritizes capturing cross-view consistency while gradually reducing the diversity with the model training, thereby capturing consistency while filtering out diversity.

---

************ For Question #2: ************

Compared with existing methods that construct a basis space to capture intra-view information in each view, our cross-view consistent basis space can directly capture cross-view consistent information, facilitating autoencoders to directly extract consistent representations from each view for promoting subsequent global contrastive fusion across views.

---

************ For Question #3: ************

If the number $c$ of basis vectors is greater than their dimension $d$, at least one basis vector $\mathbf{b}_i$ can be linearly represented by the others. As a result, samples represented by $\mathbf{b}_i$ can also be represented by other basis vectors, which intensifies the differences in sample representations and reduces the consistency of representations.

************ For Question #1: ************

When the cross-view consistent basis space is ablated, the loss function $\mathcal{L}\_{s}$ on the basis space is indeed ablated as well. To evaluate the individual impact of the consistent basis space $\mathbf{B}$, we provide additional ablation studies in Table 1, where (1) ablates both $\mathcal{L}\_{s}$ and $\mathbf{B}$, and (2) only ablates $\mathcal{L}\_{s}$. From Table 1, (2) shows better performance than (1), demonstrating the effectiveness of cross-view consistent basis space in prompting autoencoders to extract consistent representations from each view.

Table 1

---

************ For Limitation #1: ************

The model should be compared with the baseline models in 2025.

We provide an experimental comparison between our AF-UMC and the baseline model MGCCFF [1] (2025) in Table 2, where the incomplete rate is set to 0 and other parameters are set to the optimal values provided by the authors. Due to the excessive complexity of MGCCFF, Table 2 only shows the experimental comparison on small-scale datasets, with ACC as the metric. As shown in Table 2, AF-UMC outperforms MGCCFF on all small-scale datasets, demonstrating its excellent performance.

Table 2

[1] Zhao, Liang, Ziyue Wang, Xiao Wang, Zhikui Chen, and Bo Xu. "Incomplete and Unpaired Multi-View Graph Clustering with Cross-View Feature Fusion." In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 22786-22794. 2025.

************ For Weakness #1: ************

The alignment processes of existing methods are mainly divided into two categories: Learning an alignment matrix $\mathbf{P}$ and Employing the Hungarian algorithm. Assuming there are two unaligned and heterogeneous representations $(\mathbf{H}^1, \mathbf{H}^2)$ from two views, and for a specific sample instance $k$, $((\mathbf{h}^1\_i)\_k, (\mathbf{h}^2\_j)\_k)$ denote its corresponding unaligned representations in $(\mathbf{H}^1, \mathbf{H}^2)$. (1) Learning an alignment matrix $\mathbf{P}$: These approaches assume that the consistent representations from each instance $k$ have a lower featural discrepancy, and attempt to learn an alignment matrix $\mathbf{P}$ by minimizing the discrepancy loss between $\mathbf{P}\mathbf{H}^1$ and $\mathbf{H}^2$. However, due to the inherent heterogeneity between $\mathbf{H}^1$ and $\mathbf{H}^2$, the representations from the same instance often exhibit a large featural discrepancy, making it difficult to align them by discrepancy minimization, thereby hindering the alignment process. (2) Employing the Hungarian algorithm: Unlike the above methods aligning representations by extra loss terms, these methods directly employ the existing Hungarian algorithm, which calculates the alignment costs of each possible pair $(\mathbf{h}^1_i, \mathbf{h}^2_j)$ from $\mathbf{H}^1=\\{\mathbf{h}^1_i\\}\_{i=1}^N$ to $\mathbf{H}^2=\\{\mathbf{h}^2_j\\}_{j=1}^N$ by a relevant cost function, e.g., Euclidean distance, cosine distance, etc, and then select the alignment result with the lowest total costs. However, due to the heterogeneity between $\mathbf{H}^1$ and $\mathbf{H}^2$, the alignment cost for some true corresponding pairs (e.g., $((\mathbf{h}^1_i)_k, (\mathbf{h}^2_j)_k)$) is often higher than that for some false corresponding pairs (e.g., $((\mathbf{h}^1\_i)\_k, (\mathbf{h}^2\_p)\_o), k \neq o$), where it is hard for Hungarian algorithm to select the true corresponding pairs for each instance, thereby hindering the alignment process.

---

************ For Weakness #2: ************

The basis space only receives diversity at the beginning of model training. As training progresses, the diversity is gradually filtered out and will not weaken the consistency within the basis space. Specifically, the diversity only represents the information from a specific view, and has limited capacity in reconstructing samples from multiple views. Therefore, when the basis space is designed to reconstruct samples from multiple views, it tends to prioritize capturing cross-view consistency, facilitating the diverse information to fade over the model training. Besides, we constrain the basis space to be low-rank for capturing more compact consistency, which further reduces view-specific diversity.

---

************ For Weakness #3: ************

Table 1 shows the ablation study for reconstruction loss $\mathcal{L}\_{r}$ on Caltech7-5 and NoisyMNIST datasets, where (b) shows a significant performance improvement over (a), indicating that $\mathcal{L}_{r}$ plays a critical role in improving representation quality.

Table 1

---

************ For Question #1: ************

The linearly independent basis vectors have a characteristic that each of them cannot be linearly represented by others, which helps eliminate repeated consistent information (i.e., linearly dependent information) in the consistent basis space. If linearly dependent basis vectors exist, some consistent information will be repeatedly encoded across different basis vectors and samples with consistent information can be represented by different basis vectors, which intensifies the difference in sample representations and degrades the consistency of representations.

---

************ For Question #2: ************

The exponential function $e^{(\cdot)/\tau_g}$ maps the value range $(-1, 1)$ of cosine similarity into a broader range $(0, e^{1/\tau_g})$, which amplifies structural differences among basis vectors, encouraging a clearer and more distinguishable similarity structure.

---

************ For Question #3: ************

This operation will not destroy the cluster distribution and will not make the distribution indistinguishable. In addition to pulling instances toward global centers, the global contrastive fusion also introduces negative pairs to push dissimilar instances apart, preventing instances from being overly concentrated around the centers and preserving the distributional separability. Besides, Figure 3 provides the visualizations of the clustering results (see paper), where our AF-UMC shows a clear cluster distribution, confirming that this operation will not destroy the cluster distribution and will not make the distribution indistinguishable.

---

************ For Limitation #1: ************

We perform the visualization comparison with other SOTA methods (FUMC, GCFAgg, LMVSC, MFLVC, OpVuC, SCMVC) on the large-scale dataset Cifar10. Since we cannot directly provide a link to the visual results, we instead describe their distribution as follows: (1) FUMC, LMVSC, GCFAgg, MFLVC, OpVuC and SCMVC show a poor inter-class separation. (2) Our AF-UMC achieves well-separated clusters. We will provide the visualization comparison in the revised manuscript.

************ For Weakness #1: ************

Existing unaligned multi-view clustering methods fall into two main categories: feature-based methods and structure-based methods, whose reasons for the mismatched constructed basis spaces are as follows: (1) Feature-based methods: The basis space $\mathbf{Z}^v$ is constructed by intra-view sample reconstruction and only captures information from a single view $v$, as reflected by $\\|\mathbf{X}^v - \mathbf{H}^v\mathbf{Z}^v\\|\_{F}^{2}$. Due to the discrepancy in information across different views, it is unlikely for both basis vectors $(\mathbf{z}\_i^v, \mathbf{z}\_i^u)$ to coincidentally capture the matching information from different views $(v, u)$, respectively, thereby leading to mismatches across basis spaces $\\{\mathbf{Z}^v\\}\_{v=1}^V$. (2) Structure-based methods: As self-representation function $\\|\mathbf{X}^v - \mathbf{S}^v\mathbf{X}^v\\|\_{F}^{2}$, the basis space $\mathbf{X}^v$ (right) is directly constructed using the samples $\mathbf{X}^v$ (left) in each view $v$. Due to the misalignment across samples $\\{\mathbf{X}^v\\}_{v=1}^V$, the basis spaces constructed by different $\mathbf{X}^v$ are naturally mismatched across views.

---

************ For Weakness #2: ************

Thank you for the suggestions. We supplement the missing cases as follows. (1) Ablation study on reconstruction loss $\mathcal{L}_{r}$: Table 1 shows the ablation study on $\mathcal{L}\_{r}$. From Table 1, (b) shows a significant performance improvement over (a), indicating that $\mathcal{L}\_{r}$ plays a critical role in improving representation quality. (2) Ablation study on cross-view consistent basis space $\mathbf{B}$: Considering that ablating $\mathbf{B}$ also removes the loss $\mathcal{L}\_{s}$ defined on $\mathbf{B}$, it is difficult to directly evaluate the impact of the individual basis space $\mathbf{B}$. To address this issue, we design the following ablation study in Table 2, where (a) ablates both $\mathcal{L}\_{s}$ and $\mathbf{B}$, and (b) only ablates $\mathcal{L}\_{s}$. From Table 2, (b) shows better performance than (a), demonstrating the effectiveness of cross-view consistent basis space in prompting autoencoders to extract consistent representations from each view.

Table 1

Table 2

---

************ For Weakness #3: ************

We will revise the relevant sentences to make them clearer in the revised manuscript.

---

************ For Question #1: ************

Low-rank constraint can reduce the diversity components from different views and promote the basis space to focus more on capturing cross-view consistency. Specifically, consider $V$ representations $\\{\mathbf{H}^v\\}\_{v=1}^V$ from $V$ views, where $\mathbf{H}^v \in \mathbb{R}^{N \times d}$. We vertically concatenate these representations into a matrix $\mathbf{H}^{all}=[\mathbf{H}^1; \cdots; \mathbf{H}^V] \in \mathbb{R}^{VN \times d}$, and then partition it into $K$ blocks according to the ground-truth class labels:

\\ \cdots ; \\ \mathbf{M}^k; \\ \cdots ; \\ \mathbf{M}^K \end{bmatrix}

where $\mathbf{M}^k \in \mathbb{R}^{N_k \times d}$ denotes the representations of the $N_k$ samples in class $k$. In each block $\mathbf{M}^k$, diversity components encourage the representations to be dissimilar across views and drive $\mathbf{M}^k$ to be high-rank, which indicates that diversity components tend to push $\mathbf{M}^k$ into a high-rank space since the diversity components cannot be effectively represented in a low-rank space. In contrast, consistency components encourage the representations to be similar across views within each block $\mathbf{M}^k$ and can be effectively represented in a low-rank space. Therefore, by appropriately lowering the rank of the basis space $\mathbf{B}$, the diversity components can be suppressed, enabling $\mathbf{B}$ to capture cross-view consistency more effectively.

---

************ For Question #2: ************

In multi-view datasets, lower-dimensional views often provide a more limited and fragmented description of the samples, and the representations extracted from these views usually exhibit a biased or unstable global center. Consequently, these views are less suitable as the central view for cross-view global fusion. Instead, the view with the largest original feature dimension provides a more comprehensive description of the samples, promoting the extracted representations with a more stable global center. Therefore, the view with the largest original feature dimension is more suitable as the central view for cross-view global fusion.

---

************ For Question #3: ************

There are two differences between the structurally equivalent basis vectors and the normal basis vectors. (1) Intrinsic property: Structurally equivalent basis vectors can be interchanged without changing the graph structure [1], while normal basis vectors cannot. Specifically, consider a graph $G^1=(\mathbf{Z}^1, \mathbf{S}^1)$, where $\mathbf{Z}^1$ denotes the vertices (i.e., basis vectors) and $\mathbf{S}^1$ denotes the graph structure. Structurally equivalent basis vectors $(\mathbf{z}_i^1, \mathbf{z}_j^1)$ can be interchanged in $\mathbf{Z}^1$ without changing $\mathbf{S}^1$, while normal basis vectors cannot [1]. (2) Influence on structure matching: When matching $G^1$ with another graph $G^2=(\mathbf{Z}^2, \mathbf{S}^2)$ using a structure-based matching function $\varPhi(\mathbf{S}^1, \mathbf{S}^2)$, multiple matching results $\\{\mathbf{Z}^1 \xrightarrow{m} \mathbf{Z}^2\\}$ arise by interchanging structurally equivalent $(\mathbf{z}_i^1, \mathbf{z}_j^1)$ without changing the value of $\varPhi(\mathbf{S}^1, \mathbf{S}^2)$. In this case, structurally equivalent basis vectors hinder the ideal and unique matching between $\mathbf{Z}^1$ and $\mathbf{Z}^2$, whereas normal basis vectors do not have this influence.

[1] Yang, Dominic, Yurun Ge, Thien Nguyen, Denali Molitor, Jacob D. Moorman, and Andrea L. Bertozzi. "Structural equivalence in subgraph matching." IEEE Transactions on Network Science and Engineering 10, no. 4 (2023): 1846-1862.