Towards Multi-Table Learning: A Novel Paradigm for Complementarity Quantification and Integration

We would like to thank the reviewer for the careful reading and constructive suggestions. We respond to each point below.

W1: Interpretability of CS

• There is no guidance for choosing the two hyperparameters, $\alpha$ and $\gamma$. When should practitioners adjust them, and based on what criteria?
• The paper does not state the full range of $CS_t(T^k,T^l)$ and $CS_g(T^k,D \ T^k)$ , nor explain how to interpret specific values (e.g., what constitutes a "high" vs. "low" score). A discussion or examples would greatly improve clarity.

(a) How should α and γ be adjusted?

Complementarity Strength (CS) is the first metric explicitly designed to quantify how much new, non-redundant information auxiliary tables contributes to a target table. The score $CS_t(T^k,T^l)$ increases with the informativeness $\mathrm{Info}(T^l)$ and relevance $\mathrm{R_t}(T^k, T^l)$, and decreases with similarity $\mathrm{S_t}(T^k, T^l)$ in a convex manner.

Since $CS$ is computed based on pretrained column-level embeddings, the numerical distributions of relevance and similarity may vary significantly across different embedding models. To enhance the cross-task generalizability and ranking resolution of $CS$, we introduce the parameters α and γ as tunable controls:

• α adjusts the overall magnitude of the CS scores；
• γ controls the strength of non-linear penalization for similarity, thus shaping how rapidly $CS$ decays with increasing redundancy.

Practitioners may adjust these parameters based on two considerations:

• The observed range of $R_t/S_t$ produced by the pretrained embeddings;
• The desired sensitivity to redundancy, which may vary depending on application-specific tolerance for overlapping information.

In practice, we suggest fix $\alpha=1$ and adjust $\gamma\in[0.5,2]$ to adjust the sharpness of $CS$ decay. In our experiments, we adopt the pretrained embedding model from [1] and simply set α = γ = 1, which yields stable and discriminative $CS$ rankings across tables.

(b) What is the value range of CS and how should specific values be interpreted?

We appreciate the reviewer’s request for clarification on the value range and interpretation of the $CS$ score.

Range of CS. As defined in Eq.(5), $\mathrm{CS_t}(T^k, T^l)\in [0, N_l]$，$\mathrm{CS_g}(T^k,D \setminus T^k) \in \left[0,\sum\nolimits_{k=1}^K N_k\right]$, where $N_l$ denotes the number of columns in the auxiliary table $T^l$. This upper bound naturally reflects that tables with more columns can encode richer information and offer greater potential complementarity.

Interpretation of the $CS$ score. While the theoretical value range of $CS_t(T^k, T^l)$ lies in $[0, N_l]$, in practice, the effective scores tend to be much lower due to the combined effects of three scaling factors: $\mathrm{Info}(T^l) \in [0, N_l]$, $\mathrm{R_t}(T^k, T^l) \in [0, 1]$, and the term $1 - \frac{1}{N_l} \mathrm{S_t}(T^k, T^l) \in [0, 1]$.

When all components are moderately valued (e.g., $\mathrm{Info}(T^l) \approx 0.5 N_l$, $\mathrm{R_t}(T^k, T^l) \approx 0.5$, $1 - \frac{1}{N_l} \mathrm{S_t}(T^k, T^l) \approx 0.5$), the resulting score approximates $0.125 \times N_l$. Based on extensive empirical observations, we therefore suggest the following interpretation:

• $CS_t(T^k, T^l) > 0.125 \times N_l$: strong complementarity;
• $CS_t(T^k, T^l) < 0.125 \times N_l$: weak or marginal complementarity.

This threshold is validated in Table 1 of the main paper, where each candidate table contains 10 columns and a score of $CS_t > 1.25$ effectively identifies complementary tables. Further confirmation is provided by the case studies reported in Tables 6 and 7 in the appendix. We will include this interpretation and threshold discussion in the camera-ready version.

[1] Semantics-aware dataset discovery from data lakes with contextualized column-based representation learning.

W2: Pretraining Strategy

While the two multi-table pretraining strategies, multi-table reconstruction and multi-table reconstruction, are somewhat interesting, they are incremental adaptations from previous studies. Also, the authors introducing two additional hyper-parameters with the pretraining loss (Equation 9), $\lambda_{\text{rec}}$ and $\lambda_{\text{cor}}$, without any ablation or sensitivity analysis makes it hard to understand their impact.

We thank the reviewer for raising concerns about the novelty and justification of the pretraining strategies and associated hyper-parameters.

To best of our known, this is the first learning paradigm that directly learns from multiple tables — introducing the first architecture, ATCA-Net, to integrate the complementarity. The joint pretraining strategy—including multi-table reconstruction and cross-table correlation prediction—is specifically designed to support the architecture for multi-table learning, which have not been explored before. It allows the encoder to map heterogeneous tables into a unified space and model cross-table attribute and entity dependencies.

To further clarify the role of the two loss components in table alignment, we will included an ablation study in the revised version. We compare five ATCA-Net variants to assess the effect of multi-table training and each loss component:

• (S): Trained and tested on single tables only, without any fusion or pretraining.
• (P): Pretrained on individual tables, but no cross-table fusion.
• ($M$): Full multi-table model with both reconstruction and correlation objectives.
• ($M_{\lambda_{\text{cor}}=0}$): Removes correlation loss.
• ($M_{\lambda_{\text{rec}}=0}$): Removes reconstruction loss.

Table: Ablation Results

As shown in the table above, removing either objective leads to a significant performance drop, and in most cases, the model underperforms even the single-table pretraining baseline ATCA-Net (P). This confirms that both losses are essential: the representations learned from their joint training are critical for the subsequent complementarity-based fusion, and neither component can be omitted.

W3. Limited evaluation scope:

The experiments only cover table‑classification tasks. Evaluating on other downstream applications would better demonstrate the versatility and practical value of the approach.

We appreciate the reviewer’s suggestion. As the first work to define a multi-table learning paradigm—introducing the first metric (CS) to quantify complementarity and the first architecture (ATCA-Net) to integrate it—we focus on classification tasks to validate the approach. These experiments demonstrate the effectiveness of both ATCA-Net and the proposed complementarity metric. We also agree that broader downstream tasks would better showcase the generality of our method. In future work, we plan to extend this paradigm to regression and table-based question answering.

We sincerely thank the reviewer for the constructive and detailed feedback. We are encouraged by your recognition of our contributions to formalizing the multi-table learning paradigm, designing ATCA-Net, and introducing the CS metric. We respond to the key concerns below:

Q1. Necessity of the two loss components

The paper designs two types of losses: multi-table reconstruction loss and cross-table correlation prediction loss. Additional experiments and analyses are needed to specifically determine the roles and necessity of these two losses.

We thank the reviewer for highlighting the importance of understanding the role of each loss component. The two proposed loss objectives are central to enabling the encoder to embed heterogeneous tables into a unified representation space while capturing latent inter-table dependencies:

• The reconstruction loss encourages the encoder to capture attribute-level semantics within each table;
• The correlation prediction loss models the correlation between entities across different tables.

To validate their necessity, we conducted an ablation study across five variants of ATCA-Net: We conducted an ablation study comparing five variants of ATCA-Net:

• (S): Trained/tested on single tables (without pretraining or fusion);
• (P): Pretrained/Trained/tested on single table;
• ($M_{\lambda_{\text{cor}}=0}$): Trained/tested on multi tables (without correlation loss);
• ($M_{\lambda_{\text{rec}}=0}$): Trained/tested on multi tables (without reconstruction loss);
• ($M$): Full ATCA-Net with both losses.

The results (see Table below) show that removing either loss leads to consistent performance degradation, with most variants falling below even the single-table pretrained baseline (ATCA-Net (P)). This strongly indicates that Only their joint optimization enables effective complementarity-based fusion.

---

We also acknowledge the reviewer’s concern regarding the hyperparameters (e.g., α, γ) in the CS metric. While the current version adopts fixed values $\alpha=\gamma=1$, we agree that their interpretability and effect deserve further analysis. We have addressed this point in detail in our response to Reviewer MnRt (W1), including usage guidance, empirical ranges, and score interpretation. We will incorporate this discussion into the camera-ready version.

Thank you again for your thoughtful feedback and for recognizing the contributions of our work. Your comments have helped us improve the paper.

We sincerely thank the reviewer for the detailed and thoughtful feedback. We are particularly encouraged by your recognition of our contributions to formalizing the multi-table learning paradigm, motivating the framework with real-world scenarios, and validating it through controlled empirical studies. We respond to your key concerns below:

---

Q1. Generalization of pretraining

Have the authors tested whether the encoder trained on a set of tables A can transfer to a new, unseen table B from the same or different domain? Can CS still be computed in this case? A simple cross-group transfer experiment would significantly strengthen the paper's claims about generality.

We appreciate this important question. The effectiveness of computing CS scores on unseen tables fundamentally depends on the generalization capability of the column embedding model used to extract semantic representations. In our work, we adopt a pretrained column encoder from [1], which is trained on a large and diverse corpus of real-world tables. Importantly, as shown in our experiments (Table 1, Table 6, Table 7), the datasets used for evaluation (e.g., Covertype, Airline, BlastChar) are not part of the pretraining corpus of the column encoder. This empirical evidence supports the transferability of CS: the metric remains effective even when applied to previously unseen tables from different domains, without requiring retraining. We will clarify this generalization capability in the final version.

---

Q2. Clarify the encoder architecture

Please specify the architecture of the adaptive table encoder in detail. Is it a 2D Transformer trained from scratch? What pretrained model is used for column/cell embedding? How are categorical and numerical fields embedded in practice?

Thank you for raising this important point. We clarify the architecture and embedding design as follows, and will include these details in the camera-ready version.

(a) What pretrained model is used for column/cell embedding? How are categorical and numerical fields embedded?

To compute column-level embeddings for the CS score, we adopt the pretrained column encoder from [1], which was trained on a large corpus of web and enterprise tables. Within ATCA-Net, we further incorporate cell-level embeddings based on a BERT model:

• For numerical cells $c_{\text{num}}$, we first embed the corresponding column header $h_{\text{cat}}$ using BERT and apply average pooling to obtain a 768-dimensional embedding $e^h_{\text{num}}$. This is multiplied with the normalized cell value to yield the numerical cell embedding.
• For categorical cells $c_{\text{cat}}$, we concatenate the cell value and its header as "c_{cat} + '_' + h_{cat}", and input this string into BERT. The output is pooled into a 768-d embedding.

To reduce computation, we project the 768-d cell embeddings into 192 dimensions using a learned linear transformation before passing them to the ATCA-Net encoder.

(b) Is it a 2D Transformer trained from scratch?

Yes. The adaptive table encoder in ATCA-Net consists of two stacked 2D-Transformer blocks, each composed of one row-wise attention and one column-wise attention layer. This encoder is trained from scratch during the multi-table training process. We also add a row-level attention layer after the 2D blocks to further enhance entity-level alignment across tables.

We will provide an architectural diagram and implementation details in the camera-ready version for clarity and reproducibility.

---

Q3. Interpretability of attention

Could the authors provide attention heatmaps or illustrative examples showing which auxiliary rows/columns are attended to for a given query row? This would help establish that the model is indeed leveraging “complementary” information and not just aggregating noise.

We appreciate the reviewer’s interest in the interpretability of our cross-table attention mechanism. To investigate whether ATCA-Net truly attends to complementary rather than noisy information, we conducted an additional analysis on a synthetic benchmark based on the Invistico Airline dataset, where the tables are constructed by randomly sampling columns and rows. In each trial, we randomly sample one query row from the target table and 60 rows from an auxiliary table, and analysis the cross-table attention distribution over these 60 candidates.

In more than 75% of the iterations, the highest attention weights are assigned to rows from the same semantic class (e.g., same label) as the query row. This strongly suggests that the attention mechanism is not uniformly aggregating, but rather selectively focusing on informative and semantically aligned auxiliary instances.

Due to rebuttal format constraints, we are unable to include attention heatmaps here. However, we will incorporate full visualizations and case studies in the camera-ready version to further enhance the interpretability of our method.

---

Q4. Robustness to noisy tables

Have you tested performance when auxiliary tables are semantically unrelated or low-CS? Does the model degrade gracefully? This would help assess whether CS could serve as a practical filter before fusion.

Thank you for the insightful question. ATCA-Net is designed to handle noisy or low-quality auxiliary tables gracefully. Its Transformer-based fusion module $\mathcal{U}_\phi$ automatically downweights auxiliary tables that lack complementary information. When auxiliary tables offer little relevance or no complementarity, the model’s predictions remain close to those made using the target table alone, demonstrating strong robustness to irrelevant tables.

Moreover, the proposed $CS$ metric serves as a practical pre-fusion filter, effectively identifying and excluding low-complementarity tables. We will include a more detailed discussion of this property in the revised version.

Q5. Performance vs number of tables

Do more tables always help? Could the authors report AUC or accuracy vs number of integrated tables on one dataset to assess marginal utility?

Thank you for the thoughtful question. We clarify that more tables do not always yield better performance. What matters is the complementarity of the tables, not their count. In our experiments, we observed diminishing returns beyond 3–4 tables, as the attention module focuses on those with the highest Complementarity Strength (CS).

Thus we conducted a controlled experiment using a synthetic benchmark based on the BlasterChar dataset, where we fixed the number of auxiliary tables and varied their Complementarity Strength (CS). Results show that higher CS leads to consistently better AUC.

Table: AUC Performance under Varying Complementarity Strength (CS)

---

Once again, we thank the reviewer for the insightful feedback. Your comments helped us strengthen the technical clarity, empirical robustness, and practical relevance of our work. We look forward to incorporating these improvements into the final version.

[1] Semantics-aware dataset discovery from data lakes with contextualized column-based representation learning. VLDB 2023