Griffin: Towards a Graph-Centric Relational Database Foundation Model

We thank Reviewer 95m6 for acknowledging the contributions of our work and for the helpful suggestions and questions. We address each concern below. The updated experiments including 7 additional baselines about main experiment and 2 additional baselines on few-shot setting is provided at https://anonymous.4open.science/r/Griffin-Rebuttal

---

Related Work Thank you for the recommended references. In response to your suggestions, along with those from other reviewers, we have revised the related work section to include a broader range of relevant literature. We have added new subsections covering embedding-based RDB models and Table QA approaches, as well as additional works on RDB-specific models and tabular foundation models.

To better reflect the landscape and clarify our positioning, we now organize the related work into the following categories: • Single-table and RDB models for tabular predictive tasks • Table QA and SQL generation methods • Tabular and graph foundation models, including both LLM-based models and those trained from scratch

---

W1: Experimental Coverage and Baselines

Thank you for your concern regarding the experimental baselines. We have significantly extended our experiments by including the following additional baseline categories:

1. Other GNN-based models for RDBs, including GAT, PNA, and HGT.

2. Widely used single-table methods, such as XGBoost, MLP, DeepFM, and Feature Transformer, applied to RDBs via Deep Feature Synthesis (DFS) to flatten the schema.

3. Tabular foundation models, including TabPFN and TabLLM, also used with DFS.

These baselines are now incorporated into the experimental section, reinforcing its effectiveness across RDB-related tasks.

---

W2 & Q1: Design Choices and Ablation Studies

Thank you for raising questions about the justification of specific design choices. We have now addressed these through targeted ablation studies and reorganized the section to improve readability. Specifically:

• Cross-Attention vs. Mean Pooling: We compare our cross-attention mechanism with simple mean-pooling. The results show improvements, supporting the benefit of selective attention over equal-weighted aggregation.

• Max vs. Mean Aggregation: We evaluate different aggregation methods over heterogeneous relations. Max aggregation helps highlight the most informative relation types, and the ablation supports this design choice.

• Numeric Encoder/Decoder Pretraining: Our design choice is based on the goal of enabling transferability across tasks. To ensure that the model outputs remain in a consistent numerical space, we use a fixed numeric encoder and decoder during training. Since our primary focus is not on improving single-task performance but rather on transferability, we did not include ablation studies on single-task performance for this component. That said, we recognize the importance of this question and are planning a more comprehensive ablation study in future work—starting from re-pretraining the model to investigate the impact of this design on cross-task generalization.

---

Q2 & Q3: Handling Missing Values

We appreciate the questions on how missing values are handled, both in the target node and in neighboring nodes:

• For the target node’s missing column (label): We mask it using a zero placeholder during pretraining.

• For other missing values:

• Categorical/Text features are replaced with a "None" token.

• Numeric features are replaced with the column-wise mean.

This strategy generalizes to cases where multiple features or nodes have missing values, and aligns with common practices in both tabular learning and graph-based models. We have clarified this in the methodology section.

We thank Reviewer XrAk for the thoughtful and constructive feedback. We address your concerns in detail below. The updated experiments including 7 additional baselines about main experiment and 2 additional baselines on few-shot setting is provided at https://anonymous.4open.science/r/Griffin-Rebuttal

---

Claim 1: Clarifying the Challenges in Developing Foundation Models for RDBs

Thank you for pointing this out. We agree that the challenges could be stated more clearly. We now explicitly outline the key challenges in developing a foundation model for relational databases:

1. Data and task diversity: RDBs contain various data types (numerical, categorical, textual) and support diverse tasks, including regression and multi-class classification. A unified model must handle all these variations under a single framework.

2. Lack of RDB-specific GNNs: Existing GNN architectures are not tailored for relational databases and often fail to leverage the rich metadata (e.g., table schema, column descriptions, task semantics) that is critical in RDBs.

3. Missing pretraining pipeline and transferability analysis: It is not yet clear how pretraining can benefit relational data, nor what kinds of tasks or data lead to effective transferability across RDBs.

These challenges directly motivate the three components of Griffin: unified encoder/decoder design, GNN architecture enhancements, and a new pretraining pipeline. We have now added a dedicated subsection to the paper to clearly articulate these challenges and align them with our proposed solutions.

---

Experiment: Including More Baselines & W3: Ablations and Experimental Focus

Thank you for raising this point. While our primary goal is to present a unified foundation model framework for RDBs, we agree that including broader baselines and detailed ablation studies is essential for a comprehensive evaluation.

To that end, we have expanded our experimental comparisons to include three additional categories of baselines:

• Three additional GNN-based baselines: GAT, PNA, and HGT, as suggested in 4DBInfer.

• Four single-table baselines: MLP, DeepFM, Feature Transformer, and XGBoost, along with Deep Feature Synthesis (DFS)—a strong feature synthesis method that converts multi-table data into a single table by computing meaningful feature combinations.

• Two single-table baselines for few-shot settings: We include TabPFN and TabLLM as representative few-shot baselines, both of which leverage pretrained models. TabPFN is trained from scratch, while TabLLM uses LLM-based pretraining.

We have also reorganized the experimental section to make the ablation studies more prominent. These ablations highlight the contributions of key architectural choices—specifically, the cross-attention mechanism and hierarchical aggregation—to the overall performance of the model.

---

W1: Organization and Presentation

Thanks for your suggestions. We have revised the introduction to better connect the discussion of challenges with our method overview.

---

W2 & Q1: Justification for RDB-Specific Foundation Models

We appreciate this important question. While single-table models can be combined with DFS to simulate multi-table input, they inherently lack the ability to capture the complex relational structure that is central to RDBs. This limitation has been highlighted in prior benchmarks such as 4DBInfer, where structure-aware models (e.g., GNN-based) consistently outperform single-table methods with DFS.

To further support this, we have added experimental comparisons with strong single-table baselines, including XGBoost, DeepFM, TabPFN, and TabLLM (all paired with DFS). The results reinforce the need for RDB-specific model designs like Griffin.

---

Comments and Q2: Handling Temporal Data

Thank you for this valuable observation. Our approach to handling temporal information follows the setup used in 4DBInfer, where subgraphs are constructed such that all neighboring nodes have timestamps earlier than the target node. This ensures that the model only uses past information, aligning with the temporal nature of real-world relational data.

To improve clarity, we have expanded the explanation in the preliminaries and reorganized the method section to better highlight how temporal constraints are applied during subgraph construction.

Additionally, we have revised the figures and included more baselines to improve presentation.

We appreciate your feedback on presentation—it has significantly improved the clarity and accessibility of our results.

We thank Reviewer yHkP for acknowledging the contributions of our work and for the constructive questions and suggestions. The updated experiments including 7 additional baselines about main experiment and 2 additional baselines on few-shot setting is provided at https://anonymous.4open.science/r/Griffin-Rebuttal. We address the key concerns below:

---

Q1: Performance Comparison with Single-Table Baselines

Griffin is primarily designed for RDB tasks and is not optimized to outperform state-of-the-art models on purely single-table datasets. However, to better understand its generalizability—and in response to suggestions from other reviewers—we conducted two additional sets of experiments:

1. DFS + Single-Table Models on RDB Tasks:

We employed Deep Feature Synthesis (DFS) [1], which converts multi-table RDBs into single-table formats by aggregating features. This allows single-table models to be applied to RDB tasks. Building on pipelines from 4DBInfer, we evaluated advanced single-table baselines (including MLP, DeepFM, Feature Transformer, and XGBoost) on RelBench using DFS. Results indicate that even strong single-table models struggle to match Griffin’s performance, particularly on tasks where the multi-table structure plays a critical role.

2. Few-Shot Settings with Pretrained Single-Table Models:

To evaluate performance in few-shot scenarios, we introduced two representative baselines: TabPFN (trained from scratch) and TabLLM (leveraging LLM-based pretraining). Both are designed for few-shot tabular tasks. Griffin still achieves higher average performance under this setting, demonstrating its generalization ability and potential as a foundation model for RDBs.

[1] Deep Feature Synthesis: Towards Automating Data Science Endeavors https://groups.csail.mit.edu/EVO-DesignOpt/groupWebSite/uploads/Site/DSAA_DSM_2015.pdf

---

Q2: Performance Trends with Varying Numbers of Tables

Thank you for the insightful question. Griffin’s framework integrates innovations across encoder/decoder design, GNN architecture, and pretraining strategies—each contributing to improved performance under different structural complexities. Our ablation studies isolate the effect of these components, including cross-attention mechanisms and hierarchical aggregation functions.

Regarding the trend with varying numbers of tables, we do not yet have a fully consistent explanation for all scenarios where Griffin outperforms simpler models. One intuitive observation is that Griffin performs especially well in tasks that involve high-quality textual features—for example, predicting review ratings in e-commerce datasets where review content is informative.

That said, there are still cases where Griffin and simpler models (e.g., SAGE) exhibit unexpected performance patterns. Interestingly, even more expressive GNN models such as GAT, PNA, and HGT occasionally perform worse than SAGE. We believe this may stem from data-specific factors, including noisy features, varying relation types, or mismatched inductive biases—cases where simpler architectures might generalize better.

We acknowledge this as an open research question and welcome further discussion on better understanding the relationship between RDB schema complexity and model performance.

We thank Reviewer w5kU for the careful reading and detailed feedback. We address the concerns and questions below. The updated experiments including 7 additional baselines about main experiment and 2 additional baselines on few-shot setting is provided at https://anonymous.4open.science/r/Griffin-Rebuttal

---

Related Work

Thank you for the suggestions. We have included those references and also in comparison. The updated can be refered to reply to Reviewer 95m6.

---

C1: What does Griffin stand for?

Griffin stands for towards a Graph-centric RelatIonal dataset FoundatIoN model.

---

C2: Clarifying “Foundation Model” Usage

We appreciate your point. While Griffin is trained on specific datasets, we consider it to follow the general trajectory of foundation models [1] —namely, training on broad data for wide applicability. That said, we agree the claim should be stated with more care. We now consistently refer to Griffin as “towards a foundation model” or as“ a foundation model attemptation”. All inconsistent uses have been revised accordingly.

[1] On the Opportunities and Risks of Foundation Models

---

C3: Clarifying Data Curation Efforts

Thank you for raising this. The data curation involved collecting, filtering, and improving dataset quality. We started with public datasets from TPBERTa, TabLLM, UniTabE, TableGPT, TaBERT, TabNet, CARTE, and others from HuggingFace. However, many were unsuitable—some tables had very few rows, others were derived from RDBs but had lost context when flattened, and some were noisy or irrelevant (e.g., meaningless string columns without clear semantics).

We ultimately curated ~200 datasets with over 10M rows. These were selected for having rich metadata and column features not heavily preprocessed (e.g., not just hashed values). We also generated natural language task descriptions using GPT4 to improve the training signals for tasks.

---

C4 & C13: Ambiguous Adjectives

We appreciate this feedback and have revised the paper to remove or clarify ambiguous adjectives. Specifically:

• The word “seamless” was used to describe the unified modeling of both single-table and RDBs as graphs using a consistent node structure.

• The term “stable” referred to the aggregation mechanism in our heterogeneous GNN, where neighbors are first aggregated within each relation type, followed by max aggregation across types to focus on the most relevant relations. Ablations also verify the effectiveness.

For them, we have now replaced the adjective with a direct explanation of the method.

---

C5 & C6: Clarifying Section 3.1 and Decoder Design

We revised Section 3.1 to clarify the notion of “Task Unification”. Previously, it refers not only to regression/classification unification but also to input unification. Now we split the subsection into two subparts: one describing encoder design and one describing decoder design. For classification tasks, the decoder retrieves the most similar label embedding. These embeddings are aligned with both category and textual feature encodings. For regression tasks, a numerical decoder predicts normalized values.

---

C7 & C8: Numerical Feature Encoder

The ENC module is a 3-layer MLP with SiLU activations (replacing ReLU) and layer normalization. Our current pretraining task about ENC and DEC focuses on accurate numerical recovery, for which this design is sufficient. We included this paper in related work and leave further exploration on numbers to future work.

---

C9: Masking Strategy for Completion Task

We use a simple zero-filling strategy, consistent with the settings used in 4DBInfer and RelBench. Zero is also used as the placeholder during prediction.

---

C10–C11 & C14–C17: Pretraining Details

Thank you for your careful reading. We now provide a detailed breakdown of the training regimes used in our experiments:

To improve clarity in the presentation of our experiments, we have renamed the models as follows:

• Griffin-unpretrained: trained from scratch, no pretraining

• Griffin-pretrained: pretrained on single-table data (completion + joint SFT)

• Griffin-RDB-SFT: further pretrained with RDB-based joint SFT

---

C12: DFS + X Baselines

We have added these baselines in our updated experiments. A summary of the updated experimental setup is provided in our response to Reviewer zrC4 under Evaluation Concerns.

We thank reviewer zrC4 for the detailed and constructive feedback. Below, we address each of the main concerns. The updated experiments including 7 additional baselines about main experiment and 2 additional baselines on few-shot setting is provided at https://anonymous.4open.science/r/Griffin-Rebuttal

---

Claim 1: Novelty of the Proposed Approach

We appreciate the reviewer’s concern about the novelty of our approach and would like to clarify our contributions. Our goal is to take a step toward building a foundation model RDBs. We focus on three key challenges:

1. How to represent different RDBs in a unified model?

2. How to design a GNN architecture that works effectively for RDBs?

3. How to leverage abundant data, and when does it help?

To address these challenges, we propose Griffin, a unified framework with three main components: (1) encoder/decoder design, (2) GNN architecture updates, and (3) a pretraining pipeline. Then our experiments show consistent improvement through three stages:

1. Our base model (even without pretraining) outperforms prior models (initially compared only with SAGE; we have now added 7 more baselines).

2. Pretraining on single-table data further boosts performance.

3. Additional SFT with similar or diverse RDBs enhances performance, especially in low-resource settings.

The strength of Griffin lies in the integration of these three components under a unified framework aimed at RDB foundation modeling. Each component plays a distinct and necessary role: enabling diverse data usage, improving learning capacity, and supporting practical deployment. Our experiments confirm that each part contributes meaningfully to both performance and transferability.

---

Claim 2: Discussion of Related Work

We thank the reviewer for highlighting missing related work. We have now revised the related work section to include a wider range of relevant literature. The updated can be refered to reply to Reviewer 95m6.

---

Evaluation Concern: Missing Baselines and Alignment with RelBench

Thank you for raising this important point. While we initially prioritized “fair” comparisons, we agree that broader baseline coverage is necessary for a more complete evaluation. In response, we have expanded our baseline set to include:

• Three additional GNN-based baselines: GAT, PNA, and HGT, as suggested in 4DBInfer.

• Four single-table baselines: MLP, DeepFM, Feature Transformer, and XGBoost, along with Deep Feature Synthesis (DFS)—a strong feature synthesis method that converts multi-table data into a single table by computing meaningful feature combinations.

• Two single-table baselines for few-shot settings: We include TabPFN and TabLLM as representative few-shot baselines, both of which leverage pretrained models. TabPFN is trained from scratch, while TabLLM uses LLM-based pretraining. Although DFS can be computationally intensive (taking up to 7 hours for optimized pipelines in 4DBInfer and even longer using the original Featuretools implementation), making it less suitable for low-resource few-shot settings, we still include these baselines as a reference.

For evaluation, we continue to present normalized scores in the main figures to support comparison across tasks with varying scales. To ensure transparency and enable alignment with RelBench, we also provide some unnormalized results which is aligned with normalized scores, both suggesting Griffin outperforms Sage.

---

W1: Method Description and Figures

We thank the reviewer for this helpful feedback. We have made several improvements including Clarified terminology and Improved explanation of subgraph sampling and method pipeline.

---

W2: Explanation of Results

Thank you for the suggestion to provide more explanation of the results. In general, Griffin performs better when the task benefits from rich table metadata and high-quality text feature embeddings.

However, there are still cases where it is difficult to consistently explain why Griffin outperforms SAGE, or vice versa. This challenge also applies to other strong GNN baselines such as GAT, PNA, and HGT. Despite being more expressive in theory, these models sometimes perform significantly worse than SAGE. We believe this may be due to the complexity and variability in data distributions, where simpler models like SAGE may align better with the task’s inductive bias in certain cases.

We acknowledge this as an open question and an area for future work. We welcome further insights and suggestions on how to better understand and interpret these variations.