Joint Relational Database Generation via Graph-Conditional Diffusion Models

We thank the reviewer for their valuable feedback. In the following, we address the reviewer's concerns in detail. We will include all the new discussions in the camera-ready version.

---

[W1, Q1] Motivation and real-world applications

In the following, we provide concrete real-world applications of the problem of synthetic relational database generation. Note that good generative models for RDBs are a prerequisite for all the following tasks.

• Privacy-preserving research and development. Real relational databases typically contain sensitive information (e.g., personal data, financial records). Sharing them with third parties (e.g., LLM providers) or using them for product development can raise serious legal issues, e.g., on the basis of the European Union's General Data Protection Regulation (GDPR). Synthetic relational databases can be used to develop new features or valuable insights, without exposure of real user data. As a concrete example, the SyntheticMass project created a synthetic population of Massachusetts residents (based on EHR schemas) to support research and development without patient risk. For an evaluation of our model's privacy preservation capacity, we refer to our response to Reviewer s1Pb ([W1, W2, Q2] Privacy preservation and generalization).
• Data augmentation for machine learning models. Many real-world machine learning tasks leverage relational databases (e.g., fraud detection, recommendation systems). Synthetic RDBs can help augment underrepresented groups to improve the fairness of the downstream models or simulate rare events to improve their robustness.
• Training foundation models on structured data. There have been recent efforts to build foundation models for structured data [1,2]. However, training such models requires huge amounts of relational databases, which can be accelerated using generative models for RDBs.
• Adapting generative models for predictive tasks. A pre-trained diffusion model for RDBs can be easily adapted to perform conditional generation without the need for re-training [3]. This allows us to leverage a single model for different downstream prediction tasks, such as regression or classification, without the need for training different models for each task, thereby saving cost and time. For concrete results on how our model can be adapted to such cases, we refer to our response to Reviewer gCht ([Q1] Missing and incomplete tables at test time)

We will update the motivation section in the camera-ready version to include these real-world applications.

---

[W2, Q4] Many-to-many relationships

We confirm that many-to-many relationships were considered in our work. In fact, 4 out of the 6 used databases contain this type of relationship (Instacart 05, RelBench-F1, MovieLens, and CCS). As pointed out by the reviewer, relational databases cannot directly implement many-to-many relationships due to the fixed number of foreign keys a table can have. To overcome this, a so-called join table is typically introduced, which acts as an intermediary, storing the relationships between the two connected tables, essentially breaking down the many-to-many relationship into two one-to-many relationships. Examples of this type of relationship from our databases include ratings (from MovieLens), which connect multiple users with multiple movies, and transactions (from CCS), which connect multiple customers with multiple products. Since we represent databases as graphs, this implementation offers a unique advantage: the resulting graphs are simple (no multi-edges) and all the attributes are stored as node features instead of edge features (e.g., rating scores), allowing us to use standard GNN architectures out-of-the-box.

We refer to Table 1 in the paper for results on these databases. We will clarify this point in the camera-ready version.

---

[W3] Limitations

We kindly point to the last paragraph in the paper (right after the conclusion) for a detailed discussion on the limitations of our approach and promising future research directions.

---

[Q2] Augmenting real datasets

Real datasets can be biased towards a specific population or only model common events. In such cases, using generative models to augment real datasets can improve the fairness of the dataset and improve downstream performance on rare edge cases (if, e.g., used to train downstream machine learning models). As a concrete example, in the context of healthcare research, synthetic data can be generated to simulate rare diseases. Or in the case of recommendation systems, synthetic data can be used to improve the performance of downstream models on underrepresented groups.

---

[Q3] Multi-modal features

In this work, we focus on categorical and numerical data types, which cover different data formats, such as ordinal data, enums, boolean, datetime, and any discrete or continuous data. For high-dimensional data such as images, audio, or text, a natural idea would be to model low-dimensional latent representations for these data types as numerical features and directly apply our framework to generate them. Then, a suitable decoder can be applied to map back to the respective original data space.

---

References

[1] Wang, Yanbo, et al. Griffin: Towards a Graph-Centric Relational Database Foundation Model.

[2] Fey, Matthias, et al. KumoRFM: A Foundation Model for In-Context Learning on Relational Data.

[3] Lugmayr, Andreas, et al. Repaint: Inpainting using denoising diffusion probabilistic models.

---

We hope our response can address the reviewer's concerns, and we are happy to answer any further questions during the discussion period.

We thank the reviewer for their valuable feedback. In the following, we address the reviewer's concerns in detail. We will include all the new experiments and discussions in the camera-ready version.

---

[W1, Q1] Richness of Graph Generation

The cardinality metric reported in the paper compares the node indegree distributions between the real and synthetic databases. As per the reviewer's request, we now also compare higher-order structural properties, such as the correlation between node indegrees from different edge types (Correlations), e.g., the correlation between the number of actors in a movie and the number of ratings it received. In addition, we report inter-table indegree correlations (1-Hop, 2-Hop), e.g., the correlation between the number of stores in a specific region and the number of purchases these stores get.

The results show that, even though our graph generation algorithm only models first-order indegree distributions, it achieves good higher-order performance. This suggests that the used databases do not exhibit strong correlations between different types of relationships and that these relationships are, to a large extent, independent, which our approach can capture very well (which was the main motivation behind our choice in the early stages of this project). However, more complex approaches may be required for more complex databases, which is an interesting research direction, albeit orthogonal to the main focus of this work, namely the joint modeling of tables in RDBs.

Higher-order structural properties. Cardinality compares first-order node indegree distributions, Correlations compares the correlation between node indegrees from different edge types, and 1-Hop, 2-Hop compare inter-table indegree correlations:

To further evaluate whether our graph generation approach is sufficient to capture the complexity of the RDBs, we perform a new ablation by replacing the sampled graph with the ground-truth graph and using the same diffusion model to generate the attributes. The ground-truth graph achieves a perfect relational structure fidelity (cf. Cardinality metric) and provides an upper-bound on the performance achievable by graph generative models. The results show that our model achieves very similar performance to the ground-truth graph setting, showing that our graph generation procedure can effectively capture the relational structure of the RDBs.

---

[W2, Q2] Scalability to larger databases

The databases used in our evaluation cover a wide range of sizes. The number of tables ranges from 2 to 9, the number of columns from 11 to 41, and the total number of rows goes up to 2 million (see Table 3 in the paper for details). To evaluate the scalability of our model beyond the setting presented in the paper, we leverage the capability of our graph generation algorithm to generate graphs of arbitrary sizes. We use different multipliers of the original database size and generate databases with up to more than 20 million rows (10x multiplier) using the same diffusion model. The table below shows that our model maintains its good performance across different scales and that the runtime scales roughly linearly with the database size, generating the largest database (> 20 million rows) in less than 2 hours on a single GPU. Both these results can be explained by the use of our GNN in the denoising model, which operates on local subgraphs that are roughly invariant to the overall size of the RDB. Note that we used the same batch size for all scales, but the generation can be even faster with larger batch sizes, as the denoising of all rows is performed in parallel at each denoising step.

---

[W3, Q3] Handling Hard Constraints

In our framework, the database constraints are enforced through our graph-based representation of the RDB. Specifically, each row of every table is represented as a unique node in the graph, with its type defined by the table. The identity of each node in the graph can be used as its primary key in the database, ensuring the uniqueness of primary keys. On the other hand, foreign keys are represented as edges: each foreign key in the RDB corresponds to a directed edge from the child row's node to the parent row's node. Since edges are defined between nodes of different types (m-partite graph), this ensures that foreign keys in one table correctly map to existing primary keys in another. Our diffusion model is conditioned on the graph structure and only denoises the features; therefore, all these constraints are maintained during the diffusion process. Finally, given a generated graph, we get the RDB representation as follows (see Appendix D.1 for more details):

1. For each node type $i$, we assign a unique primary key $p_v \in \\{1,\dots,n_i \\}$ to each node $v$ of type $i$, where $n_i$ is the total number of nodes of type $i$. This ensures that nodes of the same type (i.e., from the same table) will have unique primary keys.
2. For each directed edge $(v_{child}, v_{parent})$ in the graph, we add $v_{parent}$'s primary key to the set of $v_{child}$'s foreign keys. This ensures that foreign keys correctly map to existing primary keys.
3. For each node type $i$, we construct the table $R^{(i)}$ as a collection of rows corresponding to all nodes of type $i$ in the graph, containing the primary keys, foreign keys, and the generated attributes.

---

We hope our response can address the reviewer's concerns, and we are happy to answer any further questions during the discussion period.

We thank the reviewer for their valuable feedback. In the following, we address the reviewer's concerns in detail. We will include all the new experiments and discussions in the camera-ready version.

---

[W1, W2, Q2] Privacy preservation and generalization

In order to evaluate the privacy preservation of our model and to test if any memorization is happening, we follow prior work [1,2] and compute the distance to closest record (DCR) [3] between the generated and the training database. Specifically, for each synthetic sample, we get the minimum L1 distance to the real records. We report the mean DCR, the average of these distances over all generated samples. Since different tables can have different feature scales and ranges, we also report the DCR values from the holdout set to the training set, which serve as "ground-truth generalizations".

We use 4 tables from different databases that contain sensitive information (but have been anonymized for these public benchmarks), such as census data containing household and individual information, sensitive financial information from a bank, and order information from an e-commerce platform. The results, provided in the table below, show that our model yields higher DCR values than the baseline (i.e., is more private) across all tables and it even yields higher DCR values than the holdout set on 3 out of 4 tables, while being close on the 4th one. These results suggest that our model is able to generate new records from the underlying distribution and is not simply memorizing the training database. Note that DCR values need to be considered along with fidelity metrics (see Table 1 in the paper), since, e.g., random noise can yield high DCR values.

Remark on W2: While we use different diffusion models for different databases, each database contains multiple tables and potentially millions of rows, which are the central entity modeled by the same diffusion model. Note the contrast to single-table settings, where a different model is used for each table. While these single-table models showcased good levels of privacy protection and generalization [1,3], it is even more challenging for our model to memorize the training database because it models multiple tables at once.

Privacy metrics achieved by our model compared to the best baseline, and to the holdout set on 4 tables (the higher, the more private):

---

[W3] Comparison to SingleTable baseline

We designed our experiments such that all models share the same hyperparameters for the common components, including model architecture and size, diffusion steps, etc. In particular, the SingleTable baseline models tables separately and needs only to capture their individual distributions, while our model additionally needs to capture the correlations between different tables, given a similar parameter budget (the only difference is the GNN component, which constitutes approximately 5% of the total number of parameters).

To better compare these two models, we provide the average single-table and multi-table metrics, respectively, in a new table below. This comparison shows that the relative drop in single-table metrics is negligible compared to the relative gain in multi-table metrics achieved by our model. Also, when averaged across all the databases, our model outperforms SingleTable on both single-table and multi-table metrics.

In addition, we report the DCR values for the SingleTable baseline in the second table below (see response to [W1, W2, Q2] for the definition of this metric) and we can see that the samples generated by SingleTable are consistently closer to the training data than our model, meaning that it generalizes less than our model, which explains the higher single-table scores.

Comparison of average single-table and multi-table metrics between SingleTable and our model (the higher, the better):

Comparison of the privacy metrics between SingleTable and our model (the higher, the more private):

---

[W4] MP-GNN details

In Appendix D.4, we provide a general formulation of heterogeneous MP-GNNs. In the following, we provide the exact equations of the heterogeneous GraphSAGE model, which we used in our experiments, by plugging in the concrete choices of the different model components (e.g. choice of the aggregation function). Our implementation is based on the official PyTorch Geometric (PyG) implementation of the heterogeneous GraphSAGE model.

We describe one layer of this model, which maps node embeddings $\\{h_v^l \\} _{v \in \mathcal{V}}$ to $\\{h_v^{l+1} \\} _{v \in \mathcal{V}}$, where $l$ denotes the layer count. We describe the GNN update for a single node, and the same operations are applied to all nodes in parallel.

1. Compute a message for each edge type. To accommodate the heterogeneity of the graph, a distinct message is computed for each edge type. For each edge type $e$, $\mathcal{N}_ e (v)$ denotes the neighbors of $v$ that are connected via the edge type $e$, and $g_e$ is a neural network (typically an MLP) that is specific to this edge type. Then, the following message is computed for each edge type $e$: $m_{v,e}^{l+1}=\sum_{w\in\mathcal{N}_e(v)}g_e(h_w^l).$
2. Combine the messages into a unified message. Let $f_e$ be another MLP that processes messages of type $e$. Then, the combined message is computed as: $m_v^{l+1}=\sum_ef_e(m_{v,e}^{l+1}).$
3. Update node embedding. The unified message is used to update the node embedding. Again, to accommodate the heterogeneity of the graph, different MLPs are applied depending on the node's type, which we denote as $\phi(v)$. The updated node embedding is: $h_v^{l+1}=f_{\phi(v)}(h_v^l,m_v^{l+1}).$

These computations are run for all nodes, defining a single layer. Multiple such layers can be stacked to map initial node embeddings $\\{h_v^0 \\} _{v \in \mathcal{V}}$ to $\\{h_v^L \\} _{v \in \mathcal{V}}$, where $L$ is the total number of layers, with different neural networks for each layer. For each node $v$, $h_v^L$ contains information about the node $v$ and its $L$-hop neighborhood and can be used for downstream processing, e.g., type-specific denoising.

---

[Q1] K=2 vs K=1

Indeed, the K=1 version achieves very strong performance and already outperforms all the baselines on the inter-table metrics, which confirms that we can safely rely on diffusion to propagate information on the graph (as long as we use K>0). The multi-hop version can be seen as a simple way to further improve the performance at the cost of some additional compute.

---

References

[1] Kotelnikov, Akim, et al. Tabddpm: Modelling tabular data with diffusion models.

[2] Pang, Wei, et al. Clavaddpm: Multi-relational data synthesis with cluster-guided diffusion models.

[3] Zhao, Zilong, et al. Ctab-gan: Effective table data synthesizing.

---

We hope our response can address the reviewer's concerns, and we are happy to answer any further questions during the discussion period.

We thank the reviewer for their valuable feedback. In the following, we address the reviewer's concerns in detail. We will include all the new experiments and discussions in the camera-ready version.

---

[W1] Evaluation of the graph generation step

To better understand the modeling capacity of our graph generation algorithm, we perform a new ablation by replacing the sampled graph with the ground-truth graph and using the same diffusion model used in the paper to generate the attributes. The ground-truth graph achieves a perfect relational structure fidelity (cf. Cardinality metric) and provides an upper-bound on the performance achievable by graph generative models. The results in the following table show that our model achieves very similar performance to the ground-truth graph setting, showing that our graph generation procedure can effectively capture the relational structure of the RDBs. We also note that existing deep generative models for graphs are limited to graphs with a few thousand nodes, and scaling them to relational graphs with millions of nodes is a very interesting research direction.

---

[W2] Discrete diffusion for categorical variables

In the early stages of the project, we experimented with multinomial diffusion [1] for categorical variables, which is based on uniform transitions between discrete states. However, we found it to perform significantly worse than unified Gaussian diffusion, as can be seen from the table below. This is likely because discrete diffusion models like uniform and masked diffusion completely ignore the relationships between the different states that can be important in, e.g., ordinal data, which is abundant in real-world RDBs (counts, discrete timesteps, ratings, etc.). On the other hand, Gaussian diffusion leverages these relationships by transitioning between more similar states with higher probability. We also compare to concurrent work, RelDiff [2], which uses masked diffusion, and show that our model outperforms it on most datasets (see response to W4).

---

[W3] Generalization to modified schemas

While we are not aware of any other model that can handle schema changes, we discuss how our model can be adapted to handle different schemas:

1. First, we need a collection of databases with different schemas to enable model generalization. We could use different databases and/or create different schemas of the same database using schema change operations.
2. To enable training on different schemas, the generation process needs to be conditioned on the schema. For example, we could model the schema as a graph (nodes=tables, edges=table-level connections) and use an encoding of this graph as input to the generation process.
3. Schema changes typically involve data type modifications, so we need a unified representation for different data types. We could use tabular foundation models pre-trained on large amounts of data to map different data types onto a unified latent space that can be used for generation.
4. With these changes, we can apply our framework to jointly model the rows of any database in a unified space conditioned on its schema. After training on a variety of schemas, the model should be able to generalize to new schemas.

Due to the limited time during the rebuttal, we leave this exciting direction for future research.

---

[W4] Comparison to RelDiff and RGCLD

Unfortunately, RelDiff [2] first appeared online 2 weeks after the NeurIPS full paper submission deadline, and at the time of writing, their code is still not available. Nonetheless, we provide a comparison to RelDiff on the datasets and metrics that are common to our evaluation setup. We also include RGCLD [3], which generates tables autoregressively. The results show that GRDM significantly outperforms both models on the vast majority of the metrics.

---

[Q1] Missing and incomplete tables at test time

To assess the performance of GRDM in the case of missing or incomplete tables at test time, we design a new experiment on the California database. First, we split the database into a train and a holdout set based on the parent table, and use the train set to train the diffusion model in the same way as presented in the paper. Next, we perform experiments on the holdout set by conditioning the generation on some sections and using the same pre-trained model to inpaint the missing sections without any modification [4]. We report the metrics comparing the generated and the holdout set. We use 3 different strategies to select the missing parts of the database:

1. Masking entire tables.
2. Masking entire rows from all tables with different rates.
3. Masking single cells (individual attributes of rows) from all tables with different rates.

The results show that GRDM consistently maintains good performance across the different conditional generation settings.

---

[Q2] Differentiable graph generation

Yes, the graph generation can be made differentiable. For instance, we can use another diffusion model to generate the graph structure. However, under our proposed factorization $p(\mathcal{G})=p(\mathcal{V},\mathcal{E})p(\mathcal{X}|\mathcal{V},\mathcal{E})$, typically the graph generative model and the conditional feature generative model are trained separately, using teacher forcing for the conditional model, similarly to how autoregressive models are trained in practice. To allow end-to-end training, we would need to directly model the joint $p(\mathcal{G})=p(\mathcal{V},\mathcal{E},\mathcal{X})$, which can pose computational challenges when scaling to very large graphs (see section 3.2 for a detailed discussion).

---

[Q3] Performance when increasing database size

Our graph generation algorithm allows us to generate graphs of arbitrary sizes. We leverage this property to test the performance of GRDM when increasing the database size. We use different multipliers of the original database size and generate databases with up to more than 20 million rows (10x multiplier) using the same diffusion model. The results, shown below, highlight that the performance does not drop when increasing the database size. This can be explained by the use of our GNN in the denoising model, which operates on local subgraphs that are roughly invariant to the overall size of the RDB. Note that all RDBs were generated within less than 2 hours.

---

References

[1] Kotelnikov, Akim, et al. Tabddpm: Modelling tabular data with diffusion models.

[2] Hudovernik, Valter, et al. RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models.

[3] Hudovernik, Valter. Relational data generation with graph neural networks and latent diffusion models.

[4] Lugmayr, Andreas, et al. Repaint: Inpainting using denoising diffusion probabilistic models.

---

We hope our response can address the reviewer's concerns, and we are happy to answer any further questions during the discussion period.