TabUnite: Efficient Encoding Schemes for Flow and Diffusion Tabular Generative Models

This paper has some weaknesses as follows:

1. Lack of novelty: The paper essentially combines proposed encoding methods with diffusion-based generative models, which are known for their strong generative capabilities. This combination, while potentially effective, does not represent a significant novel contribution to the field.

2. Insufficient analysis: The most crucial aspect of this paper is the encoding part. However, the analysis falls short of demonstrating the individual impact of each encoding method. While the overall model performance is important, the paper would have benefited from a more detailed examination of how each encoding method (Phase-Shift Keying, Dictionary, and Analog Bits) performs independently.

• Lack of Intuitive Comparison: While the paper describes the proposed encoding schemes (PSK, Dictionary, Analog Bits) in detail, it could provide a more intuitive comparison with network-based auto-encoding. For example, a straightforward baseline should be TabSyn with the latent diffusion kept the same as DDPM/Flow in TabUnite, and compare the results. Besides, TabSyn is missing in Tab 3.

• Novelty: I think the paper only concentrates on data processing, and in my point of view the novelty is limited considering ML conference. Especially, diffusion/flow based methods have been well studied in this domain.

• (W1): in my opinion, the main weakness of this work is the reported performance of the proposed encoding schemes as compared to the SOTA encoding scheme i2b. For some datasets the proposed encodings work better but provide only a marginal outperformance. For some other dataset SOTA encoding (i2b) outperform the proposed encoding schemes.

• (W2): the second weakness is the missing comparison to learned embeddings as the ones presented in [1] and [2]. Comparing the proposed encoding scheme to learned ones would support the paper's comprehensive evaluation and be helpful for future research in this domain.

• (W3): both proposed encoding schemes are rather fixed or hand-crafted in their nature and therefore limited in their potential use for encoding categorical input values. For example, PSK encoding is interesting and seems to work very well for periodic categorical values such as Mon, Tue, ect,...where the data points projected on the circular plane preserver the relationship i.e., last day of the week is next to first day of the week. However, it will not work for many other categorical values such as, e.g., city names where the relationship between data points can not be expressed through a continuous circular representation, but rather a geographical one or a learned representation. Similarly, holds true for Dictionary Encoding. This encoding does indeed preserves the intrinsic ordering among ordinal categorical data in the embedding space but again does not provide a solution to nominal categorical values such as cities (or others such as hierarchical structures represented as e.g., business unit codes), which could be represented through a learned embedding. Since this paper claims to be unifying in bringing continuous and categorical inputs together, I consider this an important point to be made.

1. $**Excessive Emphasis on Novelty**$: The paper presents three new methods for processing categorical variables and applies them to existing models, claiming this as the foundation of a new model. However, this contribution appears overstated, as the primary novelty lies in the preprocessing techniques rather than in the development of a fundamentally new model. Comparatively, the impact of these additions may be limited.

2. $**Overemphasis on the Novelty of Dictionary Encoding**$: The Dictionary Encoding approach essentially applies a min-max scaling to categorical variables, assigning them continuous values within a fixed range. While this is useful for handling ordinal data, the technique is quite similar to standard scaling methods and may not represent a significant innovation. Consequently, the novelty of the Dictionary Encoding approach may be somewhat overstated in the context of its actual contribution.

3. $**Inadequate Handling of Categorical Variables in Analog Bits Encoding**$: The Analog Bits encoding method represents categorical variables as continuous binary vectors. However, this approach may not fully capture the discrete nature of categorical data, as it implies distances between categories that don’t inherently exist. This can blur the boundaries between distinct categories, potentially reducing the model’s ability to differentiate them effectively—especially in tasks where clear categorical distinctions are important. This issue is further compounded by the decoding process, which interprets the model’s continuous outputs through a simple thresholding method, limiting precision in recreating categorical identities.

1. The authors proposed three straightforward yet effective encoding schemes, but the design choices and analysis behind them require clarification. For instance, in phase-shift keying (PSK) encoding, why is a uniformly distributed phase used? For categorical attributes like nationality, distances between categories may vary, so uniform phase distribution might not be ideal. Why is the encoding structured as in Eq. (1), and could a learnable phase bias parameter improve it? Further explanation on the design choices for each encoding scheme would enhance understanding.
2. While the proposed schemes yield improved performance, the intuition and underlying reasons for this improvement are unclear. It would be beneficial to offer more justification and explanation for these schemes, especially in comparison to one-hot encoding. The authors mention that generative models are susceptible to underfitting in high-dimensional spaces, but no supporting evidence is provided in the paper.
3. The proposed encoding schemes appear versatile and could potentially benefit various tasks, such as classification and regression. Are these schemes more effective than one-hot encoding for predictive tasks? Including results from prediction tasks would help demonstrate the broader applicability of these schemes.

My current score is low, but I am open to raising it if the authors can adequately address my concerns.