The paper presentation should be improved. In the abstract, for instance, the issues in existing OOD works do not appear novel. Extensive fine-tuning and experimental trials are not unique to this specific OOD problem but relate to many general issues. The authors need to outline issues more specific to the OOD problem.

We will revise this later. Thank you for your input.

The review of related works requires significant improvement. There is a substantial body of existing work on OOD in tabular domains; however, the authors only discuss two methods in Sec. 2.1 and one other in L38, with the most recent paper referenced from 2017. The authors should put more effort into the literature review, for example, [1].

Our related work is actually more than this; however, we've done a substantial page deduction due to the page limit. Thank you; we will add this to the revision.

The proposed TCL employs a common strategy: contrastive learning with two views generated from a sample for self-supervised learning, followed by a head for classification/regression. This framework has been used in various contrastive learning-based tabular models, such as SubTab, which modifies contrastive targets with slicing techniques for tabular structures. The performance is not superior to the selected baselines, which limits both the novelty and effectiveness of the proposed approach.

We are using full matrix representation. We have done an upcoming experiment with better results, and we will release it this week. We also include XGB, Catboost and Lightgbm.

The selected baselines are based on the work of Gorishniy et al., 2021, which was published in 2021. Since then, many advanced tabular prediction models have emerged and should be included in this paper, such as those summarized in [2]. Note that some of these models serve as tabular foundation models and may not experience the OOD issues described in this paper.

We started this project in January, while the survey paper was listed in February. We are not aware of this. Our goal is to develop an algorithm that is light enough to be used by a general user without a GPU but capable of nearing or exceeding the capability of a user with a GPU. The methods mentioned in the survey paper are based on MLP and transformer. While we do not cover the methods mentioned in the survey paper, we include base MLP and Transformer.

The authors used CPU(s) to train their model but used an H100 GPU for the baselines, which is confusing. I didn't see any specific design element that would make the proposed method more efficient; on the contrary, matrix augmentation appears to require more computational resources. Although Table 4 presents training durations, it is unclear how this table was created. For example, is the parameter space the same across all models?

Our distinctive feature is the use of a full matrix representation in the design of TCL. This approach eliminates the need for data slicing and matrix multiplication. To optimize for CPU compatibility, we refine the loss function by employing only dot matrices and narrow layers. The core concept of contrastive learning involves adding noise and enhancing generalization. By applying this strategy, we can achieve performance comparable to larger models that typically demand more powerful hardware. We will add parameters specification on the next revision.

Dear cczR.

Thank you for your response. Initially, our primary objective was to develop an efficient method, even if it required a slight compromise on accuracy. However, feedback from reviewers indicated a lack of acknowledgement of this approach. Consequently, we shifted our focus towards prioritizing accuracy, even at the expense of efficiency. This adjustment explains the modifications made to the algorithm.

 If the authors modified their proposed method, it indicates that the manuscript is not ready to be accepted.

We believe, based on the ICLR 2025 rule mentioned on https://iclr.cc/Conferences/2025/AuthorGuide :

You can upload revisions until the discussion period ends

Revision is allowed. We have made every effort to respond comprehensively and meet the expectations of all reviewers.

Regards.

Why is the Contrastive Federated Learning (CFL) reference in an anonymized link? 

-> It is due to the anonymized rule, correspondence to one or more author of this paper.

Why does the title miss some keywords like Tabular Contrastive Learning? 

-> This work focuses on how to detect and predict out-of-distribution. TCL is part of the paper but not all about TCL. Our work uses a regular dataset, predicts OOD and proof, and predicts classification or regression problems on it. This is what actually real-world users seek.

Why is there no experiment for baseline models running on CPU (Apple) to fairly compare with your TCL?

-> According to the original paper, baseline models are best performed within GPU. To support that, we follow the original paper's architecture to fit their best setting. We have a follow-up experiment with a CPU with better results. Our point is to show that with only a consumer CPU, we can achieve similar results with advanced models trained with GPU.

Thank you for your dedication and continuous effort in refining this work. Considering the competitive nature of this field, I strongly encourage the authors to explore additional related works (as the reviewers point out) and emphasize the unique innovations and advantages of your methods. Specifically, it would be beneficial to provide more details about TCL, highlight the distinctions between TCL and CFL or other comparable frameworks, and clearly articulate the motivation and compelling aspects that set the methods apart.

The experimental results are not particularly impressive. While TCL shows high efficiency, it only achieved optimal performance on three datasets compared to FT-T and ResNet.

We have done follow-up experiments with better results. We will upload it in the next revision. Thank you

The paper lacks a comprehensive comparison of methods. It would benefit from including comparisons with GBDT methods (e.g., CatBoost[1], XGBoost[2]), other state-of-the-art deep learning models (e.g., TabR[3], ExcelFormer[4], Trompt[5]), and alternative self-supervised learning algorithms (e.g., Scarf[6], VIME[7]) besides SubTab.

We have added GBDT to our experiment. We will release it in our revision We can not include every available method due to our limitation. We will add Scarf in our next revision. We will consider others, but probably for the next paper. thank you

More details should be included in the paper, such as the specific structure of TCL, the downstream classifier used, hyperparameters of the comparison methods, and whether experiments were conducted multiple times to present average results.

This work focuses on how to detect and predict out-of-distribution. TCL is part of the paper but not all about TCL. Our work uses a regular dataset, predicts OOD and proof, and predicts classification or regression problems on it. This is what real-world users actually seek. In addition, we add TCL as a prediction tool for general users without GPU.

The experiment comparing dot product and Euclidean distance indicates that dot product is more efficient on the hardware they used. However, this work does not discuss how using these two distance metrics impacts the performance of TCL and how significant the computational time for distance calculation is within the entire 

Our distinctive feature is the use of a full matrix representation in the design of TCL. This approach eliminates the need for data slicing and matrix multiplication. To optimize for CPU compatibility, we refine the loss function by employing only dot matrices and narrow layers. The core concept of contrastive learning involves adding noise and enhancing generalization. By applying this strategy, we can achieve performance comparable to larger models that typically demand more powerful hardware.

TCL process.What criteria did the authors use to select different detectors for OOD detection across various datasets?

We tried many detection algorithms, but we only used algorithms that can empirically be proven by the next step mentioned in our paper after detection.

Authors may want to assess the performance of TCL using the recently published benchmark, TabRed [1], which is a Benchmark of Tabular Machine Learning in-the-Wild with Real-World Industry-Grade Tabular Datasets.

We will definitely consider this in our revision, but for now, we will add Scraf as the method gain more popularity.

are not appropriate for this study, as these datasets do not display distribution shifts,

We deliberately use a common database. The reason behind this is In the real world, users can only use the databases that are available. For that reason, we also cover detection and steps to be taken for OOD.

Many of the main claims in the abstract lack support or are difficult to verify -- for example, "Addressing OOD data requires extensive fine-tuning and experimental trials" and "Deep learning has been sug- gested as a solution and has shown significant improvements".

We will add this in our revision. Thank you.

 The paper is missing references to many relevant works, including:

We will add this in our revision. Thank you.

The paper is also missing references to several relevant empirical studies for tabular data, including:

Our work focuses on how to detect and predict out-of-distribution. TCL is part of the paper but not all about TCL. Our work uses a regular dataset, predicts OOD and proof, and predicts classification or regression problems on it. Adding this will max out page limitation or deduct technical details of our paper.

The paper frames its use of contrastive learning as a novel contribution, however, there are two problems with this:
...
The paper simply applies an existing method (Contrastive Federated Learning, or CFL) to tabular data. This is not sufficient for acceptance, given the concerns with the empirical evaluations described above.

CFL focuses on a Federated learning network, while ours is common tabular data. In CFL, while exhibiting similar name, they use partial data augmentation as part of federated learning concept. In our CFL we use a complete full raw matrix representation. Our TCL also did not came from base understanding of CFL that mention about data in a silo similar to partial data augmentation in original contratstive learning for image. TCL is part of the paper, the whole paper is A-Z discussing about OOD treatment for general user.