Thank you for your valuable feedback! We will address your concerns in the following responses.

How do you apply modulation?

Our feature-aware temporal modulation can be flexibly applied at three stages: input features, intermediate representations, or output logits (L226). The ablation study in Table 2 explores the relationship between the application position of modulation and performance improvement, and it is found that modulation at all stages has a positive impact on performance (Sec. 6.4).

Does this mean that your model must use hidden states with the same dimensionality as your input features?

This design does not require hidden states to match input dimensionality. Modulation parameters are dynamically generated per layer, allowing dimension-agnostic adaptation.

How do you apply modulation in your experiments?

As specified in Sec. 6.2 (L274), the MLP uses full modulation across all stages, while MLP-PLR and TabM only employ single-step modulation on input features.

How do you learn $\psi$, $\gamma$, $\beta$, and $\lambda$?

All parameters of the temporal embedding and modulator are jointly optimized with the backbone model using the AdamW optimizer and standard losses (e.g., cross-entropy for classification and RMSE for regression).

You could use any ML model with this approch, right? Why do your experiments only use neural network-based methods with modulation?

The modulation mechanism is not decoupled from model training. Instead, it is learned through end-to-end optimization and relies on gradient-based methods.

Consequently, our modulation strategy is compatible with any deep learning model, but non-differentiable models (e.g., tree-based ensembles) are unable to optimize the parameters of $\psi$, $\gamma$, $\beta$, and $\lambda$ due to the absence of gradient flow.

In summary, our modulation is a deep learning-specific, gradient-based adaptation mechanism applied flexibly across network stages. Its strength lies in jointly learning temporal representations and feature transformations, which is infeasible for non-differentiable models. We appreciate your comment and will acknowledge this limitation in our revision.

We hope this response addresses your concern. Please feel free to raise any further questions or suggestions!

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with about three days remaining, I wanted to ensure that we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Reviewer,

We greatly appreciate your valuable feedback. With less than 10 hours remaining in the rebuttal phase, we sincerely hope our previous responses have addressed your concerns. If you have any further questions, we would be more than happy to clarify them within the remaining time. We look forward to hearing from you.

Thank you for your thorough review and insightful feedback! We will address your concerns in the following responses.

It would be nice to see these results applied to more “standard” tabular benchmarks.

We acknowledge that TabReD has not received the attention it deserves. However, even if the datasets in existing benchmarks contain temporal columns, they are often not suitable for direct evaluation. These problems include data leakage caused by non-temporal splitting, extensive feature engineering of timestamps, and sample grouping that is not suitable for temporal splitting. This has limited our options. The Appendix of the TabReD paper also provides a detailed analysis [1]. Here we provide results on two additional Kaggle datasets:

Unlike the dataset in TabReD, the temporal shift of these two datasets is not obvious, so directly concatenating temporal embedding may not be effective. Our method shows robustness.

It would also be interesting to see whether it improves performance by similarly modulating based on the value of another continuous (or categorical) feature.

This is a very good idea, and we've previously explored it, including combining it with ensemble methods. However, current temporal embedding already incorporates many assumptions (periodicity and trends). While features that satisfy a linear relationship or have a temporal period can be successfully used for modulation, other features with unknown periods are not. Furthermore, selecting suitable features from among all available features is a key challenge, as using unsuitable features for modulation can lead to missing input information and degraded performance. This is definitely a direction for future work, and it holds great potential in scenarios where the shift patterns can be formalized.

L258: please provide some further details here.

Thanks for pointing this out. Due to space constraints, we moved the details of this section to the appendix when we submitted the main paper. We will include more details here in the revision.

I find the colors of the points in Figure 4 quite difficult to see, especially due to the small size.

Indeed, we will enlarge the sample points, highlight the different categories, and include a time axis in the revision to improve readability.

We hope this response addresses your concern, and we will carefully address all the suggestions and correct any typos in the revision. Please feel free to raise any further questions or suggestions!

---

[1] Rubachev, I., Kartashev, N., Gorishniy, Y., and Babenko, A. Tabred: A benchmark of tabular machine learning in-the-wild. In ICLR, 2025.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with about three days remaining, I wanted to ensure that we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

We are grateful for your constructive suggestions! We will address your concerns in the following responses.

The hyperparameter for the temporal embedding dimension is briefly mentioned but not thoroughly analyzed. An additional ablation study would strengthen the claims.

We further demonstrate the model performance achieved by tuning parameters after fixing different temporal embedding dimensions. Due to the limitation of hyperparameter space, all results are lower than those in Tab. 1.

We find that our temporal modulation consistently outperforms temporal embedding. Furthermore, there is an inflection point between the dimension and performance of temporal embedding, but not our temporal modulation. This demonstrates that our approach combines strong performance with scalability. By separating the conditional input (temporal) modality from the original feature modality, we reduce the impact of temporal input on the model backbone while enabling more refined temporal feature extraction, as also declared in response to reviewer dcxr.

How sensitive is the model’s performance to changes in the temporal embedding dimension? Could you elaborate on how you selected the embedding dimension used in your experiments?

The temporal embedding dimension is indeed a very sensitive hyperparameter, as demonstrated in the experiments above. In our experiments, all hyperparameters were tuned using Optuna. The hyperparameter space for the temporal encoding dimension is int: [0, 8, 16, 32, 64, 128, 256, 512, 1024].

Doesn’t yet characterize how severe temporal shifts must be before static models fail, leaving some uncertainty about practical thresholds.

This is a critical issue and difficult to quantify. However, in the design of temporal embedding, the hyperparameter search space already includes the option of not adding temporal embedding (i.e., dimension = 0), at which point the method degenerates to conventional methods. In other words, in our experiments, whether to add temporal embedding is determined entirely through hyperparameter search.

You may also be interested in the response to the reviewer dcxr, which analyzes when to use our temporal modulation and when to use temporal embedding directly.

Some referenced adaptive methods (e.g., domain–adversarial, reweighting) are only briefly mentioned without analysis of their failure modes on temporal tasks.

Domain-adversarial methods can only address domain-to-domain shifts and require partial target domain information, making them inappropriate for the temporal shift setting used in this paper. Reweighting methods can only address covariate and label shift, not concept shift, which is often severe in temporal shift datasets. Our temporal feature modulation aims to address concept shifts primarily caused by semantic evolution.

Limited analysis of the statistical significance of the approach.

Due to the large dataset size in TabReD, there aren't enough experiments to support significant comparisons between all methods. Below are the Nemenyi tests for MLP and TabM:

It can be seen that adding our modulation method to MLP significantly improves performance, while adding temporal embedding alone does not.

Similar results are seen for TabM.

In addition, our paper presents multiple evaluation metrics, including average rank (Tab. 1) and average percentage change (Fig. 5), providing a multi-perspective evaluation.

The focus seems to be limited to one type of distribution shifts. How does the approach perform under sudden, irregular, or non-smooth temporal shifts compared to gradual shifts?

We acknowledge this limitation, but in the analysis in [1], the MMD heatmap shows that most real datasets have shifts in the form of trends and cycles. The data may indeed have unknown shifts in the test set, but for models deployed before this, there is no effective way to predict this.

Although presented as a lightweight method, there is insufficient quantitative analysis regarding computational overhead.

We will introduce more detailed quantitative analysis in the revision. Our MLP with temporal modulation still significantly outperforms other models (such as the transformer) in terms of training efficiency and achieves better performance, which in part demonstrates the lightweight nature of our approach.

What is the standard deviation over different repetitions of the model? It would be very interesting to see.

The following table provides the standard deviations of the random split results in the Appendix.

We hope this response addresses your concern. Please feel free to raise any further questions or suggestions!

---

[1] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. CoRR, abs/2502.20260, 2025.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with about three days remaining, I wanted to ensure that we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

We appreciate your thoughtful comments! We will address your concerns in the following responses.

In addition to including results on random splits, it would be great if authors provide results on the original splits from the TabReD benchmark.

We further show the model performance before and after adding our temporal modulation to the original temporal split in TabReD [1].

The results are consistent with those in paper, and our method still achieves strong performance on the original temporal split of TabReD. This shows the robustness of our method in different validation strategies.

It is interesting how the proposed technique interacts with different validation and splitting strategies.

In our understanding, the key to resolving temporal shift lies in achieving distributional and temporal extrapolation capabilities (L37-L47). The difference is that the training protocol in [2] modifies the validation split to allow the model's performance to be better extrapolated to test time (so does the random split), which is the foundation for model learning. Our temporal modulation (or directly concatenate temporal embedding) finds a solution to the extrapolation capability within a given split. We believe that the effects of these two approaches are orthogonal, affecting the final model performance in different ways. This also explains why our method has similar effects on all validation splits.

Writing is not precise enough on how the modulator is trained – I assume that it's just trained as usual.

Yes, the modulator is trained end-to-end along with the model backbone.

What precisely encourages such "normalizing" behaviour?

Comparing this phenomenon with normalization is quite instructive. Although we haven't discussed this before, we believe that our modulation method can indeed be considered a special form of normalization, but a learnable and more general form because it supports not only scale changes but also skew changes. From this perspective, we believe that this is not the result of hold-out validation (as just mentioned, these two approaches are orthogonal in our understanding), but is more similar to the RevIN method [3] in time series prediction: in time series prediction, by learning a scaling and offset coefficient for each subsequence, the distribution shift of the time series can be effectively addressed. Corresponding to our method, we first use the temporal embedding to guide the model "which samples belong to the same subsequence and should be scaled together" in temporal tabular data, and then learn the coefficients of each "subsequence" in an end-to-end form, thereby learning a transformation for each specific temporal stage.

We hope this response addresses your concern. We're open to further discussion and would appreciate any additional thoughts you may have!

---

[1] Rubachev, I., Kartashev, N., Gorishniy, Y., and Babenko, A. Tabred: A benchmark of tabular machine learning in-the-wild. In ICLR, 2025.

[2] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. CoRR, abs/2502.20260, 2025.

[3] Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift. In ICLR, 2022.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with about three days remaining, I wanted to ensure that we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Thank you for your detailed and constructive review! We will address your concerns in the following responses.

The reported improvements on TabReD are small (1–2%). To further assess the method’s distinctive strengths, I believe that the authors could construct and experiment on more controlled synthetic datasets that can show the strengths of the proposed method.

We are very grateful for your valuable suggestions and have supplemented some further analysis:

Experiment on synthetic covariate-shift, label-shift, and concept-shift scenarios.

The Fig. 4 in paper has already demonstrated the concept shift. We further constructed the above three different shifts based on a same dataset and performed numerical comparisons.

The above results show that both temporal embedding and temporal modulation focus on addressing concept shift, which is consistent with our expectations and the focus of the paper. However, for covariate shift and label shift (which we obtained by sampling different time slices of the dataset), we found that simply adding temporal embeddings resulted in performance degradation, while our temporal modulation improved performance. We believe this may be because the model's response to covariate shift and label shift primarily depends on its inductive bias, while temporal modulation directly acts on the decision boundary and always attempts to correct any potential inductive bias.

This also explains why our method is better than simply adding temporal embedding: the temporal shift in real data is often a complex superposition of the three types of shift, and our method achieves better performance on all three shifts.

These results are encouraging and we will discuss them further in the revision. In addition, we will also include visualization results in the revision.

Observe how latent representations drift over time under each shift scenario.

If we analyze the overall distribution of the latent representation, we believe this may be similar to the conclusion of the MMD heatmap in [1], that is, the model representation $\phi(x)$ is nearly stable when no temporal information is used, but the target mapping $f$ is constantly changing. Therefore, we should pay more attention to the changes in the mapping and how the modulation is corrected, rather than how the model representation itself changes. If we try to analyze it on feature level, since each dimension of the model representation is often a fusion of multiple features, it is actually difficult to know whether the type of shift in the concept of each feature is as shown in Fig. 1, or whether there is no shift at all. If you have better ideas or suggestions, please feel free to raise them!

Compare the latent space of the proposed method compared to supplying vanilla temporal embeddings to the baseline model.

This is also an interesting research direction, and we will include visualizations in the revision if possible. In our understanding, the fundamental difference between temporal modulation and simply adding temporal embedding is that temporal modulation isolates temporal information from the modality of the conditional input, rather than conflating it with the original input modality. This prevents the model input from being affected by overly complex temporal embedding and enables more refined temporal feature extraction. (For example, when simply adding temporal embedding, the embedding dimension tuned by Optuna often does not exceed 100, as the model's input feature dimension is also only ~100. However, in our temporal modulation, although we set an upper limit of 1024 for the embedding dimension, many experiments still reach this limit.) In contrast to latent representation, we believe that simply adding temporal embedding can lead to a serious curse of dimensionality, while temporal modulation fundamentally avoids this problem.

Besides, we believe that a 1-2% performance improvement is not small. Without incorporating any temporal information, the performance gap between the most basic MLP model and the strongest tested TabM model is approximately 2.7%. All deep methods except TabM and MNCA have a performance gap of less than 2% compared to MLP. This makes the dynamic MLP with our temporal modulation outperform all static deep methods except TabM and MNCA under temporal shifts. This can also be seen in Tab. 1.

The paper does not clarify the boundary between settings where the proposed method helps and where just supplying a temporal embedding could be better.

We acknowledge that this is an important question, one that the paper does not explore further. This may require more detailed experiments. Here we only list a few possible observations:

1. For ordered but discrete numerical features (e.g., int-type numerical features), temporal modulation does not necessarily yield better results than temporal embedding. This may be because the knowledge learned by the model is already discrete, and directly adding modulation to it makes it difficult to optimize, resulting in worse results than adding temporal embedding to the extra dimension.
2. Compared to categorical features, we did not observe the results in 1. This may be because we use one-hot encoding for categorical features, allowing our modulation to apply independently to each category. This provides greater flexibility.
3. For features with strong non-normality (e.g., long-tailed distributions), our temporal modulation yields greater improvements. This may be because our modulation includes a correction for skewness, which allows the model to learn better.
4. We rarely observed distribution changes such as "unimodal to bimodal" (in other words, we generally found corresponding relationships with different peaks, although the density differences may be large), but we believe this may be a difficult issue. Our time modulation may only be able to achieve this correction by transforming all distributions into more normal distributions.
5. Input dimensionality is indeed a key issue. As mentioned in the previous question, our time modulation fundamentally addresses the curse of dimensionality that can result from input temporal embedding. You may be interested in the response to reviewer kbVu, where we analyze the performance of temporal embedding and our temporal modulation when different time dimensions are fixed. The results show that our method achieves both performance and scalability. This means that temporal modulation will perform better in scenarios with more complex temporal features.

We hope this response addresses your concern. Thank you again for your constructive feedback, which will help us improve our work! Please feel free to raise any further questions or suggestions!

---

[1] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. CoRR, abs/2502.20260, 2025.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with about three days remaining, I wanted to ensure that we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.