Low-Rank Tensor Transitions (LoRT) for Transferable Tensor Regression

We sincerely thank the reviewer for the constructive and thoughtful feedback. We are encouraged that the reviewer finds our theoretical analysis and experimental design sound, and recognizes the potential of the proposed LoRT frameworks for addressing distribution shifts in tensor regression. Below, we respond to the reviewer’s concerns point by point.

On similarity to TransFusion

While LoRT is conceptually inspired by TransFusion [1], it addresses a fundamentally different and more challenging setting: tensor-valued regression, where both inputs and responses are high-order tensors. This introduces unique challenges:

1. Multi-mode low-rank structure: LoRT employs a tubal nuclear norm-based fusion regularizer to model low-rank structure along tensor modes, in contrast to the $\ell_1$-based fusion in vector settings like TransFusion.

2. Tensor-specific two-step estimation: Although both methods use two-step strategies, LoRT’s design is tailored to tensors, first estimating a shared low-tubal-rank component, then refining task-specific parts using tensor algebra.

3. Theoretical analysis under tensor structure: LoRT establishes estimation error bounds based on tensor-specific assumptions and tools, differing from the vector-based analyses in TransFusion.

These aspects make LoRT a nontrivial and technically distinct extension of fusion-based transfer learning to the tensor domain.

On comparison with CP/Tucker/convex baselines

We have conducted initial comparisons with representative CP-, Tucker-, and convex-based tensor methods under our experimental setting (https://anonymous.4open.science/r/LoRT-113D/CPtable.png) and will include full results in the final version.

On real data without known parameters

Our current study focuses on settings with known tensor coefficients, which allow precise assessment of parameter recovery and direct validation of our theoretical predictions. This experimental design is a standard and widely accepted practice in the tensor learning literature (e.g., Zhang et al., 2020; Wang et al., 2021), especially for theory-driven studies. By working with controlled synthetic data, we can rigorously examine the behavior of the low-rank transfer mechanism and its consistency with our generalization bounds.

We respect and appreciate the reviewer’s perspective that experiments on real-world data without ground-truth parameters can demonstrate practical relevance. However, such experiments serve a different purpose and are not necessary for validating the core theoretical contributions of this work. As our current goal is to establish and verify theoretical guarantees, we follow the conventional evaluation protocol in this area. We recognize the value of such experiments for illustrating broader applicability and consider them a promising direction for future work.

On computational details

We will include further algorithmic details—such as per-iteration computational cost and implementation notes—in the final version to improve clarity and reproducibility. We note that we have already provided runnable example code in the supplementary material, and we will expand the documentation to make the computational aspects more transparent.

On notation and typos

Many thanks for your careful reading! We will revise the notation system for better readability and fix all typographical errors.

On "no negative transfer" guarantee

Our current theory guarantees improvement under moderate heterogeneity, which aligns with prior work (e.g., [1]), as universal guarantees are generally infeasible without adaptive mechanisms.

That said, the structural design of LoRT naturally enables "no negative transfer"-style extensions. Specifically, the fusion regularizer allows for task-wise reweighting, and the divergence scores $h_k$ are available during training. This makes it feasible to implement heterogeneity-aware fusion, similar in spirit to [2], where adaptive regularization balances between source pooling and target-only training.

While our current focus is on estimation error in favorable regimes, we plan to explore an oracle-type guarantee of the form

$$\mathbb{E}[\text{Err}_{\text{LoRT}}] \le \\min\\{\text{Err}\_{\text{target-only}}, \text{Err}\_{\text{fusion}}\\} + \delta,$$

where $\delta$ depends on heterogeneity. We will add a discussion of this future direction in the final version.

On the assumption $r \ll d_2$

The assumption aligns with the low tubal-rank setting commonly used in t-SVD-based tensor models (e.g., Qiu et al., 2022a). Note that we assume $d_1 \ge d_2$ without loss of generality (line 111), so the rate $O(r d_1 d_3 N^{-1})$ can be more precisely written as $O(r \min\\{d_1, d_2\\} d_3 N^{-1})$ Hence, the dependence on $d_2$ is implicitly included via both $\min\\{d_1, d_2\\}$ and $r$.

We thank the reviewer for the thoughtful and constructive feedback. We appreciate the recognition of our theoretical framework, decentralized extension, and empirical evaluations. Below, we address the main concerns:

Concern 1: Conducting evaluations on real-world problems where Tensor Compressed Sensing is applicable could strengthen the claim regarding the practical utility of the proposed method. What are some real-world problems related to Tensor Compressed Sensing? Can the proposed method effectively address them?

Tensor Compressed Sensing (TCS) arises in various real-world scenarios where acquiring full tensor data is costly or constrained. Prominent examples include:

• Magnetic Resonance Imaging (MRI): Reconstructing high-resolution 3D volumes from a limited number of Fourier measurements.

• Hyperspectral Imaging: Recovering spatial-spectral data cubes under limited sampling due to hardware or bandwidth constraints.

Our method is theoretically well-suited to these scenarios: it leverages both low-rank structures and transferable information from source tasks to enhance reconstruction quality in the presence of data scarcity and distribution shifts.

In the current paper, we focus on theoretical development and controlled validation. The TCS experiments on synthetic Gaussian measurements are explicitly designed to validate the generalization guarantees and the low-rank transfer mechanism presented in Sections 4.2 and 4.3. These settings enable precise evaluation of recovery error and alignment with theoretical predictions.

Moreover, this design follows a well-established practice in the tensor learning theory literature. Many prior theoretical works (e.g., Lu et al., 2018; Wang et al., 2021) also provide guarantees under compressed sensing assumptions, while using real data primarily for completion tasks.

We fully respect the reviewer’s suggestion that applying the method to real-world TCS data could further support its practical relevance. However, such experiments are not essential to validate our main theoretical claims, and would require additional modeling assumptions (e.g., real sensing matrices, noise models) and substantial engineering efforts that fall outside the scope of this theory-focused work. Due to the limited rebuttal timeframe, adding real-world TCS experiments is unfortunately not feasible. We will clarify this rationale in the final version and discuss such extensions as promising directions for future research.

Concern 2: Are there cases where the i.i.d. assumption on the data is difficult to satisfy?

Yes, and we appreciate the reviewer raising this important point. While our current theory assumes that samples within each task are i.i.d., this is a standard assumption adopted in many theoretical works on transfer learning and tensor regression (e.g., Zhang et al., 2020; Qiu et al., 2022a). It facilitates a clear derivation of estimation bounds and highlights the key effects of low-rank structure and distribution shift.

We acknowledge, however, that real-world data often exhibit dependencies or structured sampling patterns that violate the i.i.d. assumption. Extending our theoretical framework to accommodate such cases is a promising and practically relevant direction for future work.

As an initial step in this direction, we have conducted preliminary experiments on tensor completion under non-i.i.d. settings, particularly involving mixed tube-wise and element-wise missing patterns (https://anonymous.4open.science/r/LoRT-113D/NIIDtable.png). We will provide a detailed discussion in the final version.

Concern 3: In the definition of t-SVD in Definition 2.1, shouldn't the size of $C$ be $d_1 \times d_4$?

Thank you for catching this typo. We confirm the correct dimension is $d_1 \times d_4$ and will make the correction in the final version.

We thank the reviewer for the constructive feedback and recognition of our contributions to robust tensor regression under distribution shift. We address the reviewer’s key concerns below.

Novelty: The core idea of LoRT lacks significant novelty and seems to be an incremental improvement.

We respectfully clarify that while LoRT builds on prior transfer learning principles, it tackles a substantially more challenging and underexplored setting—that of tensor-valued regression under shift. Unlike prior works which focus on vector-valued regression (e.g., TransFusion) or assume shared covariates across tasks, LoRT generalizes fusion-based transfer learning to the tensor domain, where:

• The response and predictors are both high-order tensors.

• Low-rank priors must be imposed across multiple tensor modes.

• Task heterogeneity must be handled in decentralized, distribution-shifted settings.

To address these challenges, we propose a new fusion regularizer based on the tubal nuclear norm, a two-step estimation procedure specifically designed for tensors, and estimation error bounds derived under high-dimensional tensor regression settings.

These innovations are not straightforward extensions of previous methods, and to the best of our knowledge, LoRT is the first theoretical framework to jointly address model/covariate shift and low-rank tensor regression under multi-task and decentralized settings.

Experimental diversity: Experimental diversity is limited. More general datasets from computer vision should be used to test robustness.

We acknowledge the value of broader empirical validation and note that our experiments are carefully designed to support the paper’s theoretical goals. Specifically, we focus on compressed sensing and completion tasks, which provide a controlled setting for directly assessing parameter recovery and validating the theoretical results presented in Sections 4.2 and 4.3. These tasks are intentionally selected to align with our analytical objectives, and the results already provide sufficient evidence supporting our claims.

To address the reviewer’s suggestion, we have conducted preliminary experiments on two video clips (Apply Eye Make-up and Blowing Candles) from the UCF-101 dataset, which is a standard dataset for evaluating tensor methods in computer vision [R1]. These experiments follow the setup described in Appendix A.2, and preliminary results are available at (https://anonymous.4open.science/r/LoRT-113D/CVtable.png).

[R1] Wang J, Zhao X. Functional Transform-Based Low-Rank Tensor Factorization for Multi-dimensional Data Recovery. ECCV 2024.

Baseline comparisons: Baseline comparisons only include methods published before 2021. Newer methods should be considered.

We thank the reviewer for the suggestion. To the best of our knowledge, this paper is the first to consider the proposed transferable tensor learning setting, and no existing methods are directly comparable. As explained in Footnote 6, our goal is to evaluate the benefit of transfer for tensor completion with very limited observations. In such extremely sparse regimes, all tensor formats (e.g., t-SVD, TT, TR, Tucker) are fundamentally limited by data availability, and their performance differences become marginal. Therefore, we selected TNN (Lu et al., 2019b) as a representative baseline to isolate the effect of transfer, rather than focus on fine-grained differences between tensor models.

To address the reviewer’s suggestion, we have also conducted comparisons with several recently proposed tensor completion methods [R2]–[R4]. Preliminary results are available at (https://anonymous.4open.science/r/LoRT-113D/CVtable.png), and we will clarify this point in the final version.

[R2] Qiu Y, et al. Balanced unfolding induced tensor nuclear norms for high-order tensor completion. IEEE TNNLS, 2024.

[R3] Wang A, et al. Noisy tensor completion via orientation invariant tubal nuclear norm. PJO, 2023.

[R4] Tan Z, et al. Non-convex approaches for low-rank tensor completion under tubal sampling. IEEE ICASSP, 2023.

Thank you for the constructive feedback and recognition of our theoretical framework, analysis, algorithmic design, and empirical results. Below we address the specific concerns.

Hardware implementation for distributed setting

Current implementation focuses on validating the theory, but D-LoRT is parallelizable: source gradients are computed independently and aggregated via proximal updates. It is compatible with distributed frameworks like MPI. We will release a version simulating distributed computation in the final submission.

Ablation study on the fusion regularizer

While we do not explicitly ablate individual terms in the regularizer, their effects are indirectly reflected through varying the number of nodes (Tables 1–3, Fig. 4). These experiments serve as empirical ablations. We will clarify this in the final version.

Empirical evidence for low-rank structure

The low-rank-inducing property of TNN is well established (e.g., Lu et al., 2019b). Our visualizations (https://anonymous.4open.science/r/LoRT-113D/LowRankFig.png) show rapid spectral decay in recovered tensors, confirming low-rankness.

Clarification on Line 176 for lack of CP/Tucker/TT experiments

We respectfully clarify that our paper does not claim these decompositions can be "easily" replaced, nor does it mention CP. Line 176 states that "TNN can also be replaced with other norms, such as Tucker-based (Liu et al., 2013), Tensor Train (Imaizumi et al., 2017), or Tensor Ring norms," which aims to highlight the modularity of LoRT. However, we acknowledge that the word "replace" may suggest direct applicability. “Extend to” would be more appropriate.

This paper focuses on validating LoRT under t-SVD, chosen for its computational and theoretical advantages. CP is excluded due to the NP-hardness of its nuclear norm. While LoRT can in principle be extended to Tucker or TT, such adaptations require nontrivial redesign and lie beyond the scope of this theory-oriented work.

Evaluation limited to PSNR

We use both PSNR and RE, which are standard metrics sufficient to support our core claims. That said, we agree that SSIM and LPIPS could offer complementary insights and will include them in the final version.

Code documentation and reproducibility

We will add more docstrings and ensure consistent random seeds for reproducibility.

Relation to HOSVD

Our method builds on t-SVD, which is structurally distinct but complementary to HOSVD. While HOSVD is not the focus here, our ideas can be extended to it.

Relation and comparison to TRN

TRN focuses on end-to-end deep learning with tensor layers for single-task prediction, aiming at architectural efficiency. In contrast, our work studies transfer learning across heterogeneous tasks with theoretical guarantees. Due to these fundamentally different goals and settings, a direct comparison is not appropriate. We will cite TRN and discuss its relevance in the final version.

Potential datasets and UK Biobank

This work focuses on theoretical development, and the current experiments—based on both synthetic and structured real-world data—are designed to support our theory. We believe this setting is appropriate for the scope and goals of the paper.

From a theoretical perspective, LoRT applies to a variety of structured tensor data, such as hyperspectral images, and multi-subject neuroimaging. This includes large-scale datasets like UK Biobank, where transfer across tasks or sites may be beneficial. That said, applying LoRT to such domains requires domain-specific modeling and infrastructure, which are beyond the scope of this work.

Computational complexity and scalability

LoRT is designed with a focus on theoretical guarantees rather than computational efficiency. As noted in Line 428, it may face scalability challenges in high-dimensional settings, similar to tensor-based LoRA. Such challenges are common in tensor learning and arise from the inherent complexity of modeling high-dimensional structure. Addressing them is beyond the scope of this theory-oriented work, and we will clarify this in the final version.

Detection of highly heterogeneous source tasks

Our framework assumes moderate heterogeneity, a standard setting in transfer learning. In practice, detecting large distribution gaps is important. Common approaches include computing divergence scores (e.g., MMD), domain classification, or local statistics. We will briefly discuss these options and their integration with LoRT (e.g., adaptive weighting) in the final version.

Stability of $W$ and norms of $W_k$

Thm 4.2 guarantees estimator stability, and Thm 4.3 refines the estimate via target-specific adaptation. While we do not analyze individual norms, this is an interesting extension which we will comment on in the revision.

Does D-LoRT require more data than LoRT?

Yes. Since D-LoRT transmits estimated parameters (not data), local models must be more accurate, requiring more samples per source task.