JANET: Joint Adaptive predictioN-region Estimation for Time-series

Weakness 1. The paper lacks a concrete theoretical discussion on K-FWER control for the proposed method. The relationship between the designed score and K-FWER control should be made explicit. While I understand the authors treat each time series with horizon H as a single data point and apply conformal inference accordingly, this approach may be unclear to new readers.

Weakness 2. For a single time series, the K-FWER control remains questionable. The authors should provide thorough theoretical justification, particularly in relation to Theorem 1.

Weakness 3. The background introduction is overly lengthy. Several sentences, such as those in lines 153-155 explaining $a_{(n_{cal})}$ appear redundant. The authors only present their approach in Section 5 which is quite late. I suggest that after Section 3, the authors introduce their method by illustrating the case of multiple time series first, and then discuss the single time series as the extension.

Clarity questions

• Definition 1 / Theorem 1. What is $\mathcal{A}$ and the various constants $\delta$s and $\gamma$s? They are undefined in the paper. What does $\mathcal{A}$ it output, if it only takes two datasets as input and does not take in the test sample? Overall 4.3 was not clear to me. Isn't eq 3 and 4 the same equation with an reused $\epsilon$? Theorem 1 can benefit from some explanation as well (i.e. what does the error depend on, how it behaves with different data properties etc.)

• the notations are a bit messy throughout the paper. For example, what is the arrow x in line 299? (this sequence notation is not used anywhere else, you use bold $\mathbb{X}$ instead). in Eq 5 and 6 the $\hat{\sigma}_i$ can be either a function or a scalar. In line 306, is $X$ the input at this time step or relevant history? Cleaning up these notations will make the paper easier to follow.

• Line 315: What does "inverting the conformity scores" mean?

Writing / Presentation

• deviation-scaled conformal intervals are only "adaptive" in a very narrow sense (of X*'s deviation from other training data) and doesn't always present better efficiency. In times series CP literature, adaptivity usually means adaptive to changes in nonconformity score distribution (along the lines of ACI or CAHFT).

• From my understanding, the paper's methodology contains two parts: permutation for calibration, and scaling. Both of them are existing CP algorithms adapted by the authors to the time series setting, but it is unclear what are the authors' modifications. Can you add a few sentences to each of the two sections on what are your original contributions?

• A similar concern as above, I fear that the setting of the method is not well explained. The author repeatedly emphasized that the algorithm's guarantees hold "under mild assumptions", but in reality the data has to be strongly mixing and have ergodicity. I would recommend the authors to further explain in the paper the assumptions being made, and have examples of the kind of data the algorithm is suitable for.

to summarize

I believe the paper addresses an well-motivated problem and presents a sound method, but needs significant restructuring and clarification to be at the standard of a conference like ICLR. If these concerns are addressed, I'm happy to increase my score.

• It is well-known that split conformal is a special case of full conformal([3], also highlighted directly in [1]). While split conformal offers computational advantages due to requiring only one model fitting, the motivation for framing the discussion solely from the split conformal perspective seems lacking when a more general approach could be described.
• The theoretical results are identical to those presented in [1]. A more thorough discussion differentiating this work from [1] would strengthen the paper’s contribution.
• Although the authors emphasize the applicability of their method to multivariate time series, all experiments appear to be conducted solely on univariate data.
• More detailed discussion is needed on whether the method provides significant performance gains compared to current state-of-the-art methods. Additionally, while the results suggest slight under-coverage in many cases, it is unclear if the width reduction compensates adequately for this under-coverage.
• The authors place significant emphasis on computational efficiency; however, this advantage seems to stem from the use of split conformal rather than the proposed algorithm itself. Moreover, despite this emphasis, the analysis of computational complexity is limited to a brief mention in Section 6.1.2.

Typos:

• Line 222: $l \to L_{cal}$

[3] Barber, Rina Foygel, et al. "Conformal prediction beyond exchangeability." The Annals of Statistics 51.2 (2023): 816-845.

However, my concerns are as follows

• The theoretical results are a bit limited and incremental compared with existing results;

• There are not enough comparisons with state-of-the-art methods in multi-dimensional conformal time-series analysis, such as [42] published in ICML 2024. I believe the method should at least be compared with respect to one-step ahead prediction, since it is a special case of multi-step prediction. There is no fundamental difference in constructing multi-step ahead prediction from single step ahead, if you treat the prediction for y_{t+s}, s > 1, as the response variable instead of y_{t+1}.

-There are many conformal prediction for time series paper as of now in the literature, and I feel the literature survey is incomplete and missing several key references, such as

Angelopoulos, Anastasios, Emmanuel Candes, and Ryan J. Tibshirani. "Conformal pid control for time series prediction." Advances in neural information processing systems 36 (2024).

Yang, Zitong, Emmanuel Candès, and Lihua Lei. "Bellman Conformal Inference: Calibrating Prediction Intervals For Time Series." arXiv preprint arXiv:2402.05203 (2024).