Thank you for reviewing our paper and for your thoughtful questions.

Regarding the theoretical bound in Proposition 4.1: while it is not tight, it is useful in practice for diagnosing model behavior. In particular, it helps assess whether the approximation error per step stabilizes over time. This behavior can be seen in Figure 7, where the surrogate error remains bounded throughout. Moreover, it shows that the two-step approximation incurs additive error, meaning that the errors from each approximation stage do not compound over time. We clarify this in the revised text.

Questions

R4.1.1 Why do you focus on uncertainty in predictions rather than uncertainty in parameters?
A major drawback of many Bayesian neural network methods is that they make use of misspecified priors and likelihoods, often chosen for convenience [Knoblauch2022]. These choices can lead to degraded predictive performance [Wenzel2020] and make posterior sampling slower than non-Bayesian alternatives (e.g., deep ensembles). [Lakshminarayanan2017]

By contrast, focusing directly on uncertainty in predictions avoids the need to define priors over "subjective probability distribution on parameters that may have no real-world interpretation", as noted by [Holmes2023]. Furthermore, sampling from the posterior predictive is typically much lower-dimensional and a more tractable object for decision making.

We also observe improved performance over classical TS in deep neural networks, as demonstrated in a new experiment included in the revised version (See, R4.1.2 below for results).

We clarify these points in the revised text.

R4.1.2 How does this approach compare to the traditional TS?

In traditional TS, one samples $\hat{\theta} \sim p(\theta | D_{1:t})$ and uses $\hat{y}_{t+1} = f(\hat{\theta}, x\_{t+1}, a)$ to make decisions. This requires the specification and sampling of a posterior over model parameters.

In our predictive Bayes (pBayes) approach, one samples $\hat{y}\_{t+1} \sim p(y | x_{t+1}, \theta_{t|t})$ directly and use methods like HiLoFi/LoLoFi/LRKF to update the belief state $\theta\_{t|t}$ (and also $\Sigma\_{t|t}$ for the methods we consider) .This decoupling enables faster and more stable exploration-exploitation behavior.

We clarify this distinction in the revised text, include pseudocode for the martingale-based predictive Bayes approach, and add a new experiment comparing against classical TS.

The pseudocode for our proposed predictive TS in a multiarmed contextual bandit problem is given below.

// Define posterior predictive parameterised by belief state b
// and having inputs: the context ‘x’ and action ‘a’.
def p(· | x, a, b):
    ...

For each a in actions:
   // belief state:
   // need not define a valid posterior or consider two moments only
    b(t, a) = (mu(t, a), Sigma(t, a))

For all a in actions:
    // sample from Bayesian posterior predictive  
    ŷ(t, a) ~ p(· | x(t), a, b(t, a)) 

a_next = argmax_a[ŷ(t, 1), ..., ŷ(t, A)]

// frequentist update (e.g., HiLoFi)
b(t+1, a_next) = update(b(t, a_next), y(t, a_next))

In the table below, we show results for the MNIST contextual bandit task, comparing traditional TS against our pBayes approach. Note that HiLoFi does not admit a full posterior over parameters, so we report pBayes results only. For LoFi and Laplace methods, which support both TS and pBayes, we report both.

In summary: Sampling from the posterior, as required by TS, is dramatically slower, while offering little or no performance gain. For example, Laplace-TS takes 190 minutes vs. 1.4 minutes for pBayes, with much worse reward. Even LoFi-TS is nearly 10X slower than LoFi-pBayes with similar results. pBayes offers a superior speed-performance trade-off.

Note that the significant increase in speedup is because sampling, at every step, from a 10-dimensional multivariate Gaussian is much more computationally tractable than sampling from a D-dimensional multivariate Gaussian (even if the covariance structure is diagonal-plus-low-rank, as is the case for LoFi).

We add this new experiment with results in the revised text.

R4.2 Can you provide an intuitive explanation of Figure 1, the in-between uncertainty induced by HiLoFi?
Thank you for this suggestion.

Figure 1 illustrates how explicitly modeling the last layer in full rank increases predictive uncertainty in regions without training data, where higher uncertainty is expected. Compared to fully low-rank methods (e.g., LRKF; see Figure 15 in the Appendix), HiLoFi forms more meaningful uncertainty estimates with fewer observations in these unseen regions.

This provides a better balance between expressiveness of uncertainty and computational cost, which is important for managing the exploration-exploitation trade-off in sequential decision making.

R4.3 It might be helpful to plot the cumulative regret as well, which is a more standard metric of online learning, for the bandit MNIST problem.
Thank you for this suggestion. In the revised version of the paper, we include cumulative regret plots for the Bandit MNIST problem.

[Knoblauch2022] Knoblauch, Jeremias, Jack Jewson, and Theodoros Damoulas. "An optimization-centric view on Bayes' rule: Reviewing and generalizing variational inference." Journal of Machine Learning Research 23.132 (2022): 1-109.
[Wenzel2020] Wenzel, Florian, et al. "How good is the bayes posterior in deep neural networks really?." arXiv preprint arXiv:2002.02405 (2020).
[Lakshminarayanan2017] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. "Simple and scalable predictive uncertainty estimation using deep ensembles." Advances in neural information processing systems 30 (2017)
[Holmes2023] Holmes, Chris C., and Stephen G. Walker. "Statistical inference with exchangeability and martingales." Philosophical Transactions of the Royal Society A 381.2247 (2023): 20220143.

Thank you for reviewing our paper.

R3.0
We agree with your assessment.
In the new version of the paper, we discuss the tradeoff between sample efficiency and computation time and mention that our method is designed to tackle problems in which computation-time is the main bottleneck. This is often the case in domains such as finance [Cartea2023], recommender systems [Duran-Martin2022], and Bayesian optimization where the number of acquired datapoints is reasonably large [Brunzema2024].

We also clarify in the revised version that our approach separates frequentist learning from Bayesian prediction. The model maintains a predictive distribution for decision making, updated via HiLoFi, without relying on full posterior inference. This is now reflected in the paper’s title and illustrated through added pseudocode and an additional comparison of "classical" TS and our "predictive TS" approach (pBayes).

The pseudocode for our proposed predictive TS in a multiarmed contextual bandit problem is given below.

// Define posterior predictive parameterised by belief state b
// and having inputs: the context ‘x’ and action ‘a’.
def p(· | x, a, b):
    ...

For each a in actions:
   // belief state:
   // need not define a valid posterior or consider two moments only
    b(t, a) = (mu(t, a), Sigma(t, a))

For all a in actions:
    // sample from Bayesian posterior predictive  
    ŷ(t, a) ~ p(· | x(t), a, b(t, a)) 

a_next = argmax_a[ŷ(t, 1), ..., ŷ(t, A)]

// frequentist update (e.g., HiLoFi)
b(t+1, a_next) = update(b(t, a_next), y(t, a_next))

Below we show results for the MNIST contextual bandit task, comparing traditional TS against our pBayes approach. Note that HiLoFi does not admit a full posterior over parameters, so we report pBayes results only. For LoFi and Laplace methods, which support both TS and pBayes, we report both.

In summary: Sampling from the posterior, as required by TS, is dramatically slower, while offering little or no performance gain. For example, Laplace-TS takes 190 minutes vs. 1.4 minutes for pBayes, with much worse reward. Even LoFi-TS is nearly 10X slower than LoFi-pBayes with similar results. pBayes clearly offers a superior speed-performance trade-off.

Note that the significant increase in speedup is because sampling, at every step, from a 10-dimensional multivariate Gaussian is much more computationally tractable than sampling from a D-dimensional multivariate Gaussian (even if the covariance structure is diagonal-plus-low-rank, as is the case for LoFi).

We have also introduced a new experiment with larger number of parameters ~4million (see R3.2 below) to address concerns about model scalability.

Questions

R3.1 In Figure 2, it seems that LoLoFi achieves better performance than HiLoFi. Do the authors have any hypothesis on why this happens?

The purpose of Figure 2 is to show that when uncertainty modeling is less critical (e.g., a full-information setting), modelling the last layer explicitly performs on-par with other scalable approaches.

In particular, LoLoFi outperforms HiLoFi in some cases because higher-rank models require more steps to stabilize the covariance update. With more degrees of freedom and full-information updates, convergence takes longer. In this case, lower-rank update procedures can outperform in full-information settings.

We clarify this in the revised paper.

R3.2 Intuition: why only separates the last layer? Are there ablation experiments for a larger number of layers?
Applying full-rank updates only to the last layer is a common design choice in scalable uncertainty methods and has shown strong empirical results in bandit and Bayesian deep learning settings $Harrison2024, Riquelme2018$ .

That said, we agree this warrants empirical support. We conducted a new experiment applying LRKF to VGG-style architectures with up to 4.7M parameters on CIFAR-10 after 10,000 observations. The results below show stable online classification accuracy across different low-rank approximation levels (ranks 1, 5, and 10):

For comparison, a single-pass AdamW baseline achieves 0.3442 for VGG and 0.3097 for +VGG.

These results demonstrate that LRKF scales to multi-million parameter networks while maintaining stable performance across low-rank configurations. This supports the broader applicability of our approach (including HiLoFi and LoLoFi) to large-scale architectures in vision and related domains.

In the revised paper, we also include an additional experiment of the "in-betwen uncertainty" plot as we increase the number of layers of the MLP used.

R3.3 For proposition 4.1, are there empirical validations for this error bound? Can the rank d_h be dynamically tuned to trade off accuracy and compute?
Yes, please see Figure 7 in the Appendix where we show surrogate and low-rank error as well as rolling accuracy. These plots indicate that the error per step remains stable, and learning proceeds steadily across time.

As a rule of thumb, increasing d_h leads to better performance, with running time increasing at a roughly linear rate. This trend is illustrated in Figure 6 (MNIST bandit). Furthermore, Figure 13 shows that higher d_h improves the quality of in-between uncertainty estimates.

To make this point clearer, we have also introduced a new ablation study on Bayesian optimization on the DrawNN dataset. Here, we vary d_h and report the relationship between performance and compute cost:

The table above reports the performance of HiLoFi on the DrawNN dataset (Bayesian optimization), showing runtime in seconds and the final best value (averaged over 20 runs). As shown, both runtime and performance increase approximately linearly with d_h, reinforcing the trade-off discussed above.

We clarify these point in the revised paper (including this new experiment), emphasizing that d_h offers a practical trade-off between accuracy and compute, depending on available resources.

[Cartea2023] Cartea, Álvaro, Fayçal Drissi, and Pierre Osselin. "Bandits for algorithmic trading with signals." Available at SSRN 4484004 (2023).
[Duran-Martin2022] Duran-Martin, Gerardo, Aleyna Kara, and Kevin Murphy. "Efficient online bayesian inference for neural bandits." International conference on artificial intelligence and statistics. PMLR, 2022.
[Brunzema2024] Brunzema, Paul, et al. "Bayesian optimization via continual variational last layer training." arXiv preprint arXiv:2412.09477 (2024).
[Harrison2024] Harrison, James, John Willes, and Jasper Snoek. "Variational Bayesian last layers." arXiv preprint arXiv:2404.11599 (2024).
[Riquelme2018] Riquelme, Carlos, George Tucker, and Jasper Snoek. "Deep bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling." arXiv preprint arXiv:1802.09127 (2018).

I appreciate the authors' explanation of my questions and the addition of the experiments, especially experiments for validating scalability. I also like the idea of renaming the paper to incorporate the martingale posterior, as that does make a clever connection. For the new CIFAR-10 experiment, how is the performance evaluated? Is the one-step-ahead accuracy calculated on the posterior mean?

Thank you for your review and thoughtful suggestions. Your comments have helped improve the accessibility of our paper. We summarize the revisions and respond to your questions below.

We clarify that replay-buffer methods store data for repeated use in training (e.g., BNNs) or inference (e.g., GPs). While we don’t claim these methods are inherently costly, our approach avoids them entirely, which makes our method suitable for privacy-sensitive or streaming settings.

Furthermore, in the revised version of the text, we introduce the notion of “in-between” uncertainty, originally discussed by $Foong2019$ . This concept refers to how well a model captures uncertainty in regions of input space that lie between, or away from, observed data. Intuitively, we expect uncertainty to be higher in such unexplored areas. This behavior is important for Bayesian decision-making problems, where it supports effective balancing of exploration and exploitation.

We have also made the connection between uncertainty quantification and sequential decision making more explicit (please see R2.1 and R2.2 below).

Finally, we will improve figure explanations. For Figure 5, the left panel shows best final value vs. runtime; the right panel shows rank across tasks. Error bars are included in Figure 10 (Appendix) and are referenced in the main text (L28).

R2.0 Changes to paper
A key revision was to clarify the separation between the Bayesian predictive sampling and the frequentist update. To that end, we made the following changes

• Renamed the paper to "Martingale Posterior Neural Networks for Fast Sequential Decision Making", which more succinctly expresses what we are doing: modelling uncertainty in predictions, rather than parameters. (Following $Holmes2023$ )
• Revised the text to make the role of the posterior predictive in decision-making more explicit. In particular, we now mention that our approach allows us to make Bayesian decision making in a more efficient way than the prior-times-likelihood approach.
• Introduced a new pseudoalgorithm to illustrate the key predict-act-update in contextual bandits and Bayesian optimization. For contextual bandits with one model per arm, the pseudoalgorithm has the form

// Define posterior predictive parameterised by belief state b
// and having inputs: the context ‘x’ and action ‘a’.
def p(· | x, a, b):
    ...

For each a in actions:
   // belief state:
   // need not define a valid posterior or consider two moments only
    b(t, a) = (mu(t, a), Sigma(t, a))

For all a in actions:
    // sample from Bayesian posterior predictive  
    ŷ(t, a) ~ p(· | x(t), a, b(t, a)) 

a_next = argmax_a[ŷ(t, 1), ..., ŷ(t, A)]

// frequentist update (e.g., HiLoFi)
b(t+1, a_next) = update(b(t, a_next), y(t, a_next))

This illustrates how Bayesian decision-making via the posterior predictive is combined with constant-cost frequentist updates.

Questions

R2.1 What is the benefit of combining frequentist Kalman-style updates with Bayesian predictive sampling?
The Kalman-style frequentist update provides a lightweight alternative to a posterior parameter update. Instead of tracking the full posterior, we use the predictive distribution directly for sampling-based decision making (Thompson-sampling style). This achieves efficient online learning without sacrificing exploration-exploitation quality.

This style of separating the update of model parameters from the model used to make predictions (the posterior predictive) has been called the martingale posterior [Holmes2023]. In the new version of the paper, we changed the title to "Martingale Posterior Neural Networks for Fast Sequential Decision Making", modified the introduction, and modified abstract to make this distinction more explicit.

R2.2 Why is the shift from uncertainty in parameters to uncertainty in predictions helpful for decision making?
Bayesian neural networks typically rely on priors and likelihoods that are misspecified or chosen for convenience [Knoblauch2022], which can degrade predictive performance [Wenzel2020] and make posterior sampling significantly slower than non-Bayesian alternatives like deep ensembles [Lakshminarayanan2017]. Moreover, these priors often involve subjective parameter distributions with no real-world interpretation [Holmes2023].

By shifting focus to predictive uncertainty, we bypass the need for such priors and instead work directly with the posterior predictive, which is a lower-dimensional and more tractable object for decision-making.

We introduced a new experiment on the MNIST bandit, where we observe that predictive sampling achieves comparable or better cumulative reward than traditional TS, while being up to 100x faster in high-dimensional models. We clarify this distinction and its practical benefits in the revised text.

The results are shown in the table below

R2.3 Why is Kalman filtering, especially the extended variant, a natural fit for online neural network training?
The (Extended) Kalman Filter is a form of second-order optimizer, equivalent to natural gradient descent. In practice, this means that each update is more sample-efficient than "rank-one" updates like, e.g., Adam
[Yann2018] [Khan2023] [Jones2024].

We clarify this in the main text.

R2.4 Why is replay buffer an issue in GP?
We do not claim replay buffers are inherently problematic. Instead, we emphasize that our method sidesteps this entirely. This is valuable in domains where data cannot be revisited (e.g., for privacy reasons [Tenenbaum2021]), where input distributions change over time (rendering stale data misleading [Chang2023]), or where the inputs are high-dimensional observations [Harrison2024].

We will make sure to remark this observation in the revised text.

[Knoblauch2022] Knoblauch, Jeremias, Jack Jewson, and Theodoros Damoulas. "An optimization-centric view on Bayes' rule: Reviewing and generalizing variational inference." Journal of Machine Learning Research 23.132 (2022): 1-109.
[Wenzel2020] Wenzel, Florian, et al. "How good is the bayes posterior in deep neural networks really?." arXiv preprint arXiv:2002.02405 (2020).
[Lakshminarayanan2017] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. "Simple and scalable predictive uncertainty estimation using deep ensembles." Advances in neural information processing systems 30 (2017)
[Yann2018] Ollivier, Yann. "Online natural gradient as a Kalman filter." (2018): 2930-2961.
[Khan2023] Khan, Mohammad Emtiyaz, and Håvard Rue. "The Bayesian learning rule." Journal of Machine Learning Research 24.281 (2023): 1-46.
[Jones2024] Jones, Matt, Peter Chang, and Kevin Murphy. "Bayesian online natural gradient (BONG)." Advances in Neural Information Processing Systems 37 (2024): 131104-131153.
[Tenenbaum2021] Tenenbaum, Jay, et al. "Differentially private multi-armed bandits in the shuffle model." Advances in Neural Information Processing Systems 34 (2021): 24956-24967.
[Chang2023] Chang, Peter G., et al. "Low-rank extended Kalman filtering for online learning of neural networks from streaming data." Conference on Lifelong Learning Agents. PMLR, 2023.
[Harrison2024] Harrison, James, John Willes, and Jasper Snoek. "Variational Bayesian last layers." arXiv preprint arXiv:2404.11599 (2024).
[Holmes2023] Holmes, Chris C., and Stephen G. Walker. "Statistical inference with exchangeability and martingales." Philosophical Transactions of the Royal Society A 381.2247 (2023): 20220143.
[Foong2019] Foong, Andrew YK, et al. "'In-Between'Uncertainty in Bayesian Neural Networks." arXiv preprint arXiv:1906.11537 (2019).

Thanks for explaining Markovian posterior and sharing the updated abstract.

However, I still have a few concerns.

1. From my perspective, the new title is less compelling than the previous one. The term Martingale posterior may not be widely recognized, and it’s unclear that it refers to a mechanism for uncertainty quantification and efficient belief updates.

2. I noticed that in the earlier abstract and introduction, you described your method as a combination of frequentist and Bayesian approaches. This framing seems to have been removed in the new version. Could you share your reasoning for the change?

3. In the first sentence of your new introduction, you state that "uncertainty quantification and efficient belief updates ... Bayesian approach is Thompson sampling". My understanding is that Thompson sampling is a decision-making strategy that relies on uncertainty quantification, rather than being a method for uncertainty quantification and efficient belief updates. Could you clarify why this change was made?

Thank you for your comment.

We believe your original question was about how Thompson Sampling (TS) is typically characterized in the literature, rather than about the specifics of our method:

"My understanding is that Thompson sampling is a decision-making strategy that relies on uncertainty quantification, rather than being a method for uncertainty quantification and efficient belief updates."

Our earlier response presented the classical Bayesian TS algorithm for bandits (not our own approach), and our point was simply that TS inherently involves both uncertainty quantification and belief updating. We would like to point out that this perspective is consistent with the standard literature. For example, Algorithm 4.2 in:

Russo et al. (2018), A Tutorial on Thompson Sampling, Foundations and Trends in Machine Learning, 11(1):1–96

explicitly includes all four steps: sampling, action selection, reward observation, and belief update of parameter distribution (belief update).

With regard to our method, we would like to emphasize that it is not classical TS (i.e., steps 1–4 above), but rather the variant described in our response R2.0. Specifically, our approach does not sample from the posterior over model parameters, but instead from the posterior predictive. The belief update step does not necessarily follow a Bayesian (prior-times-likelihood) rule; as you correctly noted, we use frequentist-style updates, such as those implemented in HiLoFi, LoLoFi, or LRKF. Finally, we show in a new experiment that our approach is significantly faster than classical Thompson Sampling, while achieving comparable or better performance. Please see the table in our response R2.2

We hope this resolves your concerns.

Right, my original question was about how Thompson Sampling is typically characterized in the literature and I also understand that your steps 1-4 is referring to the classical Bayesian TS. It's just I thought only steps 1-3 are usually considered part of TS while step 4 is not. That’s why I found the sentence in your revised introduction a bit confusing. I appreciate you sharing Daniel Russo’s tutorial to help clarify this.

My follow-up question—about how your method does not rely on step 4—was meant to ask whether your method also has a similar step 1-3 but without $\theta$, and also without step 4. This is the key point I would like to confirm. Could you also write the high-level idea of your method in a similar clean way?

Thank you for your review.

We believe the statement in your Strengths/Weaknesses section was left incomplete. That said, we hope our responses address your concerns.

To clarify novelty: we introduce a constant‑cost, Kalman‑style update that keeps a single theta estimate for a neural network and enables Bayesian decisions via the posterior predictive. Unlike prior cited DPM/iHMM work (i-iii above), we target tractable online neural‑learning and sequential decision‑making.

We appreciate the formatting suggestions. We will add a table of notation, remove numbering on comments in Algorithm 1, and use "parameter" only.

R1.1: Why Use a Bayesian Model If You Don’t Model Parameter Uncertainty?

We adopt the martingale posterior view [Holmes2023], in which the posterior predictive drives sequential decisions and the belief state is updated separately. Although classical TS samples parameters, it is ultimately outcome uncertainty that drives exploration. Our method captures this directly via the posterior predictive.

This new approach has several advantages:

• It enables Thompson sampling (TS)-style exploration without sampling from a high-dimensional posterior (which comes at a 10-100x speedup with comparable or increased performance. See R1.4)
• It supports plug-and-play non-Bayesian learning updates while retaining Bayesian decision logic, e.g., using HiLoFi.

As you mention, UCB and also Expected improvement are valid alternatives; we include an EI example in Figure 11 (Appendix).

We will emphasize this framing more clearly. Specifically:

• Rename the paper to "Martingale Posterior Neural Networks for Fast Sequential Decision Making",
• Update the abstract and introduction,
• Add a new pseudoalgorithm to illustrate the sampling + frequentist update loop.

For contextual bandits with one model per arm, the pseudoalgorithm is

// Define posterior predictive parameterised by belief state b
// and having inputs: the context ‘x’ and action ‘a’.
def p(· | x, a, b):
    ...

For each a in actions:
   // belief state:
   // need not define a valid posterior or consider two moments only
    b(t, a) = (mu(t, a), Sigma(t, a))

For all a in actions:
    // sample from Bayesian posterior predictive 
    ŷ(t, a) ~ p(· | x(t), a, b(t, a))

a_next = argmax_a[ŷ(t, 1), ..., ŷ(t, A)]

// frequentist update (e.g., HiLoFi)
b(t+1, a_next) = update(b(t, a_next), y(t, a_next))

R1.2 Do you assume the parameters at each time related in any way?

We do not use a hierarchical model for "borrowing strength". Instead, we assume that the parameters evolve according to a random walk, (Line 94), i.e., $\theta_t = \theta_{t-1} + u_t$, with $var(u_t) = q I_D$.

This assumption is common in adaptive filtering and online learning (see e.g. [Ismail1996] and [Chang2023]) and provides a simple way to ensure adaptivity and numerical stability.

In the special case where $q=0$, and without a low-rank assumption, we recover a static parameter model and our estimate $\theta_{t|t}$ is simply the Bayesian posterior mean given data up to time t, i.e., $\theta_{t|t} = E[\theta | y_{1:t}]$.

R1.3 How sensitive is the algorithm to misspecification of this dynamics?

The dynamics $\theta_t = \theta_{t-1} + u_t$ are misspecified by design. They are intended to serve as a regularisation mechanism for adaptivity and numerical stability, rather than reflecting the true data generating process. This dates back to, e.g., [Mehra1970].

To empirically assess sensitivity, we conducted a new experiment on the DrawNN Bayesian Optimization benchmark mentioned in the paper (using 600 steps). We varied both the hidden layer ($q_h$) and last-layer ($q_\ell$) dynamics for HiLoFi across several values and report the final maximum value found (averaged over 10 runs).

The results are robust. Some tuning helps but no catastrophic failure was observed. We’ll include these results in the revision.

R1.4 For the bandit experiments, did you compare the results with a usual TS?
Thank you for the suggestion.

Given an arm $a$, standard TS samples $\hat{\theta} \sim p(\theta | D_{1:t})$ and uses $\hat{y}\_{t+1} = f(\hat{\theta}, x\_{t+1}, a)$ to make decision. In predictive TS (pBayes), we sample directly from $\hat{y}_{t+1} \sim p(y |x\_{t+1}, \theta\_{t|t}, a)$. In the revised paper, we expand our MNIST bandit experiment to compare standard TS with pBayes approach.

We find that pBayes is 10-100x faster, than classical TS, while also giving comparative or higher cumulative reward. The speedup stems from sampling a low-dimensional predictive distribution instead of a high-dimensional parameter posterior, even when that posterior has a structured form (e.g., diagonal-plus-low-rank, as in LoFi).

The table below shows the results of comparing standard TS (LoFi, Laplace) and predictive TS (HiLoFi, LoFi, Laplace) on the MSNIST bandit task.

R1.5 How does your use of posterior predictive with point estimates differ from prior MAP-based approaches in the literature

The works suggested above (i-iii) use MAP‐plug‑in updates for nonparametric mixture models (iHMMs/DPMs) to accelerate posterior inference. Our approach departs in three key ways:

1. Those papers maintain a growing set of mixture components, whereas we keep and update only one global parameter vector $\theta_{t|t}$ that (possibly) evolves over time.
2. We target neural networks trained sequentially, with constant-time per update. These settings are not addressed by the cited DPM literature.
3. We tackle the problem of sequential decision making using so-called Martingale posteriors, which is not considered in cited works.

In the revised section, we reference each of the papers you mention and emphasize that our approach is a single-estimate parametric approach for online learning and sequential decision making.

R1.6 Can you elaborate on the roles of R, q(ell), and q(h), and how they should be tuned?

$R_t$ is the heteroskedastic noise process. Its purpose is to downweight the update information arising from natural variation of the observations y. This parameter can be estimated using an exponentially-weighted moving variance or in a burn-in period, as we do in the experiments.

The q-values, $q_\ell$ and $q_h$, are parameters for adaptation and stability:

• $q_\ell$ is the speed of adaptation for the last layer and
• $q_h$ is the speed of adaptation for the hidden layers.

As a rule of thumb: $q_h \leq q_\ell$ because, for a neural network, the capacity of the hidden layers is much greater than that of the last layer. In our experiments, these parameters are estimated during an initial phase or using alternative methods such as those in, e.g., [Titsias2023]

R1.7 Can the method be used in probabilistic models where the observation noise is not additive? Or its variance unknown (related to point 5)
Our method could be employed in parametric probabilistic models where the observation noise is not additive, however, these are better suited for sampling-based methods, similarly, for unknown variance, one can use methods such as [Agamennoni2012].

R1.8 Consistency of Proposition 4.1 when no low-rank structure is used
If no low-rank structure is used, then we would couple all layers and there is no need to use the "surrogate matrix" (shown in Equation (94), L787 in the Appendix) required for the low-rank projection. Hence, the approximation error would be zero (by definition).

R1.9 Can you say something about the estimation quality of the BLUP?
While little can be said about the true $\theta$, we can bound the estimation error by comparing a low-rank approximation (e.g., LRKF) against a full-rank update. In the linear regression case $y\_t = \theta^\intercal x\_t + e\_t$ with fixed $\theta$ and known $var(e\_t) = r$, let $\Sigma\_{t-1}$ be the full-rank covariance The one-step parameter difference between the BLUP using full-rank and using a rank-d update is upper bounded by: