Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm

We thank the reviewer for the time and valuable feedback.

would have liked to see the classification accuracy of the model on MNIST (as a percent). Predictive coding networks tend not to perform well on classification tasks.

It is true that, when used as generative models, predictive coding networks do not perform well on image classification tasks. However, when used in a discriminative direction (where the goal is to generate the label, providing the image as a very precise (Delta) prior on the first layer), PC has been shown to perform as well as backprop in complex image classification tasks:

https://arxiv.org/abs/2407.01163

However, the study of discriminative PCNs is not the focus of our work, and we have hence not considered it in our tests.

Typos

Thank you for the pointers, they have now been addressed in the final version of the manuscript.

We thank the reviewer for the time and valuable feedback.

Novelty over previous work (Kuntz 2023;2024, Naesseth 2015)

Kuntz 2023 did not construct or evaluate importance weights, while Lindsten 2017/Kuntz 2024 and Naesseth 2015 did not propose a unique decomposition of a target model into Gibbs kernels as target densities for samplers, nor did they employ gradient-based proposals. Our DCPC algorithm starts from a neuroscientific motivation to derive a new sampling algorithm: first approximate the true Gibbs kernels using gradient-based proposals and SMC, then take the nested D&C-SMC step “up” to calculate importance weights for the entire joint density, then take the pathwise derivative of their negative logarithm to estimate gradients of the complete generative model’s free-energy. Finally, as far as we know, ours is the first sampling-based predictive coding algorithm to compute a free energy that properly upper-bounds the surprisal of the sensory data, as required by the Free Energy Principle and achieved in neuronal message-passing proposals.

Is the mapping to the cortical microcircuit and other neurological constraints supposed to be the (or a) main result of the MS? These are certainly intriguing but they are not very constraining. Can the authors make predictions for neuroscience experimentalists based on their mappings? Can they explain empirical findings in the literature?

The mapping to the cortical microcircuit is not intended to be the main result of the manuscript, only a suggestion. The primary contribution is the algorithm itself, which should stand or fall on its own.

The authors propose cortical oscillations as the mechanism for synchronizing a parallel form of Gibbs sampling. At what frequency do they hypothesize these oscillations to be? Given a reasonable number of Gibbs sweeps, does this let the computation happen fast enough?

The γ-band of cortical oscillations take place at 30–150 Hz and tend to correlate with bottom-up processing of oddball or deviant stimuli, in contrast to the top-down alpha/beta band in the 8–30 Hz range. “High gamma” past 50 Hz is often more associated with “prediction error”. Without having the experimental evidence to break down gamma more finely, the oscillation frequency band we can associate with “prediction error” is faster than the frequency band associated with “prediction”. Picking the median of each frequency band, median alpha/beta would be 19 Hz while median gamma would be 90 Hz, more than 4x faster. We do suggest that if inference takes place 4x faster than generative posterior prediction, with a reasonably efficient inference algorithm, inference can converge fast enough.

More generally, can the authors give some more detail on the time complexity of the algorithm and how they would expect it to scale to problems with (say) real graph structure?

Abstracting over the time complexity of automatic differentiation and SMC, our algorithm has linear asymptotic complexity in the number of nodes in a graph, and then multiplicative factors in the number of sweeps and the total number of inference steps in the training loop. Further asymptotic guarantees would require details of model structure (ie: log-concavity, etc).

The results are very thin/missing.

We have now updated the manuscript, and added multiple, generative experiments on the Celeb64 dataset. Note that, while the results are not as good as the ones of modern generative models, they perform as well (and sometimes outperform) these of bio-plausible learning algorithms, such as other predictive coding networks such as Oliver 2024: https://www.biorxiv.org/content/10.1101/2024.02.29.581455v1.full and the particle gradient descent antecedent to ours. For more details about the experiments, we refer to the paragraph Numerical and Figure Results on CelebA, provided as a general answer to all the reviewers.

Is it necessary to write the free energy in terms of weights? Why not just let the recognition model be the product over the normalized "complete conditionals" (with the normalizers computed with Langevin dynamics)?

It is indeed mathematically necessary to write the free energy in terms of importance weights targeting the generative joint density. By definition, a variational free energy is the expected value of the negative logarithm of an importance weight. Equivalently, the VFE is the cross-entropy of the generative joint distribution, taken with respect to the proposal distribution (recognition model), minus the entropy of the recognition model. The product of normalized complete conditionals (with the normalizers estimated by importance sampling) would only estimate the entropy of the recognition model, the second term.

Typos

Thank you for the pointers, they have now been addressed in the final version of the manuscript.

We thank the reviewer for the time and valuable feedback.

Does vanilla PC really require Gaussian densities and the Laplace approximation (Table 1)? My understanding is that the framework itself doesn't require this; rather, this is just a useful assumption people use to get something simple and workable. I could be wrong, though.

We searched the literature for a definition that would clarify whether vanilla PC requires or merely uses the Laplace approximation and Gaussian densities. The definition we found in Salvatori et al 2023, now included and discussed in the manuscript, says that they are required. On a somewhat more in-depth level, Bogacz 2017 and a number of papers by Friston appear to imply that the Free Energy Principle literature first constructs the VFE/ELBO as is typically done in machine learning, and then applies the Laplace approximation and Jensen’s Inequality to the VFE itself. The resulting energy function provides a looser bound on the model evidence than the usual ELBO in machine learning, but (seemingly vitally to this earlier work) saves the resulting PC schemes from having to perform any Monte Carlo computations. Our use of up-to-date SMC and variational inference methods to construct our VFE/ELBO and evaluate its gradients appear a required ingredient for relaxing PC’s Laplace assumption.

The authors really mean local computations when they talk about biological plausibility … But biologically plausible neural computations involve neurons, and hence I think it is reasonable to expect that there is some discussion of whether or not neurons can implement the computations required.

We agree that the term “bio-plausibility” has been loosely adopted in the literature, often indeed to refer to purely local computations. In our case, we consider our list of minimal properties to consist of local computations and the lack of a global control or synchronization signal. As much of the field refers to predictive coding models as biologically plausible, we follow the mainstream view and keep this terminology for our variation upon PC. We have also added a discussion about biological plausibility in the manuscript that specifies where we are referring to local computations, as well as references to a number of papers showing that neurons across a variety of model organisms (up to and including primate cortex) can calculate derivatives of logarithms of underlying signals. We thank the reviewer for the opportunity to go into detail on an underappreciated question in the study of predictive coding algorithms.

There are some issues related to training time and speed. Can plots analyzing these features be included?

Yes. Given some time we can retrieve that data from the training logs and plot it. At an approximate estimate, since PPC has to load and unload per-batch particles from GPU memory at every iteration, it takes time to train that is more on par with minibatched mean-field variational approaches than amortized variational approaches, approximately 4x the time of amortized. This gap may well shorten when working with “full” datasets instead of minibatching, as in cognitive modeling rather than deep learning tasks.

Can the authors briefly summarize the 'new' features of their algorithm relative to other PC ideas?

Relative to contemporaries Langevin Predictive Coding and Monte Carlo Predictive Coding, to which we compare, we build a Gibbs sampling method out of the interpretation of predictive coding as gradient-based sampling. Relative to Pinchetti 2022, the first to generalize PC beyond Gaussian distributions and the Laplace assumption, we optimize a globally valid free energy/evidence lower bound, rather than a layer-wise one. Relative to all previous PC methods, we optimize a tighter bound on the model evidence (by using Monte Carlo sampling instead of applying Jensen’s Inequality a second time under the Laplace assumption) and support approximate posterior distributions with no closed form at all, well beyond Gaussian approximate posteriors.

Why particle-based gradient descent as opposed to some other approach to empirical Bayes?

Our motivation for choosing a gradient-based sampling method comes from our analogy between score functions in probability and prediction errors in neuroscience. We chose PGD in specific because it was a recent gradient-based particle method, and we chose particle methods out of a combination of neuroscientific and behavioral motivations: brains have been observed to solve non-conjugate Bayesian inference tasks. Also, compared to plain variational inference, both mean-field and amortized, particle methods achieve better likelihoods. Tentatively, as one of our VAE baselines runs, it appears to be converging to a log-likelihood two orders of magnitude (10^2) away from what PPC achieves on the same experiment. We can give similar gaps for unpublished experiments.

Concretely, how might neurons learn log-density gradients?

We have included the below text in our revised manuscript: Biological neurons often spike to represent changes in their membrane voltage, and some have even been tested and found to signal the temporal derivative of the logarithm of an underlying signal. Theorists have also proposed models under which single neurons could calculate gradients internally. In short, if neuronal circuits can represent probability densities, as many theoretical proposals and experiments suggest they can, then they can likely also calculate the prediction errors used in DCPC.

Is PPC strictly better than other PC algorithms, or is there some kind of tradeoff?

There are two trade-offs: classical PC algorithms, being deterministic, can run more quickly than PPC/DCPC, and yet for the same reason, PPC/DCPC gives a far more expressive approximation to the true posterior distribution. This is the bullet we bite using particle algorithms and empirical distributions/particle clouds to represent the posterior.

We thank the reviewer for the time and valuable feedback.

The technical sections (2 and 3) are complex.

Thank you for the pointer. We have heavily modified the explanations, restructuring the sections and adding sentences that lead to a more intuitive understanding of the algorithm. For more details, we refer to the general answer provided above, and to the updated manuscript present in the anonymous link.

Have you considered using cleanfid instead of pytorch-fid to obtain more stable FID score estimates?

We will add cleanfid to our calculations alongside our existing FID evaluator/implementation, for which we have been using torchmetrics by Lightning.ai.

It is great that you implemented it all in the Pyro framework. However, the submission seems to be lacking an (anonymous) code attachment. Will you make the code publicly available with instructions and all relevant information for reproducing your experiments?

Yes. We have anonymized our Github repository; please share and enjoy. Our training code uses the Lightning framework and our evaluations consist of running a Jupyter notebook all the way through. We will make the code publicly available and include a README.md specifically listing the combination of shell-command (for training) and notebook (for evaluation) to produce each and every figure and numerical result in the paper.

Dear reviewers,

Thank you again for your time and comments, and for having been amenable to discussion during the rebuttal period. This message is to communicate a nice improvement in our experimental evaluation: in the last days, we have also run an apples-to-apples comparison against the Langevin PC work [1], using the same model, and, more importantly, the discretized Gaussian function as a likelihood function, that they do. The results show that the discretized Gaussian largely improves the performance in our case as well. In terms of results, the authors report three different numbers for three different methods: prior sampling, and two amortized warm-start techniques, the best one using Jeffrey’s Divergence. However, we decided to not implement warm-start techniques, as they are not biologically plausible in the sense that we mean in our work, and hence only perform prior sampling. Despite that, D&C-PC achieves a FID score of 96.0473 (+/- 0.2667), that is (1) much better than the one of 120 they report for prior sampling, in the apples-to-apples comparison; and (2) better than both of the method that use warm starts, where they report a FID score on the test set of 97.49 (Table 1 in [1]).

We can thus claim that D&C PC is the variation of PC that achieves the best results in the literature when it comes to image generation tasks, even when compared against less bio-plausible PC methods. We have updated the manuscript accordingly. A second, minor, change that we have applied to the manuscript, is to write the free energy in terms of a product of complete conditionals as the recognition model Q and then consider estimating it by Monte Carlo with the whole Langevin to nested SMC procedure, as suggested by reviewer DHBv. We agree that this would further improve the clarity of our work.

We once again deeply thank the reviewers for their close engagement that has done so much to improve the paper.

[1] Zahid, U., Guo, Q. and Fountas, Z., Sample as you Infer: Predictive Coding with Langevin Dynamics. In Forty-first International Conference on Machine Learning, ICML 2024.