Improved Variational Inference in Discrete VAEs using Error Correcting Codes

Reduced error rates in inference indicate a tighter posterior approximation

In this work, we conceptualize the process of inference via $q(\boldsymbol{m}|\boldsymbol{x})$ as decoding the latent variable $\boldsymbol{m}$ given the observed data $\boldsymbol{x}$. While VAEs are often viewed as lossy compression models that aim to minimize reconstruction error while regularizing the encoding with a prior, our contribution is best understood from a generative perspective. Here, we sample a vector $\boldsymbol{m}$, generate an image $\boldsymbol{x}$, and seek to minimize the error rate in recovering $\boldsymbol{m}$ from $\boldsymbol{x}$. Achieving this requires the variational approximation $q(\boldsymbol{m}|\boldsymbol{x})$ to closely align with the true posterior. Estimating $\boldsymbol{m}$ from $\boldsymbol{x}$ thus involves approximating the true posterior $p(\boldsymbol{m}|\boldsymbol{x})$ with $q(\boldsymbol{m}|\boldsymbol{x})$. The difference between $q(\boldsymbol{m}|\boldsymbol{x})$ and $p(\boldsymbol{m}|\boldsymbol{x})$ defines the variational bound, representing the “noise” in this process. By introducing structured redundancy through the ECC, we improve the posterior $q(\boldsymbol{m}|\boldsymbol{x})$, thereby reducing the variational gap. Our results show that this approach decreases error rates (as illustrated in Table in Figure 4) and enhances model performance overall. It's worth noting that the optimal decoder (the one that minimizes the error rate in the estimation of $\boldsymbol{m}$) is the one that maximizes $p(\boldsymbol{m}|\boldsymbol{x})$; any other decoder will yield higher error rates. Thus, a reduced bit error rate (BER) in inference indicates a closer approximation to the true posterior [1].

I still lack a qualitative understanding of why this encoding approach leads to improved performance.

Following our previous comment, the key behind our method is that we exploit the known correlations introduced by the ECC in inference to refine the output of the encoder NN $g_{\boldsymbol{\eta}}(\boldsymbol{x})$, obtaining a better approximation to the posterior. The reason why this works, intuitively, is that once we introduce the ECC, the resulting codewords form an $m$-dimensional subspace of the larger $n$-dimensional space in which any two codewords $\boldsymbol{c}$ in the latent space are now separated by at least some minimum distance (i.e., it is a sphere packing). This structure, where $2^{n-m}$ of $n$-tuples are not valid codewords, allows a decoder to tolerate errors and attempt to identify the most likely codeword. For example in the case of a 5/50 repetition code, only $2^5$ of the $2^{50}$ possible combinations in the 50-dimensional space are valid (that is why introducing redundancy does not increase flexibility, since the number of available vectors remains the same). Here, the minimum distance is 50 and this large separation of codeword symbols in the latent space provides significant error correcting capability and coding gain (defined in information theory as the reduction in signal power for a target error rate over a system without coding). Indeed, given a specific channel and codeword weight distribution, one can theoretically characterize the probability of failure of the optimal decoder which must, in turn, reduce as a function of increasing minimum distance/improving weight spectrum [1].

Given an observation $\boldsymbol{x}$, ECC decoding algorithms can efficiently (attempt to) find the codeword $\boldsymbol{c}$ to maximize the probability of $p(\boldsymbol{c}|\boldsymbol{x})$ over all $2^m$ codewords and thereby maximize the probability $p(\boldsymbol{m}|\\boldsymbol{x})$ of the estimate of $\boldsymbol{m}$, since $\boldsymbol{c}$ and $\boldsymbol{m}$ are deterministically and injectively related. Specifically, for repetition codes, error detection and correction can be achieved through a straightforward majority voting approach. Since each information bit is repeated $L$ times, the original value can be recovered by selecting the most frequent value among the repetitions. In our method, in the case of repetition codes, we employ a soft majority voting strategy during inference to compute the posterior probability of each information bit in a theoretically optimal (MAP) way. This is done by evaluating the all-ones and all-zeros probabilities for the $L$ repeated bits. It is important to note that, while our paper primarily focuses on repetition codes for simplicity, the proposed method can be extended to more advanced coding schemes, as discussed in Section 6. Transitioning to more robust error-correcting codes has the potential to further enhance performance, as indicated by our preliminary results using polar codes.

If you can provide theoretical insights or quantitatively demonstrate the advantages of your approach, I will reconsider my score.

As mentioned in our previous response, it is a well-established result in information theory that the optimal decoder, which minimizes the error rate in estimating $\boldsymbol{m}$, is the one that maximizes $p(\boldsymbol{m}|\boldsymbol{x})$. Any other estimator will result in higher error rates [1]. Employing this mechanism here helps to improve the estimate of the posterior $q(\boldsymbol{m}|\boldsymbol{x})$. Estimating $\boldsymbol{m}$ from $\boldsymbol{x}$ involves approximating the true posterior $p(\boldsymbol{m}|\boldsymbol{x})$ distribution with $q(\boldsymbol{m}|\boldsymbol{x})$, our variational proposal. The closer these distributions are, the fewer errors we make when comparing $\boldsymbol{m}$ with samples from $q(\boldsymbol{m}|\boldsymbol{x})$. The gap between $q(\boldsymbol{m}|\boldsymbol{x})$ and $p(\boldsymbol{m}|\boldsymbol{x})$ is precisely the variational bound and can be considered as the “channel noise”. The structured redundancy introduced by the ECC, with large minimum distance, is exploited to improve the posterior $q(\boldsymbol{m}|\boldsymbol{x})$ and therefore reduce the variational gap to the true posterior. Our results confirm that this approach reduces error rates (see table included in Figure 4, for example), improves log-likelihoods (refer to the table in Figure 4), narrows the variational gap (see the table included in the previous comment), and enhances overall model performance.

Even without a theoretical basis, it is essential to experimentally analyze how encoding influence the distribution of the latent space and how this influence enables high accuracy to be achieved with a smaller number of trainable parameters.

We have evaluated how the introduction of ECCs influences the posterior distribution in our experiments. One key result is the evaluation of the word and bit error rates. In this experiment, we sample a number of information vectors $\boldsymbol{m}$, feed them to the decoder NN to obtain the corresponding generated images, and map these images back to the latent space. A better approximation of the posterior distribution leads to reduced error rates [1], which we see in coded models in comparison to uncoded ones.

We further analyzed the posterior distribution over the latent space through semantic accuracy and posterior uncertainty calibration experiments, focusing on two metrics: Semantic Accuracy and Confident Semantic Accuracy. Semantic Accuracy evaluates whether models successfully reconstruct images belonging to the same class as the original ones. Confident Semantic Accuracy applies the same evaluation but only considers cases where images are projected into a latent vector with a posterior probability exceeding 0.4, excluding errors when the MAP value of $q(\boldsymbol{m}|\boldsymbol{x})$ is below this threshold. In both metrics, coded models consistently outperformed uncoded ones, indicating that the posterior is more likely to map images to latent vectors of the correct semantic class. This improvement is also reflected in better reconstruction performance measured in PSNR.

We also calculated the mean entropy of the posterior distribution over the latent vectors and provided examples of incorrect reconstructions on the FMNIST and MNIST datasets. These examples included the four most probable a posteriori latent vectors alongside their posterior probabilities. Our observations showed that uncoded models often map data to incorrect class vectors with high confidence, whereas coded models exhibit greater uncertainty in such cases. This is further supported by the mean entropy across the dataset: uncoded models present low posterior uncertainty, confidently projecting data into the latent space, while coded models exhibit higher entropy, suggesting that projections are made with greater uncertainty. However, as shown in the Confident Semantic Accuracy results, when coded models project data with lower uncertainty, they are more likely to map it to a vector belonging to the correct semantic class (as evidenced by the consistent improvement of Confident Semantic Accuracy over Semantic Accuracy).

Final comment

We look forward to your feedback. If our explanations require further clarification, please let us know—we would be happy to provide further details if needed. We will also be available over the weekend to address any questions.

References

[1] Richardson, T., & Urbanke, R. (2008). Modern Coding Theory. Cambridge University Press.

Thank you for your feedback! We will address your concerns by responding to each comment individually.

What I am concerned about is that it is unclear whether the reduction of the variational gap is a theoretical result or an experimental one. If it is a theoretical result, I would appreciate it if you could include the proof in the appendix or elsewhere in the revised version. If it is an experimental result, I believe it should be explicitly stated as an empirical finding.

Our approach is definitely grounded in a theoretical base. In information theory, it is well established that “Maximum a Posteriori (MAP) decoding is the minimizer of the probability of error if all input vectors are equally probable” (we refer to Chapter 1.5 in [1]). This means that the solution that minimizes the error rate in estimating $\boldsymbol{m}$ from $\boldsymbol{x}$ is the one that maximizes $p(\boldsymbol{m}|\boldsymbol{x})$ w.r.t. $\boldsymbol{m}$. In variational inference, we do not have access to the true posterior $p(\boldsymbol{m}|\boldsymbol{x})$ but to $q(\boldsymbol{m}|\boldsymbol{x})$, which we construct using the ECC constraints on top of the encoder NN output $g_{\boldsymbol{\eta}}(\boldsymbol{x})$.

Consequently, a lower bit error rate (BER) in inference corresponds to a better approximation of the true posterior. While this result is well established in information theory, to the best of our knowledge, this is the first time it has been applied within the VAE framework. To validate our approach, we not only report lower BER values in the table presented in Figure 4 but also demonstrate a smaller gap to capacity, as previously shown to Reviewer 82yp. For completeness, we include the gap table here and will integrate it into the paper.

We would like to remind the reviewer that we have already addressed the connection between our experimental results and the underlying mechanisms by explaining the theoretical foundations and intuition behind our method, which are grounded in well-established results from information theory.

At this stage of the discussion period, it is not feasible to provide additional mathematical derivations or ablation studies; however, we will further elaborate on the intuition addressing the concerns raised by the reviewer.

Why does posterior collapse not occur?: Could you provide a clear explanation, either mathematically or through ablation studies, of the factors that prevent posterior collapse in your method?

Posterior collapse is a well-studied phenomenon in the VAE literature, though we would like to point out that it has been primarily examined in the context of continuous latent spaces. Several factors are known to contribute to posterior collapse, including the presence of local minima in the training objective and the non-identifiability of the latent space. Of particular relevance to our scenario, [1] highlights how an inexact variational approximation can lead to coding inefficiencies in VAEs, potentially resulting in posterior collapse due to a form of information preference. Building on this intuition, our method mitigates this issue by providing a tighter posterior approximation, leveraging the known structure of the ECC introduced. Note that our uncertainty calibration results demonstrate that the posterior is more open and exhibits higher variance in cases of poor reconstruction, which is a clear indication that posterior collapse is not occurring.

Why is the variational gap better than IWAE? Is it possible to formally demonstrate why your approach reduces the variational gap more effectively than IWAE? A mathematical derivation or theoretical insight would strengthen your claims.

The two approaches are fundamentally different. In IWAE, independent samples are drawn from the variational posterior and passed independently through the generative model to achieve a tighter variational bound on the marginal log-likelihood, which is then optimized. The motivation behind this approach is that the standard ELBO penalizes posterior samples that fail to explain the observations, imposing strict constraints on the model (the posterior distribution must be approximately factorial and predictable with a neural network). By relaxing these constraints, IWAE provides additional flexibility to train generative models whose posterior distributions may deviate from standard VAE assumptions.

In contrast, our method jointly propagates the output of the ECC encoder through the generative model to produce a single prediction while leveraging the introduced known correlations in the variational posterior approximation. For repetition codes, the ECC encoder outputs repeated bits (or repeated probabilities in the case of soft encoding). However, our approach extends far beyond repetition codes, paving the way for improved inference in discrete latent variable models. Unlike IWAE, we focus on enhancing the variational proposal while adhering to the VAE assumptions.

Specifically, we utilize the redundancy introduced by repetition codes to correct potential errors made by the encoder via a soft decoding approach. This leads to a more accurate approximation of $p(\boldsymbol{m}∣\boldsymbol{x})$ and an improved sampling proposal. Notably, the performance improvements achieved with our coded DVAE approach cannot be replicated by training the uncoded DVAE with the IWAE objective.

Furthermore, the hierarchical example in Section 6 demonstrates the model's ability to separate features at different levels of granularity: low-level features (e.g., class) are captured in the most protected latent space, while high-level features are represented in the least protected space. This demonstrates how, through a deterministic and low-complexity coding design, we can effectively tackle the design of hierarchical discrete deep generative models—an open challenge in the state-of-the-art.

Finally, we would like to point out that in the initial assessment, the reviewer considered our contribution excellent (4) and that the paper comprehensively evaluates coded and uncoded models. We have thoroughly addressed all the concerns raised by the reviewer, offering detailed explanations of the theoretical foundations of our method and how it promotes overall performance improvements. Therefore, we respectfully believe the current score does not reflect the merit of our work.

References

[1] Chen, X., Kingma, D. P., Salimans, T., Duan, Y., Dhariwal, P., Schulman, J., ... & Abbeel, P. (2017). Variational Lossy Autoencoder. In International Conference on Learning Representations.

Baselines

Numerous factors affect the tightness of the variational approximation, including the complexity of the true posterior, the selection of the variational family, the architectures used to parameterize the distributions, and the sampling techniques employed to evaluate the ELBO objective. Consequently, various approaches have been proposed in the literature to address these factors and reduce the gap between the variational and true posterior distributions in both continuous and discrete settings.

One popular approach to improve inference in VAEs is to define flexible priors that better align with the posterior, alleviating problems such as posterior collapse or the hole problem [1]. An example of this is the VQ-VAE [2], which fits an autoregressive model to train a prior that aligns with the inferred aggregated posterior; the VampPrior [3], where the prior is defined as a mixture of variational posteriors conditioned on learnable pseudo-inputs; or the Variational Lossy Autoencoder [4], where the authors employ an autoregressive invertible flow as prior. In this work, we introduce an alternative approach that, to the best of our knowledge, has not been previously explored in the literature. This paper aims to demonstrate that incorporating ECCs into the generative pathway can enhance inference in discrete VAEs. Since we do not tailor our solution to any specific prior distribution, this approach is fully compatible with state-of-the-art methods that utilize complex priors, such as those previously mentioned. However, we chose to focus on highlighting the significant performance improvements achievable through the introduction of ECCs alone, using a simple, fixed independent prior, which can also aid in obtaining more interpretable latent spaces. While the coded models demonstrate substantial improvements over their uncoded counterparts, we acknowledge that our results may not be directly comparable to those achieved with VQ-VAE or other advanced methods for discrete latent representations. However, establishing such comparisons is not the primary aim of this work.

We remark that even using an independent prior, the hierarchical code structure outlined in Section 6 naturally decouples information across different latent spaces at various conceptual levels. Specifically, the most relevant information (class label) is encoded by the most protected latent space, while the other space captures fine-grained features. This effect is not straightforward to enforce by direct design of a more complex prior. In fact, the DVAE [5], DVAE++ [6], and DVAE# [7] consider complex and expressive Boltzmann Machine priors coupled with a hierarchy of continuous variables modeled in an autoregressive manner to model global and local factors of variation. However, we consider that conducting an extensive comparison in terms of interpretability or the ability to decouple high- and low-level information is beyond the scope of this paper.

VQ-VAEs [2] stand out as state-of-the-art discrete latent space models, and we believe it would be valuable to highlight the differences between the two approaches. A primary distinction lies in the latent space’s structure and dimensionality. In image modeling, VQ-VAE typically employs a latent matrix where indices correspond to codewords in a codebook. This setup allows each latent code to explain a different patch of the original image, enhancing reconstruction and generation performance (the latter after training an autoregressive prior on the latent representations). However, this structure can complicate interpretability, as each data point is represented by a matrix of embeddings. Instead, our method projects all the information into a single low-dimensional vector $\boldsymbol{m}$, with the consequent information loss. Another important difference is that VQ-VAE is based on a non-probabilistic encoder, where the output is mapped to the nearest latent code based on a distance metric. This deterministic mapping limits the model's ability to quantify uncertainty in the latent space. In contrast, our method uses a fully probabilistic framework that enables uncertainty quantification in the latent representations. Moreover, all operations in our method are differentiable, enabling seamless computation and backpropagation of gradients, allowing for an end-to-end training of the model (in contrast to the two-stage training of the VQ-VAE). This also lets us use the reparameterization trick introduced in the DVAE++ [6], which we found to be more stable than continuous relaxations like the Gumbel-Softmax, used in VQ-VAEs with stochastic quantization [8].

Effective learning of low-dimensional discrete latent representations is a technically challenging problem. In this work, we propose a novel method to improve inference in discrete VAEs within a fully probabilistic framework, introducing a new perspective on the inference problem. We understand the connection that the reviewer suggests between our work and papers in the literature that analyze VAEs using rate-distortion (RD) theory. However, our approach stems from a different perspective, that is indeed compatible with all those works.

While VAEs are often viewed as lossy compression models (e.g. VAEs as source coding methods), our contribution is best understood from a generative perspective. We conceptualize the process of inference via $q(\boldsymbol{m}|\boldsymbol{x})$ as decoding the latent variable $\boldsymbol{m}$ given the observed data $\boldsymbol{x}$. We sample a vector $\boldsymbol{m}$, generate an image $\boldsymbol{x}$, and seek to minimize the error rate in recovering $\boldsymbol{m}$ from $\boldsymbol{x}$. Achieving this requires the variational approximation $q(\boldsymbol{m}|\boldsymbol{x})$ to closely align with the true posterior. Estimating $\boldsymbol{m}$ from $\boldsymbol{x}$ thus involves approximating the true posterior $p(\boldsymbol{m}|\boldsymbol{x})$ with $q(\boldsymbol{m}|\boldsymbol{x})$. We show that introducing redundancy in generation reduces error rates in estimating $\boldsymbol{m}$ and, accordingly, constructs better approximations to the latent posterior distribution. This is reflected in higher log-likelihoods, a smaller variational gap (see response to Reviewer 82yp), and enhanced generation performance.

VQ-VAEs [1] are notable for effectively learning discrete representations of data, and we believe it is useful to highlight key differences between VQ-VAEs and our approach. A primary distinction lies in the latent space’s structure and dimensionality. In image modeling, VQ-VAE typically employs a latent matrix where indices correspond to codewords in a codebook, allowing each codeword to capture specific patches of the original image, which enhances reconstruction and generation (the latter after training an autoregressive prior on the latent representations). However, this matrix representation complicates interpretability, as each data point is represented by a grid of embeddings. In our approach, the latent space encodes the whole image, instead of specific patches. Another important difference is that VQ-VAE is based on a non-probabilistic encoder, where the output is mapped to the nearest latent code based on a distance metric. This deterministic mapping limits the model's ability to quantify uncertainty in the latent space. In contrast, our method uses a fully probabilistic framework that enables uncertainty quantification in the latent representations. Moreover, all operations in our method are differentiable, enabling seamless computation and backpropagation of gradients, allowing for an end-to-end training of the model. This also lets us use the reparameterization trick introduced in the DVAE++ [2], which we found to be more stable than continuous relaxations like the Gumbel-Softmax, used in VQ-VAEs with stochastic quantization [3].

In RD theory, the focus is on compression within the latent space, typically analyzed from an encoder/decoder and reconstruction (distortion) perspective. RD theory sets theoretical limits on achievable compression rates and describes how practical models may diverge from these limits. Using RD practical compression methods, [4] demonstrates how asymmetric numeral systems (ANS) can be integrated with a VAE to improve its compression rate by jointly encoding sequences of data points, bringing performance closer to RD theoretical limits. Similarly, [5] shows that a complex prior distribution in a VAE using an autoregressive invertible flow narrows the gap between the approximate posterior and the true posterior, thereby enhancing the overall performance of the VAE. We remark that even using an independent prior, the hierarchical code structure outlined in Section 6 naturally decouples information across different latent spaces at various conceptual levels. Specifically, the most relevant information (class label) is encoded by the most protected latent space, while the other space captures fine-grained features. This effect is not straightforward to enforce by direct design of a more complex prior.

Clarification on Section 4

The coded DVAE extends the DVAE by introducing an ECC to encode the binary variable $\boldsymbol{m}$ (the information vector). The resulting codeword $\boldsymbol{c}$ is indeed a linear transformation of $\boldsymbol{m}$ as you note, computed by multiplying $\boldsymbol{m}$ with a known generator matrix $G$. This approach augments the dimensionality of the binary latent variable in a controlled and deterministic manner. During generation, we sample $\boldsymbol{m}$ and transform it into $\boldsymbol{c}$ using $G$ (refer to Fig.39 in Appendix N for the scheme). This process, known as hard encoding, is the method we use to generate new samples once the model is trained, and all presented generation results were obtained with this approach. The key aspect of our proposal is that the variational posterior approximation exploits this knowledge (i.e. certain elements of the vector must be identical) to improve inference and enhance the training of the whole model. See the “Intuition behind our approach” section below.

However, during training, soft encoding is required for reparameterization. To compute the reconstruction term of the ELBO, given a data point $\boldsymbol{x}$, a marginal posterior probability $q(m_i)$ is obtained for each latent bit $m_i$, with $i=1,..., M$. To evaluate the datum log-likelihood under this posterior, soft encoding produces a marginal probability $q(c_j)$ for each bit in the codeword, $j=1,..., M/R$, incorporating the structure of the ECC. These coded marginals are combined with the reparameterization trick in Eq.(5). In this way, marginal posterior probabilities $q(c_j)$ are directly used to sample $\boldsymbol{z}$, just as in the uncoded scenario. This method allows us to avoid the instability of alternative approaches such as the Gumbel-Softmax relaxation or the REINFORCE gradient estimator. Since the operations in the reparameterization trick and computation of the marginal posteriors are differentiable, the model can be trained via SGD.

When we say, "When comparing uncoded vs. coded DVAEs, the structure of the decoder NN is equal in both cases except for the first Multilayer Perceptron (MLP) layer that attacks the input $\boldsymbol{z}$," we mean that the decoder architecture is identical in both cases, except for the input layer, which processes the input $\boldsymbol{z}$ and therefore needs to match its dimensionality. The misunderstanding stems from wording; we’ll use clearer terminology to prevent confusion.

We have conducted a comprehensive empirical evaluation of our method, demonstrating that the tighter bound observed during training is not a result of overfitting. While we do not claim that our method directly improves generalizability or provides implicit regularization, we assert that it enhances inference narrowing the gap between the true and approximate posterior, which in turn leads to an overall improvement in performance. This is consistent with the general trends observed in the VAE literature (we would appreciate it if the Reviewers could provide any references that suggest otherwise). Additionally, we observe improvements in both the training and test sets (with training metrics provided in the Appendix), which indicate an overall enhancement in model fitting. We have evaluated not only the model's reconstruction accuracy (using metrics such as PSNR, Semantic Accuracy, and Confident Semantic Accuracy) but also its generation performance (through visual examples and FID) and the tightness of the variational posterior (measured by the variational gap, log-likelihood, BER, and WER). All results are consistent across different datasets and model configurations, providing empirical evidence that clearly supports our claims (as the Reviewers have acknowledged).

We believe this contribution is valuable and worth sharing with the community. This seminal work (supported by theoretical foundations and empirical evidence) opens the door to further theoretical analysis, which is beyond the scope of this paper, and offers a novel perspective on the inference problem that could inspire future research. In light of this, we respectfully believe that a score of ‘marginally below acceptance threshold’ would not adequately reflect the merit of this contribution, as it would likely result in the rejection of our submission.

Since there is still time until the discussion period ends, we would appreciate it if the reviewers could specify what actions we could take to encourage them to consider raising their scores.

References

[1] Tomczak, J., & Welling, M. (2018, March). VAE with a VampPrior. In International Conference on Artificial Intelligence and Statistics (pp. 1214-1223). PMLR.

[2] Vahdat, A., Macready, W., Bian, Z., Khoshaman, A., & Andriyash, E. (2018, July). Dvae++: Discrete Variational Autoencoders with Overlapping Transformations. In International Conference on Machine Learning (pp. 5035-5044). PMLR.

[3] Van Den Oord, A., & Vinyals, O. (2017). Neural Discrete Representation Learning. Advances in Neural Information Processing Systems, 30.

[4] Gatopoulos, I., & Tomczak, J. M. (2021). Self-Supervised Variational Auto-Encoders. Entropy, 23(6), 747.

[5] Wehenkel, A., & Louppe, G. (2021). Diffusion Priors In Variational Autoencoders. In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models.

[6] Sønderby, C. K., Raiko, T., Maaløe, L., Sønderby, S. K., & Winther, O. (2016). Ladder Variational autoencoders. Advances in Neural Information Processing Systems, 29.

[7] Rezende, D., & Mohamed, S. (2015, June). Variational Inference with Normalizing Flows. In International Conference on Machine Learning (pp. 1530-1538). PMLR.

[8] Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov (2016). Importance Weighted Autoencoders. In 4th International Conference on Learning Representations (ICLR).