This portal has been archived. Explore the next generation of this technology.
Note on the bias and variance of variational inference
lib:84a76ba0a9a5400b (v1.0.0)
Authors: Chin-Wei Huang,Aaron Courville
ArXiv: 1906.03708
Document: PDF DOI
Artifact development version: GitHub
Abstract URL: https://arxiv.org/abs/1906.03708v1
In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio. We show that the gap can be upper bounded by some form of dispersion measure of the likelihood ratio, which suggests the bias of variational inference can be reduced by making the distribution of the likelihood ratio more concentrated, such as via averaging and variance reduction.
ArXiv: 1906.03708
Document: PDF DOI
Artifact development version: GitHub
Abstract URL: https://arxiv.org/abs/1906.03708v1
In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio. We show that the gap can be upper bounded by some form of dispersion measure of the likelihood ratio, which suggests the bias of variational inference can be reduced by making the distribution of the likelihood ratio more concentrated, such as via averaging and variance reduction.
Relevant initiatives
Related knowledge about this paper
Reproduced results (crowd-benchmarking and competitions)
Artifact and reproducibility checklists
- ACM/cTuning artifact appendix and reproducibility checklist
- NeurIPS reproducibility checklist
- SIGPLAN's checklist for empirical evaluation
- Collective Knowledge (organizing research projects based on FAIR principles)