We are very excited to join forces with MLCommons and OctoML.ai! Contact Grigori Fursin for more details!

Compositional uncertainty in deep Gaussian processes

lib:1f427fb49c3acffb (v1.0.0)

Authors: Ivan Ustyuzhaninov,Ieva Kazlauskaite,Markus Kaiser,Erik Bodin,Neill D. F. Campbell,Carl Henrik Ek
ArXiv: 1909.07698
Document:  PDF  DOI 
Abstract URL: https://arxiv.org/abs/1909.07698v3


Gaussian processes (GPs) are nonparametric priors over functions. Fitting a GP implies computing a posterior distribution of functions consistent with the observed data. Similarly, deep Gaussian processes (DGPs) should allow us to compute a posterior distribution of compositions of multiple functions giving rise to the observations. However, exact Bayesian inference is intractable for DGPs, motivating the use of various approximations. We show that the application of simplifying mean-field assumptions across the hierarchy leads to the layers of a DGP collapsing to near-deterministic transformations. We argue that such an inference scheme is suboptimal, not taking advantage of the potential of the model to discover the compositional structure in the data. To address this issue, we examine alternative variational inference schemes allowing for dependencies across different layers and discuss their advantages and limitations.

Relevant initiatives  

Related knowledge about this paper Reproduced results (crowd-benchmarking and competitions) Artifact and reproducibility checklists Common formats for research projects and shared artifacts Reproducibility initiatives

Comments  

Please log in to add your comments!
If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!