We are very excited to join forces with MLCommons and OctoML.ai! Contact Grigori Fursin for more details!

A theory of independent mechanisms for extrapolation in generative models

lib:38f55e3afce4c458 (v1.0.0)

Authors: Michel Besserve,Rémy Sun,Dominik Janzing,Bernhard Schölkopf
ArXiv: 2004.00184
Document:  PDF  DOI 
Abstract URL: https://arxiv.org/abs/2004.00184v1


Deep generative models reproduce complex empirical data but cannot extrapolate to novel environments. An intuitive idea to promote extrapolation capabilities is to enforce the architecture to have the modular structure of a causal graphical model, where one can intervene on each module independently of the others in the graph. We develop a framework to formalize this intuition, using the principle of Independent Causal Mechanisms, and show how over-parameterization of generative neural networks can hinder extrapolation capabilities. Our experiments on the generation of human faces shows successive layers of a generator architecture implement independent mechanisms to some extent, allowing meaningful extrapolations. Finally, we illustrate that independence of mechanisms may be enforced during training to improve extrapolation.

Relevant initiatives  

Related knowledge about this paper Reproduced results (crowd-benchmarking and competitions) Artifact and reproducibility checklists Common formats for research projects and shared artifacts Reproducibility initiatives

Comments  

Please log in to add your comments!
If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!