Neural random fields (NRFs), which are defined by using neural networks to
implement potential functions in undirected models (sometimes known as
energy-based models), provide an interesting family of model spaces for machine
learning, besides various directed models such as generative adversarial
networks (GANs). In this paper we propose a new approach, the inclusive-NRF
approach, to learning NRFs for continuous data (e.g. images), by developing
inclusive-divergence minimized auxiliary generators and stochastic gradient
sampling. As demonstrations of how the new approach can be flexibly and
effectively used, specific inclusive-NRF models are developed and thoroughly
evaluated for a number of tasks - unsupervised/supervised image generation,
semi-supervised classification and anomaly detection. The proposed models
consistently achieve strong experimental results in all these tasks compared to
state-of-the-art methods. Remarkably, in addition to superior sample
generation, one fundamental additional benefit of our inclusive-NRF approach is
that, unlike GANs, it directly provides (unnormalized) density estimate for
sample evaluation. With these contributions and results, this paper
significantly advances the learning and applications of undirected models to a
new level, both theoretically and empirically, which have never been obtained
before.