Authors: Le Hou,Chen-Ping Yu,Dimitris Samaras
ArXiv: 1611.05916
Document:
PDF
DOI
Artifact development version:
GitHub
Abstract URL: http://arxiv.org/abs/1611.05916v4
In the context of single-label classification, despite the huge success of
deep learning, the commonly used cross-entropy loss function ignores the
intricate inter-class relationships that often exist in real-life tasks such as
age classification. In this work, we propose to leverage these relationships
between classes by training deep nets with the exact squared Earth Mover's
Distance (also known as Wasserstein distance) for single-label classification.
The squared EMD loss uses the predicted probabilities of all classes and
penalizes the miss-predictions according to a ground distance matrix that
quantifies the dissimilarities between classes. We demonstrate that on datasets
with strong inter-class relationships such as an ordering between classes, our
exact squared EMD losses yield new state-of-the-art results. Furthermore, we
propose a method to automatically learn this matrix using the CNN's own
features during training. We show that our method can learn a ground distance
matrix efficiently with no inter-class relationship priors and yield the same
performance gain. Finally, we show that our method can be generalized to
applications that lack strong inter-class relationships and still maintain
state-of-the-art performance. Therefore, with limited computational overhead,
one can always deploy the proposed loss function on any dataset over the
conventional cross-entropy.