Authors: Abdeslam Boularias,Jan R. Peters,Oliver B. Kroemer
Where published:
NeurIPS 2012 12
Document:
PDF
DOI
Abstract URL: http://papers.nips.cc/paper/4721-algorithms-for-learning-markov-field-policies
We present a new graph-based approach for incorporating domain knowledge in reinforcement learning applications. The domain knowledge is given as a weighted graph, or a kernel matrix, that loosely indicates which states should have similar optimal actions. We first introduce a bias into the policy search process by deriving a distribution on policies such that policies that disagree with the provided graph have low probabilities. This distribution corresponds to a Markov Random Field. We then present a reinforcement and an apprenticeship learning algorithms for finding such policy distributions. We also illustrate the advantage of the proposed approach on three problems: swing-up cart-balancing with nonuniform and smooth frictions, gridworlds, and teaching a robot to grasp new objects.