Imported from https://github.com/HumanCompatibleAI/adversarial-policies