Imported from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail