Imported from https://github.com/neka-nat/distributed_rl