python - 为连续 Action 空间实现强化(Humanoid-v2)？

Question

我已经看到 REINFORCE 又名 Vanilla 策略算法的多种实现被用于具有离散 Action 空间的强化学习任务。是否有连续 Action 空间的算法(或其他策略梯度算法)的实现？

更具体地说，是否可以实现来自 OpenAI Gym 的双足运动“Humanoid-v2”的 REINFORCE？

谢谢。

Answer 1

您可以稳定基线包:https://github.com/hill-a/stable-baselines

训练代理就这么简单:

import gym
from stable_baselines.common.policies import MlpPolicy
from stable_baselines.common.vec_env import DummyVecEnv
from stable_baselines import PPO2

my_env_id = 'Humanoid-v2'

env = gym.make(my_env_id)
# Vectorized environments allow to easily multiprocess training
# we demonstrate its usefulness in the next examples
env = DummyVecEnv([lambda: env])  # The algorithms require a vectorized environment to run

model = PPO2(MlpPolicy, env, verbose=1)
# Train the agent
model.learn(total_timesteps=10000)

# Enjoy trained agent
obs = env.reset()
for i in range(1000):
    action, _states = model.predict(obs)
    obs, rewards, dones, info = env.step(action)
    env.render()