python-3.x - Cart-Pole Python 性能比较

Question

我正在将推车和杆模拟与 python 3.7 和 Julia 1.2 进行比较。在 python 中，模拟被编写为如下所示的类对象，而在 Julia 中只是一个函数。使用 Julia 解决问题的时间是一致的 0.2 秒，这比 python 慢得多。我不太了解 Julia ，无法理解为什么。我的猜测是它与编译或循环的设置方式有关。

import math
import random
from collections import namedtuple

RAD_PER_DEG = 0.0174533
DEG_PER_RAD = 57.2958

State = namedtuple('State', 'x x_dot theta theta_dot')

class CartPole:
    """ Model for the dynamics of an inverted pendulum
    """
    def __init__(self):
        self.gravity   = 9.8
        self.masscart  = 1.0
        self.masspole  = 0.1
        self.length    = 0.5   # actually half the pole's length
        self.force_mag = 10.0
        self.tau       = 0.02  # seconds between state updates

        self.x         = 0
        self.x_dot     = 0
        self.theta     = 0
        self.theta_dot = 0

    @property
    def state(self):
        return State(self.x, self.x_dot, self.theta, self.theta_dot)

    def reset(self, x=0, x_dot=0, theta=0, theta_dot=0):
        """ Reset the model of a cartpole system to it's initial conditions
        "   theta is in radians
        """
        self.x         = x
        self.x_dot     = x_dot
        self.theta     = theta
        self.theta_dot = theta_dot

    def step(self, action):
        """ Move the state of the cartpole simulation forward one time unit
        """
        total_mass      = self.masspole + self.masscart
        pole_masslength = self.masspole * self.length

        force           = self.force_mag if action else -self.force_mag
        costheta        = math.cos(self.theta)
        sintheta        = math.sin(self.theta)

        temp = (force + pole_masslength * self.theta_dot ** 2 * sintheta) / total_mass

        # theta acceleration
        theta_dotdot = (
            (self.gravity * sintheta - costheta * temp)
            / (self.length *
               (4.0/3.0 - self.masspole * costheta * costheta /
                total_mass)))

        # x acceleration
        x_dotdot = temp - pole_masslength * theta_dotdot * costheta / total_mass

        self.x         += self.tau * self.x_dot
        self.x_dot     += self.tau * x_dotdot
        self.theta     += self.tau * self.theta_dot
        self.theta_dot += self.tau * theta_dotdot

        return self.state

为了运行模拟，使用了以下代码

from cartpole import CartPole
import time
cp = CartPole()
start = time.time()
for i in range(100000):
      cp.step(True)
end = time.time()
print(end-start)

Julia 代码是

function cartpole(state, action)
"""Cart and Pole simulation in discrete time
Inputs: cartpole( state, action )
state: 1X4 array [cart_position, cart_velocity, pole_angle, pole_velocity]
action: Boolean True or False where true is a positive force and False is a negative force
"""

gravity   = 9.8
masscart  = 1.0
masspole  = 0.1
l    = 0.5   # actually half the pole's length
force_mag = 10.0
tau       = 0.02  # seconds between state updates

# x         = 0
# x_dot     = 0
# theta     = 0
# theta_dot = 0

x         = state[1]
x_dot     = state[2]
theta     = state[3]
theta_dot = state[4]


total_mass = masspole + masscart
pole_massl = masspole * l

if action == 0
 force = force_mag
else
 force = -force_mag
end

costheta = cos(theta)
sintheta = sin(theta)

temp = (force + pole_massl * theta_dot^2 * sintheta) / total_mass

# theta acceleration
theta_dotdot = (gravity * sintheta - costheta * temp)/ (l *(4.0/3.0 - masspole * costheta * costheta / total_mass))

# x acceleration
x_dotdot = temp - pole_massl * theta_dotdot * costheta / total_mass

x         += tau * x_dot
x_dot     += tau * x_dotdot
theta     += tau * theta_dot
theta_dot += tau * theta_dotdot

new_state = [x x_dot theta theta_dot]

return new_state

end

运行代码是

@time include("cartpole.jl")


function run_sim()
"""Runs the cartpole simulation
No inputs or ouputs
Use with @time run_sim() for timing puposes.
"""
 state = [0 0 0 0]
 for i = 1:100000
  state = cartpole( state, 0)
  #print(state)
  #print("\n")
end
end

@time run_sim()

Answer 1

你的 Python 版本在我的笔记本电脑上需要 0.21 秒。以下是同一系统上原始 Julia 版本的计时结果:

julia> @time run_sim()
  0.222335 seconds (654.98 k allocations: 38.342 MiB)

julia> @time run_sim()
  0.019425 seconds (100.00 k allocations: 10.681 MiB, 37.52% gc time)

julia> @time run_sim()
  0.010103 seconds (100.00 k allocations: 10.681 MiB)

julia> @time run_sim()
  0.012553 seconds (100.00 k allocations: 10.681 MiB)

julia> @time run_sim()
  0.011470 seconds (100.00 k allocations: 10.681 MiB)

julia> @time run_sim()
  0.025003 seconds (100.00 k allocations: 10.681 MiB, 52.82% gc time)

第一次运行包括 JIT 编译，大约需要 0.2 秒，而之后每次运行需要 10-20 毫秒。这分解为大约 10 毫秒的实际计算时间和大约 10 秒的垃圾收集时间，每四个调用左右触发一次。这意味着 Julia 比 Python 快 10-20 倍，不包括 JIT 编译时间，这对于直接移植来说还不错。

基准测试时为什么不计算 JIT 时间？因为您实际上并不关心运行基准测试等快速程序需要多长时间。您正在计时小型基准问题，以推断运行速度真正重要的大型问题需要多长时间。 JIT 编译时间与您正在编译的代码量成正比，而不是与问题大小成正比。所以在解决你真正关心的大问题时，JIT 编译仍然只需要 0.2 秒，这对于大问题的执行时间来说是可以忽略不计的一小部分。

现在，让我们看看如何让 Julia 代码更快。这实际上非常简单:为您的状态使用元组而不是行向量。所以初始化状态为 state = (0, 0, 0, 0)然后类似地更新状态:

new_state = (x, x_dot, theta, theta_dot)

就是这样，否则代码是相同的。对于这个版本，时间是:

julia> @time run_sim()
  0.132459 seconds (479.53 k allocations: 24.020 MiB)

julia> @time run_sim()
  0.008218 seconds (4 allocations: 160 bytes)

julia> @time run_sim()
  0.007230 seconds (4 allocations: 160 bytes)

julia> @time run_sim()
  0.005379 seconds (4 allocations: 160 bytes)

julia> @time run_sim()
  0.008773 seconds (4 allocations: 160 bytes)

第一次运行仍然包括 JIT 时间。后续运行现在为 5-10 毫秒，比 Python 版本快约 25-40 倍。请注意，几乎没有分配——少量、固定数量的分配仅用于返回值，如果从循环中的其他代码调用，则不会触发 GC。