matplotlib.animation 的 Python 内存问题

Question

我正在创建一个复杂系统中的扩散模拟，以任意图像作为基底，允许任意创建扩散前沿，允许表面 react 以及新 Material 在起始基底上的沉积。到目前为止，我对结果感到非常自豪，您可以在此处查看我用它制作的关于粒子上的 CVD 和 SFD 沉积的影片。

CVD Movie

SFD Movie

不幸的是，我无法生成超过 50 个左右的帧，因为它耗尽了内存。我尝试在整个模拟过程中尽可能多地清除东西，但我想我一定遗漏了一些东西。总结:

我从创建一个空列表开始

ims = []

然后，每次我的“模拟”运行时，如果 frame number % frame "rate"== 0，它会生成一个帧:

• 通过plt.draw()和
使用 plt.ion()显示
• 使用 ims.append() 将渲染图添加到动画帧数组中。

在每一帧渲染之前，我运行 plt.clf() 以防止绘图中重叠的绘图数量不断增加。

如果没有 ims.append() 步骤，代码会占用 140 到 170MB 的 RAM。通过这一步，50 帧消耗了将近 1.4GB 的 RAM。显然，这是非常有限的。 50 帧很好，但我真的希望至少有 350 帧。这条路线可能无法实现，但这表明 ims 数组的内存使用量大约为每帧 24MB。

解决方法是创建框架并将其呈现为循环内的 .svg 或 .png 文件，并将其保存到磁盘。我发现这个渲染过程非常占用 CPU，所以这样做通常会使代码非常慢。此外，创建 350 个 PNG 文件然后手动将它们转换成视频非常困惑，所以我很想以某种方式将其全部放入程序本身。

有没有人知道如何在不渲染和将每一帧写入磁盘的情况下减少此示例代码的内存使用量？

在这个玩具代码中，我只是使用随机数生成器来填充评论中描述的两个数据集以加快速度。

代码:

import matplotlib.pyplot as plt
import matplotlib.animation as anim
from numpy import *
from matplotlib import *
import time

# Defines the number of frames of animation to render.
outputframes = 50

# Defines the size of the canned simulation.
nx = 800
ny = 800

# Defines the number of actual simulation timesteps
nt = 100

# This gets the number of timesteps between outputframes.
framestep = 2

# For reporting.
framenum = 0

# Creates two steps, one for the stepped simulated step,
# and one for the prior state. There are two independently
# changing materials, each of which will have half the simulation
# space containing random values here, plus 10% overlap in the
# middle.

p1 = zeros((nx, ny, 2))
p1[360:800,:,0] = random.rand(440, ny)
p2 = zeros((nx, ny, 2))
p2[0:440,:,0] = random.rand(440, ny)

# Animation colormap setup
norm = colors.Normalize(vmin=0, vmax = 1)
# And sets up two corresponding colormaps, one blue and one
# red for p1 and p2 respectively (goal is overlaid).
cmap1 = cm.Blues
cmap2 = cm.Reds

# Sets up an empty array to hold animation frames.
ims = []

# Sets up and uses ion to draw the figure without blocking.
plt.ion()
fig = plt.figure()
plt.draw()

# Run the simulation.

for t in range(nt):
    # This looks to see how far we are, and if we're at a point
    # where t is an even multiple of framestep, we should render
    # a new frame.

    if (t%framestep == 0):
        print('Frame ' + str(framenum))
        framenum = framenum + 1
        plt.clf()
        # In here I did a bunch of stuff to get special colors in
        # the colormap to get substrates and surfaces and other
        # features clearly identified. I am creating a new frame1
        # and frame2 object because in reality I will be doing a
        # log plot math to convert to the graphic frame.
        frame1 = p1[:,:,0]

        # This part is necessary in my real program because
        # I manually modify the colormap after it's created
        # to include the above mentioned special colors.
        frame1_colors = cmap1(norm(frame1))

        # This is my (not quite right) attempt to do overlaid plots.
        plt.imshow(frame1_colors, alpha = 0.5)

        # Do the same for the second set of data.
        frame2 = p2[:,:,0]
        frame2_colors = cmap2(norm(frame2))

        # The goal here was to take the combined output and make
        # it into an animation frame to append to ims, the image
        # array.

        # This is where I start to run into problems. Without the
        # ims.append, the program has constant memory usage. With
        # it, I am using 1340MB by the 50th frame. This is the
        # biggest issue. Even throwing away all other simulation
        # data, this image array for animation is *enormous*.

        # With the ims.append line replaced with the plt.imshow
        # line alone, memory usage is much smaller, ranging from
        # 140-170MB depending on execution point, but relatively
        # constant.

        ims.append([plt.imshow(frame2_colors, alpha = 0.5)])
#        plt.imshow(frame2_colors, alpha = 0.5)

        # Then try to draw updating animation to show progress
        # using draw(). As best I can tell, this basically works,
        # in that the plot is displaying with all components.
        plt.draw()


    # I'll put in a timer so that this doesn't go too fast, since
    # the actual calculation is very complex.
    time.sleep(0.01)

    # Proxy for the actual calculation. Just overwrite with new
    # random data in the overlapping ranges to show some change
    # visually.
    p1[360:800,:,1] = random.rand(440, ny)
    p2[0:440,:,1] = random.rand(440, ny)

    # In this version, it is trivial, but in the real simulation
    # p1[:,:,1] does not end up equal to p1[:,:,0], so the following
    # resets the simulation for the next timestep, overwriting the
    # old values to avoid memory overflow from the p1 and p2 arrays
    # being enormous.

    # Copy new values into old values.
    p1[:,:,0] = p1[:,:,1]
    p2[:,:,0] = p2[:,:,1]

# This is just a repeat for the final frame.
plt.clf()
frame1 = p1[:,:,0]
frame1_colors = cmap1(norm(frame1))
plt.imshow(frame1_colors, alpha = 0.5)
frame2 = p2[:,:,0]
frame2_colors = cmap2(norm(frame2))

# As above, the ims.append uses tons of memory, the imshow alone works well.
ims.append([plt.imshow(frame2_colors, alpha = 0.5)])
# plt.imshow(frame2_colors, alpha = 0.5)

plt.draw()

anim = anim.ArtistAnimation(fig, ims, blit=True)
anim.save('test.mp4', fps=10, writer='avconv')

Answer 1

最后，我决定唯一合理的方法是将我需要的每一帧渲染为 .png 文件，然后使用 avconv 从图像生成电影。

感谢所有建议，但由于未压缩图像的 RAM 使用情况，这看起来只是一个限制。