python - numpy -- 将非连续数据转换为连续数据

Question

考虑以下代码:

import numpy as np
a = np.zeros(50)
a[10:20:2] = 1
b = c = a[10:40:4]
print b.flags  # You'll see that b and c are not C_CONTIGUOUS or F_CONTIGUOUS

我的问题:

有没有一种方法(仅引用 b)使 b 和 c 连续？如果 np.may_share_memory(b,a) 在此操作后返回 False 则完全没问题。

接近但不太可行的是:np.ascontiguousarray/np.asfortranarray 因为它们将返回一个新 数组。

---

我的用例是我将非常大的 3D 字段存储在 numpy.ndarray 的子类中。为了节省内存，我想将这些字段切碎到我真正有兴趣处理的域部分:

a = a[ix1:ix2,iy1:iy2,iz1:iz2]

子类的切片比 ndarray 对象的切片更受限制，但这应该有效并且它将“做正确的事情”——附加在子类上的各种自定义元数据将按预期进行转换/保存。不幸的是，由于这会返回一个 view，numpy 之后不会释放大数组，所以我实际上并没有在这里保存任何内存。

完全清楚，我希望完成两件事:

• 保留我的类实例上的元数据。切片会起作用，但我不确定其他形式的复制。
• 使原始数组可以自由地被垃圾收集

Answer 1

According to Alex Martelli :

"The only really reliable way to ensure that a large but temporary use of memory DOES return all resources to the system when it's done, is to have that use happen in a subprocess, which does the memory-hungry work then terminates."

但是，以下似乎至少释放了一些内存:警告:我测量可用内存的方法是特定于 Linux 的:

import time
import numpy as np

def free_memory():
    """
    Return free memory available, including buffer and cached memory
    """
    total = 0
    with open('/proc/meminfo', 'r') as f:
        for line in f:
            line = line.strip()
            if any(line.startswith(field) for field in ('MemFree', 'Buffers', 'Cached')):
                field, amount, unit = line.split()
                amount = int(amount)
                if unit != 'kB':
                    raise ValueError(
                        'Unknown unit {u!r} in /proc/meminfo'.format(u=unit))
                total += amount
    return total

def gen_change_in_memory():
    """
    https://stackoverflow.com/a/14446011/190597 (unutbu)
    """
    f = free_memory()
    diff = 0
    while True:
        yield diff
        f2 = free_memory()
        diff = f - f2
        f = f2
change_in_memory = gen_change_in_memory().next

在分配大数组之前:

print(change_in_memory())
# 0

a = np.zeros(500000)
a[10:20:2] = 1
b = c = a[10:40:4]

分配大数组后:

print(change_in_memory())
# 3844 # KiB

a[:len(b)] = b
b = a[:len(b)]
a.resize(len(b), refcheck=0)
time.sleep(1)

调整大小后可用内存增加:

print(change_in_memory())
# -3708 # KiB