python - 如何使用线程检索子数组

Question

假设我有一个大小为 (10,3,4) 的 numpy 数组。现在我想在第一维中的子数组上进行一些操作，如下代码:

a = np.arange(120).reshape([10,3,4])

for batch in range(10):
    result = do_something_func( a[batch,:,:] )
    # other operations in for

# other operations

我的问题是我想使用多线程来完成此任务，而不使用索引通过 for 循环迭代和检索子数组。有谁知道该怎么做吗？

Answer 1

可能与gevent一起使用:

import gevent
from gevent.pool import Pool
import numpy as np

pool = Pool(8)

def map_subarray(sub_array):
    return sub_array

a = np.arange(120).reshape([10,3,4])

for batch in range(10):
    result = pool.map(map_subarray, a[batch,:,:])
    print(batch, list(result))

输出:

(0, [array([0, 1, 2, 3]), array([4, 5, 6, 7]), array([ 8,  9, 10, 11])])
(1, [array([12, 13, 14, 15]), array([16, 17, 18, 19]), array([20, 21, 22, 23])])
(2, [array([24, 25, 26, 27]), array([28, 29, 30, 31]), array([32, 33, 34, 35])])
(3, [array([36, 37, 38, 39]), array([40, 41, 42, 43]), array([44, 45, 46, 47])])
(4, [array([48, 49, 50, 51]), array([52, 53, 54, 55]), array([56, 57, 58, 59])])
(5, [array([60, 61, 62, 63]), array([64, 65, 66, 67]), array([68, 69, 70, 71])])
(6, [array([72, 73, 74, 75]), array([76, 77, 78, 79]), array([80, 81, 82, 83])])
(7, [array([84, 85, 86, 87]), array([88, 89, 90, 91]), array([92, 93, 94, 95])])
(8, [array([96, 97, 98, 99]), array([100, 101, 102, 103]), array([104, 105, 106, 107])])
(9, [array([108, 109, 110, 111]), array([112, 113, 114, 115]), array([116, 117, 118, 119])])

引用:

• http://sdiehl.github.io/gevent-tutorial/