python-3.x - Python 3并发.futures - 并行处理for循环

Question

如果我正确理解了 Python 3 中的 concurrent.futures 模块的工作原理，则以下代码:

import concurrent.futures
import threading

# Simple function returning a value
def test(i):

    a = 'Hello World\n'
    return a


def main():
    output1 = list()

    with concurrent.futures.ThreadPoolExecutor() as executor:

        # psdd iterator to test function
        for out1 in executor.map(test, range(0, 10)):
            # append returned result
            output1.append(out1)

            # confirm output
            print(output1)
            print("Task Executed {}".format(threading.current_thread()))


if __name__ == '__main__':
    main()

...执行以下功能:

1. 将 for 循环传递给名为 test() 的函数。
2. 并行处理循环，而不是串行处理。

但是，我真正想要的是在我的 main() 函数中并行处理循环，如下所示:

import concurrent.futures
import threading


def main():
    output1 = list()

    with concurrent.futures.ThreadPoolExecutor() as executor:

        # psdd iterator to test function
        for out1 in executor.submit(range(0, 10)):

            a = 'Hello World\n'
            # append returned result
            output1.append(a)

            # confirm output
            print(output1)
            print("Task Executed {}".format(threading.current_thread()))


if __name__ == '__main__':
    main()

...但这会产生以下错误:

Traceback (most recent call last):
  File "G:\HTPC Scripts\WebGrab Plus\TESTTESTTEST2.py", line 221, in <module>
    main()
  File "G:\HTPC Scripts\WebGrab Plus\TESTTESTTEST2.py", line 209, in main
    for out1 in executor.submit(range(0, 10)):
TypeError: 'Future' object is not iterable

我的代码需要修改什么？

谢谢

Answer 1

无论您使用 submit 还是 map，您总是必须使用可调用对象(例如函数)作为第一个参数。

Python 确实允许嵌套函数(还要注意使用 Futures 的方式)；

import concurrent.futures


def main():

    def worker(arg):
        return str(arg) + ' Hello World!'

    with concurrent.futures.ThreadPoolExecutor() as e:
        fut = [e.submit(worker, i) for i in range(10)]
        for r in concurrent.futures.as_completed(fut):
            print(r.result())


if __name__ == '__main__':
    main()

就地定义可调用的唯一方法是使用 lambda 表达式，但它们有很大的局限性。