python - 在 Python 中运行批量同步并行模型 (BSP)

Question

BSP 并行编程模型有几个好处 - 程序员无需明确关心同步，死锁变得不可能，速度推理比传统方法更容易。 SciPy 中有一个 BSPlib 的 Python 接口(interface):

import Scientific.BSP

我写了一个小程序来测试 BSP。该程序是一个简单的随机实验，它“计算” throw n 骰子产生总和为 k 的概率:

from Scientific.BSP import ParSequence, ParFunction, ParRootFunction
from sys import argv
from random import randint
n = int(argv[1]) ; m = int(argv[2]) ; k = int(argv[3])

def sumWuerfe(ws): return len([w for w in ws if sum(w)==k])
glb_sumWuerfe= ParFunction(sumWuerfe)
def ausgabe(result): print float(result)/len(wuerfe)
glb_ausgabe = ParRootFunction(output)
wuerfe = [[randint(1,6) for _ in range(n)] for _ in range(m)]
glb_wuerfe = ParSequence(wuerfe)

# The parallel calc:
ergs = glb_sumWuerfe(glb_wuerfe)
# collecting the results in Processor 0:
ergsGesamt= results.reduce(lambda x,y:x+y, 0)

glb_output(ergsGesamt)

该程序运行良好，但是:它只使用一个进程!

我的问题:有人知道如何告诉这个 Pythonb-BSP-Script 使用 4(或 8 或 16)个进程吗？我认为这个 BSP 实现会使用 MPI，但是通过 mpiexe -n 4 randExp.py 启动脚本不起作用。

Answer 1

一件小事，但是 Scientific Python != SciPy 在你的问题中......

如果您下载 ScientificPython 源代码，您将看到一个 README.BSP、一个 README.MPI 和一个 README.BSPlib。不幸的是，在线网页上并没有真正提及这些信息。

README.BSP 非常明确地说明了您需要做什么才能让 BSP 的东西在真正的并行中工作:

In order to use the module Scientific.BSP using more than one real processor, you must compile either the BSPlib or the MPI interface. See README.BSPlib and README.MPI for installation details. The BSPlib interface is probably more efficient (I haven't done extensive tests yet), and allows the use of the BSP toolset, on the other hand MPI is more widely available and might thus already be installed on your machine. For serious use, you should probably install both and make comparisons for your own applications. Application programs do not have to be modified to switch between MPI and BSPlib, only the method to run the program on a multiprocessor machine must be adapted.

To execute a program in parallel mode, use the mpipython or bsppython executable. The manual for your MPI or BSPlib installation will tell you how to define the number of processors.

README.MPI 告诉您如何获得 MPI 支持:

Here is what you have to do to get MPI support in Scientific Python:

1) Build and install Scientific Python as usual (i.e. "python setup.py install" in most cases).

2) Go to the directory Src/MPI.

3) Type "python compile.py".

4) Move the resulting executable "mpipython" to a directory on your
system's execution path.

因此，您必须显式构建更多 BSP 内容才能利用真正的并行性。好消息是您不必更改程序。这样做的原因是不同的系统安装了不同的并行库，并且在这些库之上的库必须有这样的配置/构建步骤才能利用任何可用的东西。