mpi - 为什么当消息确定发送时 MPI_Iprobe 返回 false？

Question

我想使用 MPI_Iprobe 来测试带有给定标签的消息是否已经挂起。

但是，MPI_Iprobe 的行为并不像我预期的那样。
在下面的示例中，我将消息从多个任务发送到单个任务(等级 0)。然后在等级 0 上，我等待几秒钟，让 MPI_Isends 有足够的时间完成。然后，当我运行 MPI_Iprobe 时，它​​返回标志为 false。如果我在(阻塞)MPI_Probe 之后重复，那么它返回 true。

#include "mpi.h"
#include <stdio.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
  int rank;
  int numprocs;
  int tag;
  int receive_tag;
  int flag=0;
  int number;
  int recv_number=0;

  MPI_Request request;
  MPI_Status status;

  MPI_Init(&argc,&argv);
  MPI_Comm_rank(MPI_COMM_WORLD,&rank);
  MPI_Comm_size(MPI_COMM_WORLD,&numprocs);

  // rank 0 receives messages, all others send messages
  if (rank > 0 ) {
    number = rank;
    tag = rank;
    MPI_Isend(&number, 1, MPI_INT, 0, tag, MPI_COMM_WORLD,&request); // send to rank 0
    printf("Sending tag : %d \n",tag);
   } 
   else if (rank == 0) {

   sleep(5); // [seconds] allow plenty of time for all sends from other tasks to complete

   receive_tag = 3; // just try and receive a single message from task 1

   MPI_Iprobe(MPI_ANY_SOURCE,receive_tag,MPI_COMM_WORLD,&flag,&status);
   printf("After MPI_Iprobe, flag = %d \n",flag);

   MPI_Probe(MPI_ANY_SOURCE,receive_tag,MPI_COMM_WORLD,&status);
   printf("After MPI_Probe, found message with tag : %d \n",receive_tag);

   MPI_Iprobe(MPI_ANY_SOURCE,receive_tag,MPI_COMM_WORLD,&flag,&status);
   printf("After second MPI_Iprobe, flag = %d \n",flag);

   // receive all the messages
   for (int i=1;i<numprocs;i++){    
     MPI_Recv(&recv_number, 1, MPI_INT, MPI_ANY_SOURCE, i, MPI_COMM_WORLD,&status);
     printf("Received : %d \n",recv_number);
   }

 }
 MPI_Finalize();
}

给出这个输出:

Sending tag : 4 
Sending tag : 3 
Sending tag : 2 
Sending tag : 5 
Sending tag : 1 
After MPI_Iprobe, flag = 0 
After MPI_Probe, found message with tag : 3 
After second MPI_Iprobe, flag = 1 
Received : 1 
Received : 2 
Received : 3 
Received : 4 
Received : 5 

为什么 mpi_iprobe 第一次返回“false”？

任何帮助将非常感激!

编辑:在 Hristo Iliev 的回答之后，我现在有以下代码:

#include "mpi.h"
#include <stdio.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
  int rank;
  int numprocs;
  int tag;
  int receive_tag;
  int flag=0;
  int number;
  int recv_number=0;

  MPI_Request request;
  MPI_Status status;

  MPI_Init(&argc,&argv);
  MPI_Comm_rank(MPI_COMM_WORLD,&rank);
  MPI_Comm_size(MPI_COMM_WORLD,&numprocs);

  // rank 0 receives messages, all others send messages
  if (rank > 0 ) {
    number = rank;
    tag = rank;

    MPI_Isend(&number, 1, MPI_INT, 0, tag, MPI_COMM_WORLD,&request); // send to rank 0
    printf("Sending tag : %d \n",tag);

    // do stuff

    MPI_Wait(&request,&status);
    printf("Sent tag : %d \n",tag);

   }
    else if (rank == 0) {

    sleep(5); // [seconds] allow plenty of time for all sends from other tasks to complete

    receive_tag = 3; // just try and receive a single message from task 1

    MPI_Iprobe(MPI_ANY_SOURCE,receive_tag,MPI_COMM_WORLD,&flag,&status);
    printf("After MPI_Iprobe, flag = %d \n",flag);

    MPI_Probe(MPI_ANY_SOURCE,receive_tag,MPI_COMM_WORLD,&status);
    printf("After MPI_Probe, found message with tag : %d \n",receive_tag);

    MPI_Iprobe(MPI_ANY_SOURCE,receive_tag,MPI_COMM_WORLD,&flag,&status);
    printf("After second MPI_Iprobe, flag = %d \n",flag);

    // receive all the other messages
    for (int i=1;i<numprocs;i++){   
       MPI_Recv(&recv_number, 1, MPI_INT, MPI_ANY_SOURCE, i, MPI_COMM_WORLD,&status);
    }

 }
 MPI_Finalize();
}

这给出了以下输出:

Sending tag : 5 
Sending tag : 2 
Sending tag : 1 
Sending tag : 4 
Sending tag : 3 
Sent tag : 2 
Sent tag : 1 
Sent tag : 5 
Sent tag : 4 
Sent tag : 3 
After MPI_Iprobe, flag = 0 
After MPI_Probe, found message with tag : 3 
After second MPI_Iprobe, flag = 1 

Answer 1

您正在使用 MPI_Isend为了发送消息。 MPI_Isend启动异步(后台)数据传输。实际数据传输可能不会发生，除非 MPI_Wait* 之一或 MPI_Test*已按要求调用电话。一些 MPI 实现具有(或可以配置为)后台进程线程，即使没有对请求进行等待/测试，它们也会继续发送操作，但不应依赖这种行为。

只需更换 MPI_Isend与 MPI_Send或添加 MPI_Wait(&request);在前者之后(请注意，MPI_Isend + MPI_Wait 紧随其后相当于 MPI_Send )。
MPI_Iprobe旨在用于繁忙的等待，即:

while (condition)
{
   MPI_Iprobe(...,&flag,...);
   if (flag)
   {
      MPI_Recv(...);
      ...
   }
   // Do something, e.g. background tasks
}

实际 MPI 实现中的真实消息传输是相当复杂的事情。操作通常分成多个部分，然后排队。执行该部分称为进程，它在 MPI 库中的各个点完成，例如，当进行通信调用时，或者如果库实现了后台进程线程，则在后台执行。调用 MPI_Iprobe肯定会取得进展，但不能保证一个电话就足够了。 MPI 标准规定:

The MPI implementation of MPI_PROBE and MPI_IPROBE needs to guarantee progress: if a call to MPI_PROBE has been issued by a process, and a send that matches the probe has been initiated by some process, then the call to MPI_PROBE will return, unless the message is received by another concurrent receive operation (that is executed by another thread at the probing process). Similarly, if a process busy waits with MPI_IPROBE and a matching message has been issued, then the call to MPI_IPROBE will eventually return flag = true unless the message is received by another concurrent receive operation.

注意finally的使用。如何进行进展是非常特定于实现的。将 5 次连续调用的以下输出与 MPI_Iprobe 进行比较(您的原始代码+一个紧密的循环):

打开 MPI 1.6.5 不带进度线程:

# Run 1
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1

# Run 2
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1

# Run 3
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0

观察到同一 MPI 程序的多次执行之间没有一致性，并且在第 3 次运行中，标志仍然是 false 5 次调用 MPI_Iprobe 后.

英特尔 MPI 4.1.2:

# Run 1
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1

# Run 2
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1

# Run 3
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 0
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1
After MPI_Iprobe, flag = 1

显然，英特尔 MPI 的进展与 Open MPI 不同。

两种实现之间的差异可以通过 MPI_Iprobe 来解释。应该是一个微小的探针，因此它应该花费尽可能少的时间。另一方面，进展需要时间，在单线程 MPI 实现中，唯一可能进展的时间点是对 MPI_Iprobe 的调用。 (在那种特殊情况下)。因此，MPI 实现者必须决定每次调用 MPI_Iprobe 实际进展了多少。并在通话完成的工作量和花费的时间之间取得平衡。

与 MPI_Probe事情是不同的。这是一个阻塞调用，因此它能够不断进行，直到出现匹配的消息(更具体地说是它的信封)。