c - MPI_Waitall 无效请求

Question

我有一个代码试图测量在接收端使用非阻塞 MPI_Isend 和阻塞 MPI_Send 与 MPI_Irecv 之间的时间差。 MPI_Waitall 据推测用于确保仅在所有请求处理程序都被触发后才评估时间。但是，当我运行代码时，我得到:

[user:30439] *** An error occurred in MPI_Waitall
[user:30439] *** reported by process [931725313,1]
[user:30439] *** on communicator MPI_COMM_WORLD
[user:30439] *** MPI_ERR_REQUEST: invalid request
[user:30439] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
[user:30439] ***    and potentially your MPI job)

相关答案大多指向函数调用中的参数错误，以及 MPI_Waitall 在代码中的位置。这两个我都没有在这里识别和更正。

这是我的完整代码:

#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>

#define COMM MPI_COMM_WORLD
#define REPS 1000
#define DISP 100

int main (int argc, char *argv[])
{
int numtasks, rank, buf, tag1=1, i, rc, dest, src, offset, nreqs;
double T1, T2;
MPI_Request reqs[REPS*2];
MPI_Status stats[REPS*2];

MPI_Init(&argc,&argv);
MPI_Comm_size(COMM, &numtasks);
MPI_Comm_rank(COMM, &rank);

/* Require 4 tasks */
if (rank == 0 ) {
  if (numtasks != 4) {
    printf("ERROR: Number of tasks must be 4. Quitting.\n");
    MPI_Abort(COMM, rc);
    }
  printf("Starting isend/irecv send/irecv test...\n");
  }

/* Use barriers for clean output */
MPI_Barrier(COMM);
printf("Task %d starting...\n", rank);
MPI_Barrier(COMM);

T1 = MPI_Wtime();     /* start the clock */

/* Tasks 0 and 1 do the isend/irecv test. 
*  Determine who to send/receive with. nreqs specifies how many non-blocking
*  operation request handles to capture. offset is where the task should
*  store each request as it is captured in the reqs() array.         */
if (rank < 2) {
  nreqs = REPS*2;
  if (rank == 0) {
    src = 1;
    offset = 0;
    }
  if (rank == 1) {
    src = 0;
    offset = REPS;
    }
  dest = src;

/* Do the non-blocking send and receive operations */
  for (i=0; i<REPS; i++) {
    MPI_Isend(&rank, 1, MPI_INT, dest, tag1, COMM, &reqs[offset]);
    MPI_Irecv(&buf, 1, MPI_INT, src, tag1, COMM, &reqs[offset+1]);
    offset += 2;
    if ((i+1)%DISP == 0)
      printf("Task %d has done %d isends/irecvs\n", rank, i+1);
    }
  }

/* Tasks 2 and 3 do the send/irecv test. 
   Determine who to send/receive with. nreqs specifies how many non-blocking
   operation request handles to capture. offset is where the task should
   store each request as it is captured in the reqs() array.  */
if (rank > 1) {
  nreqs = REPS;

/* Task 2 does the blocking send operation */
  if (rank == 2) {
    dest = 3;
    for (i=0; i<REPS; i++) {
      MPI_Send(&rank, 1, MPI_INT, dest, tag1, COMM);
      if ((i+1)%DISP == 0)
        printf("Task %d has done %d sends\n", rank, i+1);
      }
    }

/* Task 3 does the non-blocking receive operation */
  if (rank == 3) {
    src = 2;
    offset = 0;
    for (i=0; i<REPS; i++) {
      MPI_Irecv(&buf, 1, MPI_INT, src, tag1, COMM, &reqs[offset]);
      offset += 1;
      if ((i+1)%DISP == 0)
        printf("Task %d has done %d irecvs\n", rank, i+1);
      }
    }

  }

/* Wait for all non-blocking operations to complete and record time */
MPI_Waitall(nreqs, reqs, stats);
T2 = MPI_Wtime();     /* end time */
MPI_Barrier(COMM);

printf("Task %d time(wall)= %lf sec\n", rank, T2-T1);

MPI_Finalize();
}

提前致谢。

Answer 1

您的代码中至少有 3 个关键问题。

1. 您为 MPI_Irecv 的重复调用提供相同的缓冲区。在请求完成之前，您绝不能触及分配给异步 MPI 函数的内存。
2. 等级 2 没有请求，但调用 MPI_Waitall 声称它有 REPS 请求。
3. 等级 1 开始在 reqs 中写入从 REPS 到 2*REPS-1 - 这是没有意义的。

总的来说，您应该努力改进代码的格式和结构，以便更容易推理这些事情。同时使用 MUST MPI 正确性检查可能对您很有帮助。