c - MPI_Irecv 没有正确接收 MPI_Send 发送的数据

Question

我有一个一维矩阵数据作为 Q_send_matrix。在每次迭代中，每个处理器更新其Q_send_matrix并将其发送到前一个处理器(rank-1)，同时它接收一个新更新的矩阵作为Q_recv_matrix 来自下一个处理器(rank+1)。例如，在一次迭代中，Proc[0] 更新它的 Q_send_matrix 并将其发送到 Proc[3]，而它接收 来自 Proc[1] 的 Q_recv_matrix。正如您所估计的那样，它就像一个环形通信。请在我在下面对代码进行解释后查看下面的代码。

        MPI_Request request;
        MPI_Status status;

        // All the elements of Q_send and Q_recv buffers 
        // are set to 1.0 initially. Each processor
        // updates its Q_send buffer to prepare it
        // to be sent below.(above part is big, so it
        // is not added here...)

        /**
         * Transfer Q matrix blocks among processors
         *      + Each processor sends the Q matrix
         *  + to the previous processor while receives
         *  + the Q matrix from the next processor
         *      + It is like a ring communication
         * */


        /* Receive Q matrix with MPI_Irecv */
        source = (my_rank+1)%comm_size;
        recv_count = no_col_per_proc[source]*input_k;

        MPI_Irecv(Q_recv_matrix, recv_count,
                MPI_FP_TYPE, source,
                0, MPI_COMM_WORLD,
                &request);


        /* Send Q matrix */
        dest = (my_rank-1+comm_size)%comm_size;
        send_count = no_col_per_proc[my_rank]*input_k;

        MPI_Send(Q_send_matrix, send_count,
                MPI_FP_TYPE, dest,
                0, MPI_COMM_WORLD);


        /* Wait status */
        // MPI_Wait(request, status);

        /* Barrier */
        MPI_Barrier(MPI_COMM_WORLD);

        /* Print Q send and receive matrices */
        for( j = 0; j < send_count; j ++ )
        {
            printf("P[%d] sends Q_send[%d] to P[%d] = %.2f\n",
                    my_rank, j, dest, Q_send_matrix[j]);
        }

        for( j = 0; j < recv_count; j ++ )
        {
            printf("P[%d] receives Q_recv[%d] from P[%d] = %.2f\n",
                    my_rank, j, source, Q_recv_matrix[j]);
        }

我想以同步方式进行此通信。但是，MPI_Send 和 MPI_Recv 是不可能的，因为它们的阻塞特性会导致死锁。因此，我将 MPI_Irecv 和 MPI_Send 与 MPI_Wait 一起使用。然而，它并没有完成，所有的处理器都在等待。所以，我用 MPI_Barrier 代替 MPI_Wait 让它们同步，解决了处理器的等待问题，所以它们完成了工作。但是，它无法正常工作。以下代码的某些输出是错误的。每个处理器发送正确的数据，发送端没有问题。另一方面，接收数据缓冲区没有变化。这意味着在某些处理器中，即使从其他处理器之一接收到数据，接收缓冲区的初始值仍然存在，如下所示。

P[0] sends Q_send[0] to P[3] = -2.12
P[0] sends Q_send[1] to P[3] = -2.12
P[0] sends Q_send[2] to P[3] = 4.12
P[0] sends Q_send[3] to P[3] = 4.12
P[0] receives Q_recv[0] from P[1] = 1.00
P[0] receives Q_recv[1] from P[1] = 1.00
P[0] receives Q_recv[2] from P[1] = 1.00
P[0] receives Q_recv[3] from P[1] = 1.00

P[1] sends Q_send[0] to P[0] = -2.12
P[1] sends Q_send[1] to P[0] = -2.12
P[1] sends Q_send[2] to P[0] = 0.38
P[1] sends Q_send[3] to P[0] = 0.38
P[1] receives Q_recv[0] from P[2] = 1.00
P[1] receives Q_recv[1] from P[2] = 1.00
P[1] receives Q_recv[2] from P[2] = 1.00
P[1] receives Q_recv[3] from P[2] = 1.00

P[2] sends Q_send[0] to P[1] = 1.00
P[2] sends Q_send[1] to P[1] = 1.00
P[2] sends Q_send[2] to P[1] = -24.03
P[2] sends Q_send[3] to P[1] = -24.03
P[2] receives Q_recv[0] from P[3] = 1.00
P[2] receives Q_recv[1] from P[3] = 1.00
P[2] receives Q_recv[2] from P[3] = 1.00
P[2] receives Q_recv[3] from P[3] = 1.00

P[3] sends Q_send[0] to P[2] = 7.95
P[3] sends Q_send[1] to P[2] = 7.95
P[3] sends Q_send[2] to P[2] = 0.38
P[3] sends Q_send[3] to P[2] = 0.38
P[3] receives Q_recv[0] from P[0] = -2.12
P[3] receives Q_recv[1] from P[0] = -2.12
P[3] receives Q_recv[2] from P[0] = 4.12
P[3] receives Q_recv[3] from P[0] = 4.12

Answer 1

在从 MPI_Irecv 访问数据之前，您必须完成一个MPI_Wait 或一个成功的MPI_Test。你不能用障碍代替它。

对于环形通信，考虑使用 MPI_Sendrecv。它可以比使用异步通信更简单。