[OMPI users] Beginner's question: why multiple sends or receives don't work?

[Xianglong Kong]

Hi, all,

I’m an mpi newbie. I’m trying to connect two desktops in my office
with each other using a crossing cable and implement a parallel code
on them using MPI.

Now, the two nodes can ssh to each other without password, and can
successfully run the MPI “Hello world” code. However, when I tried to
use multiple MPI non-blocking sends or receives, the job would hang.
The problem only showed up if the two processes are launched in the
different nodes, the code can run successfully if the two processes
are launched in the same node. Also, the code can run successfully if
there are only one send or/and one receive in each process.

Here is the code that can run successfully:

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <mpi.h>

int main(int argc, char** argv) {

        int myrank, nprocs;

        MPI_Init(&argc, &argv);
        MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
        MPI_Comm_rank(MPI_COMM_WORLD, &myrank);

        printf("Hello from processor %d of %d\n", myrank, nprocs);

        MPI_Request reqs1, reqs2;
        MPI_Status stats1, stats2;

        int tag1=10;
        int tag2=11;

        int buf;
        int mesg;
        int source=1-myrank;
        int dest=1-myrank;

        if(myrank==0)
        {
                mesg=1;

                MPI_Irecv(&buf, 1, MPI_INT, source, tag1, MPI_COMM_WORLD, 
&reqs1);
                MPI_Isend(&mesg, 1, MPI_INT, dest,  tag2, MPI_COMM_WORLD, 
&reqs2);


        }

        if(myrank==1)
        {
                mesg=2;

                MPI_Irecv(&buf, 1, MPI_INT, source, tag2, MPI_COMM_WORLD, 
&reqs1);
                MPI_Isend(&mesg, 1, MPI_INT,  dest, tag1, MPI_COMM_WORLD, 
&reqs2);
        }

        MPI_Wait(&reqs1, &stats1);
        printf("myrank=%d,received the message\n",myrank);

        MPI_Wait(&reqs2, &stats2);
        printf("myrank=%d,sent the messages\n",myrank);

        printf("myrank=%d, buf=%d\n",myrank, buf);

        MPI_Finalize();
        return 0;
}

And here is the code that will hang

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <mpi.h>

int main(int argc, char** argv) {

        int myrank, nprocs;

        MPI_Init(&argc, &argv);
        MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
        MPI_Comm_rank(MPI_COMM_WORLD, &myrank);

        printf("Hello from processor %d of %d\n", myrank, nprocs);

        MPI_Request reqs1, reqs2;
        MPI_Status stats1, stats2;

        int tag1=10;
        int tag2=11;

        int source=1-myrank;
        int dest=1-myrank;

        if(myrank==0)
        {
                int buf1, buf2;

                MPI_Irecv(&buf1, 1, MPI_INT, source, tag1, MPI_COMM_WORLD, 
&reqs1);
                MPI_Irecv(&buf2, 1, MPI_INT, source, tag2, MPI_COMM_WORLD, 
&reqs2);

                MPI_Wait(&reqs1, &stats1);
                printf("received one message\n");

                MPI_Wait(&reqs2, &stats2);
                printf("received two messages\n");

                printf("myrank=%d, buf1=%d, buf2=%d\n",myrank, buf1, buf2);
        }

        if(myrank==1)
        {
                int mesg1=1;
                int mesg2=2;    

                MPI_Isend(&mesg1, 1, MPI_INT, dest, tag1, MPI_COMM_WORLD, 
&reqs1);
                MPI_Isend(&mesg2, 1, MPI_INT, dest, tag2, MPI_COMM_WORLD, 
&reqs2);

                MPI_Wait(&reqs1, &stats1);
                printf("sent one message\n");

                MPI_Wait(&reqs2, &stats2);
                printf("sent two messages\n");
        }

        MPI_Finalize();
        return 0;
}

And the output of the second failed code:
***********************************************
Hello from processor 0 of 2

Received one message

Hello from processor 1 of 2

Sent one message
*******************************************************

Can anyone help to point out why the second code didn't work?

Thanks!

Kong