[OMPI users] MPI_Type_free and non-blocking operations

[Thomas Ponweiser]

Dear friends of Open MPI,

I am currently facing a problem in connection with MPI_Ibcast and 
MPI_Type_free. I've been able to isolate the problem in a minimalistic test 
program which I attached. 

Maybe some of you can tell me what I am doing wrong or confirm that this might 
be a bug in Open MPI (I am using version 1.10.1).

Here is what I am doing:
1) I have two struct types, foo_type and bar_type, as follows:

typedef struct
{
   int v[6];
   long l;
} foo_type;

typedef struct
{
   int v[3];
   foo_type foo;
} bar_type;

2) I am creating corresponding MPI types (foo_mpitype and bar_mpitype) with 
MPI_Type_create_struct.

3) I am freeing foo_mpitype.

4) I am broadcasting a variable of type bar_type with MPI_Ibcast (using count = 
1 and datatype = bar_mpitype).

5) I am freeing bar_mpitype.

6) I am waiting for the completion of step 4) with MPI_Wait.

In step 6) I get a segmentation fault within MPI_Wait, but only if the MPI job 
is larger than 4 processes.

Testing with MPICH 3.2, the program seems to work just fine.

I found out that swapping the steps 5) and 6) helps. But I think this should 
not make any difference, according to the description of MPI_Type_free at 
http://www.mpi-forum.org/docs/mpi-1.1/mpi-11-html/node58.html: "Any 
communication that is currently using this datatype will complete normally." 
And: " Freeing a datatype does not affect any other datatype that was built 
from the freed datatype."

(In fact, doing the same thing, that is MPI_IBcast followed by MPI_Type_free 
followed by MPI_Wait, with foo_type and foo_mpitype seems to work fine).

Thanks in advance for your help, 

kind regards,
Thomas


#include <mpi.h>
#include <stdio.h>

typedef struct
{
   int v[6];
   long l;
} foo_type;

typedef struct
{
   int v[3];
   foo_type foo;
} bar_type;

int main(int argc, char* argv[])
{
   MPI_Datatype foo_mpitype, bar_mpitype;
   MPI_Request request;

   MPI_Aint base, displacements[2];
   int i, blocklengths[2];
   MPI_Datatype types[2];

   foo_type foo = {0};
   bar_type bar = {0};

   // Init MPI
   MPI_Init(&argc, &argv);

   // Create foo MPI type
   blocklengths[0] = 6; // 6 x int
   blocklengths[1] = 1; // 1 x long
   types[0] = MPI_INT;
   types[1] = MPI_LONG;
   MPI_Get_address(&foo, &base);
   MPI_Get_address(&foo.v, &displacements[0]);
   MPI_Get_address(&foo.l, &displacements[1]);
   for(i = 0; i < 2; i++) displacements[i] -= base;
   MPI_Type_create_struct(2, blocklengths, displacements, types, &foo_mpitype);
   MPI_Type_commit(&foo_mpitype);

   // Create bar MPI type
   blocklengths[0] = 3; // 3 x int
   blocklengths[1] = 1; // 1 x bar_type
   types[0] = MPI_INT;
   types[1] = foo_mpitype;
   MPI_Get_address(&bar, &base);
   MPI_Get_address(&bar.v, &displacements[0]);
   MPI_Get_address(&bar.foo, &displacements[1]);
   for(i = 0; i < 2; i++) displacements[i] -= base;
   MPI_Type_create_struct(2, blocklengths, displacements, types, &bar_mpitype);
   MPI_Type_commit(&bar_mpitype);
   
   // Non-blocking broadcast of foo
   MPI_Ibcast(&foo, 1, foo_mpitype, 0, MPI_COMM_WORLD, &request);
   MPI_Type_free(&foo_mpitype);
   MPI_Wait(&request, MPI_STATUS_IGNORE); // WORKS

   // Non-blocking broadcast of bar
   MPI_Ibcast(&bar, 1, bar_mpitype, 0, MPI_COMM_WORLD, &request);
   MPI_Type_free(&bar_mpitype);
   MPI_Wait(&request, MPI_STATUS_IGNORE); // SEGMENTATION FAULT, when number of 
processes > 4

   // Finalize
   MPI_Finalize();
}