[OMPI users] Collective MPI-IO + MPI_THREAD_MULTIPLE

[Sebastian Rettenberger]

Hi,


I am using OpenMPI 1.10.0 with support for MPI_THREAD_MULTIPLE.



Any idea how one can get around this issue?

Best regards,
Sebastian

--
Sebastian Rettenberger, M.Sc.
Technische Universität München
Department of Informatics
Chair of Scientific Computing
Boltzmannstrasse 3, 85748 Garching, Germany
http://www5.in.tum.de/

#include <mpi.h>
#include <cassert>
#include <unistd.h>
#include <pthread.h>

static int rank;
static int size;

MPI_Comm comm2;

static void* thread2(void *p)
{
	// Wait everywhere but rank 0
	if (rank != 0)
		sleep(5);
	
	MPI_File fh;
	MPI_File_open(comm2, "test2.dat", MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &fh);
	
	// Write collective
	MPI_File_set_view(fh, rank*sizeof(int), MPI_INT, MPI_INT, "native", MPI_INFO_NULL);
	MPI_File_write_all(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE);
	
	MPI_File_close(&fh);
	
	return 0L;
}

int main(int argc, char** argv)
{
	int threadSupport;
	MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, &threadSupport);
	assert(threadSupport >= MPI_THREAD_MULTIPLE);
	
	MPI_Comm_rank(MPI_COMM_WORLD, &rank);
	MPI_Comm_size(MPI_COMM_WORLD, &size);
	
	// Create 2. communicator
	MPI_Comm_dup(MPI_COMM_WORLD, &comm2);
	
	pthread_t thread;
	pthread_create(&thread, 0L, thread2, 0L);
	
	// Wait 2 sec on rank 0
	if (rank == 0)
		sleep(5);
	
	// Create a file
	MPI_File fh;
	MPI_File_open(MPI_COMM_WORLD, "test1.dat", MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &fh);
	
	// Write collective
	MPI_File_set_view(fh, rank*sizeof(int), MPI_INT, MPI_INT, "native", MPI_INFO_NULL);
	MPI_File_write_all(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE);
	
	MPI_File_close(&fh);
	
	pthread_join(thread, 0L);
	
	MPI_Comm_free(&comm2);
	
	MPI_Finalize();
	
	return 0;
}

I tried to run the attached program with OpenMPI. It works well with MPICH and Intel MPI but I get a deadlock when using OpenMPI. It seems like ROMIO uses global locks in OpenMPI which is a problem if multiple threads want to do collective I/O.

smime.p7s
Description: S/MIME Cryptographic Signature