I am trying to craft a client-server layer that needs to have 2 different modes
of operation. In the “remote server” mode, then the server runs on distinct
processes, and intercommunicator is a perfect fit for my design. In the
“local server” the server will actually run on a dedicate thread within the
same pool of processes that are running the client. What I would like is
some analog of an intercommunicator that would operate between two comms that
are essentially identical (but distinct due to use of MPI_Comm_dup()), with
one running on thread 0 and the other running on thread 1.
Now, I know the language in the standard sounds like it explicitly forbids such
overlapping groups:
"Overlap of local and remote groups that are bound into an inter-communicator
is prohibited.”
But there is then that hint of wiggle room with the parenthetical:
“(If a process is multithreaded, and MPI calls block only a thread, rather than
a process, then “dual membership” can be supported. It is then the user’s
responsibility to make sure that calls on behalf of the two “roles” of a
process are executed by two independent threads.)"
This sounds very much like what I want to do, but the following example code
crashes (apologies for Fortran source). It is intended to be run on two
threads - each of which creates a dup of MPI_COMM_WORLD. The intercomm create
would then have the “other” comm as the remote for each thread. I’m not
wed to the details here. Anything that would support the notion of an MPI
based server running on extra threads would be a potential solution. I know
that in the worst case, I could achieve the same by oversubscribing processes
on my nodes, but that pushes some issues in our batch processing system that
I’d prefer to avoid. A definitive “Can’t be done. Move along.” would also be
useful.
The following fails with OpenMPI 2.1 on OS X 10.12:
program main
use mpi
implicit none
integer, external :: omp_get_thread_num
integer :: ierror
integer :: comms(0:1)
integer :: comm
integer :: tag = 10
integer :: intercomm
integer :: provided
integer :: i
call MPI_Init_thread(MPI_THREAD_MULTIPLE, provided, ierror)
call MPI_Comm_dup(MPI_COMM_WORLD, comm, ierror)
!$omp parallel num_threads(2) default(none), private(i, intercomm, ierror),
shared(tag, comms, comm)
i = omp_get_thread_num()
if (i == 0) then
call MPI_Comm_dup(MPI_COMM_WORLD, comms(i), ierror)
end if
!$omp barrier
if (i == 1) then
call MPI_Comm_dup(MPI_COMM_WORLD, comms(i), ierror)
end if
!$omp barrier
call MPI_Intercomm_create(comms(i), 0, comm, 0, tag, intercomm, ierror)
!$omp end parallel
stop
call MPI_Finalize(ierror)
end program main
Thanks in advance.
- Tom
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
