----- Mensaje original -----
> De: "Ralf Wildenhues" <ralf.wildenh...@gmx.de>
> Para: "Jorge D'ELIA" <jde...@intec.unl.edu.ar>
> CC: "Tobias Burnus" <bur...@net-b.de>,
> "gfortran" <fort...@gcc.gnu.org>,
> "GCC Mailing List" <gcc@gcc.gnu.org>,
> "Rainer Orth" CeBiTec.Uni-Bielefeld.DE>
> Enviados: Sábado, 9 de Abril 2011 6:34:05
> Asunto: Re: [RFC] gfortran's coarray (library version):
> configure/build and the testsuite
>
> Hello,
>
> * Jorge D'ELIA wrote on Wed, Apr 06, 2011 at 01:24:58AM CEST:
> > Here there are few comments from my college Lisandro Dalcin,
> > an external developer of PETSc, e.g. see
>
> > ----- Mensaje original -----
> > > The current scheme is that the user somehow compiles
> > > the communication library (libcaf) [2] and then builds
> > > and links doing something like:
> > >
> > > mpif90 -fcoarray=lib fortran.f90 -lcaf_mpi
> > >
> > > or alternatively
> > >
> > > gfortran -fcoarray=lib fortran.f90 -lcaf_mpi
> > > -I/usr/lib64/mpi/gcc/openmpi/include
> > > -L/usr/lib64/mpi/gcc/openmpi/lib64
> > > -lmpi
> > >
> > > with some -I<dir>, -L<dir> -l<libs> are added.
> > > (Cf. "mpif90 -show" of some MPI implementations.)
> > >
> > > The resulting program is then run using, e.g.,
> > >
> > > mpiexec -n 3 ./a.out
> > >
> > > Alternatively, it could be just "-lcaf_single"
> > > which is run like normal ("./a.out").
>
> I think one of the most important things is that you allow to override
> both the running of mpif90 and the mpiexec commands, so as to allow
> batch environments (qsub, llrun). Although, parsing of output might
> need to be more complex in that case, too. But if only to allow for
> different mpiexec.* incarnations this would be good.
>
> > > (In any case, only static libraries should be created;
> > > the libraries could then be installed
> > > in $PREFIX/$lib/gcc/$target/$version/, where
> > > already libgcc.a etc. are located.)
> >
> > COMMENT: well, using shared libraries would certainly help
> > users to switch the underlying MPI implementation at runtime.
> > This is an feature should be considered.
>
> The MPI implementations I know all have pairwise incompatible ABIs,
> prohibiting any kind of switching at run time. If anything, GCC
> might consider making it easy to build and install in parallel the
> library for multiple MPI implementations.
>
> Cheers,
> Ralf
Hi Ralf,
It is clear that the present discussion is due to that
the MPI's binaries are not compatibles yet among the MPI
implementations.
To ensure that the compiler is as neutral as possible
with respect to an end user, we should see how to
overcome that problem.
Among other possibilities we consider 5 options, from
the simplest to more elaborates:
1) A static library "libcaf_mpi.a", that was proposed previously
in another thread. It uses a specific MPI implementation when
the library is built (e.g. openmpi or mpich). This possibility
is the simplest one. However, it only works with the MPI
distribution available when the library is built.
2) A dynamic library "libcaf_mpi.so" that uses a specific MPI
implementation (e.g. openmpi or mpich) when the library is
built. However, again, it only works with the MPI implementation
available when the library is built. From an usability point of
view, this option is not too much different than the previous
one.
3) A symbolic link "libcaf_mpi.so" points to "libcaf_mpich.so"
or "libcaf_openmpi.so". The sysadmin can manage the link using
tools like "alternatives". Linux distributions usually take care
of this infrastructure by themselves, no more work required
from Gfortran side. However, regular (non-root) users cannot
switch the backend MPI. This only available on POSIX systems.
4) Different dynamic libraries named "libcaf_mpi.so" are
built for each MPI implementation, but they are installed
in differente directories, e.g.
<some-prefix>/mpich/libcaf_mpi.so or
<some-prefix>/openmpi/libcaf_mpi.so.
By using the "modules" tool, users can select the preferred
MPI implementation (e.g. "module load mpich2-x86_64" in
Fedora). This works by adding entries in the LD_LIBRARY_PATH
enviroment variable. Linux distributions usually take care
of this infrastructure by themselves, no more work required
from Gfortran side. Regular users are able to choose
the prefered MPI implementation, the dynamic linker loads
the appropiate "libcaf_mpi.so" .
5) A dynamic library a "libcaf_mpi.so" built using the
dlopen() tool. This option could be practical if the number
of MPI functions would be no too large (BTW, how many?),
and with a bonus that it can be built on both Linux and
Windows systems. If there is interest, we can contribute
something in this direction.
5) A dynamic library "libcaf_mpi.so" is not linked
with MPI, but uses the dlopen() and dlsym(), to load
and access the contents of a specific "MPI-linked"
dynamic library "libcaf_mpi_{mpich2|openmpi|other}.so".
This way "libcaf_mpi.so" does not depend on any MPI
implementation, but acts as a thin wrapper to the
specific caf + MPI library. By using enviroment
variables or rc configuration files, user can choose
the preferred library to open at runtime with dlopen().
Although dlopen() is specific to POSIX systems,
similar mechanisms are available on Windows.
From all the previous options, #5 is the more robust and
convenient: regular users can switch MPI (as opposed to #3),
enviroment variables are not strictly required (as in #4).
As sysadmin could choose a default MPI by editing a
config file at /etc (for instance) and regular users
could make a different choice by editing a config file
at $HOME.
Cheers,
Lisandro / Jorge.
--
CIMEC, http://www.cimec.org.ar/
INTEC, Guemes 3450, (S3000GLN) Santa Fe, ARGENTINA
tel: +54-342-451.15.94, fax: +54-342-451.11.69
tel. +54-342-455.91.75, +54-342-455.84.50/51
Universidad Nacional del Litoral (UNL)
Consejo Nacional de Investigaciones Cientificas y Tecnicas (CONICET)
