On Fri, 2007-11-30 at 19:05 -0800, Jim Kusznir wrote:
> Thank you for your response!
>
> Just to clarify some things for my understanding:
>
> Do users load a single module that specifies both compiler and mpi
> version (as opposed to loading two different modules, one for
> complier, and one for mpi)?
We combine compiler and MPI because it's necessary that the
mpifcc/cxx/f90 be compatible with the compiler, so the correct compiler
needs to be in the user's path, and it's just easier on the users to
have them load a single module. A module can load another module, so if
you create a module for, say, the PGI compilers, the appropriate OpenMPI
module can load that module itself.
Here's an example, my ompi-pgi script (note: all my scripts were cribbed
from old ROCKS scripts; ROCKS doesn't include modules by default anymore
but they used to provide it):
#%
Module3.0#####################################################################
##
## CRAYfw mpich
##
proc ModulesHelp { } {
global version MPIhome
puts stderr "\tCRAY freeware: OpenMPI\n"
puts stderr "\tThis module loads the lastest versions of OpenMPI.\n"
puts stderr "\n\tThis adds $OpenMPI/* to several of the"
puts stderr "\tenvironment variables."
puts stderr "\n\tVersion $version\n"
}
module-whatis "loads the OpenMPI environment for PGI over Ethernet"
# for Tcl script use only
set version 1.2
set MPIhome /opt/openmpi-1.2/ethernet/pgi
prepend-path PATH $MPIhome/bin
prepend-path INCLUDE $MPIhome/include
prepend-path INFOPATH $MPIhome/share
prepend-path LD_LIBRARY_PATH $MPIhome/lib
setenv OMPI_MPIBIN $MPIhome
setenv OMPI_MPIHOME $MPIhome
setenv OMPI_MPIF77 "pgf77"
setenv OMPI_MPIF90 "pgf90"
setenv OMPI_MPICC "pgcc"
setenv OMPI_MPICXX "pgCC"
if [ module-info mode load ] {
if {! [ is-loaded pgi ] } { module load pgi }
if { [ is-loaded ompi-pgi ] } {
puts stderr "\tModule ompi-pgi already loaded\n"
}
if { [ is-loaded ompi-gnu ] } {
puts stderr "\tModule ompi-gnu superceded\n"
}
if { [ is-loaded ompi-intel ] } {
puts stderr "\tModule ompi-intel superceded\n"
}
if { [ is-loaded mpich-eth-pgi ] } {
puts stderr "\tModule mpich-eth-pgi environment superceded.\n"
}
if { [ is-loaded mpich-eth-gnu ] } {
puts stderr "\tModule mpich-eth-gnu environment superceded.\n"
}
if { [ is-loaded mpich-eth-intel ] } {
puts stderr "\tModule mpich-eth-intel environment superceded.\n"
}
}
>
> So, if the MPI implementations need to be built separately for each
> compiler, what about other software packages that users' link against?
> For example, as far as I know, there are not seperate versions of
> libc for each compiler (but I may be ignorant here....please correct
> me if I'm wrong). How about other packages that code #includes? How
> do I know when something needs to be compiled separately vs one
> version for everyone?
We build a distinct version at least for every library that has F90/95
bindings. Probably a good idea for C++ as well.
>
> Another thing that I'm still not quite clear on: Can I have all
> versions of the various MPI libraries in the ld path at once (so the
> user doesn't have to "select" the correct one somehow prior to code
> execution)?
I'll let somebody who is more of a real system administrator answer
that.
>
> Are there any tricks to building/installing all this stuff in a fairly
> automated manner?
>
We use ROCKS. You have to create an rpm for everything, but once you do
then propagating it is automated.
--
Katherine Holcomb, Ph.D. kholc...@virginia.edu
Research Computing Support Group - ITC Office Phone: (434) 982-5948
I046 BSEL, Clark Hall Center Phone: (434) 243-8799
University of Virginia 22904
