On Feb 1, 2011, at 5:02 PM, Jeffrey A Cummings wrote: > I use OpenMPI on a variety of platforms: stand-alone servers running Solaris > on sparc boxes and Linux (mostly CentOS) on AMD/Intel boxes, also Linux > (again CentOS) on large clusters of AMD/Intel boxes. These platforms all > have some version of the 1.3 OpenMPI stream. I recently requested an upgrade > on all systems to 1.4.3 (for production work) and 1.5.1 (for > experimentation). I'm getting a lot of push back from the SysAdmin folks > claiming that OpenMPI is closely intertwined with the specific version of the > operating system and/or other system software (i.e., Rocks on the clusters). > I need to know if they are telling me the truth or if they're just making > excuses to avoid the work. To state my question another way: Apparently > each release of Linux and/or Rocks comes with some version of OpenMPI bundled > in. Is it dangerous in some way to upgrade to a newer version of OpenMPI? > Thanks in advance for any insight anyone can provide. > > - Jeff_______________________________________________
Hi Jeffrey As others said, Rocks has a default MPI (some version OpenMPI built with Gnu compilers with support for Ethernet only) which comes with the "hpc" Rocks roll. You can use that MPI, but you don't have to. This doesn't prevent you to install any other version of OpenMPI (actually of any other software) with support to whatever you have (e.g. Infiniband, Torque resource manager, using other compilers than Gnu, etc). The right location to install on Rocks is the /share/apps directory of the head/frontend node, which is NFS mounted on the nodes. It is wise to use subdirectories with names identifying your version somehow, e.g. /share/apps/ompi-1.4.3/intel-11.1.020, for something compiled with intel compilers. The --prefix=/share/apps/bla/bla option of OpenMPI configure will put the installed directory tree wherever you want. 'configure --help' will tell tons of possibilities (e.g. tight coupling with Torque os SGE, Infinband support, etc). You also need to set the user environment. A simple minded way is to prepend the OpenMPI bin directory to the PATH environment variable (say in the .bashrc/.cshrc user file), and the lib directory to the LD_LIBRARY_PATH. Adding share/man to the MANPATH is not mandatory, but helpful. This is rather inflexible and requires editing those initialization files every time you want to switch the MPI version you use, though. A much better and flexible way, as was also mentioned, is to use the environment modules, but your Sys Admin must be willing to learn how to write the corresponding module files (in Tcl/Tk jargon). This will allow you switch to across different versions by just issuing a command line like 'module switch path/to/old/version path/to/new/version'. See: http://modules.sourceforge.net/ I can't speak about Solaris, but it also supports environment modules, if I am not mistaken. I hope this helps, Gus Correa
