So another question on this topic: I'm running on a 64-bit cluster. Is it important/needed/useful to maintain 32-bit and 64-bit versions of openmpi and such? At the moment, I'm using Rocks' default openmpi, which includes both 32 and 64-bit versions of the libraries. Yet, so far, my attempt to build a 32-bit on the 64-bit system has proven very difficult, and I finally resolved to just install a 32-bit build host to build the 32-bit versions if required.
Thanks! Oh, and I'd be very grateful for any samples and automation scripts to help with the whole setup. I'm pretty new to "real cluster management" :) Thanks again! --Jim On 11/15/07, Brock Palen <bro...@umich.edu> wrote: > Really modules is the only way to go, we use it to maintain no less > than 12 versions of openmpi compiled with pgi, gcc, nag, and intel. > > Yes as the other reply said just set LD_LIBRARY_PATH you can use > > ldd executable > > to see which libraries the executable is going to use under the > current environment. We also have uses submit to torque with > #PBS -V > > which copies the current environment with the job, this allows users > to submit and have multiple jobs running each with their own > environment. If your using PBS or torque look at the qsub man page. > > Another addition we came up with was after we load a default set a > modules (pbs, a compiler , mpi library) we look for a file in > > $HOME/privatemodules/default > > which just has a list of module commands. This allows users to > change their default modules for every login. > For example i have in my default file > > module load git > > Because i use git for managing files. > > We have done other hacks, our webpage of software is built out of > cron from the module files: > http://cac.engin.umich.edu/resources/systems/nyxV2/software.html > > So we dont maintain software lists online, we just generate it > dynamically. > Modules is the admins best friend. > > Brock Palen > Center for Advanced Computing > bro...@umich.edu > (734)936-1985 > > > On Nov 15, 2007, at 8:54 AM, Jim Kusznir wrote: > > > Hi all: > > > > I'm trying to set up a cluster for a group of users with very > > different needs. So far, it looks like I need gcc, pgi, and intel to > > work with openmpi and mpich, with each user able to control what > > combination they get. This is turning out to be much more difficult > > than I expected. > > > > Someone has pointed me to enviornment-modules ("Modules"), which looks > > like it will be a critical part of the solution. I even noticed that > > the provided openmpi.spec file has some direct support for modules, > > which makes me quite happy! > > > > However, I still have many questions about how to set things up. > > > > First, I get the impression that openmpi will need to be compiled with > > each compiler that will use it. If this is true, I'm not quite sure > > how to go about it. I could install in different directories for the > > user commands, but what about the libraries? I don't think I have a > > feesable way of selecting which library to use on the fly on the > > entire cluster for each user, so it seems like it would be better to > > have all the libraries available. In addition, I will need RPMs to > > deploy efficiently on the cluster. I suspect I can change the > > versioning info and build with each compiler, but at this point, I > > don't even know how to reliably select what compiler rpmbuild will use > > (I've only succeeded in using gcc). > > > > Finally, using modules, how do I set it up so that if a user changes > > compilers, but stays with openmpi, it will load the correct openmpi > > paths? I know I can set up the openmpi module file to load after the > > compiler module and based on that select different paths based on the > > currently-loaded compiler module. If the user changes the compiler > > module, will that cause the mpi module to also be reloaded so the new > > settings will be loaded? Or do I need this at all? > > > > Thanks for all your help! > > --Jim > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
