Hi Mark, I do not want to request two separate 1-node jobs, so that I can make maximum use of the MAUI queue algorithm on our local cluster, which sometimes favors jobs which utilize more resources.
I posted on this list, because someone here might have faced a similar problem before, and because how openmpi allocates cpus also depends on what options gromacs was originally compiled with (in connection to MPI). So its not strictly a 100% openmpi question. Thanks for the help -Himanshu On Nov 2, 2007 3:07 PM, Mark Abraham <[EMAIL PROTECTED]> wrote: > himanshu khandelia wrote: > > Hi, > > > > I am requesting 2 4-cpu nodes on a cluster using PBS. I want to run a > > separate GMX simulation on each 4-cpu node. However, on 2 nodes, the > > speed for each simulation decreases (50 to 100%) if compared to a > > simulation which runs in a job which requests only one node. I am > > guessing this is because openmpi fails to assign all cpus of the same > > node to one simulation ? Instead, cpus from different nodes are being > > used to run simulation. This is what I have in the PBS script > > At risk of pointing out the obvious, why not use 2 independent requests > for single 4-cpu nodes to do the job independently, or do the two jobs > as two successive 8-cpu jobs? > > > > 1. > > ######## > > mpirun -np 4 /people/disk2/hkhandel/gromacs-3.3.1/bin/mdrun_mpi -np 4 > > -v -s $mol.tpr -o $mol.trr -c $mol.gro -e $mol -g $mol.log > & > > $mol.out & > > mpirun -np 4 /people/disk2/hkhandel/gromacs-3.3.1/bin/mdrun_mpi -np 4 > > -v -s $mol2.tpr -o $mol2.trr -c $mol2.gro -e $mol2 -g $mol2.log > & > > $mol2.out & > > > > wait > > ######## > > > > OPENMPI does have a mechanism whereby one can assign specific > > processes to specific nodes > > http://www.open-mpi.org/faq/?category=running#mpirun-scheduling > > So, I have also tried all of the following in the PBS script where the > > --bynode or the --byslot option is used > > > > 2. > > ######## > > mpirun -np 4 --bynode > > /people/disk2/hkhandel/gromacs-3.3.1/bin/mdrun_mpi etc. & > > mpirun -np 4 --bynode > > /people/disk2/hkhandel/gromacs-3.3.1/bin/mdrun_mpi etc. & > > wait > > ######## > > > > 3. > > ######## > > mpirun -np 4 --byslot /people/disk2/hkhandel/gromacs-3.3.1/bin/mdrun_mpi > > etc. & > > mpirun -np 4 --byslot /people/disk2/hkhandel/gromacs-3.3.1/bin/mdrun_mpi > > etc. & > > wait > > ######## > > > > But these methods also result in similar performance losses. > > > > So how does one assign the cpus properly using mpirun if running > > different simulations in the same PBS job ?? > > This problem has got nothing to do with GROMACS, and so the people > you're asking here might well have no idea. You should probably be > asking an OpenMPI mailing list, since your problem is "how do I > allocation these 4-processor jobs to specific subsets of 4 processors", > not "how do I make GROMACS jump through hoops" :-) > > Mark > _______________________________________________ > gmx-users mailing list gmx-users@gromacs.org > http://www.gromacs.org/mailman/listinfo/gmx-users > Please search the archive at http://www.gromacs.org/search before posting! > Please don't post (un)subscribe requests to the list. Use the > www interface or send it to [EMAIL PROTECTED] > Can't post? Read http://www.gromacs.org/mailing_lists/users.php > _______________________________________________ gmx-users mailing list gmx-users@gromacs.org http://www.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to [EMAIL PROTECTED] Can't post? Read http://www.gromacs.org/mailing_lists/users.php
