IIRC, I added relative node syntax to the rankfile mapper a while back. You should then be able to specify the rank->node mapping by indicating that (e.g.) rank 0 -> first node in file, instead of specifying the node by name.
On Jun 16, 2010, at 2:21 PM, Brock Palen wrote: > We use Torque and Moab as our resource manager, > I have a user who has to mix programming models, threads and MPI, but not on > all ranks, > > Example: > Rank 0 needs 4 cores > Rank 1 needs 4 cores > Ranks 2-5 need 1 > > There is no way to give this to Torque/Moab in a way that when using TM > openMPI correctly starts the layout I need. > I found I can use a rankfile to do this, > > cat rankfile > rank 0=nyx0456 slot=0 > rank 1=nyx0457 slot=0 > rank 2=nyx0458 slot=0 > rank 3=nyx0458 slot=1 > rank 4=nyx0458 slot=2 > rank 5=nyx0458 slot=3 > > qsub -I -l nodes=4:ppn=4 > [brockp@nyx0456 tmp]$ mpirun -np 6 -rf rankfile hostname | sort | uniq -c > 1 nyx0456.engin.umich.edu > 1 nyx0457.engin.umich.edu > 4 nyx0458.engin.umich.edu > > (ignore the cpu count mismatch, I am not using all 16 cores in this example) > > I could make a script to build this rankfile, I want to know if there is a > better way to do this, > Also we don't want to use affinity, as we don't know what 'cpus' the bath > (pbs/moab) system gave us, is the above rankfile going to do the right > thing? That is the processes will float? Is there a way to tell if affinity > is set? Or to say "rank 0 goes to this host use any available cores" > > Thanks! > > Brock Palen > www.umich.edu/~brockp > Center for Advanced Computing > bro...@umich.edu > (734)936-1985 > > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
