Am 27.03.2014 um 23:59 schrieb Dave Love: > Reuti <re...@staff.uni-marburg.de> writes: > >> Do all of them have an internal bookkeeping of granted cores to slots >> - i.e. not only the number of scheduled slots per job per node, but >> also which core was granted to which job? Does Open MPI read this >> information would be the next question then. > > OMPI works with the bindings it's handed via orted (if the processes are > started that way). > >>> My understanding is that Torque delegates to OpenMPI the process placement >>> and binding (beyond the list of nodes/cpus available for >>> the job). > > Can't/doesn't torque start the MPI processes itself? Otherwise, yes, > since orted gets the binding. > >>> My guess is that OpenPBS behaves the same as Torque. >>> >>> SLURM and SGE/OGE *probably* have pretty much the same behavior. >> >> SGE/OGE: no, any binding request is only a soft request. > > I don't understand that. Does it mean the system-specific "strict" and > "non-strict" binding in hwloc, in which case I don't see how UGE can do > anything different? > >> UGE: here you can request a hard binding. But I have no clue whether this >> information is read by Open MPI too. >> >> If in doubt: use only complete nodes for each job (which is often done >> for massively parallel jobs anyway). > > There's no need with a recent SGE. All our jobs get core bindings -- > unless they use all the cores, since binding them all is equivalent to > binding none -- and OMPI inherits them. See > <http://arc.liv.ac.uk/SGE/howto/sge-configs.html#_core_binding> for the > SGE+OMPI configuration.
To avoid any misunderstanding I first discuss this last paragraph. I read http://www.slideshare.net/jsquyres/open-mpi-explorations-in-process-affinity-eurompi13-presentation which was posted on this list yesterday. And so I would phrase it: mapping to all is like mapping to none. And as they are only mapped, the kernel scheduler is free to move them around inside this set of (granted) cores. But maybe I got it wrong. -- Reuti > -- > Community Grid Engine: http://arc.liv.ac.uk/SGE/ > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
