Am 27.07.2011 um 19:43 schrieb Lane, William: > Thank you for your help Ralph and Reuti, > > The problem turned out to be the number of file descriptors was insufficient. > > The reason given by a sys admin was that since SGE isn't a user it wasn't > initially using the new > upper bound on the number of file descriptors.
Yes, the sgeexecd inherits either the limit set at startup by the starting user, or the one defined in `qconf -sconf` in the "execd_params" setting as S_DESCRIPTORS and H_DESCRIPTORS. The latter way should be honored on the fly though. -- Reuti > -Bill Lane > > > > > ________________________________________ > From: users-boun...@open-mpi.org [users-boun...@open-mpi.org] on behalf of > Ralph Castain [r...@open-mpi.org] > Sent: Tuesday, July 26, 2011 1:22 PM > To: Open MPI Users > Subject: Re: [OMPI users] Can run OpenMPI testcode on 86 or fewer slots in > cluster, but nothing more than that > > On Jul 26, 2011, at 1:58 PM, Reuti wrote: >>>>> allocation_rule $fill_up >>>> >>>> Here you specify to fill one machine after the other completely before >>>> gathering slots from the next machine. You can change this to $round_robin >>>> to get one slot form each node before taking a second from particular >>>> machines. If you prefer a fixed allocation, you could also put an integer >>>> here. >>> >>> Remember, OMPI only uses sge to launch one daemon/node. The placement of >>> MPI procs is totally up to mpirun itself, which doesn't look at any SGE >>> envar. >> >> I thought this is the purpose to use --with-sge during configure as you >> don't have to provide any hostlist at all and Open MPI will honor it by >> reading SGE envars to get the granted slots? >> > > We use the envars to determine how many slots were allocated, but not the > placement. So we'll look at the envar to get the number of slots allocated on > each node, but we then determine the layout of processes against the slots. > To the point, we don't look at an sge envar to determine how that layout is > to be done. > > I was only trying to point out the difference. I admit it can be confusing > when using sge, especially since sge doesn't actually have visibility into > the MPI procs themselves (i.e., the only processes launched by sge are the > daemons). > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > IMPORTANT WARNING: This message is intended for the use of the person or > entity to which it is addressed and may contain information that is > privileged and confidential, the disclosure of which is governed by > applicable law. If the reader of this message is not the intended recipient, > or the employee or agent responsible for delivering it to the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this information is STRICTLY PROHIBITED. If you have received this > message in error, please notify us immediately by calling (310) 423-6428 and > destroy the related message. Thank You for your cooperation. > IMPORTANT WARNING: This message is intended for the use of the person or > entity to which it is addressed and may contain information that is > privileged and confidential, the disclosure of which is governed by > applicable law. If the reader of this message is not the intended recipient, > or the employee or agent responsible for delivering it to the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this information is STRICTLY PROHIBITED. > > If you have received this message in error, please notify us immediately > by calling (310) 423-6428 and destroy the related message. Thank You for > your cooperation. > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
