Actually there was the same number of procs per node in each case. I verified this by logging into the nodes while they were running--in both cases 4 per node .
Ron --- Ron Cohen recoh...@gmail.com skypename: ronaldcohen twitter: @recohen3 On Fri, Mar 25, 2016 at 1:14 PM, Ralph Castain <r...@open-mpi.org> wrote: > >> On Mar 25, 2016, at 9:59 AM, Ronald Cohen <recoh...@gmail.com> wrote: >> >> It is very strange but my program runs slower with any of these >> choices than if IO simply use: >> >> mpirun -n 16 >> with >> #PBS -l >> nodes=n013.cluster.com:ppn=4+n014.cluster.com:ppn=4+n015.cluster.com:ppn=4+n016.cluster.com:ppn=4 >> for example. > > This command will tightly pack as many procs as possible on a node - note > that we may well not see the PBS directives regarding number of ppn. Add > —display-allocation and let’s see how many slots we think were assigned on > each node > >> >> The timing for the latter is 165 seconds, and for >> #PBS -l nodes=4:ppn=16,pmem=1gb >> mpirun --map-by ppr:4:node -n 16 >> it is 368 seconds. > > It will typically be faster if you pack more procs/node as they can use > shared memory for communication. > >> >> Ron >> >> --- >> Ron Cohen >> recoh...@gmail.com >> skypename: ronaldcohen >> twitter: @recohen3 >> >> >> On Fri, Mar 25, 2016 at 12:43 PM, Ralph Castain <r...@open-mpi.org> wrote: >>> >>>> On Mar 25, 2016, at 9:40 AM, Ronald Cohen <recoh...@gmail.com> wrote: >>>> >>>> Thank you! I will try it! >>>> >>>> >>>> What would >>>> -cpus-per-proc 4 -n 16 >>>> do? >>> >>> This would bind each process to 4 cores, filling each node with procs until >>> the cores on that node were exhausted, to a total of 16 processes within >>> the allocation. >>> >>>> >>>> Ron >>>> --- >>>> Ron Cohen >>>> recoh...@gmail.com >>>> skypename: ronaldcohen >>>> twitter: @recohen3 >>>> >>>> >>>> On Fri, Mar 25, 2016 at 12:38 PM, Ralph Castain <r...@open-mpi.org> wrote: >>>>> Add -rank-by node to your cmd line. You’ll still get 4 procs/node, but >>>>> they will be ranked by node instead of consecutively within a node. >>>>> >>>>> >>>>> >>>>>> On Mar 25, 2016, at 9:30 AM, Ronald Cohen <recoh...@gmail.com> wrote: >>>>>> >>>>>> I am using >>>>>> >>>>>> mpirun --map-by ppr:4:node -n 16 >>>>>> >>>>>> and this loads the processes in round robin fashion. This seems to be >>>>>> twice as slow for my code as loading them node by node, 4 processes >>>>>> per node. >>>>>> >>>>>> How can I not load them round robin, but node by node? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Ron >>>>>> >>>>>> >>>>>> --- >>>>>> Ron Cohen >>>>>> recoh...@gmail.com >>>>>> skypename: ronaldcohen >>>>>> twitter: @recohen3 >>>>>> >>>>>> --- >>>>>> Ronald Cohen >>>>>> Geophysical Laboratory >>>>>> Carnegie Institution >>>>>> 5251 Broad Branch Rd., N.W. >>>>>> Washington, D.C. 20015 >>>>>> _______________________________________________ >>>>>> users mailing list >>>>>> us...@open-mpi.org >>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>> Link to this post: >>>>>> http://www.open-mpi.org/community/lists/users/2016/03/28828.php >>>>> >>>>> _______________________________________________ >>>>> users mailing list >>>>> us...@open-mpi.org >>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>> Link to this post: >>>>> http://www.open-mpi.org/community/lists/users/2016/03/28829.php >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> Link to this post: >>>> http://www.open-mpi.org/community/lists/users/2016/03/28830.php >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2016/03/28831.php >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2016/03/28832.php > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2016/03/28833.php
