OK. I'll try that, too. Also,
> BTW: did you set that mpi_show_mca_params option to ensure > the app is actually seeing these params? I'm working to get to a point where I can get some time to try that. Hopefully it will be before 5PM EDT. Jim > -----Original Message----- > From: users-boun...@open-mpi.org > [mailto:users-boun...@open-mpi.org] On Behalf Of Ralph Castain > Sent: Tuesday, June 23, 2009 2:43 PM > To: Open MPI Users > Subject: Re: [OMPI users] 50% performance reduction due to > OpenMPI v 1.3.2forcing all MPI traffic over Ethernet instead > of using Infiniband > > Assuming you aren't oversubscribing your nodes, set > mpi_paffinity_alone=1. > > BTW: did you set that mpi_show_mca_params option to ensure > the app is actually seeing these params? > > > > On Tue, Jun 23, 2009 at 12:35 PM, Jim Kress > <jimkress...@kressworks.org> wrote: > > > I assume you a referring to the openmpi-mca-params.conf file > > As I indicated previously, my first run was with the line > > btl=self,openib > > As the only entry in the openmpi-mca-params.conf file. > This my default > setting and was what I used, and it worked well, for v 1.2.8 > > Then I tried > > btl=self,openib > mpi_yield_when_idle=0 > > As the only entries in the openmpi-mca-params.conf > file. No difference in > the results. > > Then I tried > > btl=self,openib > mpi_yield_when_idle=0 > > As the only entries in the openmpi-mca-params.conf file > and also set the > environment variable OMPI_MCA_mpi_leave_pinned=0 > No difference in the results. > > What else can I provide? > > By the way, did you read the message where I retracted > my assumption about > MPI traffic being forced over Ethernet? > > Jim > > > > -----Original Message----- > > From: users-boun...@open-mpi.org > > [mailto:users-boun...@open-mpi.org] On Behalf Of > Pavel Shamis (Pasha) > > Sent: Tuesday, June 23, 2009 7:24 AM > > To: Open MPI Users > > Subject: Re: [OMPI users] 50% performance reduction due to > > OpenMPI v 1.3.2 forcing all MPI traffic over Ethernet instead > > of using Infiniband > > > > Jim, > > Can you please share with us you mca conf file. > > > > Pasha. > > Jim Kress ORG wrote: > > > For the app I am using, ORCA (a Quantum Chemistry > program), when it > > > was compiled using openMPI 1.2.8 and run under > 1.2.8 with the > > > following in the openmpi-mca-params.conf file: > > > > > > btl=self,openib > > > > > > the app ran fine with no traffic over my Ethernet > network and all > > > traffic over my Infiniband network. > > > > > > However, now that ORCA has been recompiled with openMPI > > v1.3.2 and run > > > under 1.3.2 (using the same openmpi-mca-params.conf > file), the > > > performance has been reduced by 50% and all the MPI traffic > > is going > > > over the Ethernet network. > > > > > > As a matter of fact, the openMPI v1.3.2 performance now > > looks exactly > > > like the performance I get if I use MPICH 1.2.7. > > > > > > Anyone have any ideas: > > > > > > 1) How could this have happened? > > > > > > 2) How can I fix it? > > > > > > a 50% reduction in performance is just not > acceptable. Ideas/ > > > suggestions would be appreciated. > > > > > > Jim > > > > > > _______________________________________________ > > > users mailing list > > > us...@open-mpi.org > > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > > > > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > >
