Re: [OMPI users] Ideal MTU in Infiniband
Hello all, Any ideas? -- Parag Kalra On Jan 10, 2008 4:15 AM, Parag Kalra wrote: > Hello all, > > I am using Open MPI with Infiniband configured. > > What should be the ideal MTU size for infiniband? > > -- > PARAG . A . KALRA > > > >
Re: [OMPI users] Open MPI v1.2.5 released
On Jan 9, 2008, at 3:50 PM, Warner Yuen wrote: Thanks to Brian Barrett, I was able to get through some ugly Intel compiler bugs during the configure script. I now have OMPI v1.2.5 running nicely under Mac OSX v10.5 Leopard! Excellent! However, I have a question about hostfiles. I would like to manually launch MPI jobs from my headnode, but I don't want the jobs to run on the head node. In LAM/MPI I could add a "hostname schedule=no" to the hostfile, is there an equivalent in OpenMPI? I'm sure this has come up before, but I couldn't find an answer in the archives. Actually, we tried to learn how *not* do to it from LAM :-) and did it a bit better in Open MPI (at least, we think so ;-) ). In Open MPI, you can simply not list the head node in the hostfile. More specifically: in LAM, you *have* to list the node where you invoke mpirun from in the hostfile (hence the need for the schedule=no attribute). In Open MPI, there is no such requirement, so you can just lead the head node's hostname out of the hostfile. -- Jeff Squyres Cisco Systems
Re: [OMPI users] Ideal MTU in Infiniband
On Thu, Jan 10, 2008 at 06:23:50PM +0530, Parag Kalra wrote: > Hello all, > > Any ideas? Yes. The idea is that Open MPI knows what best. Run it with a default value. Usually bigger MTU is better, but some HW has bugs. Open MPI knows this and choses the best value for your HW. > > -- > Parag Kalra > > > On Jan 10, 2008 4:15 AM, Parag Kalra wrote: > > > Hello all, > > > > I am using Open MPI with Infiniband configured. > > > > What should be the ideal MTU size for infiniband? > > > > -- > > PARAG . A . KALRA > > > > > > > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Gleb.
Re: [OMPI users] Ideal MTU in Infiniband
But still can someone please tell the numerical value that is idealy set or should be set. On Jan 10, 2008 8:29 PM, Gleb Natapov wrote: > On Thu, Jan 10, 2008 at 06:23:50PM +0530, Parag Kalra wrote: > > Hello all, > > > > Any ideas? > Yes. The idea is that Open MPI knows what best. Run it with a default > value. Usually bigger MTU is better, but some HW has bugs. Open MPI > knows this and choses the best value for your HW. > > > > > -- > > Parag Kalra > > > > > > On Jan 10, 2008 4:15 AM, Parag Kalra wrote: > > > > > Hello all, > > > > > > I am using Open MPI with Infiniband configured. > > > > > > What should be the ideal MTU size for infiniband? > > > > > > -- > > > PARAG . A . KALRA > > > > > > > > > > > > > > > ___ > > users mailing list > > us...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > -- >Gleb. > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > -- Love, PARAG . A . KALRA Good judgment comes from experience, and experience comes from bad judgment http://discoverlinux.blogspot.com A Linux machine! because a 486 is a terrible thing to waste
[OMPI users] openib problems
We just updated rhel4 a few days back and now we get the following errors when trying to run on infiniband nodes with openmpi-1.2.3 and openmpi-1.2.0 [0,1,1]: OpenIB on host nyx397 was unable to find any HCAs. Another transport will be used instead, although this may result in lower performance. -- libibverbs: Warning: no userspace device-specific driver found for / sys/class/infiniband_verbs/uverbs0 libibverbs: Warning: no userspace device-specific driver found for / sys/class/infiniband_verbs/uverbs0 Has anyone ever seen this error before? We are not that knowledgeable in infiniband so any help would be great. Brock Palen Center for Advanced Computing bro...@umich.edu (734)936-1985
Re: [OMPI users] openib problems
This can mean that you have a user-level libibverbs and kernel mismatch. Do any of the OFED sample programs work properly, or perhaps the ibv_devinfo program? (ibv_devinfo should query the HCAs on your host and list the status of all the ports) On Jan 10, 2008, at 2:33 PM, Brock Palen wrote: We just updated rhel4 a few days back and now we get the following errors when trying to run on infiniband nodes with openmpi-1.2.3 and openmpi-1.2.0 [0,1,1]: OpenIB on host nyx397 was unable to find any HCAs. Another transport will be used instead, although this may result in lower performance. -- libibverbs: Warning: no userspace device-specific driver found for / sys/class/infiniband_verbs/uverbs0 libibverbs: Warning: no userspace device-specific driver found for / sys/class/infiniband_verbs/uverbs0 Has anyone ever seen this error before? We are not that knowledgeable in infiniband so any help would be great. Brock Palen Center for Advanced Computing bro...@umich.edu (734)936-1985 ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres Cisco Systems
Re: [OMPI users] Open MPI v1.2.5 released
Hi Warner. The simplest way would certainly be to launch your job with with the mpirun --nolocal option. If you're sure you want a hostfile-based way to set this, simply removing the headnode from the hostfile would also work. -- --Kris 叶ってしまう夢は本当の夢と言えん。 [A dream that comes true can't really be called a dream.] Warner Yuen wrote > Date: Wed, 9 Jan 2008 12:50:09 -0800 > From: Warner Yuen > Subject: Re: [OMPI users] Open MPI v1.2.5 released > To: us...@open-mpi.org > Message-ID: > Content-Type: text/plain; charset="us-ascii" > > Thanks to Brian Barrett, I was able to get through some ugly Intel > compiler bugs during the configure script. I now have OMPI v1.2.5 > running nicely under Mac OSX v10.5 Leopard! > > However, I have a question about hostfiles. I would like to manually > launch MPI jobs from my headnode, but I don't want the jobs to run on > the head node. In LAM/MPI I could add a "hostname schedule=no" to the > hostfile, is there an equivalent in OpenMPI? I'm sure this has come up > before, but I couldn't find an answer in the archives. > > Thanks, > > -Warner > > Warner Yuen > Scientific Computing Consultant > Apple Computer > email: wy...@apple.com > Tel: 408.718.2859 > Fax: 408.715.0133
Re: [OMPI users] Ideal MTU in Infiniband
In an Open MPI installation, have a look in $prefix/share/openmpi/mca- btl-openib-hca-params.ini, find your HCA hardware, and you'll see what value Open MPI sets the MTU to. Hope that helps. On Jan 10, 2008, at 1:56 PM, Parag Kalra wrote: But still can someone please tell the numerical value that is idealy set or should be set. On Jan 10, 2008 8:29 PM, Gleb Natapov wrote: On Thu, Jan 10, 2008 at 06:23:50PM +0530, Parag Kalra wrote: > Hello all, > > Any ideas? Yes. The idea is that Open MPI knows what best. Run it with a default value. Usually bigger MTU is better, but some HW has bugs. Open MPI knows this and choses the best value for your HW. > > -- > Parag Kalra > > > On Jan 10, 2008 4:15 AM, Parag Kalra wrote: > > > Hello all, > > > > I am using Open MPI with Infiniband configured. > > > > What should be the ideal MTU size for infiniband? > > > > -- > > PARAG . A . KALRA > > > > > > > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Gleb. ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users -- Love, PARAG . A . KALRA Good judgment comes from experience, and experience comes from bad judgment http://discoverlinux.blogspot.com A Linux machine! because a 486 is a terrible thing to waste ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres Cisco Systems
