I removed openib.so. I get this error with --mca btl_tcp_if_include p2p1: [compute-0-7.local:04799] mca: base: component_find: unable to open /share/apps/openmpi-1.8.4/lib/openmpi/mca_btl_openib: /share/apps/openmpi-1.8.4/lib/openmpi/mca_btl_openib.so: cannot open shared object file: No such file or directory (ignored)
with -mca btl ^openib ---> there is no error. Jeff, Thanks for your input. I was able to get 50% performance improvemnt forcing it to use our 10G net. > On Mar 9, 2015, at 12:19 PM, Tus <t...@riseup.net> wrote: >> >> I configured and installed 1.8.4 on my system. I was getting openfabric >> erros and started to specify-mca btl ^openib which is working but very >> slow. >> >> I would like to complile again excluding openfabric or ib support. I do >> have a 10GbE fast network in addition to 1G net. What flags are need to >> ignore ib support and how can I verify/force openmpi to use my 10GbE >> net? > > You have several options: > > 1. If you did a default Open MPI install, you can simply rm the > "mca_btl_openib.so" plugin that was installed under $prefix/lib/openmpi. > Open MPI then won't know that that plugin exists, and it won't try to use > OpenFabrics-based devices. > > 2. You can set a system-wide MCA parameter to ignore the openib BTL. That > way, you don't have to type it on the mpirun command line every time. > > 3. You can rebuild Open MPI with the --without-verbs configure command > line switch. This will ultimately have the same effect as #1 (i.e., the > openib plugin won't be in the installation tree). > > As for using your 10G, I assume you mean over TCP sockets, right? > > If so, you can use --mca btl_tcp_if_include <NAME_OF_10G_DEVICE> or > <CIDR_OF_10G_NETWORK>. E.g.: > > mpirun --mca btl_tcp_if_include eth1 ... > mpirun --mca btl_tcp_if_include 10.20.30.0/24 ... > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > >
