Hi Jeff and Ralph, Thanks. I'm really a novice user - and in cases like this one I don't really know what I'm doing. In this case, I just wanted to get my application to run without throwing strange error messages and quitting. :) That said, I would much rather learn about the components of MPI rather than taking shots in the dark.
On different clusters where I was getting error messages related to a component, the advice from this mailing list was to disable that component. Currently I'm building OpenMPI with all components, and my environment variables disable the components at runtime: I have OMPI_MCA_ras=^tm and OMPI_MCA_btl=self,sm,tcp. The latter seems to disable the advanced networking-related components that were throwing the errors. I am not sure how a BTL works for MPI ranks that are running on the same node. Do the different MPI ranks (processes) on a node still use a BTL to communicate with each other? And which one does it prefer to use (sm, tcp or something else?) Thanks, - Lee-Ping On Sep 30, 2014, at 7:29 AM, Jeff Squyres (jsquyres) <jsquy...@cisco.com> wrote: > How can you run MPI jobs at all without any BTLs? That sounds weird -- this > is not a case for which we designed the code base. > > All that being said, you're getting compile errors in the OMPI build because > of two things: > > - you selected to build static > - you didn't disable enough stuff > > Specifically, statically building verbs-based code is not for the meek (see > the FAQ). We have verbs-based code in a few places: the BTLs, and also in > the "common" framework. Hence, the linker errors you are getting are because > the "common" verbs component was still built (because it wasn't disabled), > and because building statically with verbs is... tricky (see the FAQ). > > You might have better luck with: > > ./configure --enable-mca-no-build=btl,common-verbs ... > > Or, better yet: > > ./configure --enable-mca-no-build=btl --without-verbs ... > > But again, I'm not sure how well OMPI will function without any BTLs. > > > > On Sep 29, 2014, at 11:47 PM, Ralph Castain <r...@open-mpi.org> wrote: > >> ompi_info is just the first time when an executable is built, and so it >> always is the place where we find missing library issues. It looks like >> someone has left incorrect configure logic in the system such that we always >> attempt to build Infiniband-related code, but without linking against the >> library. >> >> We'll have to try and track it down. >> >> On Sep 29, 2014, at 5:08 PM, Lee-Ping Wang <leep...@stanford.edu> wrote: >> >>> Hi Gus, >>> >>> Thank you. I did start from a completely clean directory tree every time >>> (I deleted the whole folder and re-extracted the tarball). >>> >>> I noticed that disabling any of the BTL components resulted in the same >>> error, so my solution was to build everything and disable certain >>> components at runtime. >>> >>> - Lee-Ping >>> >>> On Sep 29, 2014, at 6:03 AM, Gustavo Correa <g...@ldeo.columbia.edu> wrote: >>> >>>> Hi Lee-Ping >>>> >>>> Did you cleanup the old build, to start fresh? >>>> >>>> make distclean >>>> configure --disable-vt ... >>>> ... >>>> >>>> I hope this helps, >>>> Gus Correa >>>> >>>> On Sep 29, 2014, at 8:47 AM, Lee-Ping Wang wrote: >>>> >>>>> Hmm, the build doesn't finish - it breaks when trying to create the man >>>>> page. I guess I'll disable only a few specific BTL components that have >>>>> given me issues in the past. >>>>> >>>>> Creating ompi_info.1 man page... >>>>> CCLD ompi_info >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to >>>>> `ibv_free_device_list' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_alloc_pd' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_close_device' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_dealloc_pd' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_destroy_qp' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_create_cq' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_get_sysfs_path' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_open_device' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_create_qp' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_query_device' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to >>>>> `ibv_get_device_list' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to >>>>> `ibv_get_device_name' >>>>> ../../../ompi/.libs/libmpi.so: undefined reference to `ibv_destroy_cq' >>>>> collect2: error: ld returned 1 exit status >>>>> >>>>> Thanks, >>>>> >>>>> - Lee-Ping >>>>> >>>>> On Sep 29, 2014, at 5:27 AM, Lee-Ping Wang <leep...@stanford.edu> wrote: >>>>> >>>>>> Hi there, >>>>>> >>>>>> I'm building OpenMPI 1.8.3 on a system where I explicitly don't want any >>>>>> of the BTL components (they tend to break my single node jobs). >>>>>> >>>>>> ./configure CC=gcc CXX=g++ F77=gfortran FC=gfortran >>>>>> --prefix=$QC_EXT_LIBS/openmpi --enable-static --enable-mca-no-build=btl >>>>>> >>>>>> Building gives me this error in the vt component - it appears to be >>>>>> expecting some Infiniband stuff: >>>>>> >>>>>> CCLD otfmerge-mpi >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_free_device_list' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_alloc_pd' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_close_device' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_dealloc_pd' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_destroy_qp' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_create_cq' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_get_sysfs_path' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_open_device' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_create_qp' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_query_device' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_get_device_list' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_get_device_name' >>>>>> /u/sciteam/leeping/opt/qchem-4.2/ext-libs/openmpi-1.8.3/ompi/contrib/vt/vt/../../../.libs/libmpi.so: >>>>>> undefined reference to `ibv_destroy_cq' >>>>>> collect2: error: ld returned 1 exit status >>>>>> make[10]: *** [otfmerge-mpi] Error 1 >>>>>> >>>>>> I've decided to disable the vt component since I doubt I'm using it, but >>>>>> this could be good to know. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> - Lee-Ping >>>>> >>>>> _______________________________________________ >>>>> users mailing list >>>>> us...@open-mpi.org >>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>> Link to this post: >>>>> http://www.open-mpi.org/community/lists/users/2014/09/25399.php >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> Link to this post: >>>> http://www.open-mpi.org/community/lists/users/2014/09/25400.php >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2014/09/25409.php >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/09/25414.php > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/09/25416.php
