[OMPI users] mpif90 unable to find ibverbs
Hi, I am trying to build an application with static linking that uses openmpi. in the middle of the build, I get this mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o libiotk.a /usr/bin/ld: cannot find -libverbs collect2: ld returned 1 exit status However, such library exists on the system. [root@cluster source]# find /usr/ -name *ibverb* /usr/lib64/libibverbs.so /usr/lib64/libibverbs.so.1.0.0 /usr/lib64/libibverbs.so.1 /usr/share/doc/libibverbs-1.1.8 [root@cluster source]# mpif90 -v Using built-in specs. Target: x86_64-redhat-linux Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux Thread model: posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) Any idea for that? Regards, Mahmood ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] mpif90 unable to find ibverbs
Mahmood, since you are building a static binary, only static library (e.g. libibverbs.a) can be used. on your system, only dynamic libibverbs.so is available. simply install libibverbs.a and you should be fine. Best regards, Gilles - Original Message - Hi, I am trying to build an application with static linking that uses openmpi. in the middle of the build, I get this mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o libiotk.a /usr/bin/ld: cannot find -libverbs collect2: ld returned 1 exit status However, such library exists on the system. [root@cluster source]# find /usr/ -name *ibverb* /usr/lib64/libibverbs.so /usr/lib64/libibverbs.so.1.0.0 /usr/lib64/libibverbs.so.1 /usr/share/doc/libibverbs-1.1.8 [root@cluster source]# mpif90 -v Using built-in specs. Target: x86_64-redhat-linux Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl= http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared - -enable-threads=posix --enable-checking=release --with-system-zlib -- enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique- object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable- java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj -1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode -- with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build= x86_64-redhat-linux Thread model: posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) Any idea for that? Regards, Mahmood ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] mpif90 unable to find ibverbs
Thanks Gilles... That has been solved. Another issue is mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o libiotk.a /usr/bin/ld: cannot find -ldat The name is actually hard to google! I cannot find the library name for "dat". Have you heard of that? There is not "libdat" package as I searched. Regards, Mahmood On Wed, Sep 13, 2017 at 2:54 PM, wrote: > Mahmood, > > > > since you are building a static binary, only static library (e.g. > libibverbs.a) can be used. > > on your system, only dynamic libibverbs.so is available. > > > > simply install libibverbs.a and you should be fine. > > > > Best regards, > > > > Gilles > > - Original Message - > > Hi, > I am trying to build an application with static linking that uses openmpi. > in the middle of the build, I get this > > mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o > libiotk.a > /usr/bin/ld: cannot find -libverbs > collect2: ld returned 1 exit status > > However, such library exists on the system. > > [root@cluster source]# find /usr/ -name *ibverb* > /usr/lib64/libibverbs.so > /usr/lib64/libibverbs.so.1.0.0 > /usr/lib64/libibverbs.so.1 > /usr/share/doc/libibverbs-1.1.8 > [root@cluster source]# mpif90 -v > Using built-in specs. > Target: x86_64-redhat-linux > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man > --infodir=/usr/share/info --with-bugurl=http://bugzilla. > redhat.com/bugzilla --enable-bootstrap --enable-shared > --enable-threads=posix --enable-checking=release --with-system-zlib > --enable-__cxa_atexit --disable-libunwind-exceptions > --enable-gnu-unique-object > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada > --enable-java-awt=gtk --disable-dssi > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre > --enable-libgcj-multifile --enable-java-maintainer-mode > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib > --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 > --build=x86_64-redhat-linux > Thread model: posix > gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) > > > > Any idea for that? > Regards, > Mahmood > > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] mpif90 unable to find ibverbs
This is something related to DAPL. /* just google "libdat" */ iirc, Intel MPI uses that, but i do not recall Open MPI using it (!) are you sure you are using Open MPI ? which interconnect do you have ? Cheers, Gilles - Original Message - Thanks Gilles... That has been solved. Another issue is mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o libiotk.a /usr/bin/ld: cannot find -ldat The name is actually hard to google! I cannot find the library name for "dat". Have you heard of that? There is not "libdat" package as I searched. Regards, Mahmood On Wed, Sep 13, 2017 at 2:54 PM, wrote: Mahmood, since you are building a static binary, only static library (e.g. libibverbs.a) can be used. on your system, only dynamic libibverbs.so is available. simply install libibverbs.a and you should be fine. Best regards, Gilles - Original Message - Hi, I am trying to build an application with static linking that uses openmpi. in the middle of the build, I get this mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_ kinds.o libiotk.a /usr/bin/ld: cannot find -libverbs collect2: ld returned 1 exit status However, such library exists on the system. [root@cluster source]# find /usr/ -name *ibverb* /usr/lib64/libibverbs.so /usr/lib64/libibverbs.so.1.0.0 /usr/lib64/libibverbs.so.1 /usr/share/doc/libibverbs-1.1.8 [root@cluster source]# mpif90 -v Using built-in specs. Target: x86_64-redhat-linux Configured with: ../configure --prefix=/usr --mandir=/usr/ share/man --infodir=/usr/share/info --with-bugurl= http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared - -enable-threads=posix --enable-checking=release --with-system-zlib -- enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique- object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable- java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj -1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode -- with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build= x86_64-redhat-linux Thread model: posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) Any idea for that? Regards, Mahmood ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] mpif90 unable to find ibverbs
Beware: static linking is not for the meek. Is there a reason you need to link statically? Be sure to read this FAQ item: https://www.open-mpi.org/faq/?category=mpi-apps#static-ofa-mpi-apps (note that that FAQ item was written a long time ago; it cites the "mthca" Mellanox obverts driver; the current generation driver name is ?I think? mlx5). You'll likely also have to adapt those instructions if you're using the UCX or MXM IB libraries. > On Sep 13, 2017, at 7:21 AM, gil...@rist.or.jp wrote: > > This is something related to DAPL. > > /* just google "libdat" */ > > > iirc, Intel MPI uses that, but i do not recall Open MPI using it (!) > > are you sure you are using Open MPI ? > > which interconnect do you have ? > > > Cheers, > > > Gilles > > - Original Message - > > Thanks Gilles... That has been solved. Another issue is > > mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o libiotk.a > /usr/bin/ld: cannot find -ldat > > The name is actually hard to google! I cannot find the library name for > "dat". Have you heard of that? There is not "libdat" package as I searched. > > Regards, > Mahmood > > > On Wed, Sep 13, 2017 at 2:54 PM, wrote: > Mahmood, > > > since you are building a static binary, only static library (e.g. > libibverbs.a) can be used. > > on your system, only dynamic libibverbs.so is available. > > > simply install libibverbs.a and you should be fine. > > > Best regards, > > > Gilles > > - Original Message - > > Hi, > I am trying to build an application with static linking that uses openmpi. in > the middle of the build, I get this > > mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o libiotk.a > /usr/bin/ld: cannot find -libverbs > collect2: ld returned 1 exit status > However, such library exists on the system. > > [root@cluster source]# find /usr/ -name *ibverb* > /usr/lib64/libibverbs.so > /usr/lib64/libibverbs.so.1.0.0 > /usr/lib64/libibverbs.so.1 > /usr/share/doc/libibverbs-1.1.8 > [root@cluster source]# mpif90 -v > Using built-in specs. > Target: x86_64-redhat-linux > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man > --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla > --enable-bootstrap --enable-shared --enable-threads=posix > --enable-checking=release --with-system-zlib --enable-__cxa_atexit > --disable-libunwind-exceptions --enable-gnu-unique-object > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk > --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre > --enable-libgcj-multifile --enable-java-maintainer-mode > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib > --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 > --build=x86_64-redhat-linux > Thread model: posix > gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) > > > > Any idea for that? > Regards, > Mahmood > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users -- Jeff Squyres jsquy...@cisco.com ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] mpif90 unable to find ibverbs
>are you sure you are using Open MPI ? I am using the openmpi shipped with Rocks 6 and trying to build Quantum ESPRESSO 6.1 >Beware: static linking is not for the meek. Agree! I found that I have to install compat-dapl-static.x86_64. As can be seen from the name, it is a compatibility library. After that, I faced an error saying mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o libiotk.a /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/libpthread.a(libpthread.o): In function `sem_open': (.text+0x774d): warning: the use of `mktemp' is dangerous, better use `mkstemp' /usr/bin/ld: dynamic STT_GNU_IFUNC symbol `strcmp' with pointer equality in `/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/libc.a(strcmp.o)' can not be used when making an executable; recompile with -fPIE and relink with -pie collect2: ld returned 1 exit status With such an error, I thought it is better to forget static linking! (as it is related to libc) and work with the shared libs and LD_LIBRARY_PATH Anyway, thanks for your help. Regards, Mahmood On Wed, Sep 13, 2017 at 6:12 PM, Jeff Squyres (jsquyres) wrote: > Beware: static linking is not for the meek. > > Is there a reason you need to link statically? > > Be sure to read this FAQ item: https://www.open-mpi.org/faq/? > category=mpi-apps#static-ofa-mpi-apps (note that that FAQ item was > written a long time ago; it cites the "mthca" Mellanox obverts driver; the > current generation driver name is ?I think? mlx5). You'll likely also have > to adapt those instructions if you're using the UCX or MXM IB libraries. > > > > On Sep 13, 2017, at 7:21 AM, gil...@rist.or.jp wrote: > > > > This is something related to DAPL. > > > > /* just google "libdat" */ > > > > > > iirc, Intel MPI uses that, but i do not recall Open MPI using it (!) > > > > are you sure you are using Open MPI ? > > > > which interconnect do you have ? > > > > > > Cheers, > > > > > > Gilles > > > > - Original Message - > > > > Thanks Gilles... That has been solved. Another issue is > > > > mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o > libiotk.a > > /usr/bin/ld: cannot find -ldat > > > > The name is actually hard to google! I cannot find the library name for > "dat". Have you heard of that? There is not "libdat" package as I searched. > > > > Regards, > > Mahmood > > > > > > On Wed, Sep 13, 2017 at 2:54 PM, wrote: > > Mahmood, > > > > > > since you are building a static binary, only static library (e.g. > libibverbs.a) can be used. > > > > on your system, only dynamic libibverbs.so is available. > > > > > > simply install libibverbs.a and you should be fine. > > > > > > Best regards, > > > > > > Gilles > > > > - Original Message - > > > > Hi, > > I am trying to build an application with static linking that uses > openmpi. in the middle of the build, I get this > > > > mpif90 -g -pthread -static -o iotk_print_kinds.x iotk_print_kinds.o > libiotk.a > > /usr/bin/ld: cannot find -libverbs > > collect2: ld returned 1 exit status > > However, such library exists on the system. > > > > [root@cluster source]# find /usr/ -name *ibverb* > > /usr/lib64/libibverbs.so > > /usr/lib64/libibverbs.so.1.0.0 > > /usr/lib64/libibverbs.so.1 > > /usr/share/doc/libibverbs-1.1.8 > > [root@cluster source]# mpif90 -v > > Using built-in specs. > > Target: x86_64-redhat-linux > > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man > --infodir=/usr/share/info --with-bugurl=http://bugzilla. > redhat.com/bugzilla --enable-bootstrap --enable-shared > --enable-threads=posix --enable-checking=release --with-system-zlib > --enable-__cxa_atexit --disable-libunwind-exceptions > --enable-gnu-unique-object > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada > --enable-java-awt=gtk --disable-dssi > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre > --enable-libgcj-multifile --enable-java-maintainer-mode > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib > --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 > --build=x86_64-redhat-linux > > Thread model: posix > > gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) > > > > > > > > Any idea for that? > > Regards, > > Mahmood > > > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > > -- > Jeff Squyres > jsquy...@cisco.com > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
[OMPI users] Honor host_aliases file for tight SGE integration
Hi, I wonder whether it came ever to the discussion, that SGE can have a similar behavior like Torque/PBS regarding the mangling of hostnames. It's similiar to https://github.com/open-mpi/ompi/issues/2328, in the behavior that a node can have multiple network interfaces and each has an unique name. SGE's operation can be routed to a specific network interface by the use of a file: $SGE_ROOT/$SGE_CELL/common/host_aliases which has the format: Hence in the generated $PE_HOSTFILE the name known to SGE is listed, although the `hostname` command provides the real name. Open MPI would in this case start a `qrsh -inherit …` call instead of forking, as it thinks that these are different machines (assuming an allocation_rule of $PE_SLOTS so that the `mpiexec` is supposed to be on the same machine as the started tasks). I tried to go the "old" way to provide a start_proc_args to the PE to create a symbolic link to `hostname` in $TMPDIR, so that inside the job script an adjusted `hostname` call is available, but obviously Open MPI calls gethostname() directly and not by an external binary. So I mangled the hostname in the created machinefile in the jobscript to feed an "adjusted" $PE_HOSTFILE to Open MPI and then it's working as intended: Open MPI creates forks. Does anyone else need such a patch in Open MPI and is it suitable to be included? -- Reuti PS: Only the headnodes have more than one network interface in our case and hence it's didn't come to my attention up to now, as now there was a need to use also some cores on the headnodes. They are known internally to SGE as "login" and "master", but the external names may be "foo" and "baz" which gethostname() returns. ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
