[OMPI users] error building openmpi-dev-1031-g008755a with Sun C 5.13
Hi, today I tried to build openmpi-dev-1031-g008755a on my machines (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three platforms for both compilers. I used the following configure command for all platforms. ../openmpi-dev-1031-g008755a/configure --prefix=/usr/local/openmpi-1.9.0_64_cc \ --libdir=/usr/local/openmpi-1.9.0_64_cc/lib64 \ --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ --with-jdk-headers=/usr/local/jdk1.8.0/include \ JAVA_HOME=/usr/local/jdk1.8.0 \ LDFLAGS="-m64 -mt" \ CC="cc" CXX="CC" FC="f95" \ CFLAGS="-m64 -mt" CXXFLAGS="-m64 -library=stlport4" FCFLAGS="-m64" \ CPP="cpp" CXXCPP="cpp" \ CPPFLAGS="" CXXCPPFLAGS="" \ --enable-mpi-cxx \ --enable-cxx-exceptions \ --enable-mpi-java \ --enable-heterogeneous \ --enable-mpi-thread-multiple \ --with-hwloc=internal \ --without-verbs \ --with-wrapper-cflags="-m64 -mt" \ --with-wrapper-cxxflags="-m64 -library=stlport4" \ --with-wrapper-fcflags="-m64" \ --with-wrapper-ldflags="-mt" \ --enable-debug \ |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_cc I got the following error on my Linux machine. ... CC reachable_netlink_utils_common.lo "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 164: warning: paramete r in inline asm statement unused: %3 "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 210: warning: paramete r in inline asm statement unused: %2 "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 231: warning: paramete r in inline asm statement unused: %2 "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 252: warning: paramete r in inline asm statement unused: %2 "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 273: warning: paramete r in inline asm statement unused: %2 "../../../../../openmpi-dev-1031-g008755a/opal/mca/reachable/netlink/reachable_netlink_utils_common.c", line 322: warning: extern inline function "nl_object_priv" not defined in translation unit cc: Fatal error in /opt/sun/solarisstudio12.4/lib/compilers/acomp : Signal number = 11 make[2]: *** [reachable_netlink_utils_common.lo] Error 1 make[2]: Leaving directory `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_cc/opal/mca/r eachable/netlink' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_cc/opal' make: *** [all-recursive] Error 1 I would be grateful, if somebody could fix the problem. Thank you very much for any help in advance. Kind regards Siegmar
[OMPI users] error building openmpi-dev-1031-g008755a with gcc-4.9.2
Hi, today I tried to build openmpi-dev-1031-g008755a on my machines (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three platforms for both compilers. I used the following configure command for all platforms. ../openmpi-dev-1031-g008755a/configure --prefix=/usr/local/openmpi-1.9.0_64_gcc \ --libdir=/usr/local/openmpi-1.9.0_64_gcc/lib64 \ --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ --with-jdk-headers=/usr/local/jdk1.8.0/include \ JAVA_HOME=/usr/local/jdk1.8.0 \ LDFLAGS="-m64" CC="gcc" CXX="g++" FC="gfortran" \ CFLAGS="-m64" CXXFLAGS="-m64" FCFLAGS="-m64" \ CPP="cpp" CXXCPP="cpp" \ CPPFLAGS="" CXXCPPFLAGS="" \ --enable-mpi-cxx \ --enable-cxx-exceptions \ --enable-mpi-java \ --enable-heterogeneous \ --enable-mpi-thread-multiple \ --with-hwloc=internal \ --without-verbs \ --with-wrapper-cflags="-std=c11 -m64" \ --with-wrapper-cxxflags="-m64" \ --with-wrapper-fcflags="-m64" \ --enable-debug \ |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_gcc I got the following error on my Linux machine. ... CC pml_cm_start.lo CCLD mca_pml_cm.la make[2]: Leaving directory `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_gcc/ompi/mca/ pml/cm' Making all in mca/pml/ob1 make[2]: Entering directory `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_gcc/ompi/mca /pml/ob1' CC pml_ob1.lo In file included from ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.h:38:0, from ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c:45: ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h: In function 'ob1_hdr_ntoh': ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:204:12: error: 'mca_pml_ob1_rget_hdr _t' has no member named 'hdr_seg_cnt' (h).hdr_seg_cnt = ntohl((h).hdr_seg_cnt); \ ^ ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:440:13: note: in expansion of macro 'MCA_PML_OB1_RGET_HDR_NTOH' MCA_PML_OB1_RGET_HDR_NTOH(hdr->hdr_rget); ^ A lot more of the above messages. ... ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:366:36: error: 'mca_pml_ob1_rdma_hdr _t' has no member named 'hdr_seg_cnt' (h).hdr_seg_cnt = htonl((h).hdr_seg_cnt); \ ^ ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:495:13: note: in expansion of macro 'MCA_PML_OB1_RDMA_HDR_HTON' MCA_PML_OB1_RDMA_HDR_HTON(hdr->hdr_rdma); ^ ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c: In function 'mca_pml_ob1_send_fin': ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c:664:18: error: 'hdr' undeclared (first u se in this function) ob1_hdr_hton(hdr, MCA_PML_OB1_HDR_TYPE_FIN, proc); ^ ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:465:43: note: in definition of macro 'ob1_hdr_hton' ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p) ^ ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c:664:18: note: each undeclared identifier is reported only once for each function it appears in ob1_hdr_hton(hdr, MCA_PML_OB1_HDR_TYPE_FIN, proc); ^ ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:465:43: note: in definition of macro 'ob1_hdr_hton' ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p) ^ make[2]: *** [pml_ob1.lo] Error 1 ... I would be grateful, if somebody could fix the problem. Thank you very much for any help in advance. Kind regards Siegmar
[OMPI users] error building openmpi-dev-1031-g008755a with gcc-4.9.2
Hi, today I tried to build openmpi-dev-1031-g008755a on my machines (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three platforms for both compilers. I used the following configure command for all platforms. ../openmpi-dev-1031-g008755a/configure --prefix=/usr/local/openmpi-1.9.0_64_gcc \ --libdir=/usr/local/openmpi-1.9.0_64_gcc/lib64 \ --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ --with-jdk-headers=/usr/local/jdk1.8.0/include \ JAVA_HOME=/usr/local/jdk1.8.0 \ LDFLAGS="-m64" CC="gcc" CXX="g++" FC="gfortran" \ CFLAGS="-m64" CXXFLAGS="-m64" FCFLAGS="-m64" \ CPP="cpp" CXXCPP="cpp" \ CPPFLAGS="" CXXCPPFLAGS="" \ --enable-mpi-cxx \ --enable-cxx-exceptions \ --enable-mpi-java \ --enable-heterogeneous \ --enable-mpi-thread-multiple \ --with-hwloc=internal \ --without-verbs \ --with-wrapper-cflags="-std=c11 -m64" \ --with-wrapper-cxxflags="-m64" \ --with-wrapper-fcflags="-m64" \ --enable-debug \ |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_gcc I got the following warning on my Solaris Sparc machine. ... CC mpi-io/get_extent.lo In file included from ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/adio/include/adioi .h:950:0, from ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/adio/include/adio. h:454, from ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/mpioimpl.h: 15, from ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/get_extent. c:8: ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/get_extent.c: In function 'mca_io _romio_dist_MPI_File_get_type_extent': ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/adio/include/adioi_error.h:74:24: warnin g: 'error_code' may be used uninitialized in this function [-Wmaybe-uninitialized] error_code = MPIO_Err_return_file(fh, error_code); \ ^ ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/get_extent.c:42:9: note: 'error_c ode' was declared here int error_code; ^ CC mpi-io/get_group.lo ... I got the same error that I have already reported for my Linux machine. I would be grateful, if somebody could fix the problem. Thank you very much for any help in advance. Kind regards Siegmar
[OMPI users] error building openmpi-dev-1031-g008755a with Sun C 5.13
Hi, today I tried to build openmpi-dev-1031-g008755a on my machines (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three platforms for both compilers. I used the following configure command for all platforms. ../openmpi-dev-1031-g008755a/configure --prefix=/usr/local/openmpi-1.9.0_64_cc \ --libdir=/usr/local/openmpi-1.9.0_64_cc/lib64 \ --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ --with-jdk-headers=/usr/local/jdk1.8.0/include \ JAVA_HOME=/usr/local/jdk1.8.0 \ LDFLAGS="-m64 -mt" \ CC="cc" CXX="CC" FC="f95" \ CFLAGS="-m64 -mt" CXXFLAGS="-m64 -library=stlport4" FCFLAGS="-m64" \ CPP="cpp" CXXCPP="cpp" \ CPPFLAGS="" CXXCPPFLAGS="" \ --enable-mpi-cxx \ --enable-cxx-exceptions \ --enable-mpi-java \ --enable-heterogeneous \ --enable-mpi-thread-multiple \ --with-hwloc=internal \ --without-verbs \ --with-wrapper-cflags="-m64 -mt" \ --with-wrapper-cxxflags="-m64 -library=stlport4" \ --with-wrapper-fcflags="-m64" \ --with-wrapper-ldflags="-mt" \ --enable-debug \ |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_cc I got the following error on my Solaris x86_64 machine. ... CC base/rcache_base_frame.lo "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 164: warning: parameter i n inline asm statement unused: %3 "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 210: warning: parameter i n inline asm statement unused: %2 "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 231: warning: parameter i n inline asm statement unused: %2 "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 252: warning: parameter i n inline asm statement unused: %2 "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", line 273: warning: parameter i n inline asm statement unused: %2 "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", line 37: cannot find inclu de file: "opal/mca/rcache/base/static-components.h" "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", line 90: undefined symbol: mca_rcache_base_static_components "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", line 90: warning: improper pointer/integer combination: op "=" "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", line 90: non-constant init ializer involving a cast cc: acomp failed for ../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c make[2]: *** [base/rcache_base_frame.lo] Error 1 make[2]: Leaving directory `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-SunOS.x86_64.64_cc/opal/mca/r cache' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-SunOS.x86_64.64_cc/opal' make: *** [all-recursive] Error 1 I would be grateful, if somebody could fix the problem. Thank you very much for any help in advance. Kind regards Siegmar
Re: [OMPI users] error building openmpi-dev-1031-g008755a with Sun C 5.13
Siegmar -- This file (opal/mca/rcache/base/static-components.h) is generated during configure. I just downloaded the dev-1031 tarball from last night and ran configure on it, and the opal/mca/rcache/base/static-components.h file is there for me. Did something go wrong during your configure? > On Feb 19, 2015, at 8:03 AM, Siegmar Gross > wrote: > > Hi, > > today I tried to build openmpi-dev-1031-g008755a on my machines > (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 > x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three > platforms for both compilers. I used the following configure > command for all platforms. > > ../openmpi-dev-1031-g008755a/configure > --prefix=/usr/local/openmpi-1.9.0_64_cc \ > --libdir=/usr/local/openmpi-1.9.0_64_cc/lib64 \ > --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ > --with-jdk-headers=/usr/local/jdk1.8.0/include \ > JAVA_HOME=/usr/local/jdk1.8.0 \ > LDFLAGS="-m64 -mt" \ > CC="cc" CXX="CC" FC="f95" \ > CFLAGS="-m64 -mt" CXXFLAGS="-m64 -library=stlport4" FCFLAGS="-m64" \ > CPP="cpp" CXXCPP="cpp" \ > CPPFLAGS="" CXXCPPFLAGS="" \ > --enable-mpi-cxx \ > --enable-cxx-exceptions \ > --enable-mpi-java \ > --enable-heterogeneous \ > --enable-mpi-thread-multiple \ > --with-hwloc=internal \ > --without-verbs \ > --with-wrapper-cflags="-m64 -mt" \ > --with-wrapper-cxxflags="-m64 -library=stlport4" \ > --with-wrapper-fcflags="-m64" \ > --with-wrapper-ldflags="-mt" \ > --enable-debug \ > |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_cc > > I got the following error on my Solaris x86_64 machine. > > ... > CC base/rcache_base_frame.lo > "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 164: warning: parameter i > n inline asm statement unused: %3 > "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 210: warning: parameter i > n inline asm statement unused: %2 > "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 231: warning: parameter i > n inline asm statement unused: %2 > "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 252: warning: parameter i > n inline asm statement unused: %2 > "../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 273: warning: parameter i > n inline asm statement unused: %2 > "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", > line 37: cannot find inclu > de file: "opal/mca/rcache/base/static-components.h" > "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", > line 90: undefined symbol: > mca_rcache_base_static_components > "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", > line 90: warning: improper > pointer/integer combination: op "=" > "../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c", > line 90: non-constant init > ializer involving a cast > cc: acomp failed for > ../../../../openmpi-dev-1031-g008755a/opal/mca/rcache/base/rcache_base_frame.c > make[2]: *** [base/rcache_base_frame.lo] Error 1 > make[2]: Leaving directory > `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-SunOS.x86_64.64_cc/opal/mca/r > cache' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory > `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-SunOS.x86_64.64_cc/opal' > make: *** [all-recursive] Error 1 > > > I would be grateful, if somebody could fix the problem. Thank > you very much for any help in advance. > > > Kind regards > > Siegmar > > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26346.php -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] error building openmpi-dev-1031-g008755a with Sun C 5.13
Siegmar -- This one looks like a seg fault in your compiler. I don't know if there's much we can do about that. > On Feb 19, 2015, at 7:50 AM, Siegmar Gross > wrote: > > Hi, > > today I tried to build openmpi-dev-1031-g008755a on my machines > (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 > x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three > platforms for both compilers. I used the following configure > command for all platforms. > > ../openmpi-dev-1031-g008755a/configure > --prefix=/usr/local/openmpi-1.9.0_64_cc \ > --libdir=/usr/local/openmpi-1.9.0_64_cc/lib64 \ > --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ > --with-jdk-headers=/usr/local/jdk1.8.0/include \ > JAVA_HOME=/usr/local/jdk1.8.0 \ > LDFLAGS="-m64 -mt" \ > CC="cc" CXX="CC" FC="f95" \ > CFLAGS="-m64 -mt" CXXFLAGS="-m64 -library=stlport4" FCFLAGS="-m64" \ > CPP="cpp" CXXCPP="cpp" \ > CPPFLAGS="" CXXCPPFLAGS="" \ > --enable-mpi-cxx \ > --enable-cxx-exceptions \ > --enable-mpi-java \ > --enable-heterogeneous \ > --enable-mpi-thread-multiple \ > --with-hwloc=internal \ > --without-verbs \ > --with-wrapper-cflags="-m64 -mt" \ > --with-wrapper-cxxflags="-m64 -library=stlport4" \ > --with-wrapper-fcflags="-m64" \ > --with-wrapper-ldflags="-mt" \ > --enable-debug \ > |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_cc > > I got the following error on my Linux machine. > > ... > CC reachable_netlink_utils_common.lo > "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 164: warning: paramete > r in inline asm statement unused: %3 > "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 210: warning: paramete > r in inline asm statement unused: %2 > "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 231: warning: paramete > r in inline asm statement unused: %2 > "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 252: warning: paramete > r in inline asm statement unused: %2 > "../../../../../openmpi-dev-1031-g008755a/opal/include/opal/sys/amd64/atomic.h", > line 273: warning: paramete > r in inline asm statement unused: %2 > "../../../../../openmpi-dev-1031-g008755a/opal/mca/reachable/netlink/reachable_netlink_utils_common.c", > line > 322: warning: extern inline function "nl_object_priv" not defined in > translation unit > cc: Fatal error in /opt/sun/solarisstudio12.4/lib/compilers/acomp : Signal > number = 11 > make[2]: *** [reachable_netlink_utils_common.lo] Error 1 > make[2]: Leaving directory > `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_cc/opal/mca/r > eachable/netlink' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory > `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_cc/opal' > make: *** [all-recursive] Error 1 > > > I would be grateful, if somebody could fix the problem. Thank > you very much for any help in advance. > > > Kind regards > > Siegmar > > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26343.php -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] error building openmpi-dev-1031-g008755a with gcc-4.9.2
Looks like --enable-heterogeneous builds are broken on master. I filed https://github.com/open-mpi/ompi/issues/403. Thanks for the heads-up! > On Feb 19, 2015, at 7:50 AM, Siegmar Gross > wrote: > > Hi, > > today I tried to build openmpi-dev-1031-g008755a on my machines > (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 > x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three > platforms for both compilers. I used the following configure > command for all platforms. > > ../openmpi-dev-1031-g008755a/configure > --prefix=/usr/local/openmpi-1.9.0_64_gcc \ > --libdir=/usr/local/openmpi-1.9.0_64_gcc/lib64 \ > --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ > --with-jdk-headers=/usr/local/jdk1.8.0/include \ > JAVA_HOME=/usr/local/jdk1.8.0 \ > LDFLAGS="-m64" CC="gcc" CXX="g++" FC="gfortran" \ > CFLAGS="-m64" CXXFLAGS="-m64" FCFLAGS="-m64" \ > CPP="cpp" CXXCPP="cpp" \ > CPPFLAGS="" CXXCPPFLAGS="" \ > --enable-mpi-cxx \ > --enable-cxx-exceptions \ > --enable-mpi-java \ > --enable-heterogeneous \ > --enable-mpi-thread-multiple \ > --with-hwloc=internal \ > --without-verbs \ > --with-wrapper-cflags="-std=c11 -m64" \ > --with-wrapper-cxxflags="-m64" \ > --with-wrapper-fcflags="-m64" \ > --enable-debug \ > |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_gcc > > I got the following error on my Linux machine. > > ... > CC pml_cm_start.lo > CCLD mca_pml_cm.la > make[2]: Leaving directory > `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_gcc/ompi/mca/ > pml/cm' > Making all in mca/pml/ob1 > make[2]: Entering directory > `/export2/src/openmpi-1.9/openmpi-dev-1031-g008755a-Linux.x86_64.64_gcc/ompi/mca > /pml/ob1' > CC pml_ob1.lo > In file included from > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.h:38:0, > from > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c:45: > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h: In > function 'ob1_hdr_ntoh': > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:204:12: > error: 'mca_pml_ob1_rget_hdr > _t' has no member named 'hdr_seg_cnt' > (h).hdr_seg_cnt = ntohl((h).hdr_seg_cnt); \ >^ > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:440:13: > note: in expansion of macro > 'MCA_PML_OB1_RGET_HDR_NTOH' > MCA_PML_OB1_RGET_HDR_NTOH(hdr->hdr_rget); > ^ > > A lot more of the above messages. > > ... > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:366:36: > error: 'mca_pml_ob1_rdma_hdr > _t' has no member named 'hdr_seg_cnt' > (h).hdr_seg_cnt = htonl((h).hdr_seg_cnt); \ >^ > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:495:13: > note: in expansion of macro > 'MCA_PML_OB1_RDMA_HDR_HTON' > MCA_PML_OB1_RDMA_HDR_HTON(hdr->hdr_rdma); > ^ > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c: In > function 'mca_pml_ob1_send_fin': > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c:664:18: > error: 'hdr' undeclared (first u > se in this function) > ob1_hdr_hton(hdr, MCA_PML_OB1_HDR_TYPE_FIN, proc); > ^ > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:465:43: > note: in definition of macro > 'ob1_hdr_hton' > ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p) > ^ > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1.c:664:18: > note: each undeclared identifier > is reported only once for each function it appears in > ob1_hdr_hton(hdr, MCA_PML_OB1_HDR_TYPE_FIN, proc); > ^ > ../../../../../openmpi-dev-1031-g008755a/ompi/mca/pml/ob1/pml_ob1_hdr.h:465:43: > note: in definition of macro > 'ob1_hdr_hton' > ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p) > ^ > make[2]: *** [pml_ob1.lo] Error 1 > ... > > > I would be grateful, if somebody could fix the problem. Thank > you very much for any help in advance. > > > Kind regards > > Siegmar > > > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26343.php -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] error building openmpi-dev-1031-g008755a with gcc-4.9.2
Siegmar -- This looks like an error in ROMIO that we should report upstream. Thanks for the heads-up! > On Feb 19, 2015, at 8:00 AM, Siegmar Gross > wrote: > > Hi, > > today I tried to build openmpi-dev-1031-g008755a on my machines > (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 > x86_64) with gcc-4.9.2 and Sun C 5.13 and got errors on all three > platforms for both compilers. I used the following configure > command for all platforms. > > ../openmpi-dev-1031-g008755a/configure > --prefix=/usr/local/openmpi-1.9.0_64_gcc \ > --libdir=/usr/local/openmpi-1.9.0_64_gcc/lib64 \ > --with-jdk-bindir=/usr/local/jdk1.8.0/bin \ > --with-jdk-headers=/usr/local/jdk1.8.0/include \ > JAVA_HOME=/usr/local/jdk1.8.0 \ > LDFLAGS="-m64" CC="gcc" CXX="g++" FC="gfortran" \ > CFLAGS="-m64" CXXFLAGS="-m64" FCFLAGS="-m64" \ > CPP="cpp" CXXCPP="cpp" \ > CPPFLAGS="" CXXCPPFLAGS="" \ > --enable-mpi-cxx \ > --enable-cxx-exceptions \ > --enable-mpi-java \ > --enable-heterogeneous \ > --enable-mpi-thread-multiple \ > --with-hwloc=internal \ > --without-verbs \ > --with-wrapper-cflags="-std=c11 -m64" \ > --with-wrapper-cxxflags="-m64" \ > --with-wrapper-fcflags="-m64" \ > --enable-debug \ > |& tee log.configure.$SYSTEM_ENV.$MACHINE_ENV.64_gcc > > I got the following warning on my Solaris Sparc machine. > > ... > CC mpi-io/get_extent.lo > In file included from > ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/adio/include/adioi > .h:950:0, > from > ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/adio/include/adio. > h:454, > from > ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/mpioimpl.h: > 15, > from > ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/get_extent. > c:8: > ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/get_extent.c: > In function 'mca_io > _romio_dist_MPI_File_get_type_extent': > ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/adio/include/adioi_error.h:74:24: > warnin > g: 'error_code' may be used uninitialized in this function > [-Wmaybe-uninitialized] > error_code = MPIO_Err_return_file(fh, error_code); \ >^ > ../../../../../../openmpi-dev-1031-g008755a/ompi/mca/io/romio/romio/mpi-io/get_extent.c:42:9: > note: 'error_c > ode' was declared here > int error_code; > ^ > CC mpi-io/get_group.lo > ... > > > I got the same error that I have already reported for my > Linux machine. > > > I would be grateful, if somebody could fix the problem. Thank > you very much for any help in advance. > > > Kind regards > > Siegmar > > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26345.php -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] Help on getting CMA works
By the way, I have tried two others things: #1- I launched it as root: mpiexec --mca mca_btl_vader_single_copy_mechanism cma --allow-run-as-root -np 2 ./hw #2- Found this (http://askubuntu.com/questions/146160/what-is-the-ptrace-scope-workaround-for-wine-programs-and-are-there-any-risks) and tried this: sudo setcap cap_sys_ptrace=eip /tmp/hw On both RedHat 6.5 and OpenSuse 12.3 and still get the same error message!!! :-/ Sorry, I am not a kernel expert... What's wrong? Thanks, Eric On 02/18/2015 04:48 PM, Éric Chamberland wrote: Le 2015-02-18 15:14, Nathan Hjelm a écrit : I recommend using vader for CMA. It has code to get around the ptrace setting. Run with mca_btl_vader_single_copy_mechanism cma (should be the default). Ok, I tried it, but it gives exactly the same error message! Eric -Nathan On Wed, Feb 18, 2015 at 02:56:01PM -0500, Eric Chamberland wrote: Hi, I have configured with "--with-cma" on 2 differents OS (RedHat 6.6 and OpenSuse 12.3), but in both case, I have the following error when launching a simple mpi_hello_world.c example: /opt/openmpi-1.8.4_cma/bin/mpiexec --mca btl_sm_use_cma 1 -np 2 /tmp/hw -- WARNING: Linux kernel CMA support was requested via the btl_vader_single_copy_mechanism MCA variable, but CMA support is not available due to restrictive ptrace settings. The vader shared memory BTL will fall back on another single-copy mechanism if one is available. This may result in lower performance. Local host: compile -- Hello world from process 0 of 2 Hello world from process 1 of 2 [compile:23874] 1 more process has sent help message help-btl-vader.txt / cma-permission-denied [compile:23874] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages After I googled the subject, it seems there is a kernel parameter to modify, but I can't find it for OpenSuse 12.3 or RedHat 6.6... Here is the "config.log" issued from RedHat 6.6... http://www.giref.ulaval.ca/~ericc/ompi_bug/config.184_cma.gz Thanks, Eric ___ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2015/02/26339.php ___ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2015/02/26342.php
Re: [OMPI users] Help on getting CMA works
If you have yama installed you can try: echo 1 > /proc/sys/kernel/yama/ptrace_scope as root. -Nathan On Thu, Feb 19, 2015 at 11:06:09AM -0500, Eric Chamberland wrote: > By the way, > > I have tried two others things: > > #1- I launched it as root: > > mpiexec --mca mca_btl_vader_single_copy_mechanism cma --allow-run-as-root > -np 2 ./hw > > #2- Found this > (http://askubuntu.com/questions/146160/what-is-the-ptrace-scope-workaround-for-wine-programs-and-are-there-any-risks) > and tried this: > > sudo setcap cap_sys_ptrace=eip /tmp/hw > > On both RedHat 6.5 and OpenSuse 12.3 and still get the same error message!!! > :-/ > > Sorry, I am not a kernel expert... > > What's wrong? > > Thanks, > > Eric > > On 02/18/2015 04:48 PM, Éric Chamberland wrote: > > > >Le 2015-02-18 15:14, Nathan Hjelm a écrit : > >>I recommend using vader for CMA. It has code to get around the ptrace > >>setting. Run with mca_btl_vader_single_copy_mechanism cma (should be the > >>default). > >Ok, I tried it, but it gives exactly the same error message! > > > >Eric > > > >>-Nathan > >> > >>On Wed, Feb 18, 2015 at 02:56:01PM -0500, Eric Chamberland wrote: > >>>Hi, > >>> > >>>I have configured with "--with-cma" on 2 differents OS (RedHat 6.6 and > >>>OpenSuse 12.3), but in both case, I have the following error when > >>>launching > >>>a simple mpi_hello_world.c example: > >>> > >>>/opt/openmpi-1.8.4_cma/bin/mpiexec --mca btl_sm_use_cma 1 -np 2 /tmp/hw > >>>-- > >>> > >>>WARNING: Linux kernel CMA support was requested via the > >>>btl_vader_single_copy_mechanism MCA variable, but CMA support is > >>>not available due to restrictive ptrace settings. > >>> > >>>The vader shared memory BTL will fall back on another single-copy > >>>mechanism if one is available. This may result in lower performance. > >>> > >>> Local host: compile > >>>-- > >>> > >>>Hello world from process 0 of 2 > >>>Hello world from process 1 of 2 > >>>[compile:23874] 1 more process has sent help message > >>>help-btl-vader.txt / > >>>cma-permission-denied > >>>[compile:23874] Set MCA parameter "orte_base_help_aggregate" to 0 to > >>>see all > >>>help / error messages > >>> > >>>After I googled the subject, it seems there is a kernel parameter to > >>>modify, > >>>but I can't find it for OpenSuse 12.3 or RedHat 6.6... > >>> > >>>Here is the "config.log" issued from RedHat 6.6... > >>> > >>>http://www.giref.ulaval.ca/~ericc/ompi_bug/config.184_cma.gz > >>> > >>>Thanks, > >>> > >>>Eric > >>>___ > >>>users mailing list > >>>us...@open-mpi.org > >>>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > >>>Link to this post: > >>>http://www.open-mpi.org/community/lists/users/2015/02/26339.php > > > >___ > >users mailing list > >us...@open-mpi.org > >Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > >Link to this post: > >http://www.open-mpi.org/community/lists/users/2015/02/26342.php > > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26351.php pgpqgDxaFnCcb.pgp Description: PGP signature
Re: [OMPI users] Help on getting CMA works
On 02/19/2015 11:56 AM, Nathan Hjelm wrote: If you have yama installed you can try: Nope, I do not have it installed... is it absolutely necessary? (and would it change something when it fails when I am root?) Other question: In addition to "--with-cma" configure flag, do we have to pass any options to "mpicc" when compiling/linking an mpi application to use cma? Thanks, Eric echo 1 > /proc/sys/kernel/yama/ptrace_scope as root. -Nathan On Thu, Feb 19, 2015 at 11:06:09AM -0500, Eric Chamberland wrote: By the way, I have tried two others things: #1- I launched it as root: mpiexec --mca mca_btl_vader_single_copy_mechanism cma --allow-run-as-root -np 2 ./hw #2- Found this (http://askubuntu.com/questions/146160/what-is-the-ptrace-scope-workaround-for-wine-programs-and-are-there-any-risks) and tried this: sudo setcap cap_sys_ptrace=eip /tmp/hw On both RedHat 6.5 and OpenSuse 12.3 and still get the same error message!!! :-/ Sorry, I am not a kernel expert... What's wrong? Thanks, Eric On 02/18/2015 04:48 PM, Éric Chamberland wrote: Le 2015-02-18 15:14, Nathan Hjelm a écrit : I recommend using vader for CMA. It has code to get around the ptrace setting. Run with mca_btl_vader_single_copy_mechanism cma (should be the default). Ok, I tried it, but it gives exactly the same error message! Eric -Nathan On Wed, Feb 18, 2015 at 02:56:01PM -0500, Eric Chamberland wrote: Hi, I have configured with "--with-cma" on 2 differents OS (RedHat 6.6 and OpenSuse 12.3), but in both case, I have the following error when launching a simple mpi_hello_world.c example: /opt/openmpi-1.8.4_cma/bin/mpiexec --mca btl_sm_use_cma 1 -np 2 /tmp/hw -- WARNING: Linux kernel CMA support was requested via the btl_vader_single_copy_mechanism MCA variable, but CMA support is not available due to restrictive ptrace settings. The vader shared memory BTL will fall back on another single-copy mechanism if one is available. This may result in lower performance. Local host: compile -- Hello world from process 0 of 2 Hello world from process 1 of 2 [compile:23874] 1 more process has sent help message help-btl-vader.txt / cma-permission-denied [compile:23874] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages After I googled the subject, it seems there is a kernel parameter to modify, but I can't find it for OpenSuse 12.3 or RedHat 6.6... Here is the "config.log" issued from RedHat 6.6... http://www.giref.ulaval.ca/~ericc/ompi_bug/config.184_cma.gz Thanks, Eric ___ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2015/02/26339.php ___ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2015/02/26342.php ___ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2015/02/26351.php
Re: [OMPI users] Help on getting CMA works
On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > > On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > > > >If you have yama installed you can try: > > Nope, I do not have it installed... is it absolutely necessary? (and would > it change something when it fails when I am root?) > > Other question: In addition to "--with-cma" configure flag, do we have to > pass any options to "mpicc" when compiling/linking an mpi application to use > cma? No. CMA should work out of the box. You appear to have a setup I haven't yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER prctl. Its quite possible there are no restriction on ptrace in this setup. Can you try changing the following line at opal/mca/btl/vader/btl_vader_component.c:370 from: bool cma_happy = false; to bool cma_happy = true; and let me know if that works. If it does I will update vader to allow CMA in this configuration. -Nathan pgp8k5J9uaK7b.pgp Description: PGP signature
Re: [OMPI users] Help on getting CMA works
On 02/19/2015 02:58 PM, Nathan Hjelm wrote: On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: On 02/19/2015 11:56 AM, Nathan Hjelm wrote: If you have yama installed you can try: Nope, I do not have it installed... is it absolutely necessary? (and would it change something when it fails when I am root?) Other question: In addition to "--with-cma" configure flag, do we have to pass any options to "mpicc" when compiling/linking an mpi application to use cma? No. CMA should work out of the box. You appear to have a setup I haven't yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER prctl. Its quite possible there are no restriction on ptrace in this setup. Can you try changing the following line at opal/mca/btl/vader/btl_vader_component.c:370 from: bool cma_happy = false; to bool cma_happy = true; ok! (as of the officiel release, this is line 386.) and let me know if that works. If it does I will update vader to allow CMA in this configuration. Yep! It now works perfectly. Testing with https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my own computer (dual Xeon), I have this: Without CMA: ***Message size: 100 *** best / avg / worst (MB/sec) task pair:0 -1:8363.52 / 7946.77 / 5391.14 with CMA: task pair:0 -1:9137.92 / 8955.98 / 7489.83 Great! Now I have to bench my real application... ;-) Thanks! Eric
Re: [OMPI users] Help on getting CMA works
Great! I will add an MCA variable to force CMA and also enable it if 1) no yama and 2) no PR_SET_PTRACER. You might also look at using xpmem. You can find a version that supports 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + userspace library that can be used by vader as a single-copy mechanism. In benchmarks it performs better than CMA but it may or may not perform better with a real application. See: http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy -Nathan On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > >On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > >> > >>On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > >>> > >>>If you have yama installed you can try: > >> > >>Nope, I do not have it installed... is it absolutely necessary? (and would > >>it change something when it fails when I am root?) > >> > >>Other question: In addition to "--with-cma" configure flag, do we have to > >>pass any options to "mpicc" when compiling/linking an mpi application to use > >>cma? > > > >No. CMA should work out of the box. You appear to have a setup I haven't > >yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > >prctl. Its quite possible there are no restriction on ptrace in this > >setup. Can you try changing the following line at > >opal/mca/btl/vader/btl_vader_component.c:370 from: > > > >bool cma_happy = false; > > > >to > > > >bool cma_happy = true; > > > > ok! (as of the officiel release, this is line 386.) > > >and let me know if that works. If it does I will update vader to allow > >CMA in this configuration. > > Yep! It now works perfectly. Testing with > https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my > own computer (dual Xeon), I have this: > > Without CMA: > > ***Message size: 100 *** best / avg / worst (MB/sec) >task pair:0 -1:8363.52 / 7946.77 / 5391.14 > > with CMA: >task pair:0 -1:9137.92 / 8955.98 / 7489.83 > > Great! > > Now I have to bench my real application... ;-) > > Thanks! > > Eric > > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26355.php pgp4qiwgVsc3t.pgp Description: PGP signature
Re: [OMPI users] Help on getting CMA works
On 02/19/2015 03:53 PM, Nathan Hjelm wrote: Great! I will add an MCA variable to force CMA and also enable it if 1) no yama and 2) no PR_SET_PTRACER. cool, thanks again! You might also look at using xpmem. You can find a version that supports 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + userspace library that can be used by vader as a single-copy mechanism. In benchmarks it performs better than CMA but it may or may not perform better with a real application. See: http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy ok, I will look (and relay the information to colleagues). Thanks, Eric
Re: [OMPI users] Help on getting CMA works
Nathan, I think I already pushed a patch for this particular issue last month. I do not know if it has been back ported to release yet. See here:https://github.com/open-mpi/ompi/commit/ee3b0903164898750137d3b71a8f067e16521102 Aurelien -- ~~~ Aurélien Bouteiller, Ph.D. ~~~ ~ Research Scientist @ ICL ~ The University of Tennessee, Innovative Computing Laboratory 1122 Volunteer Blvd, suite 309, Knoxville, TN 37996 tel: +1 (865) 974-9375 fax: +1 (865) 974-8296 https://icl.cs.utk.edu/~bouteill/ > Le 19 févr. 2015 à 15:53, Nathan Hjelm a écrit : > > > Great! I will add an MCA variable to force CMA and also enable it if 1) > no yama and 2) no PR_SET_PTRACER. > > You might also look at using xpmem. You can find a version that supports > 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > userspace library that can be used by vader as a single-copy mechanism. > > In benchmarks it performs better than CMA but it may or may not perform > better with a real application. > > See: > > http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > -Nathan > > On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: >> On 02/19/2015 02:58 PM, Nathan Hjelm wrote: >>> On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > > If you have yama installed you can try: Nope, I do not have it installed... is it absolutely necessary? (and would it change something when it fails when I am root?) Other question: In addition to "--with-cma" configure flag, do we have to pass any options to "mpicc" when compiling/linking an mpi application to use cma? >>> >>> No. CMA should work out of the box. You appear to have a setup I haven't >>> yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER >>> prctl. Its quite possible there are no restriction on ptrace in this >>> setup. Can you try changing the following line at >>> opal/mca/btl/vader/btl_vader_component.c:370 from: >>> >>> bool cma_happy = false; >>> >>> to >>> >>> bool cma_happy = true; >>> >> >> ok! (as of the officiel release, this is line 386.) >> >>> and let me know if that works. If it does I will update vader to allow >>> CMA in this configuration. >> >> Yep! It now works perfectly. Testing with >> https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my >> own computer (dual Xeon), I have this: >> >> Without CMA: >> >> ***Message size: 100 *** best / avg / worst (MB/sec) >> task pair:0 -1:8363.52 / 7946.77 / 5391.14 >> >> with CMA: >> task pair:0 -1:9137.92 / 8955.98 / 7489.83 >> >> Great! >> >> Now I have to bench my real application... ;-) >> >> Thanks! >> >> Eric >> >> ___ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2015/02/26355.php > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26356.php
Re: [OMPI users] Help on getting CMA works
I don't think that will fix this issue. In this case yama is not installed and it appears PR_SET_PTRACER is not available. This forces vader to assume that CMA can not be used when that isn't always the case. I think it might be safe to assume that CMA is unrestricted here. -Nathan On Thu, Feb 19, 2015 at 04:35:00PM -0500, Aurélien Bouteiller wrote: > Nathan, > > I think I already pushed a patch for this particular issue last month. I do > not know if it has been back ported to release yet. > > See > here:https://github.com/open-mpi/ompi/commit/ee3b0903164898750137d3b71a8f067e16521102 > > Aurelien > > -- > ~~~ Aurélien Bouteiller, Ph.D. ~~~ > ~ Research Scientist @ ICL ~ > The University of Tennessee, Innovative Computing Laboratory > 1122 Volunteer Blvd, suite 309, Knoxville, TN 37996 > tel: +1 (865) 974-9375 fax: +1 (865) 974-8296 > https://icl.cs.utk.edu/~bouteill/ > > > > > > Le 19 févr. 2015 à 15:53, Nathan Hjelm a écrit : > > > > > > Great! I will add an MCA variable to force CMA and also enable it if 1) > > no yama and 2) no PR_SET_PTRACER. > > > > You might also look at using xpmem. You can find a version that supports > > 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > > userspace library that can be used by vader as a single-copy mechanism. > > > > In benchmarks it performs better than CMA but it may or may not perform > > better with a real application. > > > > See: > > > > http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > > > -Nathan > > > > On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > >> On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > >>> On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > > On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > > > > If you have yama installed you can try: > > Nope, I do not have it installed... is it absolutely necessary? (and > would > it change something when it fails when I am root?) > > Other question: In addition to "--with-cma" configure flag, do we have to > pass any options to "mpicc" when compiling/linking an mpi application to > use > cma? > >>> > >>> No. CMA should work out of the box. You appear to have a setup I haven't > >>> yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > >>> prctl. Its quite possible there are no restriction on ptrace in this > >>> setup. Can you try changing the following line at > >>> opal/mca/btl/vader/btl_vader_component.c:370 from: > >>> > >>> bool cma_happy = false; > >>> > >>> to > >>> > >>> bool cma_happy = true; > >>> > >> > >> ok! (as of the officiel release, this is line 386.) > >> > >>> and let me know if that works. If it does I will update vader to allow > >>> CMA in this configuration. > >> > >> Yep! It now works perfectly. Testing with > >> https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my > >> own computer (dual Xeon), I have this: > >> > >> Without CMA: > >> > >> ***Message size: 100 *** best / avg / worst (MB/sec) > >> task pair:0 -1:8363.52 / 7946.77 / 5391.14 > >> > >> with CMA: > >> task pair:0 -1:9137.92 / 8955.98 / 7489.83 > >> > >> Great! > >> > >> Now I have to bench my real application... ;-) > >> > >> Thanks! > >> > >> Eric > >> > >> ___ > >> users mailing list > >> us...@open-mpi.org > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > >> Link to this post: > >> http://www.open-mpi.org/community/lists/users/2015/02/26355.php > > ___ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2015/02/26356.php > > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26358.php pgp7NUNlOKKzV.pgp Description: PGP signature
Re: [OMPI users] Help on getting CMA works
Hmm, wait. Yes. Your change went in after 1.8.4 and has the same effect. If yama ins't installed it is safe to assume that the ptrace scope is effectively 0. So, your patch does fix the issue. -Nathan On Thu, Feb 19, 2015 at 02:53:47PM -0700, Nathan Hjelm wrote: > > I don't think that will fix this issue. In this case yama is not > installed and it appears PR_SET_PTRACER is not available. This forces > vader to assume that CMA can not be used when that isn't always the > case. I think it might be safe to assume that CMA is unrestricted here. > > -Nathan > > On Thu, Feb 19, 2015 at 04:35:00PM -0500, Aurélien Bouteiller wrote: > > Nathan, > > > > I think I already pushed a patch for this particular issue last month. I do > > not know if it has been back ported to release yet. > > > > See > > here:https://github.com/open-mpi/ompi/commit/ee3b0903164898750137d3b71a8f067e16521102 > > > > Aurelien > > > > -- > > ~~~ Aurélien Bouteiller, Ph.D. ~~~ > > ~ Research Scientist @ ICL ~ > > The University of Tennessee, Innovative Computing Laboratory > > 1122 Volunteer Blvd, suite 309, Knoxville, TN 37996 > > tel: +1 (865) 974-9375 fax: +1 (865) 974-8296 > > https://icl.cs.utk.edu/~bouteill/ > > > > > > > > > > > Le 19 févr. 2015 à 15:53, Nathan Hjelm a écrit : > > > > > > > > > Great! I will add an MCA variable to force CMA and also enable it if 1) > > > no yama and 2) no PR_SET_PTRACER. > > > > > > You might also look at using xpmem. You can find a version that supports > > > 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > > > userspace library that can be used by vader as a single-copy mechanism. > > > > > > In benchmarks it performs better than CMA but it may or may not perform > > > better with a real application. > > > > > > See: > > > > > > http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > > > > > -Nathan > > > > > > On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > > >> On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > > >>> On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > > > > On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > > > > > > If you have yama installed you can try: > > > > Nope, I do not have it installed... is it absolutely necessary? (and > > would > > it change something when it fails when I am root?) > > > > Other question: In addition to "--with-cma" configure flag, do we have > > to > > pass any options to "mpicc" when compiling/linking an mpi application > > to use > > cma? > > >>> > > >>> No. CMA should work out of the box. You appear to have a setup I haven't > > >>> yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > > >>> prctl. Its quite possible there are no restriction on ptrace in this > > >>> setup. Can you try changing the following line at > > >>> opal/mca/btl/vader/btl_vader_component.c:370 from: > > >>> > > >>> bool cma_happy = false; > > >>> > > >>> to > > >>> > > >>> bool cma_happy = true; > > >>> > > >> > > >> ok! (as of the officiel release, this is line 386.) > > >> > > >>> and let me know if that works. If it does I will update vader to allow > > >>> CMA in this configuration. > > >> > > >> Yep! It now works perfectly. Testing with > > >> https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my > > >> own computer (dual Xeon), I have this: > > >> > > >> Without CMA: > > >> > > >> ***Message size: 100 *** best / avg / worst (MB/sec) > > >> task pair:0 -1:8363.52 / 7946.77 / 5391.14 > > >> > > >> with CMA: > > >> task pair:0 -1:9137.92 / 8955.98 / 7489.83 > > >> > > >> Great! > > >> > > >> Now I have to bench my real application... ;-) > > >> > > >> Thanks! > > >> > > >> Eric > > >> > > >> ___ > > >> users mailing list > > >> us...@open-mpi.org > > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > >> Link to this post: > > >> http://www.open-mpi.org/community/lists/users/2015/02/26355.php > > > ___ > > > users mailing list > > > us...@open-mpi.org > > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > > Link to this post: > > > http://www.open-mpi.org/community/lists/users/2015/02/26356.php > > > > ___ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2015/02/26358.php > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26359.php pgpO1eJLnaTfh.pg
Re: [OMPI users] Help on getting CMA works
Aurélien, I should also point out your fix has already been applied to the 1.8 branch and will be included in 1.8.5. -Nathan On Thu, Feb 19, 2015 at 02:57:38PM -0700, Nathan Hjelm wrote: > > Hmm, wait. Yes. Your change went in after 1.8.4 and has the same > effect. If yama ins't installed it is safe to assume that the ptrace > scope is effectively 0. So, your patch does fix the issue. > > -Nathan > > On Thu, Feb 19, 2015 at 02:53:47PM -0700, Nathan Hjelm wrote: > > > > I don't think that will fix this issue. In this case yama is not > > installed and it appears PR_SET_PTRACER is not available. This forces > > vader to assume that CMA can not be used when that isn't always the > > case. I think it might be safe to assume that CMA is unrestricted here. > > > > -Nathan > > > > On Thu, Feb 19, 2015 at 04:35:00PM -0500, Aurélien Bouteiller wrote: > > > Nathan, > > > > > > I think I already pushed a patch for this particular issue last month. I > > > do not know if it has been back ported to release yet. > > > > > > See > > > here:https://github.com/open-mpi/ompi/commit/ee3b0903164898750137d3b71a8f067e16521102 > > > > > > Aurelien > > > > > > -- > > > ~~~ Aurélien Bouteiller, Ph.D. ~~~ > > > ~ Research Scientist @ ICL ~ > > > The University of Tennessee, Innovative Computing Laboratory > > > 1122 Volunteer Blvd, suite 309, Knoxville, TN 37996 > > > tel: +1 (865) 974-9375 fax: +1 (865) 974-8296 > > > https://icl.cs.utk.edu/~bouteill/ > > > > > > > > > > > > > > > > Le 19 févr. 2015 à 15:53, Nathan Hjelm a écrit : > > > > > > > > > > > > Great! I will add an MCA variable to force CMA and also enable it if 1) > > > > no yama and 2) no PR_SET_PTRACER. > > > > > > > > You might also look at using xpmem. You can find a version that supports > > > > 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > > > > userspace library that can be used by vader as a single-copy mechanism. > > > > > > > > In benchmarks it performs better than CMA but it may or may not perform > > > > better with a real application. > > > > > > > > See: > > > > > > > > http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > > > > > > > -Nathan > > > > > > > > On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > > > >> On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > > > >>> On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > > > > > > On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > > > > > > > > If you have yama installed you can try: > > > > > > Nope, I do not have it installed... is it absolutely necessary? (and > > > would > > > it change something when it fails when I am root?) > > > > > > Other question: In addition to "--with-cma" configure flag, do we > > > have to > > > pass any options to "mpicc" when compiling/linking an mpi > > > application to use > > > cma? > > > >>> > > > >>> No. CMA should work out of the box. You appear to have a setup I > > > >>> haven't > > > >>> yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > > > >>> prctl. Its quite possible there are no restriction on ptrace in this > > > >>> setup. Can you try changing the following line at > > > >>> opal/mca/btl/vader/btl_vader_component.c:370 from: > > > >>> > > > >>> bool cma_happy = false; > > > >>> > > > >>> to > > > >>> > > > >>> bool cma_happy = true; > > > >>> > > > >> > > > >> ok! (as of the officiel release, this is line 386.) > > > >> > > > >>> and let me know if that works. If it does I will update vader to allow > > > >>> CMA in this configuration. > > > >> > > > >> Yep! It now works perfectly. Testing with > > > >> https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on > > > >> my > > > >> own computer (dual Xeon), I have this: > > > >> > > > >> Without CMA: > > > >> > > > >> ***Message size: 100 *** best / avg / worst (MB/sec) > > > >> task pair:0 -1:8363.52 / 7946.77 / 5391.14 > > > >> > > > >> with CMA: > > > >> task pair:0 -1:9137.92 / 8955.98 / 7489.83 > > > >> > > > >> Great! > > > >> > > > >> Now I have to bench my real application... ;-) > > > >> > > > >> Thanks! > > > >> > > > >> Eric > > > >> > > > >> ___ > > > >> users mailing list > > > >> us...@open-mpi.org > > > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > > >> Link to this post: > > > >> http://www.open-mpi.org/community/lists/users/2015/02/26355.php > > > > ___ > > > > users mailing list > > > > us...@open-mpi.org > > > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > Link to this post: > > > > http://www.open-mpi.org/community/lists/users/2015/02/26356.php > > > > > > ___ > > > users
Re: [OMPI users] Help on getting CMA works
Maybe it is a stupid question, but... why it is not tested and enabled by default at configure time since it is part of the kernel? Eric On 02/19/2015 03:53 PM, Nathan Hjelm wrote: Great! I will add an MCA variable to force CMA and also enable it if 1) no yama and 2) no PR_SET_PTRACER. You might also look at using xpmem. You can find a version that supports 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + userspace library that can be used by vader as a single-copy mechanism. In benchmarks it performs better than CMA but it may or may not perform better with a real application. See: http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy -Nathan On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: On 02/19/2015 02:58 PM, Nathan Hjelm wrote: On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: On 02/19/2015 11:56 AM, Nathan Hjelm wrote: If you have yama installed you can try: Nope, I do not have it installed... is it absolutely necessary? (and would it change something when it fails when I am root?) Other question: In addition to "--with-cma" configure flag, do we have to pass any options to "mpicc" when compiling/linking an mpi application to use cma? No. CMA should work out of the box. You appear to have a setup I haven't yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER prctl. Its quite possible there are no restriction on ptrace in this setup. Can you try changing the following line at opal/mca/btl/vader/btl_vader_component.c:370 from: bool cma_happy = false; to bool cma_happy = true; ok! (as of the officiel release, this is line 386.) and let me know if that works. If it does I will update vader to allow CMA in this configuration. Yep! It now works perfectly. Testing with https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my own computer (dual Xeon), I have this: Without CMA: ***Message size: 100 *** best / avg / worst (MB/sec) task pair:0 -1:8363.52 / 7946.77 / 5391.14 with CMA: task pair:0 -1:9137.92 / 8955.98 / 7489.83 Great! Now I have to bench my real application... ;-) Thanks! Eric ___ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2015/02/26355.php
