Er... right. Duh.
> On May 1, 2017, at 11:21 AM, Prentice Bisbal <pbis...@pppl.gov> wrote: > > Jeff, > > Why IBM? This problem is caused by the PGI compilers, so shouldn't this be > directed towards NVidia, which now owns PGI? > > Prentice > > On 04/29/2017 07:37 AM, Jeff Squyres (jsquyres) wrote: >> IBM: can someone check to see if this is a compiler error? >> >> >>> On Apr 28, 2017, at 5:09 PM, Prentice Bisbal <pbis...@pppl.gov> wrote: >>> >>> Update: removing the -fast switch caused this error to go away. >>> >>> Prentice >>> >>> On 04/27/2017 06:00 PM, Prentice Bisbal wrote: >>>> I'm building Open MPI 2.1.0 with PGI 17.3, and now I'm getting 'illegal >>>> instruction' errors during 'make check': >>>> >>>> ../../config/test-driver: line 107: 65169 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math >>>> - 1 threads: Passed >>>> >>>> That's just one example of the error output. See all relevant error output >>>> below. >>>> >>>> Usually, I see these errors when trying to run an executable on a >>>> processor that doesn't support the instruction set of the executable. I >>>> used to see this all the time when I supported an IBM Blue Gene/P system. >>>> I don't think I've ever seen it on an x86 system. >>>> >>>> I'm passing the argument '-tp=x64' to pgcc to build a unified binary, so >>>> that might be part of the problem, but I've used this exact same process >>>> to build 2.1.0 with PGI 16.5 just a couple hours ago. I also built 1.10.3 >>>> with the same compiler flags with PGI 16.5 and 17.3 without this error. >>>> >>>> Any ideas? >>>> >>>> The relevant output from 'make check': >>>> >>>> make check-TESTS >>>> make[3]: Entering directory `/local/pbisbal/openmpi-2.1.0/test/asm' >>>> make[4]: Entering directory `/local/pbisbal/openmpi-2.1.0/test/asm' >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> PASS: atomic_barrier >>>> - 1 threads: Passed >>>> PASS: atomic_barrier >>>> - 2 threads: Passed >>>> PASS: atomic_barrier >>>> - 4 threads: Passed >>>> PASS: atomic_barrier >>>> - 5 threads: Passed >>>> PASS: atomic_barrier >>>> - 8 threads: Passed >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> PASS: atomic_barrier_noinline >>>> - 1 threads: Passed >>>> PASS: atomic_barrier_noinline >>>> - 2 threads: Passed >>>> PASS: atomic_barrier_noinline >>>> - 4 threads: Passed >>>> PASS: atomic_barrier_noinline >>>> - 5 threads: Passed >>>> PASS: atomic_barrier_noinline >>>> - 8 threads: Passed >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> PASS: atomic_spinlock >>>> - 1 threads: Passed >>>> PASS: atomic_spinlock >>>> - 2 threads: Passed >>>> PASS: atomic_spinlock >>>> - 4 threads: Passed >>>> PASS: atomic_spinlock >>>> - 5 threads: Passed >>>> PASS: atomic_spinlock >>>> - 8 threads: Passed >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> PASS: atomic_spinlock_noinline >>>> - 1 threads: Passed >>>> PASS: atomic_spinlock_noinline >>>> - 2 threads: Passed >>>> PASS: atomic_spinlock_noinline >>>> - 4 threads: Passed >>>> PASS: atomic_spinlock_noinline >>>> - 5 threads: Passed >>>> PASS: atomic_spinlock_noinline >>>> - 8 threads: Passed >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> ../../config/test-driver: line 107: 65169 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math >>>> - 1 threads: Passed >>>> ../../config/test-driver: line 107: 65172 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math >>>> - 2 threads: Passed >>>> ../../config/test-driver: line 107: 65176 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math >>>> - 4 threads: Passed >>>> ../../config/test-driver: line 107: 65180 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math >>>> - 5 threads: Passed >>>> ../../config/test-driver: line 107: 65185 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math >>>> - 8 threads: Passed >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> ../../config/test-driver: line 107: 65195 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math_noinline >>>> - 1 threads: Passed >>>> ../../config/test-driver: line 107: 65198 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math_noinline >>>> - 2 threads: Passed >>>> ../../config/test-driver: line 107: 65202 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math_noinline >>>> - 4 threads: Passed >>>> ../../config/test-driver: line 107: 65206 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math_noinline >>>> - 5 threads: Passed >>>> ../../config/test-driver: line 107: 65210 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_math_noinline >>>> - 8 threads: Passed >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> ../../config/test-driver: line 107: 65220 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset >>>> - 1 threads: Passed >>>> ../../config/test-driver: line 107: 65223 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset >>>> - 2 threads: Passed >>>> ../../config/test-driver: line 107: 65227 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset >>>> - 4 threads: Passed >>>> ../../config/test-driver: line 107: 65231 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset >>>> - 5 threads: Passed >>>> ../../config/test-driver: line 107: 65235 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset >>>> - 8 threads: Passed >>>> basename: extra operand `--test-name' >>>> Try `basename --help' for more information. >>>> --> Testing >>>> ../../config/test-driver: line 107: 65245 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset_noinline >>>> - 1 threads: Passed >>>> ../../config/test-driver: line 107: 65248 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset_noinline >>>> - 2 threads: Passed >>>> ../../config/test-driver: line 107: 65252 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset_noinline >>>> - 4 threads: Passed >>>> ../../config/test-driver: line 107: 65256 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset_noinline >>>> - 5 threads: Passed >>>> ../../config/test-driver: line 107: 65260 Illegal instruction "$@" > >>>> $log_file 2>&1 >>>> FAIL: atomic_cmpset_noinline >>>> - 8 threads: Passed >>>> ============================================================================ >>>> Testsuite summary for Open MPI 2.1.0 >>>> ============================================================================ >>>> # TOTAL: 8 >>>> # PASS: 4 >>>> # SKIP: 0 >>>> # XFAIL: 0 >>>> # FAIL: 4 >>>> # XPASS: 0 >>>> # ERROR: 0 >>>> ============================================================================ >>>> See test/asm/test-suite.log >>>> Please report to http://www.open-mpi.org/community/help/ >>>> ============================================================================ >>>> make[4]: *** [test-suite.log] Error 1 >>>> make[4]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test/asm' >>>> make[3]: *** [check-TESTS] Error 2 >>>> make[3]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test/asm' >>>> make[2]: *** [check-am] Error 2 >>>> make[2]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test/asm' >>>> make[1]: *** [check-recursive] Error 1 >>>> make[1]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test' >>>> make: *** [check-recursive] Error 1 >>>> >>>> >>>> The contents of test/asm/test-suite.log: >>>> >>>> $ cat test/asm/test-suite.log >>>> ============================================= >>>> Open MPI 2.1.0: test/asm/test-suite.log >>>> ============================================= >>>> >>>> # TOTAL: 8 >>>> # PASS: 4 >>>> # SKIP: 0 >>>> # XFAIL: 0 >>>> # FAIL: 4 >>>> # XPASS: 0 >>>> # ERROR: 0 >>>> >>>> .. contents:: :depth: 2 >>>> >>>> FAIL: atomic_math >>>> ================= >>>> >>>> FAIL atomic_math (exit status: 132) >>>> >>>> FAIL: atomic_math_noinline >>>> ========================== >>>> >>>> FAIL atomic_math_noinline (exit status: 132) >>>> >>>> FAIL: atomic_cmpset >>>> =================== >>>> >>>> FAIL atomic_cmpset (exit status: 132) >>>> >>>> FAIL: atomic_cmpset_noinline >>>> ============================ >>>> >>>> FAIL atomic_cmpset_noinline (exit status: 132) >>>> >>>> >>>> And my configure command: >>>> >>>> ./configure \ >>>> --prefix=/usr/pppl/pgi/17.3-pkgs/openmpi-2.1.0 \ >>>> --disable-silent-rules \ >>>> --enable-mpi-fortran \ >>>> --enable-mpi-cxx \ >>>> --enable-shared \ >>>> --enable-static \ >>>> --enable-mpi-thread-multiple \ >>>> --with-cuda=/usr/pppl/cuda/cudatoolkit/6.5.14 \ >>>> --with-pmix \ >>>> --with-verbs \ >>>> --with-hwloc \ >>>> --with-pmi=/usr/pppl/slurm/15.08.8 \ >>>> --with-slurm \ >>>> --with-psm \ >>>> CPP=cpp \ >>>> CC=pgcc \ >>>> CFLAGS="-fpic -tp=x64 -fast" \ >>>> CXX=pgc++ \ >>>> CXXFLAGS="-fpic -tp=x64 -fast" \ >>>> FC=pgfortran \ >>>> FCFLAGS="-fpic -tp=x64 -fast" \ >>>> 2>&1 | tee configure.log >>>> >>> _______________________________________________ >>> users mailing list >>> users@lists.open-mpi.org >>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users >> > > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://rfd.newmexicoconsortium.org/mailman/listinfo/users -- Jeff Squyres jsquy...@cisco.com _______________________________________________ users mailing list users@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/users
