Thanks, Howard.
I downloaded a current version of UCX (1.11.2) and installed it with OpenMPI
4.1.1. When I try to specify the "-mca pml ucx" for a simple, 2-process
benchmark problem, I get:
--------------------------------------------------------------------------
No components were able to be opened in the pml framework.
This typically means that either no components of this type were
installed, or none of the installed components can be loaded.
Sometimes this means that shared libraries required by these
components are unable to be found/loaded.
Host: bl1311
Framework: pml
--------------------------------------------------------------------------
[bl1311:20168] PML ucx cannot be selected
[bl1311:20169] PML ucx cannot be selected
------------------------------------------------------------
I've attached my ucx_info -d output, as well as the ucx configuration
information. I'm not sure I follow everything on the UCX FAQ page, but it seems
like everything is being routed over TCP, which is probably not what I want.
Any thoughts as to what I might be doing wrong?
Thanks,
Greg
From: Pritchard Jr., Howard <howa...@lanl.gov>
Sent: Wednesday, October 13, 2021 12:28 PM
To: Open MPI Users <users@lists.open-mpi.org>
Cc: Fischer, Greg A. <fisch...@westinghouse.com>
Subject: Re: [EXTERNAL] [OMPI users] OpenMPI 3.1.6 openib failure: "mlx4_0
errno says Success"
[External Email]
HI Greg,
It's the aging of the openib btl.
You may be able to apply the attached patch. Note the 3.1.x release stream is
no longer supported.
You may want to try using the 4.1.1 release, in which case you'll want to use
UCX.
Howard
From: users
<users-boun...@lists.open-mpi.org<mailto:users-boun...@lists.open-mpi.org>> on
behalf of "Fischer, Greg A. via users"
<users@lists.open-mpi.org<mailto:users@lists.open-mpi.org>>
Reply-To: Open MPI Users
<users@lists.open-mpi.org<mailto:users@lists.open-mpi.org>>
Date: Wednesday, October 13, 2021 at 10:06 AM
To: "users@lists.open-mpi.org<mailto:users@lists.open-mpi.org>"
<users@lists.open-mpi.org<mailto:users@lists.open-mpi.org>>
Cc: "Fischer, Greg A."
<fisch...@westinghouse.com<mailto:fisch...@westinghouse.com>>
Subject: [EXTERNAL] [OMPI users] OpenMPI 3.1.6 openib failure: "mlx4_0 errno
says Success"
Hello,
I have compiled OpenMPI 3.1.6 from source on SLES12-SP3, and I am seeing the
following errors when I try to use the openib btl:
WARNING: There was an error initializing an OpenFabrics device.
Local host: bl1308
Local device: mlx4_0
--------------------------------------------------------------------------
[bl1308][[44866,1],5][../../../../../openmpi-3.1.6/opal/mca/btl/openib/btl_openib_component.c:1671:init_one_device]
error obtaining device attributes for mlx4_0 errno says Success
I have disabled UCX ("--without-ucx") because the UCX installation we have
seems to be too out-of-date. ofed_info says "MLNX_OFED_LINUX-4.1-1.0.2.0". I've
attached the detailed output of ofed_info and ompi_info.
This issue seems similar to Issue #7461
(https://github.com/open-mpi/ompi/issues/7461<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopen-mpi%2Fompi%2Fissues%2F7461&data=04%7C01%7Cfischega%40westinghouse.com%7Cfe8eac2c9dfb4f26781a08d98e667521%7C516ec17ab92f438b8594e11b6f6bec79%7C0%7C0%7C637697392985500288%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uZVYaEU3YA7hcUD%2F4Mrtarmo26J64O41I9WlPDPpXLk%3D&reserved=0>),
which I don't see a resolution for.
Does anyone know what the likely explanation is? Is the version of OFED on the
system badly out-of-sync with contemporary OpenMPI?
Thanks,
Greg
________________________________
This e-mail may contain proprietary information of the sending organization.
Any unauthorized or improper disclosure, copying, distribution, or use of the
contents of this e-mail and attached document(s) is prohibited. The information
contained in this e-mail and attached document(s) is intended only for the
personal and private use of the recipient(s) named above. If you have received
this communication in error, please notify the sender immediately by email and
delete the original e-mail and attached document(s).
________________________________
This e-mail may contain proprietary information of the sending organization.
Any unauthorized or improper disclosure, copying, distribution, or use of the
contents of this e-mail and attached document(s) is prohibited. The information
contained in this e-mail and attached document(s) is intended only for the
personal and private use of the recipient(s) named above. If you have received
this communication in error, please notify the sender immediately by email and
delete the original e-mail and attached document(s).
#
# Memory domain: posix
# Component: posix
# allocate: unlimited
# remote key: 24 bytes
# rkey_ptr is supported
#
# Transport: posix
# Device: memory
# System device: <unknown>
#
# capabilities:
# bandwidth: 0.00/ppn + 12179.00 MB/sec
# latency: 80 nsec
# overhead: 10 nsec
# put_short: <= 4294967295
# put_bcopy: unlimited
# get_bcopy: unlimited
# am_short: <= 100
# am_bcopy: <= 8256
# domain: cpu
# atomic_add: 32, 64 bit
# atomic_and: 32, 64 bit
# atomic_or: 32, 64 bit
# atomic_xor: 32, 64 bit
# atomic_fadd: 32, 64 bit
# atomic_fand: 32, 64 bit
# atomic_for: 32, 64 bit
# atomic_fxor: 32, 64 bit
# atomic_swap: 32, 64 bit
# atomic_cswap: 32, 64 bit
# connection: to iface
# device priority: 0
# device num paths: 1
# max eps: inf
# device address: 8 bytes
# iface address: 8 bytes
# error handling: ep_check
#
#
# Memory domain: sysv
# Component: sysv
# allocate: unlimited
# remote key: 12 bytes
# rkey_ptr is supported
#
# Transport: sysv
# Device: memory
# System device: <unknown>
#
# capabilities:
# bandwidth: 0.00/ppn + 12179.00 MB/sec
# latency: 80 nsec
# overhead: 10 nsec
# put_short: <= 4294967295
# put_bcopy: unlimited
# get_bcopy: unlimited
# am_short: <= 100
# am_bcopy: <= 8256
# domain: cpu
# atomic_add: 32, 64 bit
# atomic_and: 32, 64 bit
# atomic_or: 32, 64 bit
# atomic_xor: 32, 64 bit
# atomic_fadd: 32, 64 bit
# atomic_fand: 32, 64 bit
# atomic_for: 32, 64 bit
# atomic_fxor: 32, 64 bit
# atomic_swap: 32, 64 bit
# atomic_cswap: 32, 64 bit
# connection: to iface
# device priority: 0
# device num paths: 1
# max eps: inf
# device address: 8 bytes
# iface address: 8 bytes
# error handling: ep_check
#
#
# Memory domain: self
# Component: self
# register: unlimited, cost: 0 nsec
# remote key: 0 bytes
#
# Transport: self
# Device: memory0
# System device: <unknown>
#
# capabilities:
# bandwidth: 0.00/ppn + 6911.00 MB/sec
# latency: 0 nsec
# overhead: 10 nsec
# put_short: <= 4294967295
# put_bcopy: unlimited
# get_bcopy: unlimited
# am_short: <= 8K
# am_bcopy: <= 8K
# domain: cpu
# atomic_add: 32, 64 bit
# atomic_and: 32, 64 bit
# atomic_or: 32, 64 bit
# atomic_xor: 32, 64 bit
# atomic_fadd: 32, 64 bit
# atomic_fand: 32, 64 bit
# atomic_for: 32, 64 bit
# atomic_fxor: 32, 64 bit
# atomic_swap: 32, 64 bit
# atomic_cswap: 32, 64 bit
# connection: to iface
# device priority: 0
# device num paths: 1
# max eps: inf
# device address: 0 bytes
# iface address: 8 bytes
# error handling: ep_check
#
#
# Memory domain: tcp
# Component: tcp
# register: unlimited, cost: 0 nsec
# remote key: 0 bytes
#
# Transport: tcp
# Device: ib0
# System device: <unknown>
#
# capabilities:
# bandwidth: 11.90/ppn + 0.00 MB/sec
# latency: 10960 nsec
# overhead: 50000 nsec
# put_zcopy: <= 18446744073709551590, up to 6 iov
# put_opt_zcopy_align: <= 1
# put_align_mtu: <= 0
# am_short: <= 8K
# am_bcopy: <= 8K
# am_zcopy: <= 64K, up to 6 iov
# am_opt_zcopy_align: <= 1
# am_align_mtu: <= 0
# am header: <= 8037
# connection: to ep, to iface
# device priority: 1
# device num paths: 1
# max eps: 256
# device address: 6 bytes
# iface address: 2 bytes
# ep address: 10 bytes
# error handling: peer failure, ep_check, keepalive
#
# Transport: tcp
# Device: lo
# System device: <unknown>
#
# capabilities:
# bandwidth: 11.91/ppn + 0.00 MB/sec
# latency: 10960 nsec
# overhead: 50000 nsec
# put_zcopy: <= 18446744073709551590, up to 6 iov
# put_opt_zcopy_align: <= 1
# put_align_mtu: <= 0
# am_short: <= 8K
# am_bcopy: <= 8K
# am_zcopy: <= 64K, up to 6 iov
# am_opt_zcopy_align: <= 1
# am_align_mtu: <= 0
# am header: <= 8037
# connection: to ep, to iface
# device priority: 1
# device num paths: 1
# max eps: 256
# device address: 18 bytes
# iface address: 2 bytes
# ep address: 10 bytes
# error handling: peer failure, ep_check, keepalive
#
# Transport: tcp
# Device: eth0
# System device: <unknown>
#
# capabilities:
# bandwidth: 113.16/ppn + 0.00 MB/sec
# latency: 5776 nsec
# overhead: 50000 nsec
# put_zcopy: <= 18446744073709551590, up to 6 iov
# put_opt_zcopy_align: <= 1
# put_align_mtu: <= 0
# am_short: <= 8K
# am_bcopy: <= 8K
# am_zcopy: <= 64K, up to 6 iov
# am_opt_zcopy_align: <= 1
# am_align_mtu: <= 0
# am header: <= 8037
# connection: to ep, to iface
# device priority: 0
# device num paths: 1
# max eps: 256
# device address: 6 bytes
# iface address: 2 bytes
# ep address: 10 bytes
# error handling: peer failure, ep_check, keepalive
#
#
# Connection manager: tcp
# max_conn_priv: 2064 bytes
#
# Memory domain: cma
# Component: cma
# register: unlimited, cost: 9 nsec
#
# Transport: cma
# Device: memory
# System device: <unknown>
#
# capabilities:
# bandwidth: 0.00/ppn + 11145.00 MB/sec
# latency: 80 nsec
# overhead: 400 nsec
# put_zcopy: unlimited, up to 16 iov
# put_opt_zcopy_align: <= 1
# put_align_mtu: <= 1
# get_zcopy: unlimited, up to 16 iov
# get_opt_zcopy_align: <= 1
# get_align_mtu: <= 1
# connection: to iface
# device priority: 0
# device num paths: 1
# max eps: inf
# device address: 8 bytes
# iface address: 4 bytes
# error handling: peer failure, ep_check
#
checking for gcc... icc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether icc accepts -g... yes
checking for icc option to accept ISO C89... none needed
checking how to run the C preprocessor... icc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for git... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking whether make supports nested variables... yes
checking whether UID '2513' is supported by ustar format... yes
checking whether GID '250' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking dependency style of icc... gcc3
checking whether make supports nested variables... (cached) yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for gcc... (cached) icc
checking whether we are using the GNU C compiler... (cached) yes
checking whether icc accepts -g... (cached) yes
checking for icc option to accept ISO C89... (cached) none needed
checking whether we are using the GNU C++ compiler... yes
checking whether icpc accepts -g... yes
checking dependency style of icpc... gcc3
checking for icc option to support OpenMP... -fopenmp
checking dependency style of icc... gcc3
checking whether ln -s works... yes
checking for a sed that does not truncate output... /usr/bin/sed
checking build system type... x86_64-suse-linux-gnu
checking host system type... x86_64-suse-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for ld used by icc... /usr/x86_64-suse-linux/bin/ld
checking if the linker (/usr/x86_64-suse-linux/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 3458764513820540925
checking how to convert x86_64-suse-linux-gnu file names to
x86_64-suse-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-suse-linux-gnu file names to toolchain format...
func_convert_file_noop
checking for /usr/x86_64-suse-linux/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from icc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if icc supports -fno-rtti -fno-exceptions... yes
checking for icc option to produce PIC... -fPIC -DPIC
checking if icc PIC flag -fPIC -DPIC works... yes
checking if icc static flag -static works... yes
checking if icc supports -c -o file.o... yes
checking if icc supports -c -o file.o... (cached) yes
checking whether the icc linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64)
supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... icpc -E
checking for ld used by icpc... /usr/x86_64-suse-linux/bin/ld -m elf_x86_64
checking if the linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64) is GNU
ld... yes
checking whether the icpc linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64)
supports shared libraries... yes
checking for icpc option to produce PIC... -fPIC -DPIC
checking if icpc PIC flag -fPIC -DPIC works... yes
checking if icpc static flag -static works... yes
checking if icpc supports -c -o file.o... yes
checking if icpc supports -c -o file.o... (cached) yes
checking whether the icpc linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64)
supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for ANSI C header files... (cached) yes
checking for cos in -lm... yes
checking for C/C++ restrict keyword... __restrict
checking whether strerror_r is declared... yes
checking for strerror_r... yes
checking whether strerror_r returns char *... yes
checking for pkg-config... /usr/bin/pkg-config
checking if ln -s supports --relative... yes
checking for dot... yes
checking for doxygen... /usr/bin/doxygen
checking for perl... /usr/bin/perl
checking for pdflatex... /usr/bin/pdflatex
checking for makeindex... /usr/bin/makeindex
checking for bibtex... /usr/bin/bibtex
checking for egrep... /usr/bin/egrep
checking for size_t... yes
checking compiler flag -diag-error 10006... yes
checking whether -diag-disable 1478 overrides deprecated declarations... yes
checking whether -Wno-deprecated-declarations overrides deprecated
declarations... yes
checking compiler flag -diag-disable 269... yes
checking compiler flag -fmax-type-align=16... no
configure: Detected CPU implementation:
configure: Detected CPU architecture:
configure: Detected CPU variant:
configure: Detected CPU part:
checking for __attribute__(optimize)... 1
checking compiler flag -funwind-tables... no
configure: compiling without unwind tables
checking if icpc works... yes
checking c++11 support... yes
checking gnu++11 support... yes
checking whether _GLIBCXX_NOTHROW is declared... yes
checking compiler flag --display_error_number... no
checking compiler flag --diag_suppress 1... no
checking compiler flag --diag_suppress 68... no
checking compiler flag --diag_suppress 111... no
checking compiler flag --diag_suppress 167... no
checking compiler flag --diag_suppress 181... no
checking compiler flag --diag_suppress 188... no
checking compiler flag --diag_suppress 381... no
checking compiler flag --diag_suppress 1215... no
checking compiler flag --diag_suppress 1901... no
checking compiler flag --diag_suppress 1902... no
checking compiler flag -pedantic... yes
checking compiler flag -Wno-missing-field-initializers... yes
checking compiler flag -Wno-unused-parameter... yes
checking compiler flag -Wno-unused-label... yes
checking compiler flag -Wno-long-long... yes
checking compiler flag -Wno-endif-labels... yes
checking compiler flag -Wno-sign-compare... yes
checking compiler flag -Wno-multichar... yes
checking compiler flag -Wno-deprecated-declarations... yes
checking compiler flag -Winvalid-pch... yes
checking compiler flag -Wno-pointer-sign... yes
checking compiler flag -Werror-implicit-function-declaration... yes
checking compiler flag -Wno-format-zero-length... yes
checking compiler flag -Wnested-externs... yes
checking compiler flag -Wshadow... yes
checking compiler flag -Werror=declaration-after-statement... yes
checking for working alloca.h... yes
checking for alloca... yes
checking for shm_open in -lrt... yes
checking for timer_create in -lrt... yes
checking libgen.h usability... yes
checking libgen.h presence... yes
checking for libgen.h... yes
checking whether asprintf is declared... yes
checking whether basename is declared... yes
checking whether fmemopen is declared... yes
checking sys/cpuset.h usability... no
checking sys/cpuset.h presence... no
checking for sys/cpuset.h... no
checking whether CPU_ZERO is declared... yes
checking whether CPU_ISSET is declared... yes
checking for cpu_set_t... yes
checking for cpuset_t... no
checking for sighandler_t... yes
checking for __sighandler_t... yes
checking pthread_np.h usability... no
checking pthread_np.h presence... no
checking for pthread_np.h... no
checking for library containing pthread_create... -lpthread
checking for library containing pthread_atfork... none required
checking for clearenv... yes
checking for malloc_trim... yes
checking for memalign... yes
checking for posix_memalign... yes
checking for mremap... yes
checking for sched_setaffinity... yes
checking for sched_getaffinity... yes
checking for cpuset_setaffinity... no
checking for cpuset_getaffinity... no
checking whether F_SETOWN_EX is declared... yes
checking whether ethtool_cmd_speed is declared... yes
checking whether SPEED_UNKNOWN is declared... yes
checking whether __ppc_get_timebase_freq is declared... no
checking sys/platform/ppc.h usability... no
checking sys/platform/ppc.h presence... no
checking for sys/platform/ppc.h... no
checking for using Google C++ Testing Framework... no
checking numa.h usability... no
checking numa.h presence... no
checking for numa.h... no
checking numaif.h usability... no
checking numaif.h presence... no
checking for numaif.h... no
checking for mbind in -lnuma... no
checking for struct bitmask... no
configure: WARNING: NUMA support not found, this many impact library
performance.
configure: WARNING: Please consider installing libnuma-devel package.
checking malloc hooks... yes
checking sys/capability.h usability... no
checking sys/capability.h presence... no
checking for sys/capability.h... no
checking whether PR_SET_PTRACER is declared... yes
checking for struct in6_addr.s6_addr32... yes
checking for struct in6_addr.__u6_addr.__u6_addr32... no
checking for struct iphdr.daddr.s_addr... no
checking for struct ip.ip_dst.s_addr... yes
checking for struct sigevent._sigev_un._tid... yes
checking for struct sigevent.sigev_notify_thread_id... no
checking for struct sigaction.sa_restorer... yes
checking sys/epoll.h usability... yes
checking sys/epoll.h presence... yes
checking for sys/epoll.h... yes
checking sys/eventfd.h usability... yes
checking sys/eventfd.h presence... yes
checking for sys/eventfd.h... yes
checking sys/event.h usability... no
checking sys/event.h presence... no
checking for sys/event.h... no
checking sys/thr.h usability... no
checking sys/thr.h presence... no
checking for sys/thr.h... no
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking malloc_np.h usability... no
checking malloc_np.h presence... no
checking for malloc_np.h... no
checking endian.h, usability... no
checking endian.h, presence... no
checking for endian.h,... no
checking sys/endian.h usability... no
checking sys/endian.h presence... no
checking for sys/endian.h... no
checking linux/mman.h usability... yes
checking linux/mman.h presence... yes
checking for linux/mman.h... yes
checking linux/ip.h usability... yes
checking linux/ip.h presence... yes
checking for linux/ip.h... yes
checking linux/futex.h usability... yes
checking linux/futex.h presence... yes
checking for linux/futex.h... yes
checking for net/ethernet.h... yes
checking for netinet/ip.h... yes
configure: Memory allocator is ptmalloc-2.8.6 version
checking for malloc_get_state... yes
checking for malloc_set_state... yes
checking whether MADV_FREE is declared... no
checking whether MADV_REMOVE is declared... yes
checking whether POSIX_MADV_DONTNEED is declared... yes
checking whether SYS_mmap is declared... yes
checking whether SYS_munmap is declared... yes
checking whether SYS_mremap is declared... yes
checking whether SYS_brk is declared... yes
checking whether SYS_madvise is declared... yes
checking whether SYS_shmat is declared... yes
checking whether SYS_shmdt is declared... yes
checking whether SYS_ipc is declared... no
checking for __curbrk... yes
checking for tc_malloc in -ltcmalloc... no
Package fuse3 was not found in the pkg-config search path.
Perhaps you should add the directory containing `fuse3.pc'
to the PKG_CONFIG_PATH environment variable
No package 'fuse3' found
Package fuse3 was not found in the pkg-config search path.
Perhaps you should add the directory containing `fuse3.pc'
to the PKG_CONFIG_PATH environment variable
No package 'fuse3' found
checking whether fuse_open_channel is declared... no
checking whether fuse_mount is declared... no
checking whether fuse_unmount is declared... no
checking for fuse_open_channel... no
checking for fuse_mount... no
checking for fuse_unmount... no
checking for mvn... no
checking for java... yes
configure: WARNING: Disabling Java support - java or mvn not in path.
checking cuda.h usability... no
checking cuda.h presence... no
checking for cuda.h... no
checking cuda_runtime.h usability... no
checking cuda_runtime.h presence... no
checking for cuda_runtime.h... no
configure: WARNING: CUDA not found
configure: ROCm path was not specified. Guessing ...
checking hsa.h usability... no
checking hsa.h presence... no
checking for hsa.h... no
configure: WARNING: ROCm not found
checking for hipFree in -lhip_hcc... no
checking hip_runtime.h usability... no
checking hip_runtime.h presence... no
checking for hip_runtime.h... no
configure: WARNING: HIP Runtime not found
checking whether inotify_init is declared... yes
checking whether inotify_add_watch is declared... yes
checking whether IN_ATTRIB is declared... yes
checking bfd.h usability... yes
checking bfd.h presence... yes
checking for bfd.h... yes
checking for bfd_openr in -lbfd... no
configure: WARNING: bfd library not found
checking for dlopen in -ldl... yes
checking for main in -lintl... no
configure: WARNING: intl library not found
checking for struct dl_phdr_info... yes
checking whether bfd_get_section_flags is declared... yes
checking whether bfd_section_flags is declared... no
checking whether bfd_get_section_vma is declared... yes
checking whether bfd_section_vma is declared... yes
checking bfd_section_size API version... 2-args API
configure: WARNING: detailed backtrace is not supported
checking __attribute__((constructor))... yes
configure: enabling builtin memcpy
checking for __clear_cache... yes
checking for __aarch64_sync_cache_range... no
checking gdrapi.h usability... no
checking gdrapi.h presence... no
checking for gdrapi.h... no
configure: WARNING: GDR_COPY not found
configure: Compiling with verbs support from /usr
checking infiniband/verbs.h usability... yes
checking infiniband/verbs.h presence... yes
checking for infiniband/verbs.h... yes
checking for ibv_get_device_list in -libverbs... yes
checking whether ibv_wc_status_str is declared... yes
checking whether ibv_event_type_str is declared... yes
checking whether ibv_query_gid is declared... yes
checking whether ibv_get_device_name is declared... yes
checking whether ibv_create_srq is declared... yes
checking whether ibv_get_async_event is declared... yes
checking infiniband/verbs_exp.h usability... yes
checking infiniband/verbs_exp.h presence... yes
checking for infiniband/verbs_exp.h... yes
checking for struct ibv_exp_device_attr.exp_device_cap_flags... yes
checking for struct ibv_exp_device_attr.odp_caps... yes
checking for struct
ibv_exp_device_attr.odp_caps.per_transport_caps.dc_odp_caps... yes
checking for struct ibv_exp_device_attr.odp_mr_max_size... yes
checking for struct ibv_exp_qp_init_attr.max_inl_recv... yes
checking for struct ibv_async_event.element.dct... yes
checking whether IBV_CREATE_CQ_ATTR_IGNORE_OVERRUN is declared... no
checking whether IBV_EXP_CQ_IGNORE_OVERRUN is declared... yes
configure: Checking for legacy bare-metal support
checking infiniband/mlx5_hw.h usability... yes
checking infiniband/mlx5_hw.h presence... yes
checking for infiniband/mlx5_hw.h... yes
checking whether ibv_mlx5_exp_get_qp_info is declared... yes
checking whether ibv_mlx5_exp_get_cq_info is declared... yes
checking whether ibv_mlx5_exp_get_srq_info is declared... yes
checking whether ibv_mlx5_exp_update_cq_ci is declared... yes
checking whether MLX5_WQE_CTRL_SOLICITED is declared... yes
checking for struct mlx5_srq.cmd_qp... yes
checking for struct mlx5_ah.ibv_ah... yes
checking for struct ibv_mlx5_qp_info.bf.need_lock... yes
checking whether IBV_EXP_QP_INIT_ATTR_RES_DOMAIN is declared... yes
checking whether IBV_EXP_RES_DOMAIN_THREAD_MODEL is declared... yes
checking whether ibv_exp_create_res_domain is declared... yes
checking whether ibv_exp_destroy_res_domain is declared... yes
configure: Checking for DV bare-metal support
checking for mlx5dv_query_device in -lmlx5-rdmav2... yes
checking for infiniband/mlx5dv.h... yes
checking whether mlx5dv_init_obj is declared... yes
checking whether mlx5dv_create_qp is declared... no
checking whether mlx5dv_is_supported is declared... no
checking whether mlx5dv_devx_subscribe_devx_event is declared... no
checking whether MLX5DV_CQ_INIT_ATTR_MASK_CQE_SIZE is declared... no
checking whether MLX5DV_QP_CREATE_ALLOW_SCATTER_TO_CQE is declared... no
checking whether MLX5DV_UAR_ALLOC_TYPE_BF is declared... no
checking whether MLX5DV_UAR_ALLOC_TYPE_NC is declared... no
checking for struct mlx5dv_cq.cq_uar... no
checking whether MLX5DV_OBJ_AH is declared... no
checking whether MLX5DV_DCTYPE_DCT is declared... no
checking whether ibv_alloc_td is declared... no
checking whether ibv_alloc_td is declared... (cached) no
checking whether MLX5DV_CONTEXT_FLAGS_DEVX is declared... no
configure: Compiling with mlx5 bare-metal support
checking whether IBV_LINK_LAYER_INFINIBAND is declared... yes
checking whether IBV_LINK_LAYER_ETHERNET is declared... yes
checking whether IBV_EVENT_GID_CHANGE is declared... yes
checking whether ibv_create_qp_ex is declared... yes
checking whether ibv_create_srq_ex is declared... yes
checking whether ibv_query_device_ex is declared... yes
checking whether IBV_EXP_ACCESS_ALLOCATE_MR is declared... yes
checking whether IBV_EXP_ACCESS_ON_DEMAND is declared... yes
checking whether IBV_EXP_DEVICE_MR_ALLOCATE is declared... yes
checking whether IBV_EXP_WR_NOP is declared... yes
checking whether IBV_EXP_DEVICE_DC_TRANSPORT is declared... yes
checking whether IBV_EXP_ATOMIC_HCA_REPLY_BE is declared... yes
checking whether IBV_EXP_PREFETCH_WRITE_ACCESS is declared... yes
checking whether IBV_EXP_QP_OOO_RW_DATA_PLACEMENT is declared... yes
checking whether IBV_EXP_DCT_OOO_RW_DATA_PLACEMENT is declared... yes
checking whether IBV_EXP_CQ_MODERATION is declared... yes
checking whether IBV_EXP_DEVICE_ATTR_PCI_ATOMIC_CAPS is declared... no
checking whether ibv_exp_reg_mr is declared... yes
checking whether ibv_exp_create_qp is declared... yes
checking whether ibv_exp_prefetch_mr is declared... yes
checking whether ibv_exp_create_srq is declared... yes
checking whether ibv_exp_setenv is declared... yes
checking whether ibv_exp_query_gid_attr is declared... yes
checking whether ibv_exp_query_device is declared... yes
checking whether ibv_exp_post_send is declared... yes
checking whether IBV_EXP_WR_EXT_MASKED_ATOMIC_CMP_AND_SWP is declared... yes
checking whether IBV_EXP_WR_EXT_MASKED_ATOMIC_FETCH_AND_ADD is declared... yes
checking whether IBV_EXP_QP_INIT_ATTR_ATOMICS_ARG is declared... yes
checking whether IBV_EXP_SEND_EXT_ATOMIC_INLINE is declared... yes
checking whether IBV_EXP_DEVICE_ATTR_RESERVED_2 is declared... no
checking whether IBV_EXP_MR_INDIRECT_KLMS is declared... yes
checking whether IBV_EXP_QP_CREATE_UMR is declared... yes
checking for struct ibv_exp_qp_init_attr.umr_caps... no
checking whether IBV_EXP_MR_FIXED_BUFFER_SIZE is declared... no
checking for struct ibv_device_attr_ex.pci_atomic_caps... no
checking for struct
ibv_exp_masked_atomic_params.masked_log_atomic_arg_sizes_network_endianness...
yes
checking whether IBV_EXP_ODP_SUPPORT_IMPLICIT is declared... yes
checking whether IBV_EXP_ACCESS_ON_DEMAND is declared... (cached) yes
checking whether IBV_ACCESS_ON_DEMAND is declared... yes
checking whether IBV_EXP_ODP_SUPPORT_IMPLICIT is declared... (cached) yes
checking whether IBV_ODP_SUPPORT_IMPLICIT is declared... no
checking whether IBV_ACCESS_RELAXED_ORDERING is declared... no
checking whether IBV_QPF_GRH_REQUIRED is declared... no
checking whether ibv_exp_prefetch_mr is declared... (cached) yes
checking whether ibv_advise_mr is declared... no
checking for struct mlx5_wqe_av.base... yes
checking for struct mlx5_grh_av.rmac... yes
checking for struct mlx5_cqe64.ib_stride_index... no
checking whether IBV_EXP_QPT_DC_INI is declared... yes
checking infiniband/tm_types.h usability... no
checking infiniband/tm_types.h presence... no
checking for infiniband/tm_types.h... no
checking for struct ibv_exp_tmh.tag... yes
checking for struct ibv_tmh.tag... no
checking for struct ibv_exp_create_srq_attr.dc_offload_params... no
checking whether ibv_exp_alloc_dm is declared... no
checking whether ibv_alloc_dm is declared... no
checking whether ibv_cmd_modify_qp is declared... yes
configure: Checking OFED valgrind libs /usr/lib64/mlnx_ofed/valgrind
checking /usr/include/rdma/rdma_cma.h usability... yes
checking /usr/include/rdma/rdma_cma.h presence... yes
checking for /usr/include/rdma/rdma_cma.h... yes
checking for rdma_establish in -lrdmacm... no
configure: WARNING: RDMACM requested but librdmacm is not found or does not
provide rdma_establish() API
checking sys/uio.h usability... yes
checking sys/uio.h presence... yes
checking for sys/uio.h... yes
checking for process_vm_readv... yes
configure: KNEM path was not found, guessing ...
checking whether KNEM_CMD_GET_INFO is declared... yes
configure: XPMEM - failed to open the requested location (guess), guessing ...
checking cray-ugni... no
checking whether IPPROTO_TCP is declared... yes
checking whether SOL_SOCKET is declared... yes
checking whether SO_KEEPALIVE is declared... yes
checking whether TCP_KEEPCNT is declared... yes
checking whether TCP_KEEPIDLE is declared... yes
checking whether TCP_KEEPINTVL is declared... yes
checking compiler flag -fno-exceptions... yes
checking compiler flag -fno-rtti... yes
checking compiler flag --no_exceptions... no
checking compiler flag -fno-tree-vectorize... yes
checking compiler flag --diag_suppress 186... no
checking compiler flag --diag_suppress 236... no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating src/ucm/cuda/Makefile
config.status: creating src/ucm/rocm/Makefile
config.status: creating src/ucm/Makefile
config.status: creating src/ucs/vfs/sock/Makefile
config.status: creating src/ucs/vfs/fuse/Makefile
config.status: creating src/ucs/Makefile
config.status: creating src/uct/cuda/gdr_copy/Makefile
config.status: creating src/uct/cuda/Makefile
config.status: creating src/uct/ib/rdmacm/Makefile
config.status: creating src/uct/ib/Makefile
config.status: creating src/uct/rocm/gdr/Makefile
config.status: creating src/uct/rocm/Makefile
config.status: creating src/uct/sm/scopy/cma/Makefile
config.status: creating src/uct/sm/scopy/knem/Makefile
config.status: creating src/uct/sm/scopy/Makefile
config.status: creating src/uct/sm/mm/xpmem/Makefile
config.status: creating src/uct/sm/mm/Makefile
config.status: creating src/uct/sm/Makefile
config.status: creating src/uct/ugni/Makefile
config.status: creating src/uct/Makefile
config.status: creating src/tools/perf/lib/Makefile
config.status: creating src/tools/perf/cuda/Makefile
config.status: creating src/tools/perf/rocm/Makefile
config.status: creating src/tools/perf/Makefile
config.status: creating test/gtest/ucm/test_dlopen/Makefile
config.status: creating test/gtest/ucm/test_dlopen/rpath-subdir/Makefile
config.status: creating test/gtest/ucs/test_module/Makefile
config.status: creating test/gtest/Makefile
config.status: creating Makefile
config.status: creating docs/doxygen/header.tex
config.status: creating src/uct/api/version.h
config.status: creating ucx.spec
config.status: creating ucx.pc
config.status: creating contrib/rpmdef.sh
config.status: creating debian/rules
config.status: creating debian/control
config.status: creating debian/changelog
config.status: creating src/ucp/Makefile
config.status: creating src/ucp/api/ucp_version.h
config.status: creating src/ucp/core/ucp_version.c
config.status: creating src/tools/vfs/Makefile
config.status: creating src/tools/info/Makefile
config.status: creating src/tools/profile/Makefile
config.status: creating test/apps/Makefile
config.status: creating test/apps/iodemo/Makefile
config.status: creating test/apps/sockaddr/Makefile
config.status: creating test/apps/profiling/Makefile
config.status: creating test/mpi/Makefile
config.status: creating bindings/java/Makefile
config.status: creating bindings/java/pom.xml
config.status: creating bindings/java/src/main/native/Makefile
config.status: creating examples/Makefile
config.status: creating test/mpi/run_mpi.sh
config.status: creating config.h
config.status: linking /local/fischega/ucx/ucx-1.11.2/contrib/../debian/compat
to debian/compat
config.status: linking
/local/fischega/ucx/ucx-1.11.2/contrib/../debian/copyright to debian/copyright
config.status: linking
/local/fischega/ucx/ucx-1.11.2/contrib/../debian/ucx.prerm to debian/ucx.prerm
config.status: executing depfiles commands
config.status: executing libtool commands
configure: =========================================================
configure: UCX build configuration:
configure: Build prefix: /home/fischega/install/ucx/ucx-1.11.2
configure: Configuration dir: ${prefix}/etc/ucx
configure: Preprocessor flags: -DCPU_FLAGS="" -I${abs_top_srcdir}/src
-I${abs_top_builddir} -I${abs_top_builddir}/src
configure: C compiler: icc -O3 -g -Wall -Werror -diag-error 10006
-diag-disable 269 -Wno-missing-field-initializers -Wno-unused-parameter
-Wno-unused-label -Wno-long-long -Wno-endif-labels -Wno-sign-compare
-Wno-multichar -Wno-deprecated-declarations -Winvalid-pch -Wno-pointer-sign
-Werror-implicit-function-declaration -Wno-format-zero-length -Wnested-externs
-Wshadow -Werror=declaration-after-statement
configure: C++ compiler: icpc -O3 -g -Wall -Werror -diag-error 10006
-diag-disable 269 -Wno-missing-field-initializers -Wno-unused-parameter
-Wno-unused-label -Wno-long-long -Wno-endif-labels -Wno-sign-compare
-Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
configure: Multi-thread: disabled
configure: NUMA support: disabled
configure: MPI tests: disabled
configure: VFS support: no
configure: Devel headers: no
configure: io_demo CUDA support: no
configure: Bindings: < >
configure: UCS modules: < >
configure: UCT modules: < ib cma knem >
configure: CUDA modules: < >
configure: ROCM modules: < >
configure: IB modules: < >
configure: UCM modules: < >
configure: Perf modules: < >
configure: =========================================================
