Hi Markus,
I had to install additionally this RPM from the Intel OPA software suite
(I disable the CentOS 7.4 RPMs):
IntelOPA-Tools.RHEL74-x86_64.10.6.1.0.2/RPMS/x86_64/opa-opa-libopamgt-devel-10.6.1.0-2.el7.x86_64.rpm
I also had to edit the CMakeLists.txt to replace SLURM_PREFIX /usr/local
by /usr because we install the default Slurm RPMs from SchedMD.
Now opa2slurm builds without errors and works very nicely:
ccmake .
make
$ ./opa2slurm | grep '^Switch'
SwitchName=opa-L2 Nodes=niflfs[1-3],niflmgt,niflopt1,x[025-048]
SwitchName=opa-c1 Switches=opa-L[1-8]
SwitchName=opa-L1 Nodes=x[001-024]
SwitchName=opa-L3 Nodes=x[049-072]
SwitchName=opa-L4 Nodes=x[073-096]
SwitchName=opa-L6 Nodes=x[121-144]
SwitchName=opa-L5 Nodes=x[097-120]
SwitchName=opa-c2 Switches=opa-L[1-8]
SwitchName=opa-c3 Switches=opa-L[1-8]
SwitchName=opa-L7 Nodes=x[169-192]
SwitchName=opa-L8 Nodes=x[145-168]
Jeffrey: It would be very nice if you could document in detail how to
configure opa2slurm and list all prerequisite RPMs in your README.md.
Thanks a lot,
Ole
On 06/14/2018 10:07 AM, Marcus Wagner wrote:
Hi Ole, hi Jeffrey.
Thanks Jeffrey for that tool, for me it is working. I changed a little
bit the CMakeLists.txt such that slurm can be found also in non standard
install paths ;)
replaced
SET (SLURM_PREFIX "/usr/local" CACHE PATH "Directory in which SLURM is
installed.")
with
FIND_PATH(SLURM_PREFIX NAMES sbatch)
and changed
FIND_PATH(SLURM_INCLUDE_DIR NAMES slurm/slurm.h)
FIND_LIBRARY(SLURM_LIBRARY NAMES libslurm.so)
to
FIND_PATH(SLURM_INCLUDE_DIR NAMES slurm/slurm.h HINTS
${SLURM_PREFIX}/../include)
FIND_LIBRARY(SLURM_LIBRARY NAMES libslurm.so HINTS
${SLURM_PREFIX}/../lib64)
@Ole,
I also got first compile errors:
CMakeFiles/opa2slurm.dir/opa2slurm.c.o: In function `main':
opa2slurm.c:(.text+0x15ee): undefined reference to `omgt_status_totext'
opa2slurm.c:(.text+0x16df): undefined reference to `omgt_status_totext'
I used the libopamgt coming with CentOS 7.5 (10.5.0.0.140-2.el7), after
changing this to version 10.6.1.0-2.el7 everything went through:
$> cmake . && make
-- The C compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Found SLURM: /opt/slurm/lib64/libslurm.so
-- Found OPA: /usr/lib/libopamgt.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mw445520/git/opa2slurm
Scanning dependencies of target opa2slurm
[ 25%] Building C object CMakeFiles/opa2slurm.dir/ptr_list.c.o
[ 50%] Building C object CMakeFiles/opa2slurm.dir/opa_host.c.o
[ 75%] Building C object CMakeFiles/opa2slurm.dir/opa_switch.c.o
[100%] Building C object CMakeFiles/opa2slurm.dir/opa2slurm.c.o
Linking C executable opa2slurm
[100%] Built target opa2slurm
the only additional prerequisites for me have been the libopamgt and
libopamgt-devel packages.
So, since you seem to use the same version as me, I'm not sure why you
have these linking problems :/
Best
Marcus
On 06/14/2018 09:17 AM, Ole Holm Nielsen wrote:
Hi Jeffrey,
On 06/13/2018 10:35 PM, Jeffrey Frey wrote:
Intel's OPA doesn't include the old IB net discovery library/API;
instead, they have their own library to enumerate nodes, links, etc.
I've started a rewrite of ye olde "ib2slurm" utility to make use of
Intel's new enumeration library.
https://gitlab.com/jtfrey/opa2slurm
Thanks for sharing this tool! We have an OPA fabric, and I'd like to
try out your tool to generate topology.conf. Unfortunately I'm unable
to build opa2slurm because of missing headers and libraries:
1. The RPM package opa-libopamgt-devel is required for header files.
This RPM may or may not be installed from the
IntelOPA-Basic.RHEL74-x86_64.10.6.1.0.2.tgz tar-ball. I had a node
where the OPA FM software was installed, including the required header
files.
2. After the CMake configuration the make fails:
$ make
Scanning dependencies of target opa2slurm
[ 20%] Building C object CMakeFiles/opa2slurm.dir/ptr_list.c.o
[ 40%] Building C object CMakeFiles/opa2slurm.dir/opa_host.c.o
[ 60%] Building C object CMakeFiles/opa2slurm.dir/opa_switch.c.o
[ 80%] Building C object CMakeFiles/opa2slurm.dir/opa2slurm.c.o
[100%] Linking C executable opa2slurm
CMakeFiles/opa2slurm.dir/opa2slurm.c.o: In function `opa_link_speed':
opa2slurm.c:(.text+0xe7): undefined reference to
`omgt_sa_get_portinfo_records'
opa2slurm.c:(.text+0x1ae): undefined reference to `omgt_sa_free_records'
CMakeFiles/opa2slurm.dir/opa2slurm.c.o: In function `main':
opa2slurm.c:(.text+0x734): undefined reference to `omgt_open_port'
opa2slurm.c:(.text+0x791): undefined reference to `omgt_open_port_by_num'
opa2slurm.c:(.text+0x7e1): undefined reference to
`omgt_open_port_by_guid'
opa2slurm.c:(.text+0x88a): undefined reference to
`omgt_sa_get_node_records'
opa2slurm.c:(.text+0xa12): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0xa5d): undefined reference to
`omgt_sa_get_node_records'
opa2slurm.c:(.text+0xbd8): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0xc19): undefined reference to
`omgt_sa_get_link_records'
opa2slurm.c:(.text+0xee5): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0x15e9): undefined reference to `omgt_status_totext'
opa2slurm.c:(.text+0x1623): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0x163e): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0x1659): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0x169a): undefined reference to `omgt_close_port'
opa2slurm.c:(.text+0x16d6): undefined reference to `omgt_status_totext'
opa2slurm.c:(.text+0x1717): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0x1732): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0x174d): undefined reference to `omgt_sa_free_records'
opa2slurm.c:(.text+0x178e): undefined reference to `omgt_close_port'
collect2: error: ld returned 1 exit status
make[2]: *** [opa2slurm] Error 1
make[1]: *** [CMakeFiles/opa2slurm.dir/all] Error 2
make: *** [all] Error 2
Could you kindly add detailed building instructions including software
prerequisites to your Gitlab page?
Thanks a lot,
Ole
--
Ole Holm Nielsen
PhD, Senior HPC Officer
Department of Physics, Technical University of Denmark,
Building 307, DK-2800 Kongens Lyngby, Denmark
E-mail: ole.h.niel...@fysik.dtu.dk
Homepage: http://dcwww.fysik.dtu.dk/~ohnielse/
Tel: (+45) 4525 3187 / Mobile (+45) 5180 1620
