The primary problem I've had with ib2slurm is that it segfaults. There's a bug in the ibnetdiscover library -- ib2slurm passes a NULL config pointer to the ibnd_discover_fabric() which is supposed to be okay according to the documentation, but that function actually requires a config structure to be passed to it.
I've forked and updated ib2slurm:
https://github.com/jtfrey/ib2slurm
There doesn't appear to be much movement on the original project, so I haven't
put in a pull request against my fork.
> On May 8, 2017, at 10:55 AM, Damien François <[email protected]>
> wrote:
>
> Hi
>
> many thanks for the tools, it works flawlessly here.
>
> I just patched to send the output that does not belong to the topology.conf
> to stderr so I could simply redirect to topology.conf in an automated Slurm
> install procedure
>
> 16c16
> < echo Verify the Infiniband interface: >&2
> ---
>> echo Verify the Infiniband interface:
> 18c18
> < if $IBSTAT -l >&2
> ---
>> if $IBSTAT -l
> 20c20
> < echo Infiniband interface OK >&2
> ---
>> echo Infiniband interface OK
> 22c22
> < echo Infiniband interface NOT OK >&2
> ---
>> echo Infiniband interface NOT OK
> 26c26
> < cat <<EOF >&2
> ---
>> cat <<EOF
>
> Sincerely
>
> damien
>
>> On 07 May 2017, at 14:29, Ole Holm Nielsen <[email protected]>
>> wrote:
>>
>>
>> I'm announcing an initial version 0.1 of an Infiniband topology tool
>> "slurmibtopology.sh" for Slurm.
>>
>> I had to create a Slurm topology.conf file and needed an automated way to
>> get the correct node and switch Infiniband connectivity. The manual page
>> https://slurm.schedmd.com/topology.conf.html refers to an outdated tool
>> ib2slurm.
>>
>> Inspired by the script in
>> https://unix.stackexchange.com/questions/255472/text-processing-building-a-slurm-topology-conf-file-from-ibnetdiscover-output
>> I decided to write a simpler and more understandable tool. It parses the
>> output of the OFED command "ibnetdiscover" and generates an initial
>> topology.conf file (which you may want to edit for readability).
>>
>> Download the script https://ftp.fysik.dtu.dk/Slurm/slurmibtopology.sh
>>
>> Example: Running this script on our Infiniband network:
>>
>> # slurmibtopology.sh
>> Verify the Infiniband interface:
>> mlx4_0
>> Infiniband interface OK
>>
>> Generate the Slurm topology.conf file for Infiniband switches:
>>
>> # IB switch no. 1: MF0;mell02:IS5030/U1
>> SwitchName=ibsw1 Nodes=i[001-028]
>> # IB switch no. 2: MF0;mell03:IS5030/U1
>> SwitchName=ibsw2 Nodes=finbul,i[029-051],niflfs[1-2],niflopt1
>> # IB switch no. 3: MF0;mell01:SX6036/U1
>> SwitchName=ibsw3 Nodes=g[081-100,102-112],h[001-002]
>> # IB switch no. 4: MF0;mell04:SX6036/U1
>> # NOTICE: This switch has no attached nodes (empty hostlist)
>> SwitchName=ibsw4 Nodes=""
>> # IB switch no. 5: Mellanox 4036 # volt01
>> SwitchName=ibsw5 Nodes=g[005-008,013-014,016,021-024,029-032,037-040]
>> # IB switch no. 6: Mellanox 4036 # volt03
>> SwitchName=ibsw6 Nodes=g[045-048,053-056,061-064,069-072,077-080]
>> # IB switch no. 7: Mellanox 4036 # volt04
>> SwitchName=ibsw7 Nodes=g[041-044,049-052,057-060,065-068,073-076]
>> # IB switch no. 8: Mellanox 4036 # volt02
>> SwitchName=ibsw8 Nodes=g[001-004,009-012,017-020,025-028,033-036]
>> # Merging all switches in a top-level spine switch
>> SwitchName=spineswitch Switches=ibsw[1-8]
>>
>> It would be great if other sites could test this tool on their Infiniband
>> network and report bugs or suggest improvements.
>>
>> --
>> Ole Holm Nielsen
>> Department of Physics, Technical University of Denmark
>
::::::::::::::::::::::::::::::::::::::::::::::::::::::
Jeffrey T. Frey, Ph.D.
Systems Programmer V / HPC Management
Network & Systems Services / College of Engineering
University of Delaware, Newark DE 19716
Office: (302) 831-6034 Mobile: (302) 419-4976
::::::::::::::::::::::::::::::::::::::::::::::::::::::
signature.asc
Description: Message signed with OpenPGP using GPGMail
