Sorry for this. This was an try and ERROR ERROR Problem. It was a mismatch of OFED versions and kernel updates.
Now I installed a fresh centOS 6.4 (with default kernel NO KENELUPDATE). Then installed the official MELLANOX OFED Driver and compiled openMPI (without options). And now it works fine. Before it was a mismatch of first installing centOS 6.3, then updating to 6.4 and maybe some files of the old OFED drivers survived. Thank you Christian On 07/30/2013 04:40 PM, Reuti wrote: > Am 30.07.2013 um 15:01 schrieb christian schmitt: > >> I´m trying to get openmpi(1.6.5) running with/over infiniband. >> My system is a centOS 6.3. I have installed the Mellanox OFED driver >> (2.0) and everything seems working. ibhosts shows all hosts and the switch. >> A "hca_self_test.ofed" shows: >> >> ---- Performing Adapter Device Self Test ---- >> Number of CAs Detected ................. 1 >> PCI Device Check ....................... PASS >> Kernel Arch ............................ x86_64 >> Host Driver Version .................... MLNX_OFED_LINUX-2.0-2.0.5 >> (OFED-2.0-2.0.5): 2.6.32-279.el6.x86_64 >> Host Driver RPM Check .................. PASS >> Firmware on CA #0 VPI .................. v2.11.500 >> Firmware Check on CA #0 (VPI) .......... PASS >> Host Driver Initialization ............. PASS" >> Number of CA Ports Active .............. 1 >> Port State of Port #1 on CA #0 (VPI)..... UP 4X QDR (InfiniBand) >> Error Counter Check on CA #0 (VPI)...... PASS >> Kernel Syslog Check .................... PASS >> Node GUID on CA #0 (VPI) ............... 00:02:c9:03:00:1f:a4:e0 >> >> >> A "ompi_info | grep openib" shows: >> MCA btl: openib (MCA v2.0, API v2.0, Component v1.6.5) >> >> So I now compiled openmpi with the option "--with-openib" and tried to >> run the intel MPI test. > > What's the "intel MPI test" - is this an application from Intel's MPI library > which is included as source and you recompiled it with Open MPI? > > -- Reuti > > >> But it still uses the Ethernet interface to >> communicate. Only when I configure ipoib (ib0) and start my job with >> "--mca btl ^openib --mca btl_tcp_if_include ib0" it runs with >> infiniband. But when I´m right, it should work without the ib0 interface. >> I´m quiet new to infiniband so maybe I forgot something. >> I'm grateful for any information that help me solving this problem. >> >> Thank you, >> >> Christian >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
