It turns out that the machine size listed in the test plan (Standard_D2ps_v6) does not include a MANA interface. With the assistance of Konstantin Taranov at Microsoft, I was able to obtain temporary access to a machine of size Standard_ND128isr_NDR_GB200_v6 running Noble. On that machine, I installed the following package versions:
linux-azure-nvidia=6.8.0-1013 rdma-core=50.0-2ubuntu0.1 rdmacm-utils=50.0-2ubuntu0.1 librdmacm1t64=50.0-2ubuntu0.1 ibverbs-providers=50.0-2ubuntu0.1 ibverbs-utils=50.0-2ubuntu0.1 libibverbs1=50.0-2ubuntu0.1 I then ran the rping test described in the test plan. The client and server were able to communicate: ``` # Server $ rping -s -C 10 -v -d created cm_id 0xafb08acbd010 rdma_bind_addr successful rdma_listen cma_event type RDMA_CM_EVENT_CONNECT_REQUEST cma_id 0xeddea4000ce0 (child) child cma 0xeddea4000ce0 created pd 0xafb08acb9010 created channel 0xafb08acb8fd0 created cq 0xafb08acb8e50 created qp 0xafb08acbd690 rping_setup_buffers called on cb 0xafb08acb07c0 allocated & registered buffers... accepting client connection request cq_thread started. cma_event type RDMA_CM_EVENT_ESTABLISHED cma_id 0xeddea4000ce0 (child) ESTABLISHED recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion recv completion Received rkey d500 addr c3435f47e120 len 64 from peer server received sink adv server posted rdma read req rdma read completion server received read complete server ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA server posted go ahead send completion recv completion Received rkey d400 addr c3435f461590 len 64 from peer server received sink adv rdma write from lkey dd00 laddr afb08acf1870 len 64 rdma write completion server rdma write complete server posted go ahead send completion cma_event type RDMA_CM_EVENT_DISCONNECTED cma_id 0xeddea4000ce0 (child) server DISCONNECT EVENT... wait for RDMA_READ_ADV state 10 rping_free_buffers called on cb 0xafb08acb07c0 destroy cm_id 0xafb08acbd010 # Client $ rping -c -a 10.0.0.4 -C 10 -v -d created cm_id 0xc3435f46d760 cma_event type RDMA_CM_EVENT_ADDR_RESOLVED cma_id 0xc3435f46d760 (parent) cma_event type RDMA_CM_EVENT_ROUTE_RESOLVED cma_id 0xc3435f46d760 (parent) rdma_resolve_addr - rdma_resolve_route successful created pd 0xc3435f469720 created channel 0xc3435f4696e0 created cq 0xc3435f469560 created qp 0xc3435f46dde0 rping_setup_buffers called on cb 0xc3435f4607c0 allocated & registered buffers... cq_thread started. cma_event type RDMA_CM_EVENT_ESTABLISHED cma_id 0xc3435f46d760 (parent) ESTABLISHED rdma_connect successful RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz RDMA addr c3435f47e120 rkey d500 len 64 send completion recv completion RDMA addr c3435f461590 rkey d400 len 64 send completion recv completion ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA rping_free_buffers called on cb 0xc3435f4607c0 cma_event type RDMA_CM_EVENT_DISCONNECTED cma_id 0xc3435f46d760 (parent) client DISCONNECT EVENT... destroy cm_id 0xc3435f46d760 ``` -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2100089 Title: rdma-core in latest Ubuntu LTS does not support Microsoft Azure Network Adapter To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/2100089/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
