Public bug reported:
GPU direct RDMA does not work with infiband, following is AI generated
explanation:
On Ubuntu 26.04 with kernel 7.0.0-34-generic, I installed
doca-ofed-3.4-dkms 3.4.0+0.8.5.0-0ubuntu1~26.04.1 from
resolute-updates/universe and nvidia-dkms-580-server
580.178.04-0ubuntu0.26.04.1 from the Ubuntu repository.
The installed DOCA-OFED ib_uverbs module exports
ib_register_peer_memory_client and
ib_unregister_peer_memory_client. However, rebuilding the NVIDIA
DKMS modules without a manually supplied OFED Module.symvers builds
nvidia-peermem without those imports. The resulting peermem module
does not provide the peer-memory integration needed here.
The DOCA DKMS build log shows that mlnx-ofed-kernel/Module.symvers is
generated during the build. After installation, I could not find
that file at the paths checked by the NVIDIA 580 nvidia-peermem build,
including /usr/src/ofa_kernel/<kernel-version>/Module.symvers.
The active top-level DOCA dkms.conf invokes build_all.sh but has no
POST_BUILD hook to preserve the file. The bundled
mlnx-ofed-kernel/dkms.conf does contain a POST_BUILD script for
copying build headers and symbols, but it is not invoked by the
top-level DKMS configuration used by this package.
To reproduce:
1. Install doca-ofed-3.4-dkms and nvidia-dkms-580-server on the
kernel above.
2. Ensure there is no manually created /usr/src/ofa_kernel directory.
3. Run:
sudo dkms build -m nvidia-srv -v 580.178.04 \
-k 7.0.0-34-generic --force
4. Inspect the built nvidia-peermem module:
modprobe --dump-modversions \
/var/lib/dkms/nvidia-srv/580.178.04/7.0.0-34-generic/x86_64/module/nvidia-peermem.ko.zst
|
grep -E 'ib_register_peer_memory_client|ib_unregister_peer_memory_client'
Observed: the grep produces no output; peermem was built without the
DOCA peer-memory interface.
Expected: the matching OFED build symbols should be available to
dependent module builds, or the packages should provide a documented,
supported integration procedure for nvidia-peermem.
As a diagnostic workaround, I supplied a Module.symvers file whose
exported-symbol CRCs were checked against the selected DOCA kernel
modules. Rebuilding NVIDIA then produced a peermem module importing
both peer-memory symbols. I have since reverted that workaround.
Please advise whether the missing build artifact is a packaging
issue in doca-ofed-3.4-dkms or whether another supported package or
DKMS configuration is required. This report does not attribute a
separate PCIe system hang to this packaging issue.
ProblemType: Bug
DistroRelease: Ubuntu 26.04
Package: doca-ofed-3.4-dkms 3.4.0+0.8.5.0-0ubuntu1~26.04.1
ProcVersionSignature: Ubuntu 7.0.0-34.34-generic 7.0.14
Uname: Linux 7.0.0-34-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.34.1-0ubuntu0.1
Architecture: amd64
CasperMD5CheckResult: unknown
Date: Mon Sep 28 10:17:01 2026
PackageArchitecture: all
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=xterm-256color
XDG_RUNTIME_DIR=<set>
SourcePackage: doca-ofed-3.4-dkms
UpgradeStatus: No upgrade log present (probably fresh install)
** Affects: doca-ofed-3.4-dkms (Ubuntu)
Importance: Undecided
Status: New
** Tags: amd64 apport-bug resolute third-party-packages
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2168737
Title:
doca-ofed-3.4-dkms does not expose Module.symvers for nvidia-peermem
builds
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/doca-ofed-3.4-dkms/+bug/2168737/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
