https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=298485
Bug ID: 298485
Summary: [regression] fc0a5ae09430 (linuxkpi device_add) hides
class sysctl nodes, breaking all InfiniBand/RoCE
userspace: ibv_devices/ibv_devinfo find no devices
Product: Base System
Version: CURRENT
Hardware: amd64
OS: Any
Status: New
Severity: Affects Some People
Priority: ---
Component: kern
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected], [email protected],
[email protected], [email protected],
[email protected]
OVERVIEW
Commit fc0a5ae09430 ("linuxkpi: Add device under parent, not under class")
changes LinuxKPI device_add() to register a device's kobject under
dev->parent instead of dev->class when a parent is set.
On FreeBSD the sysfs emulation is sysctl-backed and has no symlinks, so this
does not *add* a hierarchy view, it removes the class view. Any consumer that
looks devices up by class stops finding them.
This silently breaks the entire InfiniBand/RoCE userspace. ibv_devices and
ibv_devinfo report no devices on a fully functional adapter.
I have confirmed the cause by reverting only this commit. Details under
VALIDATION below.
AFFECTED VERSIONS
Bad: main-n289089-e46a7d842a75 (built 2026-09-12) -- fails
Good: main-n287270-3e3752ce73b7 (built 2026-07-07) -- works
SYMPTOMS
# ibv_devices
device node GUID
------ ----------------
<- empty list, exit status 0
# ibv_devinfo
No IB devices found
THE KERNEL SIDE IS HEALTHY
All four adapters attach, mlx5ib registers four IB devices, and the
character devices are present:
/dev/uverbs0 /dev/uverbs1 /dev/uverbs2 /dev/uverbs3
/dev/umad0 /dev/umad1 /dev/umad2 /dev/umad3
/dev/rdma_cm
ROOT CAUSE
FreeBSD's libibverbs has no sysfs. ibv_read_sysfs_file() in
contrib/ofed/libibverbs/sysfs.c translates a sysfs path into a sysctl name
by replacing '/' with '.':
for (s = &path[0]; *s != '\0'; s++)
if (*s == '/')
*s = '.';
ret = sysctlbyname(&path[1], buf, &len, NULL, 0);
Device discovery in find_sysfs_devs() (contrib/ofed/libibverbs/init.c)
iterates uverbs0..255 and reads
<sysfs>/class/infiniband_verbs/uverbsN/ibdev, i.e. sysctl
sys.class.infiniband_verbs.uverbsN.ibdev. Enumeration therefore depends
entirely on the CLASS view of the tree.
VALIDATION
Reversing only this commit's own hunk with 'git apply -R', rebuilding and
reloading restores full functionality.
REPRODUCTION
On an affected main system with any RDMA-capable adapter:
kldload mlx5ib # pulls in ibcore
ibv_devices # empty list, exit status 0
sysctl -N sys.class.infiniband_verbs sys.class.infiniband
sysctl -n sys.device.mlx5_core0.uverbs0.ibdev # -> mlx5_0, misplaced node
sysctl -n sys.class.infiniband_mad.umad0.ibdev # -> mlx5_0, still correct
WORKAROUND
Boot a kernel predating 2026-09-09, or revert fc0a5ae09430 and rebuild both
linuxkpi.ko and ibcore.ko. There is no userspace workaround: libibverbs
honours the SYSFS_PATH environment variable, but the "class/infiniband_verbs"
path component is hardcoded, so the class subtree cannot be redirected.
SUGGESTED DIRECTION
The DRM motivation for the change is legitimate, so a plain revert is
probably not wanted. Options, roughly in order of preference:
1. Have device_add() register into BOTH hierarchies when dev->class !=
NULL -- the parent for the device tree plus an entry under the class --
emulating what the Linux symlink provides and matching existing
device_register() behaviour.
2. Keep parent-based placement but make it opt-in for the DRM case, so
existing class-based consumers are untouched.
3. Fix ibcore to register under its class explicitly. This works but leaves
the device_add()/device_register() inconsistency in place, and any
future LinuxKPI driver that sets dev->parent hits the same trap.
It would also be worth auditing other LinuxKPI consumers that set
dev->parent and whose userspace looks devices up by class.
ENVIRONMENT
# uname -a
FreeBSD 16.0-CURRENT #139 main-n289089-e46a7d842a75:
Sat Sep 12 16:10:10 IDT 2026 amd64
uname -K: 1600025 uname -U: 1600013
Hardware: Dell PowerEdge R760, 4x Mellanox ConnectX-7, firmware 28.51.0256,
ports in Ethernet (RoCE) mode.
# pciconf -lv | grep -A2 '^mlx5_core'
mlx5_core0@pci0:97:0:0: class=0x020000 vendor=0x15b3 device=0x1021
vendor = 'Mellanox Technologies'
device = 'MT2910 Family [ConnectX-7]'
mlx5_core1@pci0:97:0:1: (same)
mlx5_core2@pci0:160:0:0: (same)
mlx5_core3@pci0:160:0:1: (same)
Kernel configuration (sys/amd64/conf/LATEST):
include GENERIC
ident LATEST
nooptions COMPAT_LINUXKPI
nodevice mlx5
nodevice mlxfw
nodevice mlx5en
options IPSEC_OFFLOAD
include ../../conf/std.nodebug
Relevant dmesg (normal, shown to demonstrate the kernel side is fine):
mlx5_core0: <mlx5_core> mem 0x25fff8000000-0x25fff9ffffff at device 0.0
numa-domain 0 on pci11
mlx5: Mellanox Core driver 3.7.1 (November 2021)
mlx5_core0: INFO: mlx5_port_module_event: Module 0, status: plugged and
enabled
mlx5_core: INFO: (mlx5_core0): E-Switch: Total vports 17
--
You are receiving this mail because:
You are the assignee for the bug.
