>>> On Sun, Nov 14, 2021 at 10:40:28PM +0100, na...@poczta.fm wrote:
>>> Hi misc@,
>>>
>>> While testing the U-blox MPCI-L210 LTE modem on OpenBSD -current
>>> (the modem is attached as a cdce(4) device, the router mode is enabled;
>>> other modes are also available), I noticed that the automatic configuration
>>> via DHCP works correctly, but the MAC address assigned to the corresponding
>>> cdce0 network interface is different from the MAC address reported by
>>> lsusb -v. Unfortunately, due to this mismatch, incoming Ethernet frames
>>> having the actual MAC address of the device set as the destination address
>>> in the header are dropped by the kernel in the ether_input() function
>>> (/src/sys/net/if_ethersubr.c).
>>>
>>> Based on an analysis of the source code (/sys/dev/usb/if_cdce.c) and the
>>> results of the subsequent experiments, I was able to confirm that the cdce
>>> driver is unable to get the MAC address of the device (in particular,
>>> the usbd_get_string_desc() function returns USBD_STALLED instead of
>>> USBD_NORMAL_COMPLETION), and as a result, a semi-random fake Ethernet
>>> address is set:
>>>
>>> -----------------------------------------------------------------------------
>>> 306 if (!ethd || usbd_get_string_desc(sc->cdce_udev, ethd->iMacAddress, 0,
>>> 307 &eaddr_str, &len)) {
>>> 308 ether_fakeaddr(ifp); // <---- Set a semi-random MAC
>>> address
>>> 309 } else {
>>> -----------------------------------------------------------------------------
>>
>> Can you find out what happens on Linux?
>>
>> Their cdc_ether driver seems to be doing exactly the same thing. So with
>> this driver you would also end up with a random MAC if the device fails
>> to respond to a mac address query. In this case I have no idea what else
>> could be done apart from using the workaround you have already found.
>>
>> But Linux has more CDC sub-drivers than we do, and runs some devices in
>> specialized modes.
>> Perhaps Linux performs a mode switch and uses a different driver for e.g.
>> MBIM mode? If that is the case then we could change our drivers to try to
>> switch this device into a mode where it works better.
>>
>>
>
> Hi Stefan,
>
> Thank you for your prompt reply.
>
> Good point -- I remember that I tried to use this modem on Linux some time
> ago and it worked with no noticeable issues. I will take a closer look again
> on Friday this week, and I will summarize my findings in my next email.
>
> Best wishes,
> Andrzej
>
>
Hi Stefan, hi misc@,
Since my last email, I followed your suggestion and investigated how Linux
(Debian 11) handles the same modem operating in the CDCE mode, on the same
hardware platform. Please find below the corresponding summary.
On system startup (or whenever the modem device is restarted using the
following AT command: echo "AT+CFUN=16" > /dev/ttyACM0), the modem is
detected and attached properly by the cdc_ether/cdc_acm drivers (associated
with the data and control interfaces, respectively), as shown below:
-----------------------------------------------------------------------------
usb 1-1.3: new high-speed USB device number 11 using ehci-pci
usb 1-1.3: New USB device found, idVendor=1546, idProduct=1143, bcdDevice= 1.00
usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-1.3: Product: MODEM-LTE
usb 1-1.3: Manufacturer: u-blox
usb 1-1.3: SerialNumber: 000000000100
cdc_ether 1-1.3:1.0 wwan0: register 'cdc_ether' at usb-0000:00:13.0-1.3, \
Mobile Broadband Network Device, <mac-address>
cdc_acm 1-1.3:1.2: ttyACM0: USB ACM device
cdc_acm 1-1.3:1.4: ttyACM1: USB ACM device
cdc_acm 1-1.3:1.6: ttyACM2: USB ACM device
cdc_ether 1-1.3:1.0 wwx<mac-address>: renamed from wwan0
-----------------------------------------------------------------------------
At this point, the corresponding network interface can be configured
automatically using dhclient, and then, the Internet connection works as
expected.
I confirm that during my experiments on Linux, the network interface backed
by the cdc_ether driver was always assigned the same MAC address that could
be found in the dmesg/lsusb output related to the modem device (at the same
time, you may recall from my initial email that OpenBSD would assign
a semi-random MAC address instead, due to the stalled USB transfer following
an attempt to fetch the MAC string from the device).
To take a closer look at what exactly was happening during the device
attachment on Linux, I first analyzed the source code of the cdc_ether
driver and found the usbnet_cdc_bind() function in which the
usbnet_get_ethernet_addr() function is called:
-----------------------------------------------------------------------------
https://github.com/torvalds/linux/blob/master/drivers/net/usb/cdc_ether.c
442 int usbnet_cdc_bind(struct usbnet *dev, struct usb_interface *intf)
443 {
(...)
453
454 status = usbnet_get_ethernet_addr(dev, info->ether->iMACAddress);
455 if (status < 0) {
456 usb_set_intfdata(info->data, NULL);
457 usb_driver_release_interface(driver_of(intf), info->data);
458 return status;
459 }
460
461 return 0;
462 }
-----------------------------------------------------------------------------
As expected, the usbnet_get_ethernet_addr() function attempts to get the
MAC string from the device, and if it succeeds, it configures the
corresponding network interface accordingly:
-----------------------------------------------------------------------------
https://github.com/torvalds/linux/blob/master/drivers/net/usb/usbnet.c
172 ret = usb_string(dev->udev, iMACAddress, buf, sizeof buf);
173 if (ret == 12)
174 tmp = hex2bin(addr, buf, 6);
(...)
182 eth_hw_addr_set(dev->net, addr);
183 return 0;
-----------------------------------------------------------------------------
In the next step, using dynamic tracing, I was able to confirm that this code
path was indeed being followed whenever the modem device was attached by the
kernel (and by the cdc_ether driver, in particular):
-----------------------------------------------------------------------------
# trace-bpfcc -K 'usbnet_get_ethernet_addr(struct usbnet *dev, \
int iMACAddress) "iMACAddress: %d", iMACAddress'
PID TID COMM FUNC -
582 582 kworker/2:0 usbnet_get_ethernet_addr MAC: 6
b'usbnet_get_ethernet_addr+0x1 [usbnet]'
b'usbnet_cdc_bind+0x35 [cdc_ether]'
b'usbnet_probe+0x33e [usbnet]'
b'usb_probe_interface+0xe2 [usbcore]'
b'really_probe+0x37b [kernel]'
b'driver_probe_device+0xe1 [kernel]'
b'bus_for_each_drv+0x7e [kernel]'
b'__device_attach+0xd8 [kernel]'
b'bus_probe_device+0x8e [kernel]'
b'device_add+0x399 [kernel]'
b'usb_set_configuration+0x46f [usbcore]'
b'usb_generic_driver_probe+0x4c [usbcore]'
b'usb_probe_device+0x39 [usbcore]'
b'really_probe+0x37b [kernel]'
b'driver_probe_device+0xe1 [kernel]'
b'bus_for_each_drv+0x7e [kernel]'
b'__device_attach+0xd8 [kernel]'
b'bus_probe_device+0x8e [kernel]'
b'device_add+0x399 [kernel]'
b'usb_new_device.cold+0x111 [usbcore]'
b'hub_event+0x146a [usbcore]'
b'process_one_work+0x1b6 [kernel]'
b'worker_thread+0x53 [kernel]'
b'kthread+0x11b [kernel]'
b'ret_from_fork+0x22 [kernel]'
-----------------------------------------------------------------------------
We also see in the subsequent example that the usbnet_get_ethernet_addr()
function returned 0 in this context:
-----------------------------------------------------------------------------
# trace-bpfcc 'r::usbnet_get_ethernet_addr "ret: %d", retval'
PID TID COMM FUNC -
136 136 kworker/2:2 usbnet_get_ethernet_addr ret: 0
-----------------------------------------------------------------------------
Based on the above observation as well as on the fact that the MAC addresses
reported by dmesg and lsusb were exactly the same (no workaround was needed),
it appears that the corresponding MAC string had successfully been fetched
from the device upon its attachment.
After all, I think this might be a positive sign for us, and I would be happy
to provide additional in-depth information and perform further tests, if that
would help us solve the underlying issue on OpenBSD.
I would appreciate your comments and suggestions.
Best wishes,
Andrzej
