On Wed, Jan 11, 2023 at 11:53 AM Hrvoje Popovski <hrv...@srce.hr> wrote:
> Hi all,
>
> claudio@ suggested to try same setup with FreeBSD. So I've tried FreeBSD
> and Linux and results are same, 1G BiDi from FS.COM coded for Cisco
> won't work with Intel 82599 10G card.
> If I boot FreeBSD and Linux with that sfp in 82599 or X710 card that
> interface will be disabled at boot and won't show up in OS.
> On OpenBSD i could see it but mac address is 00:00:00:00:00:00
>
> I've tried all combinations with unsupported_sfp statements for FreeBSD
> and Linux but I couldn't see that interface in OS.
> If I put that sfp in card after boot status is no carrier whatever I do,
> at least on FreeBSD. I'm still playing with Linux..
>
>
> it seems that I've missed out all football here :)
Definitely seems like the "Intel NIC unsupported SFP" behavior,
nothing specific to BiDi. (But thanks to everyone in this thread for
educating me about BiDi! I doubt I'll be using it any time soon, but
it's neat how it works, and I definitely see the appeal if you have a
limited number of fibres.)
Note that in Linux, it actually disables the card until you unload and
reload the driver, for example:
Dec 8 17:01:19 lambda kernel: [ 1428.680975] ixgbe 0000:24:00.0
enp36s0f0: NIC Link is Down
Dec 8 17:01:19 lambda systemd-networkd[83876]: enp36s0f0: Lost carrier
Dec 8 17:01:24 lambda kernel: [ 1433.830686] ixgbe 0000:24:00.0:
failed to initialize because an unsupported SFP+ module type was
detected.
Dec 8 17:01:24 lambda kernel: [ 1433.832843] ixgbe 0000:24:00.0:
Reload the driver after installing a supported module.
Dec 8 17:01:24 lambda kernel: [ 1433.852297] ixgbe 0000:24:00.0:
removed PHC on enp36s0f0
Dec 8 17:01:24 lambda systemd-networkd[83876]: enp36s0f0: Link DOWN
That was with an Intel X520-DA2 (82599). I had to 'rmmod ixgbe' and
then 'modprobe ixgbe' (optionally with 'allow_unsupported_sfp=1') to
get the card active again, even with an Intel SFP+ module. Apparently
there is a bit you can toggle in the EEPROM to make that setting
permanent in all OSes:
https://forums.servethehome.com/index.php?threads/patching-intel-x520-eeprom-to-unlock-all-sfp-transceivers.24634/page-3#post-324427
Note that this only applies to the 500 series cards (ix), the 700
series (ixl) have a different EEPROM that doesn't easily allow
unsupported modules, although in my experience they do allow more
modules, e.g. Finisar, and AFAIK don't require reloading the driver
after swapping in a "supported" SFP.
I'm not exactly sure how the vendor-lock works, but I think it looks
at the "Vendor OUI" bits in the SFP's EEPROM. My FS.com generic
10GBASE-SR modules have 00:00:00 there, while an "Intel" one from FS
has 00:1b:21, but still the FS vendor name and PN. I don't know if
OpenBSD's 'ifconfig <intf> transceiver' reports all of that, but Linux
'ethtool -m <intf>' does.
-Andrew
