Am 02.05.21 um 01:37 schrieb Greg Rivers: > On Saturday, 1 May 2021 16:45:03 CDT Stefan Esser wrote: >> Am 01.05.21 um 21:48 schrieb Greg Rivers via freebsd-current: >>> On Saturday, 1 May 2021 14:09:46 CDT Nilton Jose Rizzo wrote: >>>> I using a FreeBSD 14-Current and get random error with my NIC. The >>>> watchdog timer send a timeout message and I loose connection temporaly. In logs show only this message: >>>> >>> Switch to the official Realtek driver in ports: net/realtek-re-kmod >> >> The "official" RealTek driver is based on a very old version of "our" >> driver that was written by Bill Paul. >> >> It lacks many features that have been introduced in FreeBSD in the >> last decade (or even earlier) like NETMAP-Support. >> >> The RealTek-driver has special cases for some 50 variants of RealTek >> Ethernet chips and contains individual firmware patches for nearly all >> of them. >> >> I had started to merge chip specific changes from the official driver >> to the FreeBSD driver in the hope to get it to support the RTL8125A/B >> chips. But I have stopped that project for lack of RTL8125 documentation, >> especially regarding the PHY, which has its own driver module in our >> version but not the RealTek code. (And somebody claimed to know that >> another FreeBSD developer was working on RTL8125 support but did not >> tell who that might be and whether he had documentation.) >> >> Anyway, there are changes regarding the initialization and error recovery >> of different RealTek chips in the official driver that could be merged >> into our version. But I do not know whether these changes require the >> firmware changes provided by the RealTek driver to correctly work. >> > Thanks for the information Stefan, and for your work on FreeBSD. My use > of the term "official" was apparently inaccurate. I was not aware of the > deficiencies in the RealTek driver. I would prefer to use the FreeBSD > driver, but I don't for purely pragmatic reasons: the FreeBSD driver > continually locks up and resets under load (as described by the OP), > while the RealTek driver does not.
Hi Greg, the RealTek driver is "official" in the sense that it is provided by the vendor and written with knowledge about all the (many!) deficiencies of the RealTek Ethernet chips. And yes, the FreeBSD drived definitely needs work to fully support all variants of the RealTek chip. I guess that due to uncovered chip specifics or hardware issues, the FreeBSD driver will often lack the special code (or firmware patches) and will have to recover chip operations be going through a hard reset. If you look at the "official" driver sources, you'll find #ifdefs for FreeBSD versions before 4.9, but that is not the reason the main driver source file is more than 30000 lines long. The driver distinguishes between more than 70 different chip versions (identified by MACFG_3 to MACFG_84 with some IDs missing). And each one has specific requirements regarding firmware patches, initialization and reset behavior, error handling, ... I have analyzed these differences (see the attached file) but for lack of RTL8125 documentation not preceded with this project at this time. (The column lx_fw identifies firmware patches used by the Linux driver, while rt_fw identifies those embedded into the RealTek driver for FreeBSD - and those differ somewhat, and I have no idea why ...). I have local modifications of the re driver in my sources, but have one other project that I really want to get ready in the next few months (after working on it for nearly 2 years) and I do not want to become responsible for issues of the RealTek driver in base (after committing fixes that also might cause regressions, if they need to be accompanied by firmware patches ...) > FWIW, here are the particulars on the RealTek chip-set that I've got: > > re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port > 0xe000-0xe0ff mem 0xb0804000-0xb0804fff,0xb0800000-0xb0803fff irq 16 at > device 0.0 on pci1 > re0: Using 1 MSI-X message > re0: Chip rev. 0x2c800000 > re0: MAC rev. 0x00100000 The Chip rev. indicates that you have got a RTL8168E_VL, which does not need a firmware patch according to RealTek driver, but gets one in Linux. It is identified by MACFG_38 in the RealTek driver, BTW, and there are only a few chip specific code fragments relevant to that chip. It seems, it needs special handling when the MAC address is programmed, but I did not spot any other special code for that particular chip in the "official" driver. > re0@pci0:1:0:0: class=0x020000 rev=0x06 hdr=0x00 vendor=0x10ec device=0x8168 > subvendor=0x1458 subdevice=0xe000 > vendor = 'Realtek Semiconductor Co., Ltd.' > device = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller' > class = network > subclass = ethernet > > Would the FreeBSD Foundation be able to help with getting documentation from RealTek? I have no idea - there is a RTL8125 driver in OpenBSD (which is separate from the RTL8111/8168 drivers, there) and that driver is actually relatively short, but it consists just of sequences of register reads and writes without any comments that could help understand the purpose of these operations. And there are not so many differences between the RTL8125 and prior chips (except that the RTL8125 expects 32 bit wide register writes, while the RTL8111 was limited to 16 bit reads/writes - but this is trivially solved by having 2 sets of low-level routines without any impact on the higher level code). There is also driver code in Linux that can be used to reverse engineer device specific setup and operation procedures. But that driver is ugly (IMHO at least) and I'm afraid that there might be legal issues if I use initialization sequences, firmware patches or other information that can be found in that driver. Therefore I have only taken a cursory look and identified the internal chip IDs in the Linux driver and the firmware patches used for each supported chip. If there was detailed documentation (detailed register set specification and operation details like initialization sequence, delays/timeouts that are required, etc.) this would greatly help getting the RTL8125 into our version of the "re" driver (with NETMAP and other features that we have that are not present in the driver from RealTek). I might try to get support from the author of the OpenBSD driver, but he might have received documentation under an NDA that does not allow to share it with us. Anyway, since the RealTek driver packages works quite well, I can use my RTL8125 chip (on an AMD B550 mainboard) and it was not that urgent (for me) to get that chip supported in base. But if I get documentation, I might be able to integrate the RTL8125 (and at the same time apply all the special handling and firmware patching the RealTek driver does - it is still BSD licensed, but in the 4-clause form) during summer. Best regards, STefan
RealTek Version:
#define RL_FLAG_MSI 0x00000001
#define RL_FLAG_PHYWAKE_PM 0x00000004
#define RL_FLAG_DESCV2 0x00000040
#define RL_FLAG_MSIX 0x00000800
#define RL_FLAG_MAGIC_PACKET_V2 0x20000000
#define RL_FLAG_PCIE 0x40000000
#define RL_FLAG_MAGIC_PACKET_V3 0x80000000
FreeBSD Version:
#define RL_FLAG_MSI 0x00000001 // ?
#define RL_FLAG_AUTOPAD 0x00000002
#define RL_FLAG_PHYWAKE_PM 0x00000004 // ?
#define RL_FLAG_PHYWAKE 0x00000008
#define RL_FLAG_JUMBOV2 0x00000010
#define RL_FLAG_PAR 0x00000020 // Ethernet-Address
Access Mode
#define RL_FLAG_DESCV2 0x00000040
#define RL_FLAG_MACSTAT 0x00000080
#define RL_FLAG_FASTETHER 0x00000100
#define RL_FLAG_CMDSTOP 0x00000200
#define RL_FLAG_MACRESET 0x00000400
#define RL_FLAG_MSIX 0x00000800 // ?
#define RL_FLAG_WOLRXENB 0x00001000
#define RL_FLAG_MACSLEEP 0x00002000
#define RL_FLAG_WAIT_TXPOLL 0x00004000
#define RL_FLAG_CMDSTOP_WAIT_TXQ 0x00008000
#define RL_FLAG_WOL_MANLINK 0x00010000
#define RL_FLAG_EARLYOFF 0x00020000
#define RL_FLAG_8168G_PLUS 0x00040000
#define RL_FLAG_MAGIC_PACKET_V2 0x20000000 // "new" register
layout (RTL8139C+ and newer)
#define RL_FLAG_PCIE 0x40000000 // ?
#define RL_FLAG_LINK 0x80000000 // ?
#define RL_FLAG_MAGIC_PACKET_V3 0x80000000 // 8125 only ??? XXX
duplicate def
chip rev macfg linux jumbo cmd cfgmode power magic
lx_fw rt_fw rl_flags
----------------------------------------------------------------------------------------------------------------------------------------
?8169
MACRESET
8169S 0x008 3 2 7 0xff00 cfg1 - -
MACRESET
8110S 0x040 3 3 7 0xff00 cfg1 - -
MACRESET
8169_8110SB 0x100 4 4 7 0xff00 cfg1 - -
MACRESET|PHYWAKE
8169_8110SBL?
MACRESET|PHYWAKE
8169_8110SC 0x180 5 5 7 0xff00 cfg1 - -
MACRESET|PYHWAKE
8169_8110SCE 0x980 6 6 7 0xff00 cfg1 - -
MACRESET|PHYWAKE
8100E 0x308 ? 14 0 0xe700 cfg1 - -
FASTETHER|AUTOPAD
8101E 0x340 11 13 0 0xe700 cfg1 - -
FASTETHER|AUTOPAD
8101E 0x342 12 16 0 0xe700 cfg1 - -
FASTETHER|AUTOPAD
8101E 0x343 13 10 0 0xe700 cfg1 - -
FASTETHER|AUTOPAD
?8102E 0x348 ? 7 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102E 0x349 14 8 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102E 0x34A 15 9 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102E 0x34B 16 9 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8103E 0x34C 17 9 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|MACSLEEP
8103E 0x34D 18 9 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|MACSLEEP
8103E 0x34E 19 9 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|MACSLEEP
?8102EL 0x248 ? 7 0 0xe700 cfg2 - -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL 0x249 14 8 0 0xe700 cfg2 pm -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL 0x24A 15 9 0 0xe700 cfg2 pm -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL 0x24B 16 9 0 0xe700 cfg2 pm -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL_SPIN1 0x24C 17 9 0 0xe700 cfg2 pm -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL_SPIN1 0x24D 18 9 0 0xe700 cfg2 pm -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL_SPIN1 0x24E 19 9 0 0xe700 cfg2 pm -
PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8168B_SPIN1 0x300 21 11 4 0xe700 cfg2 - -
PYHWAKE|MACSTAT|WOLRXENB
8168B_SPIN2 0x380 22 12 4 0xe700 cfg2 - -
PYHWAKE|MACSTAT|WOLRXENB
8168B_SPIN3 0x384 23 17 4 0xe700 cfg2 - -
PYHWAKE|MACSTAT
8168C 0x3C0 24 19 6 0xc700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP?
8168C 0x3C2 25 20 6 0xc700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP?
?8168C 0x3C3 ? 21 6 0xc700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP?
8168C_SPIN2 0x3C4 26 22 6 0xc700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP
?8168CP 0x3C8 ? 18 6 0xc700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168CP 0x3C9 27 23 6 0xc700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168CP 0x3CB 28 24 6 0xc700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168D 0x281 31 25 9 0x8700 cfg2 pm -
8168d_1
PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168D 0x282 32 26 9 0x8700 cfg2 pm -
8168d_2
PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168D 0x283 33 26 9 0x8700 cfg2 pm -
8168d_2
PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168DP 0x288 63 27 9 0x8700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168DP 0x289 64 ? 9 0x8700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168DP 0x28A 65 28 9 0x8700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168DP 0x28B 66 31 9 0x8700 cfg2 pm -
PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168E 0x2C1 36 32 9 0x8700 cfg2 pm -
8168e_1
PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168E 0x2C2 37 33 9 0x8700 cfg2 pm -
8168e_2
PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168E_VL 0x2C8 38 34 9 0x8700 cfg2 pm mp2
8168e_3
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168E_VL 0x2C9 39 34 9 0x8700 cfg2 pm mp2
8168e_3
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8401E 0x240 41 13 0 0xe700 cfg2 pm -
PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E 0x409 42 29 0 0xe700 cfg2 pm -
8105E_1 PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E 0x40A 43 30 0 0xe700 cfg2 pm -
8105e_1 PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E 0x40B 43 30 0 0xe700 cfg2 pm -
8105e_1 PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E_SPIN1 0x40C 43 30 0 0xe700 cfg2 pm -
8105e_1 PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8168F 0x480 50 35 9 0xbf00 cfg2 pm mp2
8168f_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168F 0x481 51 36 9 0xbf00 cfg2 pm mp2
8168f_2
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8411 0x488 52 38 9 0xbf00 cfg2 pm mp2
8411_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK
8402 0x440 53 37 0 0xe700 cfg2 pm mp2
8402_1
PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ
8106E 0x448 54 39 0 0xe700 cfg2 pm -
8106e_1 PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8106E 0x449 55 39 0 0xe700 cfg2 pm -
8106e_1 PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8106EUS ? ? 43
8106e_2
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|FASTETHER
8107E ? ? 47
8107e_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|FASTETHER
8107E ? ? 48
8107e_2
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|FASTETHER
8168G 0x4C0 56 40 9 0xcf00 cfg2 pm mp2
8168g_2 8168g_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168G 0x4C1 57 41 9 0xcf00 cfg2 pm mp2
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8411B 0x5C8 60 44 9 0xcf00 cfg2 pm mp2
8411_2 8411b_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168EP 0x500 61 49 9 0xcf00 cfg2 pm mp2
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168EP 0x501 62 50 9 0xcf00 cfg2 pm mp2
8168ep_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168EP 0x502 67 51 9 0xcf00 cfg2 pm mp2
8168ep_2
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168GU 0x508 58 ? 9 0xcf00 cfg2 pm mp2
8168gu_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168GU 0x509 59 42 9 0xcf00 cfg2 pm mp2
8168g_3 8168gu_2
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168H 0x540 68 45 9 0xcf00 cfg2 pm mp2
8168h_1 6168h_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168H 0x541 69 46 9 0xcf00 cfg2 pm mp2
8168h_2 6168h_1
PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168FP 0x549 70 ? 9 0xcf00 cfg2 pm mp2
6168fp_1/2
?PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168FP(8117?) 0x54A 71 52 9 0xcf00 cfg2 pm mp2
8168fp_3 8168fp_3
?PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168FP 0x54B 72 9 0xcf00 cfg2 pm mp2
8168fp_4
?PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8125A 0x608 80 60 9 0xcf00 cfg3 pm mp3
8125a_1 ?DESCV3
8125A 0x609 81 61 9 0xcf00 cfg3 pm mp3
8125a_3 8125a_2 ?DESCV3
8125B 0x640 82 9 0xcf00 cfg3 pm mp3
8125b_1 ?DESCV3
8125B 0x641 83 9 0xcf00 cfg3 pm mp3
8125b_2 ?DESCV3
OpenPGP_signature
Description: OpenPGP digital signature
