Ethernet device with shared mdio
The following device tree specifies a shared mdio.
The ffec driver uses miibus.
When there is a shared mdio, one of the device instances will not be able to
properly configure the PHY, as it needs to use the other devices resource to
read/write the PHY.
&fec1 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_fec1>;
phy-mode = "rgmii-id";
phy-handle = <ðphy0>;
fsl,magic-packet;
status = "okay";
mdio {
#address-cells = <1>;
#size-cells = <0>;
ethphy0: ethernet-phy@0 {
compatible =
"ethernet-phy-ieee802.3-c22";
reg = <0>;
};
ethphy1: ethernet-phy@1 {
compatible =
"ethernet-phy-ieee802.3-c22";
reg = <1>;
};
};
};
&fec2 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_fec2>;
phy-mode = "rgmii-txid";
phy-handle = <ðphy1>;
phy-supply = <®_fec2_supply>;
nvmem-cells = <&fec_mac1>;
nvmem-cell-names = "mac-address";
rx-internal-delay-ps = <2000>;
fsl,magic-packet;
status = "okay";
};
Does FreeBSD have any plans for supporting hardware that specifies a shared
mdio in the dtb?
Just knowing the general approach being considered would be helpful.
--
This transmission (including any attachments) may contain confidential
information, privileged material (including material protected by the
solicitor-client or other applicable privileges), or constitute non-public
information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error,
please immediately reply to the sender and delete this information from your
system. Use, dissemination, distribution, or reproduction of this transmission
by unintended recipients is not authorized and may be unlawful.
Re: Ethernet device with shared mdio
Thank you for the response and for sharing your scenario.
We’ve also hacked up the cgem and the ffec driver to support a shared mdio.
That was not too difficult, but we have a new scenario where the mdio is now
being shared between two different devices that use different drivers (ffec and
eqos).
This presents a few extra challenges.
I was hoping that FreeBSD may have considered supporting a shared mdio. We can
come up with something, but if there is an existing architecture/approach in
the works…we would like to use a consistent approach. At first glance,
miiproxy did not seem like a fit.
I do not have the hardware. I am trying to help somebody else with this. I
have seen the dtb.
It’s a Variscite DAR-MX8M-PLUS.
Regards,
Mike.
From: owner-freebsd-...@freebsd.org on behalf
of Milan Obuch
Date: Friday, September 13, 2024 at 3:08 AM
To: freebsd-net@freebsd.org
Subject: Re: Ethernet device with shared mdio
CAUTION - This email is from an external source. Please be cautious with links
and attachments. (go/taginfo)
On Fri, 6 Sep 2024 18:03:39 +
Mike Belanger wrote:
> The following device tree specifies a shared mdio.
> The ffec driver uses miibus.
> When there is a shared mdio, one of the device instances will not be
> able to properly configure the PHY, as it needs to use the other
> devices resource to read/write the PHY.
>
> &fec1 {pinctrl-names = "default";
>pinctrl-0 = <&pinctrl_fec1>;
>phy-mode = "rgmii-id";
>phy-handle = <ðphy0>;
>fsl,magic-packet;
>status = "okay";
>
>mdio {
> #address-cells = <1>;
> #size-cells = <0>;
>
> ethphy0: ethernet-phy@0 {
> compatible =
> "ethernet-phy-ieee802.3-c22"; reg = <0>;
> };
>
> ethphy1: ethernet-phy@1 {
> compatible =
> "ethernet-phy-ieee802.3-c22"; reg = <1>;
> };
>};
> };
>
> &fec2 {
>pinctrl-names = "default";
>pinctrl-0 = <&pinctrl_fec2>;
>phy-mode = "rgmii-txid";
>phy-handle = <ðphy1>;
>phy-supply = <®_fec2_supply>;
>nvmem-cells = <&fec_mac1>;
>nvmem-cell-names = "mac-address";
>rx-internal-delay-ps = <2000>;
>fsl,magic-packet;
>status = "okay";
> };
>
> Does FreeBSD have any plans for supporting hardware that specifies a
> shared mdio in the dtb? Just knowing the general approach being
> considered would be helpful.
>
I can't speak for FreeBSD project, I just can share my experience with
similar case. It is described in my post to hackers mailing list (see
https://urldefense.com/v3/__https://lists.freebsd.org/archives/freebsd-hackers/2021-December/000649.html__;!!JoeW-IhCUkS0Jg!fv0DHFN5Xb4FbKwre1H4UDCUvbmhAoO1y5HgQiDkN6wuv2t3B4pyS1akuKuCn6ZqO1AfbrCaFsVsJibdfui4KfJQGw$<https://urldefense.com/v3/__https:/lists.freebsd.org/archives/freebsd-hackers/2021-December/000649.html__;!!JoeW-IhCUkS0Jg!fv0DHFN5Xb4FbKwre1H4UDCUvbmhAoO1y5HgQiDkN6wuv2t3B4pyS1akuKuCn6ZqO1AfbrCaFsVsJibdfui4KfJQGw$>
for details), unfortunately, no response received. Another attempt to
get some attention a week later on net mailing list was done, see
https://urldefense.com/v3/__https://lists.freebsd.org/archives/freebsd-net/2021-December/001114.html__;!!JoeW-IhCUkS0Jg!fv0DHFN5Xb4FbKwre1H4UDCUvbmhAoO1y5HgQiDkN6wuv2t3B4pyS1akuKuCn6ZqO1AfbrCaFsVsJibdfujKN3_xCA$<https://urldefense.com/v3/__https:/lists.freebsd.org/archives/freebsd-net/2021-December/001114.html__;!!JoeW-IhCUkS0Jg!fv0DHFN5Xb4FbKwre1H4UDCUvbmhAoO1y5HgQiDkN6wuv2t3B4pyS1akuKuCn6ZqO1AfbrCaFsVsJibdfujKN3_xCA$>
for the post, with no response either.
As you see, my case was similar, just the mdio block was attached to
second controller. This makes it a bit more problematic - you can't use
mdio controller before being initialized, naturally.
I was not able to use miiproxy approach as noted in my post to hackers
mailing list, additionally, miiproxy was removed from the tree with
MIPS arch some time later. I resolved the issue by modifying cgem driver
and mii layer. This was just a proof of concept with some hacks, but I
was able to use both ports with proper link state change detection. I
did not continue the work because vendor changed hardware design and
there was no shared mdio anymore.
If you are interested I can dig for the sources, big part of my changes
would not be necessary, just the idea of decoupling MDIO and MII
interfaces still applies, I think. By the way, which board are you
working on? Is it accessible for general audience?
Regards,
Milan
--
Re: [EXTERNAL] - Re: Race condition in ether_ifattach
In our reported case a startup script is loading the driver and bringing the interface up with ifconfig. Since they are putting these commands to the background, so ifconfig is not properly waiting for the driver load to fully complete. When ifconfig is successful, it will send the IPv6 neighbour discovery packets…and this can result in a crash if ether_ifattach is not complete (ifp->if_output is NULL). We are considering breaking up if_attach_internal, so that ether_ifattach can call the first part and then call the end part after the ifp is fully setup. We can reproduce the issue by adding an artificial delay after the if_attach in ether_ifattach. Mike. From: owner-freebsd-...@freebsd.org on behalf of Zhenlei Huang Date: Saturday, May 3, 2025 at 9:34 PM To: Mike Belanger Cc: freebsd-net@freebsd.org , Gleb Smirnoff Subject: [EXTERNAL] - Re: Race condition in ether_ifattach CAUTION - This email is from an external source. Please be cautious with links and attachments. (go/taginfo) Hi Mike, On May 1, 2025, at 9:13 PM, Mike Belanger mailto:mibelan...@qnx.com>> wrote: There appears to be a race condition in ether_ifattach (if_ethersubr.c). The ether_ifattach() function calls if_attach, where the interface will get announced, and then ether_ifattach continues with the initialization of the ifp. I also noticed this while working on https://reviews.freebsd.org/D49359<https://urldefense.com/v3/__https:/reviews.freebsd.org/D49359__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ3FMtaxAA$>. There's an attempt for the attaching process https://reviews.freebsd.org/D49358<https://urldefense.com/v3/__https:/reviews.freebsd.org/D49358__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ30mbVejw$> . > then ether_ifattach continues with the initialization of the ifp. In most cases that should not matter, as at that moment the interface has not been flagged up ( IFF_UP ) yet. Is there any guarantee in FreeBSD that this race condition cannot be exposed. We have been running the FreeBSD stack for some time under QNX and have just recently run into an issue with this race condition. We are considering a modification where we have the option of deferring the interface announcement in if_attach. Can you elaborate how the race condition happens and how that affect you ? Before opening a FreeBSD bug, I wanted to check if this issue would not be valid in a FreeBSD system. It’s very clear that there is a potential race when looking at the code, but perhaps there is a mitigation that is not obvious. This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. Best regards, Zhenlei -- This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation.
Re: [EXTERNAL] - Re: Race condition in ether_ifattach
Let me know if there is anything else I can do for this issue (e.g. open a bug). From: Zhenlei Huang Date: Tuesday, May 6, 2025 at 10:30 PM To: Mike Belanger Cc: freebsd-net@freebsd.org , Gleb Smirnoff Subject: Re: [EXTERNAL] - Re: Race condition in ether_ifattach CAUTION - This email is from an external source. Please be cautious with links and attachments. (go/taginfo) On May 5, 2025, at 9:54 PM, Mike Belanger mailto:mibelan...@qnx.com>> wrote: In our reported case a startup script is loading the driver and bringing the interface up with ifconfig. Since they are putting these commands to the background, so ifconfig is not properly waiting for the driver load to fully complete. When ifconfig is successful, it will send the IPv6 neighbour discovery packets…and this can result in a crash if ether_ifattach is not complete (ifp->if_output is NULL). I think I see the problem. We are considering breaking up if_attach_internal, so that ether_ifattach can call the first part and then call the end part after the ifp is fully setup. We can reproduce the issue by adding an artificial delay after the if_attach in ether_ifattach. Mike. From: owner-freebsd-...@freebsd.org<mailto:owner-freebsd-...@freebsd.org> mailto:owner-freebsd-...@freebsd.org>> on behalf of Zhenlei Huang mailto:z...@freebsd.org>> Date: Saturday, May 3, 2025 at 9:34 PM To: Mike Belanger mailto:mibelan...@qnx.com>> Cc: freebsd-net@freebsd.org<mailto:freebsd-net@freebsd.org> mailto:freebsd-net@freebsd.org>>, Gleb Smirnoff mailto:gleb...@freebsd.org>> Subject: [EXTERNAL] - Re: Race condition in ether_ifattach CAUTION - This email is from an external source. Please be cautious with links and attachments. (go/taginfo) Hi Mike, On May 1, 2025, at 9:13 PM, Mike Belanger mailto:mibelan...@qnx.com>> wrote: There appears to be a race condition in ether_ifattach (if_ethersubr.c). The ether_ifattach() function calls if_attach, where the interface will get announced, and then ether_ifattach continues with the initialization of the ifp. I also noticed this while working on https://reviews.freebsd.org/D49359<https://urldefense.com/v3/__https:/reviews.freebsd.org/D49359__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ3FMtaxAA$>. There's an attempt for the attaching process https://reviews.freebsd.org/D49358<https://urldefense.com/v3/__https:/reviews.freebsd.org/D49358__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ30mbVejw$> . > then ether_ifattach continues with the initialization of the ifp. In most cases that should not matter, as at that moment the interface has not been flagged up ( IFF_UP ) yet. Is there any guarantee in FreeBSD that this race condition cannot be exposed. We have been running the FreeBSD stack for some time under QNX and have just recently run into an issue with this race condition. We are considering a modification where we have the option of deferring the interface announcement in if_attach. Can you elaborate how the race condition happens and how that affect you ? Before opening a FreeBSD bug, I wanted to check if this issue would not be valid in a FreeBSD system. It’s very clear that there is a potential race when looking at the code, but perhaps there is a mitigation that is not obvious. This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. Best regards, Zhenlei This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation. Best regards, Zhenlei -- This email and any attachments are intended solely for the use of the individual or entity to whom they are addres
