On 04/12/2020 10:55:16-0800, Jakub Kicinski wrote:
> On Fri, 4 Dec 2020 18:12:51 +0000 Vladimir Oltean wrote:
> > On Fri, Dec 04, 2020 at 10:00:21AM -0800, Jakub Kicinski wrote:
> > > On Fri, 4 Dec 2020 19:51:25 +0200 Vladimir Oltean wrote:
> > > > Currently ocelot_set_rx_mode calls ocelot_mact_learn directly, which has
> > > > a very nice ocelot_mact_wait_for_completion at the end. Introduced in
> > > > commit 639c1b2625af ("net: mscc: ocelot: Register poll timeout should be
> > > > wall time not attempts"), this function uses readx_poll_timeout which
> > > > triggers a lot of lockdep warnings and is also dangerous to use from
> > > > atomic context, leading to lockups and panics.
> > > >
> > > > Steen Hegelund added a poll timeout of 100 ms for checking the MAC
> > > > table, a duration which is clearly absurd to poll in atomic context.
> > > > So we need to defer the MAC table access to process context, which we do
> > > > via a dynamically allocated workqueue which contains all there is to
> > > > know about the MAC table operation it has to do.
> > > >
> > > > Fixes: 639c1b2625af ("net: mscc: ocelot: Register poll timeout should
> > > > be wall time not attempts")
> > > > Signed-off-by: Vladimir Oltean <vladimir.olt...@nxp.com>
> > > > Reviewed-by: Florian Fainelli <f.faine...@gmail.com>
> > > > ---
> > > > Changes in v2:
> > > > - Added Fixes tag (it won't backport that far, but anyway)
> > > > - Using get_device and put_device to avoid racing with unbind
> > >
> > > Does get_device really protect you from unbind? I thought it only
> > > protects you from .release being called, IOW freeing struct device
> > > memory..
> >
> > Possibly.
> > I ran a bind && unbind loop for a while, and I couldn't trigger any
> > concurrency.
>
> You'd need to switch to a delayed work or add some other sleep for
> testing, maybe?
>
> > > More usual way of handling this would be allocating your own workqueue
> > > and flushing that wq at the right point.
> >
> > Yeah, well I more or less deliberately lose track of the workqueue as
> > soon as ocelot_enqueue_mact_action is over, and that is by design. There
> > is potentially more than one address to offload to the hardware in progress
> > at the same time, and any sort of serialization in .ndo_set_rx_mode (so
> > I could add the workqueue to a list of items to cancel on unbind)
> > would mean
> > (a) more complicated code
> > (b) more busy waiting
>
> Are you sure you're not confusing workqueue with a work entry?
>
> You can still put multiple work entries on the queue.
>
> > > > drivers/net/ethernet/mscc/ocelot_net.c | 83 +++++++++++++++++++++++++-
> > > > 1 file changed, 80 insertions(+), 3 deletions(-)
> > >
> > > This is a little large for a rc7 fix :S
> >
> > Fine, net-next it is then.
>
> If this really is the fix we want, it's the fix we want, and it should
> go into net. We'll just need to test it very well is all.
>
> > > What's the expected poll time? maybe it's not that bad to busy wait?
> > > Clearly nobody noticed the issue in 2 years (you mention lockdep so
> > > not a "real" deadlock) which makes me think the wait can't be that long?
> >
> > Not too much, but the sleep is there.
> > Also, all 3 of ocelot/felix/seville are memory-mapped devices. But I
> > heard from Alex a while ago that he intends to add support for a switch
> > managed over a slow bus like SPI, and to use the same regmap infrastructure.
> > That would mean that this problem would need to be resolved anyway.
>
> So it's MMIO but the other end is some firmware running on the device?
>
No, the SoC can also expose its registers as a pcie endpoint or a SPI
device. In that case, obviously, the on board MIPS CPU is not enabled.
IIRC, you can even access all the registers using MDIO.
> > > Also for a reference - there are drivers out there with busy poll
> > > timeout of seconds :/
> >
> > Yeah, not sure if that tells me anything. I prefer avoiding that from
> > atomic context, because our cyclictest numbers are not that great anyway,
> > the last thing I want is to make them worse.
>
> Fair.
On that topic, we could certainly play with regmap fast_io, I dont
remember if this is enabled by default or even remove regmap locking
completely when using MMIO.
--
Alexandre Belloni, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
