On Thu, Aug 14, 2025 at 2:56 AM Niklas Schnelle <schne...@linux.ibm.com> wrote:
>
> On Wed, 2025-08-13 at 07:11 +0200, Lukas Wunner wrote:
> > When Advanced Error Reporting was introduced in September 2006 by commit
> > 6c2b374d7485 ("PCI-Express AER implemetation: AER core and aerdriver"), it
> > sought to adhere to the recovery flow and callbacks specified in
> > Documentation/PCI/pci-error-recovery.rst.
> >
> --- snip ---
> > Signed-off-by: Lukas Wunner <lu...@wunner.de>
> > ---
> > drivers/pci/pcie/err.c | 17 ++++++++++-------
> > 1 file changed, 10 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
> > index de6381c690f5..e795e5ae6b03 100644
> > --- a/drivers/pci/pcie/err.c
> > +++ b/drivers/pci/pcie/err.c
> > @@ -217,15 +217,10 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
> > pci_walk_bridge(bridge, pci_pm_runtime_get_sync, NULL);
> >
> > pci_dbg(bridge, "broadcast error_detected message\n");
> > - if (state == pci_channel_io_frozen) {
> > + if (state == pci_channel_io_frozen)
> > pci_walk_bridge(bridge, report_frozen_detected, &status);
> > - if (reset_subordinates(bridge) != PCI_ERS_RESULT_RECOVERED) {
> > - pci_warn(bridge, "subordinate device reset failed\n");
> > - goto failed;
> > - }
> > - } else {
> > + else
> > pci_walk_bridge(bridge, report_normal_detected, &status);
> > - }
> >
> > if (status == PCI_ERS_RESULT_CAN_RECOVER) {
> > status = PCI_ERS_RESULT_RECOVERED;
>
> On s390 PCI errors leave the device with MMIO blocked until either the
> error state is cleared or we reset via the firmware interface. With
> this change and the pci_channel_io_frozen case AER would now do the
> report_mmio_enabled() before the reset with nothing happening between
> report_frozen_detected() and report_mmio_enabled() is MMIO enabled at
> this point? I think this callback really only makes sense if you have
> an equivalent to s390's clearing of the error state that enables MMIO
> but doesn't otherwise reset. Similarly EEH has eeh_pci_enable(pe,
> EEH_OPT_THAW_MMIO).
The original intent was that if the channel locked up e.g. due to some
uncorrectable ECC error or some transient errors due to electrical
problems on the bus (bad reflection of some pulse off some poorly
terminated connector) then such an error would almost surely be
transient and very very unlikely to repeat.
Thus it would be OK to re-enable the MMIO (without otherwise resetting
any channel controller state) and let the device driver examine the PCI
config registers. If they all look good, don't contain any scrambled addrs
or bitflags, then completely normal operations could be resumed without
any further messing around, resetting, invalidating etc.
But first, the device driver needs to examine the config registers and
that cannot be done unless MMIO is enabled. If MMIO is enabled,
and the PCI config regs appear to contain garbage, then that garbage
can be logged in some error report or crash dump. After this
got done, the device driver would invalidate any pending i/o (for example,
half-finished blocks in some s390 orb, irb, schib, ioccw, whatever)
make sure that assorted channel subsystems are actually halted,
and then attempt a reset of the bus, the bus controllers (390 channel
or subchannel) and probably the device as well. If that reset succeeds,
then the device driver can restart with a fresh, clean device and a working
channel. And maybe, if we're lucky, start handling any pending i/o requests.
In practice, this worked great for network adapters. However, if the affected
device was some storage controller for e.g. some mounted filesystem then
(way back when) it was hopeless, because the Linux block subsystem did
not know how to deal with transient errors like this. Trying to figure out how
to unscramble the block subsystem, and keep mounted filesystems shielded
from this chaos was the one thing I couldn't figure out how to solve. It seemed
important. But we've come a long way since then, so I dunno.
-- Linas
