On 3/14/20 9:18 AM, Nicholas Piggin wrote:
Ganesh Goudar's on March 14, 2020 12:04 am:
MCE handling on pSeries platform fails as recent rework to use common
code for pSeries and PowerNV in machine check error handling tries to
access per-cpu variables in realmode. The per-cpu variables may be
outside the RMO region on pSeries platform and needs translation to be
enabled for access. Just moving these per-cpu variable into RMO region
did'nt help because we queue some work to workqueues in real mode, which
again tries to touch per-cpu variables.
Which queues are these? We should not be using Linux workqueues, but the
powerpc mce code which uses irq_work.
Yes, irq work queues accesses memory outside RMO.
irq_work_queue()->__irq_work_queue_local()->[this_cpu_ptr(&lazy_list) |
this_cpu_ptr(&raised_list)]
Also fwnmi_release_errinfo()
cannot be called when translation is not enabled.
Why not?
It crashes when we try to get RTAS token for "ibm, nmi-interlock" device
tree node. But yes we can avoid it by storing it rtas_token somewhere but
haven't
tried it, here is the backtrace I got when fwnmi_release_errinfo() called from
realmode handler.
[ 70.856908] BUG: Unable to handle kernel data access on read at
0xc0000001ffffa8f8
[ 70.856918] Faulting instruction address: 0xc000000000853920
[ 70.856927] Oops: Kernel access of bad area, sig: 11 [#1]
[ 70.856935] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries
[ 70.856943] Modules linked in: mcetest_slb(OE+) bridge stp llc
ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sg
pseries_rng ip_tables xfs libcrc32c sd_mod t10_pi ibmvscsi ibmveth
scsi_transport_srp
[ 70.856975] CPU: 13 PID: 6480 Comm: insmod Kdump: loaded Tainted: G
OE 5.6.0-rc2-ganesh+ #6
[ 70.856985] NIP: c000000000853920 LR: c000000000853a14 CTR: c0000000000376b0
[ 70.856994] REGS: c000000007e4b870 TRAP: 0300 Tainted: G OE
(5.6.0-rc2-ganesh+)
[ 70.857003] MSR: 8000000000001003 <SF,ME,RI,LE> CR: 88000422 XER: 00000009
[ 70.857015] CFAR: c000000000853a10 DAR: c0000001ffffa8f8 DSISR: 40000000
IRQMASK: 1
[ 70.857015] GPR00: c000000000853a14 c000000007e4bb00 c000000001372b00
c0000001ffffa8c8
[ 70.857015] GPR04: c000000000cf8728 0000000000000000 0000000000000002
c008000000420810
[ 70.857015] GPR08: 0000000000000000 0000000000000000 0000000000000001
0000000000000001
[ 70.857015] GPR12: 0000000000000000 c000000007f92000 c0000001f8113d70
c00800000059070d
[ 70.857015] GPR16: 00000000000004f8 c008000000421080 000000000000fff1
c008000000421038
[ 70.857015] GPR20: c00000000125eb20 c000000000d1d1c8 c008000000590000
0000000000000000
[ 70.857015] GPR24: 4000000000000510 c008000008000000 c0000000012355d8
c008000000420940
[ 70.857015] GPR28: c008000008000011 0000000000000000 c000000000cf8728
c00000000169a098
[ 70.857097] NIP [c000000000853920] __of_find_property+0x30/0xd0
[ 70.857106] LR [c000000000853a14] of_find_property+0x54/0x90
[ 70.857113] Call Trace:
[ 70.857117] Instruction dump:
[ 70.857124] 3c4c00b2 3842f210 2c230000 418200bc 7c0802a6 fba1ffe8 fbc1fff0
7cbd2b78
[ 70.857136] fbe1fff8 7c9e2378 f8010010 f821ffc1 <ebe30030> 2fbf0000 409e0014
48000064
[ 70.857152] ---[ end trace 13755f7502f3150b ]---
[ 70.864199]
[ 70.864226] Sending IPI to other CPUs
[ 82.011761] ERROR: 15 cpu(s) not responding
This patch fixes this by enabling translation in the exception handler
when all required real mode handling is done. This change only affects
the pSeries platform.
Not supposed to do this, because we might not be in a state
where the MMU is ready to be turned on at this point.
I'd like to understand better which accesses are a problem, and whether
we can fix them all to be in the RMO.
I faced three such access problems,
* accessing per-cpu data (like mce_event,mce_event_queue and mce_event_queue),
we can move this inside RMO.
* calling fwnmi_release_errinfo().
* And queuing work to irq_work_queue, not sure how to fix this.
Thanks,
Nick
