> From: Paul E. McKenney <[email protected]> > Sent: Thursday, November 26, 2020 3:55 PM > To: Dexuan Cui <[email protected]> > Cc: [email protected]; Ingo Molnar <[email protected]>; > [email protected]; vkuznets <[email protected]>; Michael Kelley > <[email protected]>; [email protected] > Subject: Re: kdump always hangs in rcu_barrier() -> wait_for_completion() > > On Thu, Nov 26, 2020 at 10:59:19PM +0000, Dexuan Cui wrote: > > > From: Paul E. McKenney <[email protected]> > > > Sent: Thursday, November 26, 2020 1:42 PM > > > > > > > > Another possibility is that rcu_state.gp_kthread is non-NULL, but that > > > > > something else is preventing RCU grace periods from completing, but in > > > > > > > > It looks like somehow the scheduling is not working here: in > > > > rcu_barrier() > > > > , if I replace the wait_for_completion() with > > > > wait_for_completion_timeout(&rcu_state.barrier_completion, 30*HZ), > the > > > > issue persists. > > > > > > Have you tried using sysreq-t to see what the various tasks are doing? > > > > Will try it. > > > > BTW, this is a "Generation 2" VM on Hyper-V, meaning sysrq only starts to > > work after the Hyper-V para-virtualized keyboard driver loads... So, at this > > early point, sysrq is not working. :-( I'll have to hack the code and use a > > virtual NMI interrupt to force the sysrq handler to be called. > > Whatever works! > > > > Having interrupts disabled on all CPUs would have the effect of disabling > > > the RCU CPU stall warnings. > > > Thanx, Paul > > > > I'm sure the interrupts are not disabled. Here the VM only has 1 virtual > > CPU, > > and when the hang issue happens the virtual serial console is still > > responding > > when I press Enter (it prints a new line) or Ctrl+C (it prints ^C). > > > > Here the VM does not use the "legacy timers" (PIT, Local APIC timer, etc.) > > at > all. > > Instead, the VM uses the Hyper-V para-virtualized timers. It looks the > Hyper-V > > timer never fires in the kdump kernel when the hang issue happens. I'm > > looking into this... I suspect this hang issue may only be specific to > > Hyper-V. > > Fair enough, given that timers not working can also suppress RCU CPU > stall warnings. ;-) > > Thanx, Paul
FYI: the issue has been fixed by this fix: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fff7b5e6ee63c5d20406a131b260c619cdd24fd1 Thanks, -- Dexuan
