Re: new panic in cpu_reset() with WITNESS

2012-05-17 Thread Attilio Rao
2012/5/17, Andriy Gapon : > on 25/01/2012 23:52 Andriy Gapon said the following: >> on 24/01/2012 14:32 Gleb Smirnoff said the following: >>> Yes, now: >>> >>> Rebooting... >>> lock order reversal: >>> 1st 0x80937140 smp rendezvous (smp rendezvous) @ >>> /usr/src/head/sys/kern/kern_shutdow

Re: new panic in cpu_reset() with WITNESS

2012-05-17 Thread Andriy Gapon
on 25/01/2012 23:52 Andriy Gapon said the following: > on 24/01/2012 14:32 Gleb Smirnoff said the following: >> Yes, now: >> >> Rebooting... >> lock order reversal: >> 1st 0x80937140 smp rendezvous (smp rendezvous) @ >> /usr/src/head/sys/kern/kern_shutdown.c:542 >> 2nd 0xfe0001f5d838

Re: new panic in cpu_reset() with WITNESS

2012-01-25 Thread Andriy Gapon
on 24/01/2012 14:32 Gleb Smirnoff said the following: > Yes, now: > > Rebooting... > lock order reversal: > 1st 0x80937140 smp rendezvous (smp rendezvous) @ > /usr/src/head/sys/kern/kern_shutdown.c:542 > 2nd 0xfe0001f5d838 uart_hwmtx (uart_hwmtx) @ > /usr/src/head/sys/dev/uart/uart

Re: new panic in cpu_reset() with WITNESS

2012-01-24 Thread Gleb Smirnoff
On Tue, Jan 24, 2012 at 02:09:03PM +0200, Andriy Gapon wrote: A> on 24/01/2012 13:54 Gleb Smirnoff said the following: A> > On Mon, Jan 23, 2012 at 06:56:08PM +0200, Andriy Gapon wrote: A> > A> on 23/01/2012 18:46 Gleb Smirnoff said the following: A> > A> > On Mon, Jan 23, 2012 at 06:43:23PM +0200,

Re: new panic in cpu_reset() with WITNESS

2012-01-24 Thread Andriy Gapon
on 24/01/2012 13:54 Gleb Smirnoff said the following: > On Mon, Jan 23, 2012 at 06:56:08PM +0200, Andriy Gapon wrote: > A> on 23/01/2012 18:46 Gleb Smirnoff said the following: > A> > On Mon, Jan 23, 2012 at 06:43:23PM +0200, Andriy Gapon wrote: > A> > A> > db> bt > A> > A> > Tracing pid 1 tid 1000

Re: new panic in cpu_reset() with WITNESS

2012-01-24 Thread Gleb Smirnoff
On Mon, Jan 23, 2012 at 06:56:08PM +0200, Andriy Gapon wrote: A> on 23/01/2012 18:46 Gleb Smirnoff said the following: A> > On Mon, Jan 23, 2012 at 06:43:23PM +0200, Andriy Gapon wrote: A> > A> > db> bt A> > A> > Tracing pid 1 tid 11 td 0xfe0001d5e000 A> > A> > kdb_enter() at kdb_enter+0x3b

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Andriy Gapon
on 23/01/2012 18:46 Gleb Smirnoff said the following: > On Mon, Jan 23, 2012 at 06:43:23PM +0200, Andriy Gapon wrote: > A> > db> bt > A> > Tracing pid 1 tid 11 td 0xfe0001d5e000 > A> > kdb_enter() at kdb_enter+0x3b > A> > panic() at panic+0x1c7 > A> > _mtx_lock_spin_flags() at _mtx_lock_spi

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Gleb Smirnoff
On Mon, Jan 23, 2012 at 06:43:23PM +0200, Andriy Gapon wrote: A> > db> bt A> > Tracing pid 1 tid 11 td 0xfe0001d5e000 A> > kdb_enter() at kdb_enter+0x3b A> > panic() at panic+0x1c7 A> > _mtx_lock_spin_flags() at _mtx_lock_spin_flags+0x10f A> > cnputs() at cnputs+0x7a A> > vprintf() at vprin

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Andriy Gapon
on 23/01/2012 18:26 Gleb Smirnoff said the following: > Sorry. I was testing wrong kernel. The substitute doesn't help. Panic > trace is almost the same: > > db> bt > Tracing pid 1 tid 11 td 0xfe0001d5e000 > kdb_enter() at kdb_enter+0x3b > panic() at panic+0x1c7 > _mtx_lock_spin_flags() at

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Gleb Smirnoff
On Mon, Jan 23, 2012 at 08:24:10PM +0400, Gleb Smirnoff wrote: T> On Mon, Jan 23, 2012 at 04:01:20PM +0200, Andriy Gapon wrote: T> A> > A> Can you try to change printfs in witness to db_printfs? Perhaps this will allow T> A> > A> to get the details of the LOR in uart_cnputc. Maybe that will rev

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Gleb Smirnoff
On Mon, Jan 23, 2012 at 04:01:20PM +0200, Andriy Gapon wrote: A> > A> Can you try to change printfs in witness to db_printfs? Perhaps this will allow A> > A> to get the details of the LOR in uart_cnputc. Maybe that will reveal some A> > A> important additional details. A> > A> > Should I do s/

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Andriy Gapon
on 23/01/2012 15:07 Gleb Smirnoff said the following: > On Mon, Jan 23, 2012 at 10:21:57AM +0200, Andriy Gapon wrote: > A> > I can confirm that I can reboot succesfully with a new kernel > A> > and kern.stop_scheduler_on_panic=0. > A> > A> This is very puzzling. You observation means that stop_sc

Re: locks under printf(9) and WITNESS [Was: new panic in cpu_reset() with WITNESS]

2012-01-23 Thread Andriy Gapon
on 23/01/2012 15:04 Gleb Smirnoff said the following: > On Sat, Jan 21, 2012 at 07:26:55PM +0200, Andriy Gapon wrote: > A> > BTW, we have a quite strange situation with spin locks in console output > path. > A> > cnputs_mtx is marked as MTX_NOWITNESS, supposedly because cnputs > (printf) can be >

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Gleb Smirnoff
On Mon, Jan 23, 2012 at 10:21:57AM +0200, Andriy Gapon wrote: A> > I can confirm that I can reboot succesfully with a new kernel A> > and kern.stop_scheduler_on_panic=0. A> A> This is very puzzling. You observation means that stop_scheduler_on_panic has A> an effect on the code outside the panic

Re: locks under printf(9) and WITNESS [Was: new panic in cpu_reset() with WITNESS]

2012-01-23 Thread Gleb Smirnoff
On Sat, Jan 21, 2012 at 07:26:55PM +0200, Andriy Gapon wrote: A> > BTW, we have a quite strange situation with spin locks in console output path. A> > cnputs_mtx is marked as MTX_NOWITNESS, supposedly because cnputs (printf) can be A> > called in any locking context (even during normal operation)

Re: new panic in cpu_reset() with WITNESS

2012-01-23 Thread Andriy Gapon
on 22/01/2012 18:35 Gleb Smirnoff said the following: > On Sat, Jan 21, 2012 at 03:38:59PM +0200, Andriy Gapon wrote: > A> on 20/01/2012 17:41 Gleb Smirnoff said the following: > A> > > A> > On Tue, Jan 17, 2012 at 03:02:42PM +0400, Gleb Smirnoff wrote: > A> > T> New panic has been introduced so

Re: new panic in cpu_reset() with WITNESS

2012-01-22 Thread Gleb Smirnoff
On Sat, Jan 21, 2012 at 03:38:59PM +0200, Andriy Gapon wrote: A> on 20/01/2012 17:41 Gleb Smirnoff said the following: A> > A> > On Tue, Jan 17, 2012 at 03:02:42PM +0400, Gleb Smirnoff wrote: A> > T> New panic has been introduced somewhere between A> > T> r229851 and r229932, that happens on shu

Re: locks under printf(9) and WITNESS [Was: new panic in cpu_reset() with WITNESS]

2012-01-21 Thread Andriy Gapon
on 21/01/2012 16:37 Andriy Gapon said the following: > > BTW, we have a quite strange situation with spin locks in console output path. > cnputs_mtx is marked as MTX_NOWITNESS, supposedly because cnputs (printf) can > be > called in any locking context (even during normal operation). But there a

Re: new panic in cpu_reset() with WITNESS

2012-01-21 Thread Andriy Gapon
on 21/01/2012 15:38 Andriy Gapon said the following: > on 20/01/2012 17:41 Gleb Smirnoff said the following: >> >> On Tue, Jan 17, 2012 at 03:02:42PM +0400, Gleb Smirnoff wrote: >> T> New panic has been introduced somewhere between >> T> r229851 and r229932, that happens on shutdown if >> T> kern

locks under printf(9) and WITNESS [Was: new panic in cpu_reset() with WITNESS]

2012-01-21 Thread Andriy Gapon
BTW, we have a quite strange situation with spin locks in console output path. cnputs_mtx is marked as MTX_NOWITNESS, supposedly because cnputs (printf) can be called in any locking context (even during normal operation). But there are a number of console-specific locks (scrlock, uart_hwmtx, "sys

Re: new panic in cpu_reset() with WITNESS

2012-01-21 Thread Andriy Gapon
on 20/01/2012 17:41 Gleb Smirnoff said the following: > > On Tue, Jan 17, 2012 at 03:02:42PM +0400, Gleb Smirnoff wrote: > T> New panic has been introduced somewhere between > T> r229851 and r229932, that happens on shutdown if > T> kernel has WITNESS and doesn't have WITNESS_SKIPSPIN. > > I've

Re: new panic in cpu_reset() with WITNESS

2012-01-20 Thread Gleb Smirnoff
On Tue, Jan 17, 2012 at 03:02:42PM +0400, Gleb Smirnoff wrote: T> New panic has been introduced somewhere between T> r229851 and r229932, that happens on shutdown if T> kernel has WITNESS and doesn't have WITNESS_SKIPSPIN. I've run through binary search and panic was introduced by r229854. --

Re: new panic in cpu_reset() with WITNESS

2012-01-17 Thread Andriy Gapon
on 17/01/2012 17:34 m...@freebsd.org said the following: > 2012/1/17 Gleb Smirnoff : >> New panic has been introduced somewhere between >> r229851 and r229932, that happens on shutdown if >> kernel has WITNESS and doesn't have WITNESS_SKIPSPIN. [**] >> Uptime: 1h0m17s >> Rebooting... >> panic: mtx

Re: new panic in cpu_reset() with WITNESS

2012-01-17 Thread Gleb Smirnoff
On Tue, Jan 17, 2012 at 07:34:23AM -0800, m...@freebsd.org wrote: m> 2012/1/17 Gleb Smirnoff : m> >  New panic has been introduced somewhere between m> > r229851 and r229932, that happens on shutdown if m> > kernel has WITNESS and doesn't have WITNESS_SKIPSPIN. m> > m> > Uptime: 1h0m17s m> > Reboot

Re: new panic in cpu_reset() with WITNESS

2012-01-17 Thread mdf
2012/1/17 Gleb Smirnoff : >  New panic has been introduced somewhere between > r229851 and r229932, that happens on shutdown if > kernel has WITNESS and doesn't have WITNESS_SKIPSPIN. > > Uptime: 1h0m17s > Rebooting... > panic: mtx_lock_spin: recursed on non-recursive mutex cnputs_mtx @ > /usr/src

new panic in cpu_reset() with WITNESS

2012-01-17 Thread Gleb Smirnoff
New panic has been introduced somewhere between r229851 and r229932, that happens on shutdown if kernel has WITNESS and doesn't have WITNESS_SKIPSPIN. Uptime: 1h0m17s Rebooting... panic: mtx_lock_spin: recursed on non-recursive mutex cnputs_mtx @ /usr/src/head/sys/kern/kern_cons.c:500 cpuid = 0