On Mon, Sep 14, 2009 at 03:36:48PM -0400, Carlos O'Donell wrote: > On Mon, Sep 14, 2009 at 1:26 PM, dann frazier <da...@dannf.org> wrote: > > After about a week of loaded uptime, penalosa returned to instability > > over the weekend. > > > > It began with an Illegal instruction leading to a panic: > > http://people.debian.org/~dannf/penalosa/penalosa.0 > > > > I then rebooted it, and the console was hung immediately after boot: > > http://people.debian.org/~dannf/penalosa/penalosa.1 > > > > Note the rc.local segfault there. > > > > I rebooted it again, and it crashed after only a few minutes: > > http://people.debian.org/~dannf/penalosa/penalosa.2 > > Exactly how did you install the kernel and what sources were used to build it?
I used dpkg -i to install it, sources are here: http://people.debian.org/~dannf/tmp/ To create an unpacked source tree, you can: dget http://people.debian.org/~dannf/tmp/linux-2.6_2.6.31~rc6-1~experimental.1~snapshot.14160.dsc cd linux-2.6-2.6.31~rc6 ./debian/rules setup It is a pretty vanilla 2.6.31-rc6. > The trace looks familiar (crash in unwind_once), and I thought it had > been fixed by Kyle already. I will put together a kernel for you to > boot on penalosa. I don't see any obvious fixes between 2.6.31-rc6 & 2.6.31 - is the fix in kyle's tree? > > peri (identical hardware/kernel) has been up for about 20 days now. > > Please issue a sysrq-t (ctrl^c, send break, t) on peri's console and > tell me if it crashes the kernel? Yes, it does. (echo t > /proc/sysrq-trigger didn't though, for whatever reason). peri login: [1714067.644000] SysRq : Show State [1714067.644000] task PC stack pid father [1714067.644000] init S 0000000040103568 0 1 0 0x00000010 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<00000000401ce628>] __follow_mount+0x50/0xe0 [1714067.644000] [<00000000401ce854>] do_lookup+0x7c/0x280 [1714067.644000] [<00000000401ce5c8>] path_put+0x38/0x48 [1714067.644000] [<0000000040104ed4>] syscall_exit+0x0/0x14 [1714067.644000] [1714067.644000] kthreadd S 0000000040103568 0 2 0 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<00000000401c06c8>] kmem_cache_alloc+0x70/0xc0 [1714067.644000] [1714067.644000] ksoftirqd/0 S 0000000040103568 0 3 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] events/0 S 0000000040103568 0 4 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] cpuset S 0000000040103568 0 5 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] khelper S 0000000040103568 0 6 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] netns S 0000000040103568 0 9 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] async/mgr S 0000000040103568 0 10 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] kpowerswd S 0000000040103568 0 52 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] kintegrityd/0 S 0000000040103568 0 85 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] kblockd/0 S 0000000040103568 0 87 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] kseriod S 0000000040103568 0 91 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] khungtaskd S 0000000040103568 0 113 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] pdflush S 0000000040103568 0 114 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] pdflush S 0000000040103568 0 115 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] kswapd0 S 0000000040103568 0 116 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] aio/0 S 0000000040103568 0 117 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] crypto/0 S 0000000040103568 0 118 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] led_wq S 0000000040103568 0 123 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] scsi_eh_0 S 0000000040103568 0 304 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] scsi_eh_1 S 0000000040103568 0 309 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] ata/0 S 0000000040103568 0 454 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] khubd S 0000000040103568 0 464 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] ata_aux S 0000000040103568 0 505 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] md0_raid1 S 0000000040103568 0 667 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] kjournald S 0000000040103568 0 695 2 0x00000000 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<000000004013ce10>] set_user_nice+0x170/0x208 [1714067.644000] [1714067.644000] syslog-ng S 0000000040103568 0 1398 1 0x00000010 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<00000000401d6734>] poll_select_set_timeout+0x54/0xa8 [1714067.644000] [<00000000401d6ffc>] sys_poll+0xb4/0x130 [1714067.644000] [1714067.644000] sshd S 0000000040103568 0 1408 1 0x00000010 [1714067.644000] Backtrace: [1714067.644000] [<00000000401121d8>] schedule+0x398/0x428 [1714067.644000] [<00000000401dbbc4>] iput+0x84/0x98 [1714067.644000] timer_interrupt(CPU 0): delayed! cycles 15D4E7F37 rem 188649 next/now 4915154F419A3/4915154DB935A [1714075.580000] Backtrace: [1714075.580000] [<000000004030531c>] io_serial_out+0x2c/0x38 [1714075.580000] [<00000000402a1050>] sprintf+0x48/0x58 [1714075.580000] [1714075.580000] [1714075.580000] Kernel Fault: Code=15 regs=00000000405b54d0 (Addr=000000002e6aca70) [1714075.580000] [1714075.580000] YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI [1714075.580000] PSW: 00001000000001001111111100001110 Not tainted [1714075.580000] r00-03 000000ff0804ff0e 0000000040613e80 000000006fc00000 00000000405b4f78 [1714075.580000] r04-07 0000000037de0000 ffffffffffe00008 00000000405b4f78 0000000040613680 [1714075.580000] r08-11 000000000fc212c1 000000006bc23fd9 0000000000000007 000000000800000e [1714075.580000] r12-15 000000000800000f 00000000405b4af8 0000000000000004 00000000405b4af9 [1714075.580000] r16-19 00000000000000f3 000000000000001e 0000000000000002 000000012e6ac9c0 [1714075.580000] r20-23 00000000ffffe000 00000000401dbb44 000000004047d784 0000000000000010 [1714075.580000] r24-27 0000000000000000 00000000401dbae0 00000000401dbbc4 0000000040604e80 [1714075.580000] r28-31 000000002e6aca70 00000000000000d0 00000000405b54d0 000000002e6aca80 [1714075.580000] sr00-03 0000000000002000 0000000000000000 0000000000000000 0000000000002000 [1714075.580000] sr04-07 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [1714075.580000] [1714075.580000] IASQ: 0000000000000000 0000000000000000 IAOQ: 000000004011fc10 000000004011fc14 [1714075.580000] IIR: 0f8010dc ISR: 0000000000000000 IOR: 000000002e6aca70 [1714075.580000] CPU: 0 CR30: 00000000405b4000 CR31: 0000000011111111 [1714075.580000] ORIG_R28: 0000000000000010 [1714075.580000] IAOQ[0]: unwind_once+0x370/0x3d0 [1714075.580000] IAOQ[1]: unwind_once+0x374/0x3d0 [1714075.580000] RP(r2): 0x6fc00000 [1714075.580000] Backtrace: [1714075.580000] [<000000004030531c>] io_serial_out+0x2c/0x38 [1714075.580000] [<00000000402a1050>] sprintf+0x48/0x58 [1714075.580000] [1714075.580000] Kernel panic - not syncing: Kernel Fault [1714075.580000] Backtrace: [1714075.580000] Backtrace: [1714075.580000] [<000000004030531c>] io_serial_out+0x2c/0x38 [1714075.580000] [<0000000040117428>] parisc_terminate+0x140/0x148 [1714075.580000] [1714075.580000] [1714075.580000] Bad Address (null pointer deref?): Code=15 regs=00000000405b6350 (Addr=c00007ff405b6270) [1714075.580000] [1714075.580000] YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI [1714075.580000] PSW: 00001000000001101111111100001110 Not tainted [1714075.580000] r00-03 000000ff0806ff0e 0000000040613e80 000000006fc00000 00000000405b5df8 [1714075.580000] r04-07 0000000037de0000 ffffffffffe00008 00000000405b5df8 0000000040613680 [1714075.580000] r08-11 000000000fc212c1 000000006bc23fd9 0000000000000007 000000000800000e [1714075.580000] r12-15 000000000800000f 00000000405b4af8 0000000000000004 00000000405b4af9 [1714075.580000] r16-19 00000000405b54d0 000000000000001e 0000000000000002 00000000405b6180 [1714075.580000] r20-23 00000000ffffe000 000000004011672c 0000000040471f84 0000000000000010 [1714075.580000] r24-27 0000000000000000 0000000040116680 0000000040116d00 0000000040604e80 [1714075.580000] r28-31 ffffffff405b6270 0000000000000350 00000000405b6350 ffffffff405b6280 [1714075.580000] sr00-03 0000000000002000 0000000000000000 0000000000000000 0000000000002000 [1714075.580000] sr04-07 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [1714075.580000] [1714075.580000] IASQ: 0000000000000000 0000000000000000 IAOQ: 000000004011fc10 000000004011fc14 [1714075.580000] IIR: 0f8010dc ISR: 000000003ffff800 IOR: c00007ff405b6270 [1714075.580000] CPU: 0 CR30: 00000000405b4000 CR31: 0000000011111111 [1714075.580000] ORIG_R28: 0000000040614680 [1714075.580000] IAOQ[0]: unwind_once+0x370/0x3d0 [1714075.580000] IAOQ[1]: unwind_once+0x374/0x3d0 [1714075.580000] RP(r2): 0x6fc00000 [1714075.580000] Backtrace: [1714075.580000] [<000000004030531c>] io_serial_out+0x2c/0x38 [1714075.580000] [<0000000040117428>] parisc_terminate+0x140/0x148 [1714075.580000] [1714075.580000] Kernel panic - not syncing: Bad Address (null pointer deref?) [1714075.580000] Backtrace: [1714075.580000] Backtrace: [1714075.580000] [<0000000040111890>] printk+0x40/0x50 [1714075.580000] [<0000000040116c34>] do_show_stack+0x5c/0xe0 [1714075.580000] Backtrace: [1714075.580000] [<0000000040155c9c>] __kernel_text_address+0xac/0xc0 [1714075.580000] [1714075.580000] [1714075.580000] Bad Address (null pointer deref?): Code=15 regs=00000000405b7db0 (Addr=c00007ff405b72e0) [1714075.580000] [1714075.580000] YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI [1714075.580000] PSW: 00001000000001101111111100001110 Not tainted [1714075.580000] r00-03 000000ff0806ff0e 0000000040613e80 000000006fc00000 00000000405b7b08 [1714075.580000] r04-07 0000000037de0000 ffffffffffe00008 00000000405b7b08 0000000040613680 ... -- dann frazier -- To UNSUBSCRIBE, email to debian-release-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
