Awoke this morning to another fault (x2) and I've attached the log.
-- Kind regards, Michael
# This fault resulted in: # # root 2039 0.0 0.0 22168 1020 ? Ss Jun10 0:00 /usr/sbin/cron # root 15252 0.0 0.0 45292 1272 ? S 02:55 0:00 \_ /USR/SBIN/CRON # munin 15256 0.0 0.0 3888 556 ? Ss 02:55 0:00 \_ /bin/sh -c if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi # munin 15258 0.0 0.0 3888 540 ? S 02:55 0:00 \_ /bin/sh /usr/bin/munin-cron # munin 15891 0.0 0.1 107372 13604 ? SN 02:55 0:00 \_ /usr/bin/perl /usr/share/munin/munin-graph --cron # munin 15899 0.0 0.0 0 0 ? DN 02:55 0:00 | \_ [munin-graph] # munin 15892 0.0 0.0 5912 520 ? S 02:55 0:00 \_ fgrep -v *** attempt to put segment in horiz list twice # # box rebooted at ~08:10 Jun 11 02:45:01 gaea /USR/SBIN/CRON[13808]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi) Jun 11 02:45:01 gaea /USR/SBIN/CRON[13809]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) Jun 11 02:45:01 gaea /USR/SBIN/CRON[13810]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi) Jun 11 02:50:01 gaea /USR/SBIN/CRON[14532]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi) Jun 11 02:50:01 gaea /USR/SBIN/CRON[14533]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi) Jun 11 02:55:01 gaea /USR/SBIN/CRON[15254]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi) Jun 11 02:55:01 gaea /USR/SBIN/CRON[15255]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) Jun 11 02:55:01 gaea /USR/SBIN/CRON[15256]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi) Jun 11 02:55:12 gaea kernel: [57509.073252] general protection fault: 0000 [#1] SMP Jun 11 02:55:12 gaea kernel: [57509.073256] last sysfs file: /sys/devices/system/cpu/cpu3/cpufreq/stats/time_in_state Jun 11 02:55:12 gaea kernel: [57509.073258] CPU 1 Jun 11 02:55:12 gaea kernel: [57509.073259] Modules linked in: ppdev lp parport tun sit tunnel4 xt_multiport iptable_filter ip_tables x_tables powernow_k8 cpufreq_powersave cpufreq_userspace cpufreq_stats cpufreq_conservative binfmt_misc vboxnetadp vboxnetflt vboxdrv fuse nfsd nfs lockd fscache nfs_acl auth_rpcgss sunrpc xfs exportfs it87 hwmon_vid loop firewire_sbp2 snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event nouveau snd_seq ttm drm_kms_helper snd_timer snd_seq_device drm i2c_algo_bit snd soundcore i2c_piix4 edac_core snd_page_alloc edac_mce_amd i2c_core pcspkr evdev xhci tpm_tis tpm tpm_bios button processor ext3 jbd mbcache raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx raid1 md_mod sr_mod cdrom usbhid sd_mod crc_t10dif ata_generic firewire_ohci firewire_core sg hid pata_jmicron ohci_hcd r8169 crc_itu_t ehci_hcd pata_atiixp ahci mii libata usbcore nls_base thermal thermal_sys scsi_m Jun 11 02:55:12 gaea kernel: od [last unloaded: scsi_wait_scan] Jun 11 02:55:12 gaea kernel: [57509.073300] Pid: 15899, comm: munin-graph Not tainted 2.6.32-5-amd64 #1 GA-790XTA-UD4 Jun 11 02:55:12 gaea kernel: [57509.073302] RIP: 0010:[<ffffffff810ee7b3>] [<ffffffff810ee7b3>] sys_write+0x29/0x6e Jun 11 02:55:12 gaea kernel: [57509.073308] RSP: 0018:ffff8802007a1f48 EFLAGS: 00010282 Jun 11 02:55:12 gaea kernel: [57509.073309] RAX: f7ff8801f48cee40 RBX: f7ff8801f48cee40 RCX: 7672657320646568 Jun 11 02:55:12 gaea kernel: [57509.073311] RDX: ffff88022f065350 RSI: ffff8802007a1f5c RDI: f7ff8801f48cee40 Jun 11 02:55:12 gaea kernel: [57509.073312] RBP: fffffffffffffff7 R08: 0a2934202a206365 R09: 706172472032313a Jun 11 02:55:12 gaea kernel: [57509.073314] R10: 7672657320646568 R11: 0000000000000246 R12: 0000000000000038 Jun 11 02:55:12 gaea kernel: [57509.073315] R13: 00000000024bb660 R14: 0000000000000038 R15: 00000000024bb660 Jun 11 02:55:12 gaea kernel: [57509.073317] FS: 00007f2fb8a7f6f0(0000) GS:ffff880008c80000(0000) knlGS:0000000000000000 Jun 11 02:55:12 gaea kernel: [57509.073318] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 11 02:55:12 gaea kernel: [57509.073320] CR2: 00007f2fb8634260 CR3: 00000002004fc000 CR4: 00000000000006e0 Jun 11 02:55:12 gaea kernel: [57509.073321] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jun 11 02:55:12 gaea kernel: [57509.073323] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Jun 11 02:55:12 gaea kernel: [57509.073324] Process munin-graph (pid: 15899, threadinfo ffff8802007a0000, task ffff88022d2df810) Jun 11 02:55:12 gaea kernel: [57509.073326] Stack: Jun 11 02:55:12 gaea kernel: [57509.073326] 0000000000000002 ffffffff812fab75 0000000000000002 0000000001b74010 Jun 11 02:55:12 gaea kernel: [57509.073329] <0> 00000000024bb660 0000000001bb6260 0000000000000003 ffffffff81010b42 Jun 11 02:55:12 gaea kernel: [57509.073331] <0> 0000000000000246 7672657320646568 706172472032313a 0a2934202a206365 Jun 11 02:55:12 gaea kernel: [57509.073333] Call Trace: Jun 11 02:55:12 gaea kernel: [57509.073337] [<ffffffff812fab75>] ? page_fault+0x25/0x30 Jun 11 02:55:12 gaea kernel: [57509.073341] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b Jun 11 02:55:12 gaea kernel: [57509.073342] Code: 5d c3 41 55 49 89 f5 41 54 49 89 d4 55 48 c7 c5 f7 ff ff ff 53 48 83 ec 18 48 8d 74 24 14 e8 eb 03 00 00 48 85 c0 48 89 c3 74 37 <48> 8b 40 40 48 8d 4c 24 08 4c 89 e2 4c 89 ee 48 89 df 48 89 44 Jun 11 02:55:12 gaea kernel: [57509.073357] RIP [<ffffffff810ee7b3>] sys_write+0x29/0x6e Jun 11 02:55:12 gaea kernel: [57509.073359] RSP <ffff8802007a1f48> Jun 11 02:55:12 gaea kernel: [57509.073361] ---[ end trace e472713dd7682be2 ]--- Jun 11 02:55:12 gaea kernel: [57509.074095] general protection fault: 0000 [#2] SMP Jun 11 02:55:12 gaea kernel: [57509.074096] last sysfs file: /sys/devices/system/cpu/cpu3/cpufreq/stats/time_in_state Jun 11 02:55:12 gaea kernel: [57509.074098] CPU 1 Jun 11 02:55:12 gaea kernel: [57509.074098] Modules linked in: ppdev lp parport tun sit tunnel4 xt_multiport iptable_filter ip_tables x_tables powernow_k8 cpufreq_powersave cpufreq_userspace cpufreq_stats cpufreq_conservative binfmt_misc vboxnetadp vboxnetflt vboxdrv fuse nfsd nfs lockd fscache nfs_acl auth_rpcgss sunrpc xfs exportfs it87 hwmon_vid loop firewire_sbp2 snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event nouveau snd_seq ttm drm_kms_helper snd_timer snd_seq_device drm i2c_algo_bit snd soundcore i2c_piix4 edac_core snd_page_alloc edac_mce_amd i2c_core pcspkr evdev xhci tpm_tis tpm tpm_bios button processor ext3 jbd mbcache raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx raid1 md_mod sr_mod cdrom usbhid sd_mod crc_t10dif ata_generic firewire_ohci firewire_core sg hid pata_jmicron ohci_hcd r8169 crc_itu_t ehci_hcd pata_atiixp ahci mii libata usbcore nls_base thermal thermal_sys scsi_m Jun 11 02:55:12 gaea kernel: od [last unloaded: scsi_wait_scan] Jun 11 02:55:12 gaea kernel: [57509.074134] Pid: 15899, comm: munin-graph Tainted: G D 2.6.32-5-amd64 #1 GA-790XTA-UD4 Jun 11 02:55:12 gaea kernel: [57509.074136] RIP: 0010:[<ffffffff810ec37e>] [<ffffffff810ec37e>] filp_close+0x7/0x62 Jun 11 02:55:12 gaea kernel: [57509.074139] RSP: 0018:ffff8802007a1db8 EFLAGS: 00010282 Jun 11 02:55:12 gaea kernel: [57509.074141] RAX: ffff88022f0653f0 RBX: ffff88022f065340 RCX: ffff88022b7cfbe0 Jun 11 02:55:12 gaea kernel: [57509.074142] RDX: 0000000000000002 RSI: ffff88022f065340 RDI: f7ff8801f48cee40 Jun 11 02:55:12 gaea kernel: [57509.074143] RBP: 0000000000000007 R08: 0000000000000000 R09: ffff88022ed35f00 Jun 11 02:55:12 gaea kernel: [57509.074145] R10: 0000000000000002 R11: 0000000000000b61 R12: ffff88022f065340 Jun 11 02:55:12 gaea kernel: [57509.074146] R13: 0000000000000000 R14: ffff88022f065350 R15: 0000000000000000 Jun 11 02:55:12 gaea kernel: [57509.074148] FS: 00007f2fb8a7f6f0(0000) GS:ffff880008c80000(0000) knlGS:0000000000000000 Jun 11 02:55:12 gaea kernel: [57509.074149] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 11 02:55:12 gaea kernel: [57509.074150] CR2: 00007f2fb8634260 CR3: 0000000001001000 CR4: 00000000000006e0 Jun 11 02:55:12 gaea kernel: [57509.074152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jun 11 02:55:12 gaea kernel: [57509.074153] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Jun 11 02:55:12 gaea kernel: [57509.074155] Process munin-graph (pid: 15899, threadinfo ffff8802007a0000, task ffff88022d2df810) Jun 11 02:55:12 gaea kernel: [57509.074156] Stack: Jun 11 02:55:12 gaea kernel: [57509.074157] ffff88022f065340 0000000000000007 0000000000000003 ffffffff8104f768 Jun 11 02:55:12 gaea kernel: [57509.074158] <0> ffff88022c68c000 000000000000000b ffff88022d2df810 0000000000000246 Jun 11 02:55:12 gaea kernel: [57509.074161] <0> 00000000024bb660 0000000000000038 0000000000000001 ffffffff81051016 Jun 11 02:55:12 gaea kernel: [57509.074163] Call Trace: Jun 11 02:55:12 gaea kernel: [57509.074166] [<ffffffff8104f768>] ? put_files_struct+0x64/0xc1 Jun 11 02:55:12 gaea kernel: [57509.074168] [<ffffffff81051016>] ? do_exit+0x225/0x6b5 Jun 11 02:55:12 gaea kernel: [57509.074170] [<ffffffff8104e271>] ? release_console_sem+0x17e/0x1af Jun 11 02:55:12 gaea kernel: [57509.074172] [<ffffffff81014a82>] ? oops_end+0xaf/0xb4 Jun 11 02:55:12 gaea kernel: [57509.074175] [<ffffffff812fab45>] ? general_protection+0x25/0x30 Jun 11 02:55:12 gaea kernel: [57509.074177] [<ffffffff810ee7b3>] ? sys_write+0x29/0x6e Jun 11 02:55:12 gaea kernel: [57509.074179] [<ffffffff810ee7ab>] ? sys_write+0x21/0x6e Jun 11 02:55:12 gaea kernel: [57509.074181] [<ffffffff812fab75>] ? page_fault+0x25/0x30 Jun 11 02:55:12 gaea kernel: [57509.074183] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b Jun 11 02:55:12 gaea kernel: [57509.074184] Code: 42 08 48 83 3c c8 00 74 04 0f 0b eb fe 48 8b 42 08 4c 89 2c c8 66 ff 85 80 00 00 00 5e 5b 5d 41 5c 41 5d c3 41 54 49 89 f4 55 53 <48> 8b 47 30 48 89 fb 48 85 c0 75 10 48 c7 c7 1b d6 3b 81 31 ed Jun 11 02:55:12 gaea kernel: [57509.074199] RIP [<ffffffff810ec37e>] filp_close+0x7/0x62 Jun 11 02:55:12 gaea kernel: [57509.074201] RSP <ffff8802007a1db8> Jun 11 02:55:12 gaea kernel: [57509.074202] ---[ end trace e472713dd7682be3 ]--- Jun 11 02:55:12 gaea kernel: [57509.074203] Fixing recursive fault but reboot is needed! Jun 11 02:55:13 gaea kernel: [57509.836759] BUG: Bad page state in process munin-graph pfn:1c6d11 Jun 11 02:55:13 gaea kernel: [57509.836764] page:ffffea000637dbb8 flags:0200000000000000 count:0 mapcount:-134217728 mapping:(null) index:2683 Jun 11 02:55:13 gaea kernel: [57509.836767] Pid: 15943, comm: munin-graph Tainted: G D 2.6.32-5-amd64 #1 Jun 11 02:55:13 gaea kernel: [57509.836769] Call Trace: Jun 11 02:55:13 gaea kernel: [57509.836775] [<ffffffff810b7268>] ? bad_page+0x116/0x129 Jun 11 02:55:13 gaea kernel: [57509.836778] [<ffffffff810b9267>] ? get_page_from_freelist+0x4fd/0x760 Jun 11 02:55:13 gaea kernel: [57509.836781] [<ffffffff810b9824>] ? __alloc_pages_nodemask+0x10f/0x5cf Jun 11 02:55:13 gaea kernel: [57509.836785] [<ffffffff8118d1b8>] ? cpumask_any_but+0x28/0x34 Jun 11 02:55:13 gaea kernel: [57509.836788] [<ffffffff810ca732>] ? do_wp_page+0x386/0x707 Jun 11 02:55:13 gaea kernel: [57509.836791] [<ffffffff810edd42>] ? do_sync_write+0xce/0x113 Jun 11 02:55:13 gaea kernel: [57509.836794] [<ffffffff8101166e>] ? apic_timer_interrupt+0xe/0x20 Jun 11 02:55:13 gaea kernel: [57509.836796] [<ffffffff810cc445>] ? handle_mm_fault+0x742/0x7a5 Jun 11 02:55:13 gaea kernel: [57509.836800] [<ffffffff81032847>] ? do_page_fault+0x266/0x282 Jun 11 02:55:13 gaea kernel: [57509.836803] [<ffffffff812fab75>] ? page_fault+0x25/0x30 Jun 11 02:57:04 gaea smartd[2204]: Device: /dev/sde [SAT], SMART Usage Attribute: 195 Hardware_ECC_Recovered changed from 44 to 43 Jun 11 02:57:23 gaea ovpn-linode[1726]: TLS: tls_process: killed expiring key Jun 11 02:57:24 gaea ovpn-linode[1726]: TLS: soft reset sec=0 bytes=36907/0 pkts=703/0 Jun 11 02:57:25 gaea ovpn-linode[1726]: VERIFY OK: depth=1, /C=US/ST=TX/L=San_Antonio/O=vpn.12.am/CN=vpn.12.am_CA/emailaddress=r...@linode.12.am Jun 11 02:57:25 gaea ovpn-linode[1726]: VERIFY OK: nsCertType=SERVER Jun 11 02:57:25 gaea ovpn-linode[1726]: VERIFY OK: depth=0, /C=US/ST=TX/L=San_Antonio/O=vpn.12.am/CN=vpn.12.am/emailaddress=r...@linode.12.am Jun 11 02:57:26 gaea ovpn-linode[1726]: Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key Jun 11 02:57:26 gaea ovpn-linode[1726]: Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Jun 11 02:57:26 gaea ovpn-linode[1726]: Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key Jun 11 02:57:26 gaea ovpn-linode[1726]: Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Jun 11 02:57:26 gaea ovpn-linode[1726]: Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA Jun 11 03:00:01 gaea /USR/SBIN/CRON[15984]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi) Jun 11 03:00:01 gaea /USR/SBIN/CRON[15985]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi) Jun 11 03:05:01 gaea /USR/SBIN/CRON[16705]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi) Jun 11 03:05:01 gaea /USR/SBIN/CRON[16706]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) Jun 11 03:05:01 gaea /USR/SBIN/CRON[16707]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi)
