Hi, We experienced a similar lockup tonight.
At 3:30 the system started to have problems: Aug 14 03:39:33 camelot kernel: [7646288.524370] ------------[ cut here ]------------ Aug 14 03:39:33 camelot kernel: [7646288.524402] kernel BUG at arch/x86/kvm/mmu.c:615! Aug 14 03:39:33 camelot kernel: [7646288.524427] invalid opcode: 0000 [1] SMP Aug 14 03:39:33 camelot kernel: [7646288.524459] CPU 1 Aug 14 03:39:33 camelot kernel: [7646288.524485] Modules linked in: iptable_filter ip_tables x_tables tun kvm_intel kvm ipv6 bridge loop snd_pcm snd_timer snd soundcore snd_page_al loc parport_pc parport iTCO_wdt i2c_i801 pcspkr i2c_core rng_core i5000_edac edac_core container button shpchp pci_hotplug evdev ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mo d ide_cd_mod cdrom usb_storage piix ide_pci_generic ide_core ses enclosure sd_mod ata_generic ehci_hcd libata dock uhci_hcd tg3 aacraid scsi_mod thermal processor fan thermal_sys Aug 14 03:39:33 camelot kernel: [7646288.525219] Pid: 6140, comm: kvm Not tainted 2.6.26-2-amd64 #1 Aug 14 03:39:33 camelot kernel: [7646288.525219] RIP: 0010:[<ffffffffa027b844>] [<ffffffffa027b844>] :kvm:kvm_mmu_get_page +0x2bd/0x3cf Aug 14 03:39:33 camelot kernel: [7646288.525219] RSP: 0018:ffff81017ebc3b48 EFLAGS: 00010246 Aug 14 03:39:33 camelot kernel: [7646288.525219] RAX: ffff81011c2dc210 RBX: ffffc200049d1fd8 RCX: 0000000000000000 Aug 14 03:39:33 camelot kernel: [7646288.525219] RDX: 87e8c88be7c689e6 RSI: ffff8101c5f8b880 RDI: 0000000000000000 Aug 14 03:39:33 camelot kernel: [7646288.525219] RBP: ffff8102f6545e48 R08: 000000000000000a R09: 0000000000000002 Aug 14 03:39:33 camelot kernel: [7646288.525219] R10: 000000000000000d R11: ffffffffa0296fb4 R12: ffff81017d8c0000 Aug 14 03:39:33 camelot kernel: [7646288.525219] R13: ffff81017e0d02c0 R14: 000000000001bcfb R15: 000000000001bcfb Aug 14 03:39:33 camelot kernel: [7646288.525219] FS: 0000000000000000(0000) GS:ffff81033e48b8c0(0000) knlGS:000000009a39fb90 Aug 14 03:39:33 camelot kernel: [7646288.525219] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033 Aug 14 03:39:33 camelot kernel: [7646288.525219] CR2: 0000000068ba0000 CR3: 000000017ed88000 CR4: 00000000000026e0 Aug 14 03:39:33 camelot kernel: [7646288.525219] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 03:39:33 camelot kernel: [7646288.525219] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Aug 14 03:39:33 camelot kernel: [7646288.525219] Process kvm (pid: 6140, threadinfo ffff81017ebc2000, task ffff81031da6c1c0) Aug 14 03:39:33 camelot kernel: [7646288.525219] Stack: 0000000000000000 0000000000000002 0000000000000000 ffff81017e0d02c0 Aug 14 03:39:33 camelot kernel: [7646288.525219] ffff8101d5009a28 0000000000000007 0000000000000000 ffffffffa027d137 Aug 14 03:39:33 camelot kernel: [7646288.525219] ffff8101d5009a28 ffff81017ebc3c64 0000000000000000 00000000000015de Aug 14 03:39:33 camelot kernel: [7646288.525219] Call Trace: Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa027d137>] ? :kvm:paging64_page_fault+0x2b4/0x3bf Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa0274b01>] ? :kvm:kvm_read_guest_atomic+0x4b/0x6c Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa027ccf2>] ? :kvm:paging64_prefetch_page+0x3b/0x77 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa0295499>] ? :kvm_intel:vmx_set_cr3+0xe5/0xec Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa027bc16>] ? :kvm:kvm_mmu_page_fault+0x19/0x81 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa0279318>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x463/0x615 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff8022c246>] ? try_to_wake_up+0x118/0x129 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa027461b>] ? :kvm:kvm_vcpu_ioctl+0x36/0x3a9 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa02746d8>] ? :kvm:kvm_vcpu_ioctl+0xf3/0x3a9 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff802284eb>] ? __wake_up_common+0x41/0x74 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff8024fb23>] ? futex_wake+0x74/0xfa Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff80250718>] ? do_futex+0xa6/0x777 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff80248d5e>] ? ktime_get_ts+0x22/0x4b Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffffa027436e>] ? :kvm:kvm_vm_ioctl+0x19c/0x1b5 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff8022f0d5>] ? hrtick_set+0x88/0xf7 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff802125e7>] ? read_tsc+0x9/0x20 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff802c8180>] ? compat_sys_ioctl+0xc8/0x33f Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff80244d9c>] ? posix_ktime_get_ts+0xc/0x10 Aug 14 03:39:33 camelot kernel: [7646288.525219] [<ffffffff80224c52>] ? sysenter_do_call+0x1b/0x66 Aug 14 03:39:33 camelot kernel: [7646288.525219] Aug 14 03:39:33 camelot kernel: [7646288.525219] Aug 14 03:39:33 camelot kernel: [7646288.525219] Code: e7 49 89 c6 e8 75 ed ff ff 31 d2 48 89 c3 48 89 c6 4c 89 e7 e8 04 e8 ff ff c7 44 24 04 00 00 00 00 eb 2e 48 8b 10 f6 c2 01 75 04 <0f> 0b eb fe f6 c2 02 74 0f 48 83 e2 fd 48 89 10 c7 44 24 04 01 Aug 14 03:39:33 camelot kernel: [7646288.525219] RIP [<ffffffffa027b844>] :kvm:kvm_mmu_get_page+0x2bd/0x3cf Aug 14 03:39:33 camelot kernel: [7646288.525219] RSP <ffff81017ebc3b48> Aug 14 03:39:33 camelot kernel: [7646288.526757] ---[ end trace 0d943ec153fc0496 ]--- After that, system started to have problems every minute: Aug 14 03:40:39 camelot kernel: [7646353.902579] BUG: soft lockup - CPU#2 stuck for 61s! [kvm:6137] Aug 14 03:40:39 camelot kernel: [7646353.902579] Modules linked in: iptable_filter ip_tables x_tables tun kvm_intel kvm ipv6 bridge loop snd_pcm snd_timer snd soundcore snd_page_al loc parport_pc parport iTCO_wdt i2c_i801 pcspkr i2c_core rng_core i5000_edac edac_core container button shpchp pci_hotplug evdev ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mod ide_cd_mod cdrom usb_storage piix ide_pci_generic ide_core ses enclosure sd_mod ata_generic ehci_hcd libata dock uhci_hcd tg3 aacraid scsi_mod thermal processor fan thermal_sys Aug 14 03:40:39 camelot kernel: [7646353.902579] CPU 2: Aug 14 03:40:39 camelot kernel: [7646353.902579] Modules linked in: iptable_filter ip_tables x_tables tun kvm_intel kvm ipv6 bridge loop snd_pcm snd_timer snd soundcore snd_page_alloc parport_pc parport iTCO_wdt i2c_i801 pcspkr i2c_core rng_core i5000_edac edac_core container button shpchp pci_hotplug evdev ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mod ide_cd_mod cdrom usb_storage piix ide_pci_generic ide_core ses enclosure sd_mod ata_generic ehci_hcd libata dock uhci_hcd tg3 aacraid scsi_mod thermal processor fan thermal_sys Aug 14 03:40:39 camelot kernel: [7646353.902579] Pid: 6137, comm: kvm Tainted: G D 2.6.26-2-amd64 #1 Aug 14 03:40:39 camelot kernel: [7646353.902579] RIP: 0010:[<ffffffff804295cc>] [<ffffffff804295cc>] _spin_lock+0x12/0x15 Aug 14 03:40:39 camelot kernel: [7646353.902579] RSP: 0018:ffff81017eff1cb0 EFLAGS: 00000297 Aug 14 03:40:39 camelot kernel: [7646353.902579] RAX: 0000000000008d8c RBX: 0000000000000000 RCX: ffff81033e7d6880 Aug 14 03:40:39 camelot kernel: [7646353.902579] RDX: 0000000000000001 RSI: ffff81017e804338 RDI: ffff81017d8c0018 Aug 14 03:40:39 camelot kernel: [7646353.902579] RBP: 00000000ffffffff R08: ffff81017e804280 R09: 0000000000000003 Aug 14 03:40:39 camelot kernel: [7646353.902579] R10: ffff81017eff0000 R11: ffffffff802f2039 R12: ffffffffa0294eb9 Aug 14 03:40:39 camelot kernel: [7646353.902579] R13: ffff8102295cc000 R14: 0000000200000202 R15: ffff81033b9be348 Aug 14 03:40:39 camelot kernel: [7646353.902579] FS: 0000000000000000(0000) GS:ffff81033e48b0c0(0063) knlGS:000000009aba3b90 Aug 14 03:40:39 camelot kernel: [7646353.902579] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Aug 14 03:40:39 camelot kernel: [7646353.902579] CR2: 00000000927b7004 CR3: 000000017ed88000 CR4: 00000000000026e0 Aug 14 03:40:39 camelot kernel: [7646353.902579] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 03:40:39 camelot kernel: [7646353.902579] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Aug 14 03:40:39 camelot kernel: [7646353.902579] Aug 14 03:40:39 camelot kernel: [7646353.902579] Call Trace: Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffffa027aa68>] ? :kvm:mmu_free_roots+0x25/0xfb Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffffa02790fc>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x247/0x615 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffffa027461b>] ? :kvm:kvm_vcpu_ioctl+0x36/0x3a9 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffffa02746d8>] ? :kvm:kvm_vcpu_ioctl+0xf3/0x3a9 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff802284eb>] ? __wake_up_common+0x41/0x74 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff8024fb23>] ? futex_wake+0x74/0xfa Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff8023e248>] ? __dequeue_signal+0x19/0x15a Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff80248d5e>] ? ktime_get_ts+0x22/0x4b Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff8023decf>] ? recalc_sigpending+0xe/0x38 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff8023f1c3>] ? dequeue_signal+0x8d/0x111 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff8025d482>] ? compat_sys_rt_sigtimedwait+0xfc/0x26a Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff802125e7>] ? read_tsc+0x9/0x20 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff802c8180>] ? compat_sys_ioctl+0xc8/0x33f Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff80244d9c>] ? posix_ktime_get_ts+0xc/0x10 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffff80224c52>] ? sysenter_do_call+0x1b/0x66 Aug 14 03:40:39 camelot kernel: [7646353.902579] [<ffffffffa0296f60>] ? :kvm_intel:handle_io+0x0/0x54 Aug 14 03:40:39 camelot kernel: [7646353.902579] We arrived at 8:15 to office. Server behaviour was clunky at best. It ceased to respond tcp/ip connections shortly after that, after an apt-get update was issued. We had to cold-reboot the system. System has been working without problems since January. It has been rebooted maybe 2 or 3 times since then. Shall I submit another bug? Thanks, Eneko -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
