Package: qemu-kvm
Version: 1.7.0+dfsg-2
Severity: important
Hi out there,
I tried to give nested virtualisation a test, because this could allow me to
organise my virtual machines more like a data center.
Virtual hypervisors would form "racks" and house a set of vm's, which in turn
could be migrated between racks and full racks be migrated to different
hardware.
For testing I use an older laptop with an Intel(R) Core(TM)2 Duo CPU P8400 @
2.26GHz, with 4Gigs of Ram.
Nesting is switched on for kvm_intel on the bare metal.
I passed all host cpu flags to the vm.
The guest hypervisor starts without any problems and loads kvm_intel accepting
it has hypervisor capabilities.
Trying to start kvm or qemu-system-i386 (or -x86_64) with --enable-kvm freezes
the guest hypervisor vm to death immediately.
Neither does any window open nor are any messages displayed when starting in a
terminal.
The guest hypervisor VM starts to happily use 100% CPU until switched off the
hard way.
In one instance I left it running overnight to see if it would recover, but it
didn't.
Using qemu without enabling kvm works just fine.
I have tried several CPUs for the guest hypervisor including kvm64, Penryn,
core2duo, always seeing the same result.
CPU selection for the L2 guest seems to have no effect either, although I did
not test this exhaustively.
While researching the above, I found bug #697551, which by the sounds of it is
a very similar problem.
This bug was closed with the remark the problem being fixed in V1.2.
In turn I upgraded qemu from testing 1.1.2+dfsg-6 to unstable 1.7.0+dfsg-2.
I have tested both 1.7 on Bare Metal and 1.1.2 on L1 Hypervisor as well as 1.7
on L0 and 1.7 on L1.
Unfortunately I have the same result for all combinations, kvm crashes the L1
into a loop consuming 100% cpu.
Below is a bit of technical information, happy to provide more or run some more
tests.
regards
Vanis
---------------------------------------------------------------------------------------------------------
Both, L0 and L1 use the same Linux kernel:
Linux gliese 3.11-2-amd64 #1 SMP Debian 3.11.8-1 (2013-11-13) x86_64 GNU/Linux
---------------------------------------------------------------------------------------------------------
Libvirt generated command to start the guest hypervisor (L1)
qemu-system-x86_64 -enable-kvm -name TestNest -S -machine
pc-1.1,accel=kvm,usb=off -cpu
kvm64,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme
-m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid
11111111-55555-55555-1234-9564564131 -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/TestNest.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot order=dc,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2
-drive file=/dev/sdd,if=none,id=drive-virtio-disk1,format=raw,cache=none
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,id
=virtio-disk1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev
tap,fd=24,id=hostnet0 -device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:8a:aa:f4,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
-vnc 127.0.0.1:0 -device qxl-
vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
---------------------------------------------------------------------------------------------------------
kvm_intel settings on bare metal (L0)
BareMetal:/sys/module/kvm_intel# for i in `find . -type f` ; do echo $i : `cat
$i` ; done
../notes/.note.gnu.build-id : GNU!6+H�8��dǘ�s���
../taint :
../initstate : live
../coresize : 130397
../sections/.bss : 0xffffffffa0813580
../sections/.init.text : 0xffffffffa0819000
../sections/.smp_locks : 0xffffffffa0812d24
../sections/.data : 0xffffffffa0813000
../sections/.text : 0xffffffffa07f8000
../sections/__param : 0xffffffffa0812d90
../sections/__ex_table : 0xffffffffa0811f60
../sections/.data.unlikely : 0xffffffffa0813300
../sections/.rodata : 0xffffffffa0804040
../sections/.strtab : 0xffffffffa081ca78
../sections/.symtab : 0xffffffffa081a000
../sections/.text.unlikely : 0xffffffffa08030c8
../sections/.gnu.linkonce.this_module : 0xffffffffa0813320
../sections/.rodata.str1.1 : 0xffffffffa081288c
../sections/.rodata.str1.8 : 0xffffffffa0812948
../sections/__bug_table : 0xffffffffa0812d84
../sections/.note.gnu.build-id : 0xffffffffa0804000
../sections/.fixup : 0xffffffffa0801c82
../sections/.exit.text : 0xffffffffa080328b
../sections/.parainstructions : 0xffffffffa0812580
../sections/.data..read_mostly : 0xffffffffa0813301
../parameters/ept : N
../parameters/vpid : N
../parameters/enable_shadow_vmcs : N
../parameters/eptad : N
../parameters/flexpriority : Y
../parameters/fasteoi : Y
../parameters/enable_apicv : N
../parameters/unrestricted_guest : N
../parameters/emulate_invalid_guest_state : Y
../parameters/nested : Y
../parameters/vmm_exclusive : Y
../parameters/ple_window : 4096
../parameters/ple_gap : 0
../refcnt : 3
cat: ./uevent: Permission denied
../uevent :
../initsize : 0
---------------------------------------------------------------------------------------------------------
dmesg on bare-metal has some messages that I believe are unrelated
[ 1071.302327] kvm [8458]: vcpu0 unhandled wrmsr: 0x62 data 0
[ 1071.302332] kvm [8458]: vcpu0 unhandled wrmsr: 0x43 data 0
[ 1071.302338] kvm [8458]: vcpu0 unhandled wrmsr: 0x63 data 0
[ 7827.681545] kvm: zapping shadow pages for mmio generation wraparound
---------------------------------------------------------------------------------------------------------
All info below from guest hypervisor (L1)
root@Testbed:~# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 6
model name : Common KVM processor
stepping : 1
microcode : 0x1
cpu MHz : 2260.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ss syscall nx lm constant_tsc nopl pni vmx
cx16 hypervisor
bogomips : 4521.99
clflush size : 64
cache_alignment : 128
address sizes : 40 bits physical, 48 bits virtual
power management:
---------------------------------------------------------------------------------------------------------
root@Testbed:~# lsmod | grep kvm
kvm_intel 130397 0
kvm 354353 1 kvm_intel
---------------------------------------------------------------------------------------------------------
root@Testbed:~# dmesg | grep kvm
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr 0:3fff2001, boot clock
[ 0.000000] kvm-clock: cpu 0, msr 0:3fff2001, primary cpu clock
[ 0.000000] kvm-stealtime: cpu 0, msr 3fc0e040
[ 0.260023] Switched to clocksource kvm-clock
---------------------------------------------------------------------------------------------------------
root@Testbed:~# cd /sys/module/kvm_intel/
root@Testbed:/sys/module/kvm_intel# for i in `find . -type f` ; do echo $i :
`cat $i` ; done
../notes/.note.gnu.build-id : GNU!6+H�8��dǘ�s���
../taint :
../initstate : live
../coresize : 130397
../sections/.bss : 0xffffffffa02c5580
../sections/.init.text : 0xffffffffa02cb000
../sections/.smp_locks : 0xffffffffa02c4d24
../sections/.data : 0xffffffffa02c5000
../sections/.text : 0xffffffffa02aa000
../sections/__param : 0xffffffffa02c4d90
../sections/__ex_table : 0xffffffffa02c3f60
../sections/.data.unlikely : 0xffffffffa02c5300
../sections/.rodata : 0xffffffffa02b6040
../sections/.strtab : 0xffffffffa02cea78
../sections/.symtab : 0xffffffffa02cc000
../sections/.text.unlikely : 0xffffffffa02b50c8
../sections/.gnu.linkonce.this_module : 0xffffffffa02c5320
../sections/.rodata.str1.1 : 0xffffffffa02c488c
../sections/.rodata.str1.8 : 0xffffffffa02c4948
../sections/__bug_table : 0xffffffffa02c4d84
../sections/.note.gnu.build-id : 0xffffffffa02b6000
../sections/.fixup : 0xffffffffa02b3c82
../sections/.exit.text : 0xffffffffa02b528b
../sections/.parainstructions : 0xffffffffa02c4580
../sections/.data..read_mostly : 0xffffffffa02c5301
../parameters/ept : N
../parameters/vpid : N
../parameters/enable_shadow_vmcs : N
../parameters/eptad : N
../parameters/flexpriority : N
../parameters/fasteoi : Y
../parameters/enable_apicv : N
../parameters/unrestricted_guest : N
../parameters/emulate_invalid_guest_state : Y
../parameters/nested : N
../parameters/vmm_exclusive : Y
../parameters/ple_window : 4096
../parameters/ple_gap : 0
../refcnt : 0
cat: ./uevent: Permission denied
../uevent :
../initsize : 0
root@Testbed:/sys/module/kvm_intel#
-- System Information:
Debian Release: jessie/sid
APT prefers testing-proposed-updates
APT policy: (500, 'testing-proposed-updates'), (500, 'unstable'), (500,
'testing')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 3.11-2-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages qemu-kvm depends on:
ii qemu-system-x86 1.7.0+dfsg-2
qemu-kvm recommends no packages.
qemu-kvm suggests no packages.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
