Public bug reported:
We are occasionally running into the below qemu error on our Ubuntu16
KVM hypervisors managed using Openstack, where the qemu guest instance
gets paused unexpectedly.
pooja@kvm14:~$ sudo cat
/var/log/libvirt/qemu/f8a20654-4c96-4446-95b3-24b8d28fab7f.log.1
error: kvm run failed Bad address
RAX=0000000000000000 RBX=ffff8d12ffc01b00 RCX=0000000000000000
RDX=ffff8d14d111c040
RSI=000000000000000e RDI=ffff8d15bffda000 RBP=ffff8d156afbb898
RSP=ffff8d156afbb870
R8 =000000000000001e R9 =0000000000000000 R10=000000000000001d
R11=ffffffffffffffd0
R12=ffff8d14d111c000 R13=fffff7de0d444700 R14=ffff8d14d111c000
R15=0000000000000001
RIP=ffffffff9ba1911b RFL=00010082 [--S----] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 0000000000000000 000fffff 00000000
CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0000 0000000000000000 ffffffff 00c00000
DS =0000 0000000000000000 000fffff 00000000
FS =0000 00007f8c7e8ac700 000fffff 00000000
GS =0000 ffff8d15bfd00000 000fffff 00000000
LDT=0000 0000000000000000 000fffff 00000000
TR =0040 ffff8d15bfd04000 00002087 00008b00 DPL=0 TSS64-busy
GDT= ffff8d15bfd0c000 0000007f
IDT= ffffffffff528000 00000fff
CR0=80050033 CR2=000000001699e002 CR3=00000000b0d54000 CR4=003606e0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
DR3=0000000000000000
DR6=00000000fffe0ff0 DR7=0000000000000400
EFER=0000000000000d01
Code=44 39 f8 48 63 43 20 0f 8e e1 00 00 00 48 63 53 18 4c 01 e2 <49> 89 14 04
41 0f b7 55 1a 48 63 43 18 41 83 c7 01 66 81 e2 ff 7f 49 01 c4 0f b7 c2 44 39
2019-07-16T23:59:30.354240Z qemu-system-x86_64: terminating on signal 15 from
pid 7487 (/usr/sbin/libvirtd)
2019-07-16 23:59:31.549+0000: shutting down, reason=destroyed
We also saw some swap related errors in /var/log/syslog previously:
Jul 4 08:40:18 kvm14 kernel: \[8084318.769268] audit: type=1400
audit(1562229618.904:4385): apparmor="STATUS"
operation="profile_replace" profile="unconfined" name="libvirt-a89c8a67
-24ae-45cb-a885-8880b49f86fd" pid=53770 comm="apparmor_parser"
Jul 4 08:40:18 kvm14 libvirtd\[20274]: 2019-07-04 08:40:18.911+0000:
20277: warning : AppArmorSetFDLabel:1164 : could not find path for
descriptor /proc/self/fd/29, skippingJul 4 08:40:19 kvm13 kernel:
\[8084318.930088] print_req_error: critical medium error, dev nvme0n1,
sector 392710192
Jul 4 08:40:19 kvm14 kernel: \[8084318.937754] Read-error on swap-
device (259:0:392710200)
Jul 4 08:40:19 kvm14 kernel: \[8084318.943157] Read-error on swap-
device (259:0:392710208)
Wondering if these swap read errors happening intermittently, cause qemu
to pause the guest instance due to memory overcommitment issues.
pooja@kvm14:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial
pooja@kvm14:~$ /usr/bin/qemu-system-x86_64 --version
QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.10~cloud0)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
pooja@kvm14:~$ dpkg -l | grep libvirt
ii libvirt-bin 4.0.0-1ubuntu8.8~cloud0
amd64 programs for the libvirt library
Sample qemu process args for an instance created using Openstack Nova (Newton
release):
pooja@kvm14:~$ sudo ps -ef | grep qemu | tail -1
libvirt+ 52655 1 83 Jun27 ? 22-23:37:19 qemu-system-x86_64
-enable-kvm -name guest=157b8c3d-87c6-4a62-a05b-8d1aae47e890,debug-threads=on
-S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-567-157b8c3d-87c6-4a62-a/master-key.aes
-machine pc-i440fx-bionic,accel=kvm,usb=off,dump-guest-core=off -cpu
Skylake-Server-IBRS -m 8192 -realtime mlock=off -smp
2,sockets=2,cores=1,threads=1 -uuid 157b8c3d-87c6-4a62-a05b-8d1aae47e890
-smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack
Nova,version=2013.2.1,serial=1a2fd1a2-3df9-4ed9-ad7f-b4c2479a01ca,uuid=157b8c3d-87c6-4a62-a05b-8d1aae47e890,family=Virtual
Machine -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-567-157b8c3d-87c6-4a62-a/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew
-global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/dev/disk/by-path/ip-192.168.5.47:3260-iscsi-iqn.2013-05.com.daterainc:tc:01:sn:722e8e82b38019b5-lun-0,format=raw,if=none,id=drive-virtio-disk0,serial=1acefaf2-4096-4e85-9a39-aabf3841d022,cache=none,aio=native
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=53,id=hostnet0,vhost=on,vhostfd=56 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:cd:4e:e1,bus=pci.0,addr=0x3
-add-fd set=2,fd=58 -chardev file,id=charserial0,path=/dev/fdset/2,append=on
-device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1
-device isa-serial,chardev=charserial1,id=serial1 -device
usb-tablet,id=input0,bus=usb.0,port=1 -vnc 127.0.0.1:23 -k en-us -device
cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
** Affects: linux (Ubuntu)
Importance: Undecided
Status: Confirmed
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1837810
Title:
qemu instance gets paused with error: kvm run failed Bad address
Status in linux package in Ubuntu:
Confirmed
Bug description:
We are occasionally running into the below qemu error on our Ubuntu16
KVM hypervisors managed using Openstack, where the qemu guest instance
gets paused unexpectedly.
pooja@kvm14:~$ sudo cat
/var/log/libvirt/qemu/f8a20654-4c96-4446-95b3-24b8d28fab7f.log.1
error: kvm run failed Bad address
RAX=0000000000000000 RBX=ffff8d12ffc01b00 RCX=0000000000000000
RDX=ffff8d14d111c040
RSI=000000000000000e RDI=ffff8d15bffda000 RBP=ffff8d156afbb898
RSP=ffff8d156afbb870
R8 =000000000000001e R9 =0000000000000000 R10=000000000000001d
R11=ffffffffffffffd0
R12=ffff8d14d111c000 R13=fffff7de0d444700 R14=ffff8d14d111c000
R15=0000000000000001
RIP=ffffffff9ba1911b RFL=00010082 [--S----] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 0000000000000000 000fffff 00000000
CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0000 0000000000000000 ffffffff 00c00000
DS =0000 0000000000000000 000fffff 00000000
FS =0000 00007f8c7e8ac700 000fffff 00000000
GS =0000 ffff8d15bfd00000 000fffff 00000000
LDT=0000 0000000000000000 000fffff 00000000
TR =0040 ffff8d15bfd04000 00002087 00008b00 DPL=0 TSS64-busy
GDT= ffff8d15bfd0c000 0000007f
IDT= ffffffffff528000 00000fff
CR0=80050033 CR2=000000001699e002 CR3=00000000b0d54000 CR4=003606e0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
DR3=0000000000000000
DR6=00000000fffe0ff0 DR7=0000000000000400
EFER=0000000000000d01
Code=44 39 f8 48 63 43 20 0f 8e e1 00 00 00 48 63 53 18 4c 01 e2 <49> 89 14
04 41 0f b7 55 1a 48 63 43 18 41 83 c7 01 66 81 e2 ff 7f 49 01 c4 0f b7 c2 44 39
2019-07-16T23:59:30.354240Z qemu-system-x86_64: terminating on signal 15 from
pid 7487 (/usr/sbin/libvirtd)
2019-07-16 23:59:31.549+0000: shutting down, reason=destroyed
We also saw some swap related errors in /var/log/syslog previously:
Jul 4 08:40:18 kvm14 kernel: \[8084318.769268] audit: type=1400
audit(1562229618.904:4385): apparmor="STATUS"
operation="profile_replace" profile="unconfined" name="libvirt-
a89c8a67-24ae-45cb-a885-8880b49f86fd" pid=53770 comm="apparmor_parser"
Jul 4 08:40:18 kvm14 libvirtd\[20274]: 2019-07-04 08:40:18.911+0000:
20277: warning : AppArmorSetFDLabel:1164 : could not find path for
descriptor /proc/self/fd/29, skippingJul 4 08:40:19 kvm13 kernel:
\[8084318.930088] print_req_error: critical medium error, dev nvme0n1,
sector 392710192
Jul 4 08:40:19 kvm14 kernel: \[8084318.937754] Read-error on swap-
device (259:0:392710200)
Jul 4 08:40:19 kvm14 kernel: \[8084318.943157] Read-error on swap-
device (259:0:392710208)
Wondering if these swap read errors happening intermittently, cause
qemu to pause the guest instance due to memory overcommitment issues.
pooja@kvm14:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial
pooja@kvm14:~$ /usr/bin/qemu-system-x86_64 --version
QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.10~cloud0)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
pooja@kvm14:~$ dpkg -l | grep libvirt
ii libvirt-bin 4.0.0-1ubuntu8.8~cloud0
amd64 programs for the libvirt library
Sample qemu process args for an instance created using Openstack Nova (Newton
release):
pooja@kvm14:~$ sudo ps -ef | grep qemu | tail -1
libvirt+ 52655 1 83 Jun27 ? 22-23:37:19 qemu-system-x86_64
-enable-kvm -name guest=157b8c3d-87c6-4a62-a05b-8d1aae47e890,debug-threads=on
-S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-567-157b8c3d-87c6-4a62-a/master-key.aes
-machine pc-i440fx-bionic,accel=kvm,usb=off,dump-guest-core=off -cpu
Skylake-Server-IBRS -m 8192 -realtime mlock=off -smp
2,sockets=2,cores=1,threads=1 -uuid 157b8c3d-87c6-4a62-a05b-8d1aae47e890
-smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack
Nova,version=2013.2.1,serial=1a2fd1a2-3df9-4ed9-ad7f-b4c2479a01ca,uuid=157b8c3d-87c6-4a62-a05b-8d1aae47e890,family=Virtual
Machine -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-567-157b8c3d-87c6-4a62-a/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew
-global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/dev/disk/by-path/ip-192.168.5.47:3260-iscsi-iqn.2013-05.com.daterainc:tc:01:sn:722e8e82b38019b5-lun-0,format=raw,if=none,id=drive-virtio-disk0,serial=1acefaf2-4096-4e85-9a39-aabf3841d022,cache=none,aio=native
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=53,id=hostnet0,vhost=on,vhostfd=56 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:cd:4e:e1,bus=pci.0,addr=0x3
-add-fd set=2,fd=58 -chardev file,id=charserial0,path=/dev/fdset/2,append=on
-device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1
-device isa-serial,chardev=charserial1,id=serial1 -device
usb-tablet,id=input0,bus=usb.0,port=1 -vnc 127.0.0.1:23 -k en-us -device
cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1837810/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
