For 1. I mostly agree, the default is currently off in code and in 2.11 there
is this for backwards compatability:
smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_ON;
We would have to
- Moving that "keep the old default" entry to 2.10 (to cover <=2.10)
spapr_machine_2_10_class_options
smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_ON;
- And we would set it explicit off in 2.11 (which is what pseries-bionic refers
to)
spapr_machine_2_11_class_options
smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF;
- 2.12 we would not change IMHO, that might become whatever it becomes with
2.12 development
For #2 - this is a bug fix so it does not fall under the Feature Freeze
(tomorrow).
But I don't know how much lead time the kernel team needs.
Given that kernel fixes are involved this clearly needs a kernel task for them
to know about - adding ...
@Kernel - please read the context - what is the last date you'd need to
have this commit upstream by IBM to be able to pick it and still be in
the initial 18.04 release kernel (not in -updates)?
@IBM - how about this approach:
A) We switch the default to HTM=off in qemu "now" (as soon as you ack this) to
be safe
B) If you get the kernel fixes upstream fast enough for the kernel Team to pick
up in time:
B1) a fixed kernel will be pushed (before 18.04 release)
B2) we unroll this change in qemu (before 18.04 release)
That way we would surely have something that "works" by default via (A) and if
(B) is in time we can switch back to "working but with HTM enabled".
And if (B) is too late we will keep HTM disabled in the 2.11/Bionic machine
type.
** Also affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Changed in: qemu (Ubuntu)
Status: New => Incomplete
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1752026
Title:
Ubuntu18.04:POWER9:DD2.2 - Unable to start a KVM guest with default
machine type(pseries-bionic) complaining "KVM implementation does not
support Transactional Memory, try cap-htm=off" (kvm)
Status in The Ubuntu-power-systems project:
Incomplete
Status in linux package in Ubuntu:
New
Status in qemu package in Ubuntu:
Incomplete
Bug description:
== Comment: #0 - Satheesh Rajendran <[email protected]> - 2018-02-23
08:31:06 ==
---Problem Description---
libvirt unable to start a KVM guest complaining about cap-htm machine
property to be off
Host Env:
# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 160
On-line CPU(s) list: 0-159
Thread(s) per core: 4
Core(s) per socket: 20
Socket(s): 2
NUMA node(s): 2
Model: 2.2 (pvr 004e 1202)
Model name: POWER9 (raw), altivec supported
CPU max MHz: 3800.0000
CPU min MHz: 2166.0000
L1d cache: 32K
L1i cache: 32K
L2 cache: 512K
L3 cache: 10240K
NUMA node0 CPU(s): 0-79
NUMA node8 CPU(s): 80-159
ii qemu-kvm 1:2.11+dfsg-1ubuntu2
ppc64el QEMU Full virtualization on x86 hardware
ii libvirt-bin 4.0.0-1ubuntu3
ppc64el programs for the libvirt library
# lsmcode
Version of System Firmware :
Product Name : OpenPOWER Firmware
Product Version : open-power-SUPERMICRO-P9DSU-V1.03-20180205-imp
Product Extra : occ-577915f
Product Extra : skiboot-v5.9-240-g081882690163-pcbedce4
Product Extra : petitboot-v1.6.6-p019c87e
Product Extra : sbe-095e608
Product Extra : machine-xml-fb5f933
Product Extra : hostboot-9bfb201
Product Extra : linux-4.14.13-openpower1-p78d7eee
Contact Information = [email protected]
---uname output---
4.15.0-10-generic
Machine Type = power9 boston 2.2 (pvr 004e 1202)
---Debugger---
A debugger is not configured
---Steps to Reproduce---
1. Boot a guest from libvirt with default pseries machine type or
pseries-bionic
/usr/bin/virt-install --connect=qemu:///system --hvm --accelerate --name
'virt-tests-vm1' --machine pseries --memory=32768
--vcpu=32,sockets=1,cores=32,threads=1 --import --nographics --serial pty
--memballoon model=virtio --controller type=scsi,model=virtio-scsi --disk
path=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,bus=scsi,size=10,format=qcow2
--network=bridge=virbr0,model=virtio,mac=52:54:00:77:78:79 --noautoconsole
WARNING No operating system detected, VM performance may suffer. Specify an
OS with --os-variant for optimal results.
Starting install...
ERROR internal error: process exited while connecting to monitor:
,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26
-device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1
-chardev pty,id=charserial0 -device
spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on
2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not
support Transactional Memory, try cap-htm=off
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh --connect qemu:///system start virt-tests-vm1
otherwise, please restart your installation.
2. Fails to boot..
Note: if we specify machine type as pseries=2.12 it boots fine like
below
/usr/bin/virt-install --connect=qemu:///system --hvm --accelerate --name
'virt-tests-vm1' --machine pseries-2.12 --memory=32768
--vcpu=32,sockets=1,cores=32,threads=1 --import --nographics --serial pty
--memballoon model=virtio --controller type=scsi,model=virtio-scsi --disk
path=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,bus=scsi,size=10,format=qcow2
--network=bridge=virbr0,model=virtio,mac=52:54:00:77:78:79 --noautoconsole
WARNING No operating system detected, VM performance may suffer. Specify an
OS with --os-variant for optimal results.
qemu-cmd line:
libvirt+ 4283 1 99 09:26 ? 00:00:38 qemu-system-ppc64
-enable-kvm -name guest=virt-tests-vm1,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4
-virt-tests-vm1/master-key.aes -machine pseries-2.12,accel=kvm,usb=off
,dump-guest-core=off -m 32768 -realtime mlock=off -smp
32,sockets=1,cores=32,threads=1 -uuid
108ac2b5-e8b2-4399-a925-a707e8020871 -display none -no-user-config
-nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-virt-tests-
vm1/monitor.sock,server,nowait -mon
chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device
virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive
file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-
wrapper/data/avocado-
vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive-
scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-
id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev
tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-
pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1
-chardev pty,id=charserial0 -device spapr-
vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-
balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on
Userspace tool common name: ii libvirt-bin
4.0.0-1ubuntu3 ppc64el programs for the libvirt
library
The userspace tool has the following bit modes: both
Userspace rpm: ii libvirt-bin
4.0.0-1ubuntu3 ppc64el programs for the
libvirt library
Userspace tool obtained from project website: na
*Additional Instructions for [email protected]:
-Post a private note with access information to the machine that the bug is
occuring on.
-Attach ltrace and strace of userspace application.
== Comment: #1 - Satheesh Rajendran <[email protected]> - 2018-02-23
08:35:17 ==
vm qemu log for failed and passed cases:
Failed:(pseries-bionic)
2018-02-23 14:21:10.806+0000: starting up libvirt version: 4.0.0, package:
1ubuntu3 (Christian Ehrhardt <[email protected]> Mon, 19 Feb
2018 14:18:44 +0100), qemu version: 2.11.1(Debian 1:2.11+dfsg-1ubuntu2),
hostname: ltc-boston8.aus.stglabs.ibm.com
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
QEMU_AUDIO_DRV=none /usr/bin/kvm -name guest=virt-tests-vm1,debug-threads=on -S
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-virt-tests-vm1/master-key.aes
-machine pseries-bionic,accel=kvm,usb=off,dump-guest-core=off -m 32768
-realtime mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid
36c37d3b-fb24-4350-94f9-3271b257f75c -display none -no-user-config -nodefaults
-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-virt-tests-vm1/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device
virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive
file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
-netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1
-chardev pty,id=charserial0 -device
spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on
2018-02-23T14:21:10.909242Z qemu-system-ppc64: -chardev pty,id=charserial0:
char device redirected to /dev/pts/1 (label charserial0)
2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not
support Transactional Memory, try cap-htm=off
2018-02-23 14:21:18.857+0000: shutting down, reason=failed
Passed:(pseries-2.12)
2018-02-23 14:26:07.047+0000: starting up libvirt version: 4.0.0, package:
1ubuntu3 (Christian Ehrhardt <[email protected]> Mon, 19 Feb
2018 14:18:44 +0100), qemu version: 2.11.1(Debian 1:2.11+dfsg-1ubuntu2),
hostname: ltc-boston8.aus.stglabs.ibm.com
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
QEMU_AUDIO_DRV=none /usr/bin/kvm -name guest=virt-tests-vm1,debug-threads=on -S
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-virt-tests-vm1/master-key.aes
-machine pseries-2.12,accel=kvm,usb=off,dump-guest-core=off -m 32768 -realtime
mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid
108ac2b5-e8b2-4399-a925-a707e8020871 -display none -no-user-config -nodefaults
-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-virt-tests-vm1/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device
virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive
file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
-netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1
-chardev pty,id=charserial0 -device
spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on
2018-02-23T14:26:07.116991Z qemu-system-ppc64: -chardev pty,id=charserial0:
char device redirected to /dev/pts/1 (label charserial0)
Regards,
-Satheesh
== Comment: #8 - VIPIN K. PARASHAR <[email protected]> - 2018-02-25
23:38:29 ==
Starting install...
ERROR internal error: process exited while connecting to monitor:
,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26
-device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1
-chardev pty,id=charserial0 -device
spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on
2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not
support Transactional Memory, try cap-htm=off
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh --connect qemu:///system start virt-tests-vm1
otherwise, please restart your installation.
As per above message, qemu is reporting TM to be not supported by KVM on this
hardware
and thus recommending to turn off cap-htm.
== Comment: #12 - Suraj Jitindar Singh <[email protected]> - 2018-02-26
23:35:02 ==
I don't know what a pseries-bionic is, there is no reference to it upstream.
What you are seeing is expected behaviour as far as I can tell. POWER9
currently does not support HTM for a guest and thus it must not be
turned on, otherwise qemu will fail to start.
HTM can be disabled from the qemu command line by setting cap-htm=off,
as stated in the the error message. The pseries-2.12 machine type has
htm disabled by default and thus with that machine type there is no
requirement to set cap-htm=off on the command line to get qemu to
start.
So depending on what machine pseries-bionic is based on it will be
required to disable htm on the command line (with cap-htm=off) if it
is not disabled by default for the machine.
== Comment: #13 - Satheesh Rajendran <[email protected]> - 2018-02-27
00:05:12 ==
Had a chat with Suraj and here is the summary
1. Currently Power9 DD2.2(host kernel) does not support HTM, so guest should
be booted with cap-htm=off, looks like host kernel patch rework in progress-->
Initial patch, https://www.spinics.net/lists/kvm-ppc/msg13378.html
2. Libvirt does not know about this cap-htm yet and currently it does not set
any default values?
3. pseries-2.12 does have cap-htm=off by default but not the older machine
types, so we see the guest is booting from libvirt with pseries-2.12
4. Once 1 is fixed, we can boot cap-htm=on, I guess by that time pseries-2.12
to be changed to cap-htm=on bydefault.
---> 3. Immediate fix can be Canonical defaults their machine
type(pseries-bioic) to pseries-2.12...
---> Future 1 and 4 to be addressed, not sure about 2?
Needs a mirror to Canonical to address this.
Regards,
-Satheesh
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1752026/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : [email protected]
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
