** Description changed:
+ SRU Justification
+
+ [Impact]
+
+ Hot removal of disks under kvm on s390 does not result in the kernel
+ removing the block device, which can lead to hung tasks and other
+ issues.
+
+ [Test Plan]
+
+ See steps to reproduce the bug in the original description below. To
+ test, execute these steps and confirm that the block device gets removed
+ as expected.
+
+ [Where problems could occur]
+
+ The fix is a revert of the changes which introduced this regression. The
+ original commit was a removal of supposedly unused code, but it seems a
+ mistake was made in the logic around unregistering of disks. Reverting
+ the changes could have potential to introduce bugs related to other virt
+ devices, especially if it interacts badly with subsequent driver
+ changes. However, the patch reverted cleanly, and reverting restores the
+ code to the state which has been working well in previous kernels and
+ seems like the lowest risk option until a proper fix is available
+ upstream.
+
+ ---
+
Repro:
#1 Get a guest
$ uvt-kvm create --disk 5 --password=ubuntu h release=hirsute arch=s390x
label=daily
$ uvt-kvm wait h release=hirsute arch=s390x label=daily
#2 Attach and Detach disk
$ sudo qemu-img create -f qcow2 /var/lib/libvirt/images/test.qcow2 10M
$ virsh attach-disk h /var/lib/libvirt/images/test.qcow2 vdc
$ virsh detach-disk h vdc
-
From libvirts POV it is gone at this point
$ virsh domblklist h
Target Source
------------------------------------------------------------------
vda /var/lib/uvtool/libvirt/images/hirsute-2nd-zfs.qcow
vdb /var/lib/uvtool/libvirt/images/hirsute-2nd-zfs-ds.qcow
But the guest thinks still it is present
$ uvt-kvm ssh --insecure hirsute-2nd-zfs lsblk
...
vdc 252:32 0 20M 0 disk
This even remains a while after (not a race).
Any access to it in the guest will hang (as you'd expect of a non-existing
blockdev)
4 0 1758 1739 20 0 12140 4800 - S+ pts/0 0:00 |
\_ sudo mkfs.ext4 /dev/vdc
4 0 1759 1758 20 0 6924 1044 - D+ pts/0 0:00 |
\_ mkfs.ext4 /dev/vdc
The result above was originally found with hirsute-guest@hirsute-host on
s390x
I do NOT see the same with groovy-guest@hirsute-host on s390x
I DO see the same with hirsute-guest@groovy-host on s390x
=> Guest version dependent not Host/Hipervisor dependent
I DO see the same with ZFS disks AND LVM disks being added&removed
=> not type dependent
I do NOT see the same on x86.
=> Arch dependent ??
... the evidence slowly points towards an issue in the guest, damn we are so
close to release - but non-fully detaching disks are critical in my POV :-/
Filing this as-is for awareness, but certainly this will need more debugging.
Unsure where this is going to eventually I'll now file it for
kernel/udev/systemd.
If there are any known issues/components that are related let me know please!
- ---
+ ---
ProblemType: Bug
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code
2: ls: cannot access '/dev/snd/': No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu65
Architecture: s390x
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
CRDA: N/A
CasperMD5CheckResult: unknown
DistroRelease: Ubuntu 21.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lspci:
-
+
Lspci-vt: -[0000:00]-
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Lsusb-t: Error: command ['lsusb', '-t'] failed with exit code 1:
/sys/bus/usb/devices: No such file or directory
Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1:
Package: udev
PackageArchitecture: s390x
PciMultimedia:
-
+
ProcFB:
-
+
ProcKernelCmdLine: root=LABEL=cloudimg-rootfs
ProcVersionSignature: User Name 5.11.0-14.15-generic 5.11.12
RelatedPackageVersions:
- linux-restricted-modules-5.11.0-14-generic N/A
- linux-backports-modules-5.11.0-14-generic N/A
- linux-firmware N/A
+ linux-restricted-modules-5.11.0-14-generic N/A
+ linux-backports-modules-5.11.0-14-generic N/A
+ linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: hirsute uec-images
Uname: Linux 5.11.0-14-generic s390x
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm audio cdrom dialout dip floppy lxd netdev plugdev sudo video
_MarkForUpload: True
acpidump:
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1925211
Title:
Hot-unplug of disks leaves broken block devices around in Hirsute on
s390x
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1925211/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
