** Description changed:
== Justification ==
Running the ftrace in ubuntu_kernel_selftests repetitively against x86 Cosmic
kernel will cause system hang.
+ When this happens, you won't be able to ssh into this system, and no log
+ can be found in syslog.
+
This hang is caused by one of the sub-test: kprobe/multiple_kprobes
- Upstream discussion could be found here:
- https://lkml.org/lkml/2018/12/3/1219
+ Masami's comment from upstream discussion
(https://lkml.org/lkml/2018/12/3/1219):
+ In arch/x86/kernel/kprobes/opt.c, copy_optimized_instructions() does a copy
loop, but only update src and dest cursors, but not update real address which
is used for adjusting RIP relative instruction.
== Fix ==
43a1b0cb4 (kprobes/x86: Fix instruction patching corruption when copying more
than one RIP-relative instruction)
This patch is already in D.
- For B/C, they all have this ill-commit 63fef14 and this patch can be
cherry-picked. Note that for Bionic kernel it can only be triggered in this way
with kernel built with GCC-8.
+ For B/C, they all have this ill-commit 63fef14 and this patch can be
cherry-picked. Note that for Bionic kernel it can only be triggered in this way
with a kernel built with GCC-8.
For X, the ill-commit 63fef14 does not exist.
== Test ==
Test kernel for Cosmic and Bionic built with GCC-8:
http://people.canonical.com/~phlin/kernel/lp-1826385-ftrace-hang/
+ (To verify this for the Bionic, you will need to build a kernel with
+ GCC-8.)
+
Patch tested with a bare-metal and a KVM node, both of them can pass the
beating repetitively.
== Regression Potential ==
- Upstream fix limited to x86 architecture, and specific for kprobe.
+ Low, upstream fix specific for kprobe and limited to x86 architecture.
+
+ Although it's a bit difficult to trigger this on Bionic, I think it
+ worth this fix as it's quite straightforward.
+
== Original Bug Report ==
This issue is a bit strange.
The test has passed with Cosmic 4.18.0-18.19 generic kernel on AMD64
node during our SRU testing process.
https://pastebin.ubuntu.com/p/HN2vN6fCXn/
However, Tyler found that this test will hang after:
[30] Kretprobe dynamic event with maxactive [PASS]
[31] Register/unregister many kprobe events [PASS]
And this is 100% reproducible.
No relevant output in syslog.
This will need further investigation.
ProblemType: Bug
DistroRelease: Ubuntu 18.10
Package: linux-image-4.18.0-18-generic 4.18.0-18.19
ProcVersionSignature: User Name 4.18.0-18.19-generic 4.18.20
Uname: Linux 4.18.0-18-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Apr 25 07:16 seq
crw-rw---- 1 root audio 116, 33 Apr 25 07:16 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.10-0ubuntu13.2
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord':
'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
Date: Thu Apr 25 07:18:51 2019
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
PciMultimedia:
ProcFB: 0 cirrusdrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.18.0-18-generic
root=UUID=2f68c627-8ab4-40d5-8c06-6563436d0f96 ro
RelatedPackageVersions:
linux-restricted-modules-4.18.0-18-generic N/A
linux-backports-modules-4.18.0-18-generic N/A
linux-firmware 1.175.3
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: Ubuntu-1.8.2-1ubuntu1
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-xenial
dmi.modalias:
dmi:bvnSeaBIOS:bvrUbuntu-1.8.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-xenial:cvnQEMU:ct1:cvrpc-i440fx-xenial:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.version: pc-i440fx-xenial
dmi.sys.vendor: QEMU
** Also affects: linux (Ubuntu Bionic)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1826385
Title:
ftrace in ubuntu_kernel_selftests hang with Cosmic kernel
Status in ubuntu-kernel-tests:
In Progress
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Bionic:
In Progress
Status in linux source package in Cosmic:
In Progress
Bug description:
== Justification ==
Running the ftrace in ubuntu_kernel_selftests repetitively against x86 Cosmic
kernel will cause system hang.
When this happens, you won't be able to ssh into this system, and no
log can be found in syslog.
This hang is caused by one of the sub-test: kprobe/multiple_kprobes
Masami's comment from upstream discussion
(https://lkml.org/lkml/2018/12/3/1219):
In arch/x86/kernel/kprobes/opt.c, copy_optimized_instructions() does a copy
loop, but only update src and dest cursors, but not update real address which
is used for adjusting RIP relative instruction.
== Fix ==
43a1b0cb4 (kprobes/x86: Fix instruction patching corruption when copying more
than one RIP-relative instruction)
This patch is already in D.
For B/C, they all have this ill-commit 63fef14 and this patch can be
cherry-picked. Note that for Bionic kernel it can only be triggered in this way
with a kernel built with GCC-8.
Although it's a bit difficult to trigger this on Bionic, I think it
worth this fix as it's quite straightforward.
For X, the ill-commit 63fef14 does not exist.
== Test ==
Test kernel for Cosmic and Bionic built with GCC-8:
http://people.canonical.com/~phlin/kernel/lp-1826385-ftrace-hang/
(To verify this for the Bionic, you will need to build a kernel with
GCC-8.)
Patch tested with a bare-metal and a KVM node, both of them can pass
the beating repetitively.
== Regression Potential ==
Low, upstream fix specific for kprobe and limited to x86 architecture.
== Original Bug Report ==
This issue is a bit strange.
The test has passed with Cosmic 4.18.0-18.19 generic kernel on AMD64
node during our SRU testing process.
https://pastebin.ubuntu.com/p/HN2vN6fCXn/
However, Tyler found that this test will hang after:
[30] Kretprobe dynamic event with maxactive [PASS]
[31] Register/unregister many kprobe events [PASS]
And this is 100% reproducible.
No relevant output in syslog.
This will need further investigation.
ProblemType: Bug
DistroRelease: Ubuntu 18.10
Package: linux-image-4.18.0-18-generic 4.18.0-18.19
ProcVersionSignature: User Name 4.18.0-18.19-generic 4.18.20
Uname: Linux 4.18.0-18-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Apr 25 07:16 seq
crw-rw---- 1 root audio 116, 33 Apr 25 07:16 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.10-0ubuntu13.2
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord':
'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
Date: Thu Apr 25 07:18:51 2019
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
PciMultimedia:
ProcFB: 0 cirrusdrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.18.0-18-generic
root=UUID=2f68c627-8ab4-40d5-8c06-6563436d0f96 ro
RelatedPackageVersions:
linux-restricted-modules-4.18.0-18-generic N/A
linux-backports-modules-4.18.0-18-generic N/A
linux-firmware 1.175.3
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: Ubuntu-1.8.2-1ubuntu1
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-xenial
dmi.modalias:
dmi:bvnSeaBIOS:bvrUbuntu-1.8.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-xenial:cvnQEMU:ct1:cvrpc-i440fx-xenial:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.version: pc-i440fx-xenial
dmi.sys.vendor: QEMU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1826385/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
