** Description changed: == Justification == Running the ftrace in ubuntu_kernel_selftests repetitively against x86 Cosmic kernel will cause system hang. + When this happens, you won't be able to ssh into this system, and no log + can be found in syslog. + This hang is caused by one of the sub-test: kprobe/multiple_kprobes - Upstream discussion could be found here: - https://lkml.org/lkml/2018/12/3/1219 + Masami's comment from upstream discussion (https://lkml.org/lkml/2018/12/3/1219): + In arch/x86/kernel/kprobes/opt.c, copy_optimized_instructions() does a copy loop, but only update src and dest cursors, but not update real address which is used for adjusting RIP relative instruction. == Fix == 43a1b0cb4 (kprobes/x86: Fix instruction patching corruption when copying more than one RIP-relative instruction) This patch is already in D. - For B/C, they all have this ill-commit 63fef14 and this patch can be cherry-picked. Note that for Bionic kernel it can only be triggered in this way with kernel built with GCC-8. + For B/C, they all have this ill-commit 63fef14 and this patch can be cherry-picked. Note that for Bionic kernel it can only be triggered in this way with a kernel built with GCC-8. For X, the ill-commit 63fef14 does not exist. == Test == Test kernel for Cosmic and Bionic built with GCC-8: http://people.canonical.com/~phlin/kernel/lp-1826385-ftrace-hang/ + (To verify this for the Bionic, you will need to build a kernel with + GCC-8.) + Patch tested with a bare-metal and a KVM node, both of them can pass the beating repetitively. == Regression Potential == - Upstream fix limited to x86 architecture, and specific for kprobe. + Low, upstream fix specific for kprobe and limited to x86 architecture. + + Although it's a bit difficult to trigger this on Bionic, I think it + worth this fix as it's quite straightforward. + == Original Bug Report == This issue is a bit strange. The test has passed with Cosmic 4.18.0-18.19 generic kernel on AMD64 node during our SRU testing process. https://pastebin.ubuntu.com/p/HN2vN6fCXn/ However, Tyler found that this test will hang after: [30] Kretprobe dynamic event with maxactive [PASS] [31] Register/unregister many kprobe events [PASS] And this is 100% reproducible. No relevant output in syslog. This will need further investigation. ProblemType: Bug DistroRelease: Ubuntu 18.10 Package: linux-image-4.18.0-18-generic 4.18.0-18.19 ProcVersionSignature: User Name 4.18.0-18.19-generic 4.18.20 Uname: Linux 4.18.0-18-generic x86_64 AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Apr 25 07:16 seq crw-rw---- 1 root audio 116, 33 Apr 25 07:16 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.10-0ubuntu13.2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Date: Thu Apr 25 07:18:51 2019 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: QEMU Standard PC (i440FX + PIIX, 1996) PciMultimedia: ProcFB: 0 cirrusdrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.18.0-18-generic root=UUID=2f68c627-8ab4-40d5-8c06-6563436d0f96 ro RelatedPackageVersions: linux-restricted-modules-4.18.0-18-generic N/A linux-backports-modules-4.18.0-18-generic N/A linux-firmware 1.175.3 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: Ubuntu-1.8.2-1ubuntu1 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-xenial dmi.modalias: dmi:bvnSeaBIOS:bvrUbuntu-1.8.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-xenial:cvnQEMU:ct1:cvrpc-i440fx-xenial: dmi.product.name: Standard PC (i440FX + PIIX, 1996) dmi.product.version: pc-i440fx-xenial dmi.sys.vendor: QEMU
** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1826385 Title: ftrace in ubuntu_kernel_selftests hang with Cosmic kernel Status in ubuntu-kernel-tests: In Progress Status in linux package in Ubuntu: Fix Released Status in linux source package in Bionic: In Progress Status in linux source package in Cosmic: In Progress Bug description: == Justification == Running the ftrace in ubuntu_kernel_selftests repetitively against x86 Cosmic kernel will cause system hang. When this happens, you won't be able to ssh into this system, and no log can be found in syslog. This hang is caused by one of the sub-test: kprobe/multiple_kprobes Masami's comment from upstream discussion (https://lkml.org/lkml/2018/12/3/1219): In arch/x86/kernel/kprobes/opt.c, copy_optimized_instructions() does a copy loop, but only update src and dest cursors, but not update real address which is used for adjusting RIP relative instruction. == Fix == 43a1b0cb4 (kprobes/x86: Fix instruction patching corruption when copying more than one RIP-relative instruction) This patch is already in D. For B/C, they all have this ill-commit 63fef14 and this patch can be cherry-picked. Note that for Bionic kernel it can only be triggered in this way with a kernel built with GCC-8. Although it's a bit difficult to trigger this on Bionic, I think it worth this fix as it's quite straightforward. For X, the ill-commit 63fef14 does not exist. == Test == Test kernel for Cosmic and Bionic built with GCC-8: http://people.canonical.com/~phlin/kernel/lp-1826385-ftrace-hang/ (To verify this for the Bionic, you will need to build a kernel with GCC-8.) Patch tested with a bare-metal and a KVM node, both of them can pass the beating repetitively. == Regression Potential == Low, upstream fix specific for kprobe and limited to x86 architecture. == Original Bug Report == This issue is a bit strange. The test has passed with Cosmic 4.18.0-18.19 generic kernel on AMD64 node during our SRU testing process. https://pastebin.ubuntu.com/p/HN2vN6fCXn/ However, Tyler found that this test will hang after: [30] Kretprobe dynamic event with maxactive [PASS] [31] Register/unregister many kprobe events [PASS] And this is 100% reproducible. No relevant output in syslog. This will need further investigation. ProblemType: Bug DistroRelease: Ubuntu 18.10 Package: linux-image-4.18.0-18-generic 4.18.0-18.19 ProcVersionSignature: User Name 4.18.0-18.19-generic 4.18.20 Uname: Linux 4.18.0-18-generic x86_64 AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Apr 25 07:16 seq crw-rw---- 1 root audio 116, 33 Apr 25 07:16 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.10-0ubuntu13.2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Date: Thu Apr 25 07:18:51 2019 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: QEMU Standard PC (i440FX + PIIX, 1996) PciMultimedia: ProcFB: 0 cirrusdrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.18.0-18-generic root=UUID=2f68c627-8ab4-40d5-8c06-6563436d0f96 ro RelatedPackageVersions: linux-restricted-modules-4.18.0-18-generic N/A linux-backports-modules-4.18.0-18-generic N/A linux-firmware 1.175.3 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: Ubuntu-1.8.2-1ubuntu1 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-xenial dmi.modalias: dmi:bvnSeaBIOS:bvrUbuntu-1.8.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-xenial:cvnQEMU:ct1:cvrpc-i440fx-xenial: dmi.product.name: Standard PC (i440FX + PIIX, 1996) dmi.product.version: pc-i440fx-xenial dmi.sys.vendor: QEMU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1826385/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp