Bionic/xfstests ====== Original/patched kernel versions: $ grep -h ^PLATFORM xfstests.log.* | sort -u PLATFORM -- Linux/s390x mfo-s390x-bionic 4.15.0-122-generic #124+test20201026b1 SMP Mon Oct 26 13:10:37 -03 2020 PLATFORM -- Linux/s390x mfo-s390x-bionic 4.15.0-122-generic #124-Ubuntu SMP Thu Oct 15 13:02:05 UTC 2020
Original kernel: --- $ VERSION='4.15.0-122-generic #124-Ubuntu' Number of runs: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep -c Failures: 10 $ RUNS=10 Consistent fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w $RUNS 10 btrfs/186 10 btrfs/193 10 btrfs/200 10 btrfs/202 10 btrfs/204 10 btrfs/205 10 btrfs/214 10 btrfs/219 10 btrfs/220 10 btrfs/221 10 btrfs/222 10 generic/260 10 generic/286 10 generic/301 10 generic/371 10 generic/405 10 generic/407 10 generic/465 10 generic/480 10 generic/488 10 generic/502 10 generic/513 10 generic/526 10 generic/527 10 generic/546 10 generic/562 10 generic/565 Flaky fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w -v $RUNS 1 btrfs/028 1 generic/089 1 generic/464 2 generic/175 3 generic/476 4 generic/531 7 btrfs/078 8 btrfs/010 8 generic/074 8 generic/142 8 generic/143 9 generic/133 9 generic/475 Patched kernel: --- $ VERSION='4.15.0-122-generic #124+test20201026b1' Number of runs: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep -c Failures: 10 Consistent fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w $RUNS 10 btrfs/186 10 btrfs/193 10 btrfs/200 10 btrfs/202 10 btrfs/204 10 btrfs/205 10 btrfs/214 10 btrfs/219 10 btrfs/220 10 btrfs/221 10 btrfs/222 10 generic/260 10 generic/286 10 generic/301 10 generic/371 10 generic/405 10 generic/407 10 generic/465 10 generic/475 <<-- New but not regression; failed 9/10 in original kernel. 10 generic/480 10 generic/488 10 generic/502 10 generic/513 10 generic/526 10 generic/527 10 generic/546 10 generic/562 10 generic/565 Flaky fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w -v $RUNS 1 btrfs/028 1 generic/089 1 generic/102 1 generic/269 2 generic/166 2 generic/464 2 generic/476 4 btrfs/010 4 generic/074 4 generic/142 4 generic/143 4 generic/531 6 generic/133 8 generic/175 << Increased from 2/10 to 8/10 but it seems a regression from other btrfs patches; see Focal. 9 btrfs/078 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1902254 Title: Bionic: btrfs: kernel BUG at /build/linux- eTBZpZ/linux-4.15.0/fs/btrfs/ctree.c:3233! Status in linux package in Ubuntu: Fix Committed Status in linux source package in Bionic: In Progress Status in linux source package in Focal: In Progress Status in linux source package in Groovy: In Progress Bug description: [Impact] * Users of btrfs started hitting a kernel BUG() (below) after upgrade from 4.15.0-99.100 to 4.15.0-109.110, which has 55 btrfs changes. kernel BUG at /build/linux-eTBZpZ/linux-4.15.0/fs/btrfs/ctree.c:3233! ... Krnl PSW : 00000000be9cb874 00000000ef3786e8 (btrfs_set_item_key_safe+0x152/0x1c0 [btrfs]) ... [...] Call Trace: [...] btrfs_set_item_key_safe+0x11c/0x1c0 [btrfs]) [...] __btrfs_drop_extents+0xb5a/0xda8 [btrfs] [...] btrfs_log_changed_extents+0x35c/0xaf0 [btrfs] [...] btrfs_log_inode+0x9ee/0x1080 [btrfs] [...] btrfs_log_inode_parent+0x224/0xa10 [btrfs] [...] btrfs_log_dentry_safe+0x80/0xa8 [btrfs] [...] btrfs_sync_file+0x392/0x550 [btrfs] [...] do_fsync+0x5e/0x90 [...] SyS_fdatasync+0x32/0x48 [...] system_call+0xd8/0x2c8 $ git log --oneline Ubuntu-4.15.0-99.100..Ubuntu-4.15.0-109.110 -- fs/btrfs/ | wc -l 55 * The error happens at random moments, regardless of a particular activity/load. Workaround is to downgrade. [Fix] * This BUG()/function is addressed in patch 4/4 [1] of series 'btrfs: Enhanced runtime defence against fuzzed images' [2], after issues in the real world, not just crafted fs images: 'one internal report has hit one BUG_ON() with real world fs' kernel BUG at fs/btrfs/ctree.c:3188! ... RIP: 0010:btrfs_set_item_key_safe+0x16c/0x180 * The patch/set [3] is applied in v5.10-rc1 and Ubuntu Unstable: - d16c702fe4f2 btrfs: ctree: check key order before merging tree blocks - 07cce5cf3b48 btrfs: extent-tree: kill the BUG_ON() in insert_inline_extent_backref() - 1c2a07f598d5 btrfs: extent-tree: kill BUG_ON() in __btrfs_free_extent() - f98b6215d7d1 btrfs: extent_io: do extra check for extent buffer read write functions [Test Case] * There is working synthetic reproducer for this issue, which is hard to reproduce as reported in commit [4] that introduces debugging for the issue. * Regression tests with xfstests and stress-ng shows no regressions between un/patched kernels. [Other Info] * Trivial backports (only refreshing a few context lines) with 3 more dependency patches on Bionic and 1 on Focal. And Bionic needed one extra hunk to '#include' a header. Groovy all apply cleanly. [1] https://lore.kernel.org/linux-btrfs/20200819063550.62832-5-...@suse.com/ [2] https://lore.kernel.org/linux-btrfs/20200819063550.62832-1-...@suse.com/ [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d16c702fe4f274bd77b47d3ab737eadcf24e0b93 [4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7c15d41016dc886cc011e3854d855e219759ae68 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1902254/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp