Test result for B-4.15 before this patch set. ** Description changed:
+ == Justification == + In the ubuntu_kernel_selftests, the skipped test will uses return code 4 (KSFT_SKIP). + + However the code to handle this non-zero return code was not implemented + in the kselftest framework. And this will generate some false-positives + results as those skipped tests were treated as a failure. + + For example the raw_skew test in timers on 4.15: + 1..0 # Skipped: The clock was adjusted externally. Shutdown NTPd or other time sync daemons + not ok 1..7 selftests: raw_skew [FAIL] + + And the test_user_copy.sh test in user: + user: module test_user_copy is not found [SKIP] + not ok 1..1 selftests: test_user_copy.sh [FAIL] + + == Fix == + 771cbc3b (selftests: lib.mk set KSFT_TAP_LEVEL to prevent nested TAP headers) + a3355440 (selftests: Fix lib.mk run_tests target shell script) + 42b44c34 (selftests: lib.mk: cleanup RUN_TESTS define and make it readable) + 3f4435b5 (selftests: lib.mk: add SKIP handling to RUN_TESTS define) + + All these can be cherry-picked into Bionic. + + The crucial patch is commit 3f4435b5, the other three patches just paves + the road for it to be cherry-picked. + + == Test == + Tested on a Bionic amd64 node, the result shows positive outcome. + + A test summary before the patch: + END GOOD ubuntu_kernel_selftests.setup + END GOOD ubuntu_kernel_selftests.breakpoints + END GOOD ubuntu_kernel_selftests.cpu-hotplug + END GOOD ubuntu_kernel_selftests.efivarfs + END GOOD ubuntu_kernel_selftests.memfd + END GOOD ubuntu_kernel_selftests.memory-hotplug + END GOOD ubuntu_kernel_selftests.mount + END GOOD ubuntu_kernel_selftests.net + END GOOD ubuntu_kernel_selftests.ptrace + END GOOD ubuntu_kernel_selftests.seccomp + END ERROR ubuntu_kernel_selftests.timers + END GOOD ubuntu_kernel_selftests.powerpc + END GOOD ubuntu_kernel_selftests.user + END GOOD ubuntu_kernel_selftests.ftrace + + A test summary after the patch: + END GOOD ubuntu_kernel_selftests.setup + END GOOD ubuntu_kernel_selftests.breakpoints + END GOOD ubuntu_kernel_selftests.cpu-hotplug + END GOOD ubuntu_kernel_selftests.efivarfs + END GOOD ubuntu_kernel_selftests.memfd + END GOOD ubuntu_kernel_selftests.memory-hotplug + END GOOD ubuntu_kernel_selftests.mount + END GOOD ubuntu_kernel_selftests.net + END GOOD ubuntu_kernel_selftests.ptrace + END GOOD ubuntu_kernel_selftests.seccomp + END GOOD ubuntu_kernel_selftests.timers + END GOOD ubuntu_kernel_selftests.powerpc + END GOOD ubuntu_kernel_selftests.user + END GOOD ubuntu_kernel_selftests.ftrac + + The raw_skew test in timers is now marked as [SKIPPED] and no longer + causing failure. + + A complete test report could be found in the bug report attachment. + + == Regression Potential == + Low, this patch set just added support to handle the skip return code and changes are limited to the kselftest framework itself, no actual impact to normal users. + + In the selftests tool, we are using a KSFT_SKIP=4 return code for those skipped tests in the latest upstream kernel tree. But the corresponding code to take care of this return value does not exist in Bionic and Xenial kernel, this will cause the test still being marked as failed: selftests: test_user_copy.sh ======================================== user: module test_user_copy is not found [SKIP] not ok 1..1 selftests: test_user_copy.sh [FAIL] ** Attachment added: "B-client.DEBUG" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1812352/+attachment/5303235/+files/B-client.DEBUG -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1812352 Title: Handle the skip return code in kernel_selftests Status in ubuntu-kernel-tests: In Progress Status in linux package in Ubuntu: Confirmed Status in linux source package in Trusty: Invalid Status in linux source package in Xenial: New Status in linux source package in Bionic: In Progress Bug description: == Justification == In the ubuntu_kernel_selftests, the skipped test will uses return code 4 (KSFT_SKIP). However the code to handle this non-zero return code was not implemented in the kselftest framework. And this will generate some false-positives results as those skipped tests were treated as a failure. For example the raw_skew test in timers on 4.15: 1..0 # Skipped: The clock was adjusted externally. Shutdown NTPd or other time sync daemons not ok 1..7 selftests: raw_skew [FAIL] And the test_user_copy.sh test in user (failure could be found in KVM kernels): user: module test_user_copy is not found [SKIP] not ok 1..1 selftests: test_user_copy.sh [FAIL] == Fix == 771cbc3b (selftests: lib.mk set KSFT_TAP_LEVEL to prevent nested TAP headers) a3355440 (selftests: Fix lib.mk run_tests target shell script) 42b44c34 (selftests: lib.mk: cleanup RUN_TESTS define and make it readable) 3f4435b5 (selftests: lib.mk: add SKIP handling to RUN_TESTS define) All these can be cherry-picked into Bionic. The crucial patch is commit 3f4435b5, the other three patches just paves the road for it to be cherry-picked. == Test == Tested on a Bionic amd64 node, the result shows positive outcome. A test summary before the patch: END GOOD ubuntu_kernel_selftests.setup END GOOD ubuntu_kernel_selftests.breakpoints END GOOD ubuntu_kernel_selftests.cpu-hotplug END GOOD ubuntu_kernel_selftests.efivarfs END GOOD ubuntu_kernel_selftests.memfd END GOOD ubuntu_kernel_selftests.memory-hotplug END GOOD ubuntu_kernel_selftests.mount END GOOD ubuntu_kernel_selftests.net END GOOD ubuntu_kernel_selftests.ptrace END GOOD ubuntu_kernel_selftests.seccomp END ERROR ubuntu_kernel_selftests.timers END GOOD ubuntu_kernel_selftests.powerpc END GOOD ubuntu_kernel_selftests.user END GOOD ubuntu_kernel_selftests.ftrace A test summary after the patch: END GOOD ubuntu_kernel_selftests.setup END GOOD ubuntu_kernel_selftests.breakpoints END GOOD ubuntu_kernel_selftests.cpu-hotplug END GOOD ubuntu_kernel_selftests.efivarfs END GOOD ubuntu_kernel_selftests.memfd END GOOD ubuntu_kernel_selftests.memory-hotplug END GOOD ubuntu_kernel_selftests.mount END GOOD ubuntu_kernel_selftests.net END GOOD ubuntu_kernel_selftests.ptrace END GOOD ubuntu_kernel_selftests.seccomp END GOOD ubuntu_kernel_selftests.timers END GOOD ubuntu_kernel_selftests.powerpc END GOOD ubuntu_kernel_selftests.user END GOOD ubuntu_kernel_selftests.ftrac The raw_skew test in timers is now marked as [SKIPPED] and no longer causing failure. A complete test report could be found in the bug report attachment. == Regression Potential == Low, this patch set just added support to handle the skip return code and changes are limited to the kselftest framework itself, no actual impact to normal users. In the selftests tool, we are using a KSFT_SKIP=4 return code for those skipped tests in the latest upstream kernel tree. But the corresponding code to take care of this return value does not exist in Bionic and Xenial kernel, this will cause the test still being marked as failed: selftests: test_user_copy.sh ======================================== user: module test_user_copy is not found [SKIP] not ok 1..1 selftests: test_user_copy.sh [FAIL] To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1812352/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : [email protected] Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
