Re: [PATCH v3 1/2] kunit: support failure from dynamic analysis tools

2021-02-16 Thread Daniel Latypov
On Thu, Feb 11, 2021 at 1:33 PM 'Brendan Higgins' via KUnit Development wrote: > > On Thu, Feb 11, 2021 at 12:58 PM Daniel Latypov wrote: > > > > On Thu, Feb 11, 2021 at 7:40 AM Alan Maguire > > wrote: > > > > > > On Thu, 11 Feb 2021, David G

Re: [PATCH v3 1/2] kunit: add a KUnit test for SLUB debugging functionality

2021-04-08 Thread Daniel Latypov
On Thu, Apr 8, 2021 at 3:30 AM Marco Elver wrote: > > On Tue, 6 Apr 2021 at 12:57, Vlastimil Babka wrote: > > > > > > On 4/1/21 11:24 PM, Marco Elver wrote: > > > On Thu, 1 Apr 2021 at 21:04, Daniel Latypov wrote: > > >> > } > >

[PATCH v4] lib: add basic KUnit test for lib/math

2021-04-08 Thread Daniel Latypov
meterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy Signed-off-by: Daniel Latypov --- Changes since v3: * fix `checkpatch.pl --strict` warnings * add test cases for gcd(0,0) and lcm(0,0) * minor: don&#x

Re: [PATCH v3 1/2] kunit: support failure from dynamic analysis tools

2021-02-11 Thread Daniel Latypov
On Thu, Feb 11, 2021 at 7:40 AM Alan Maguire wrote: > > On Thu, 11 Feb 2021, David Gow wrote: > > > On Wed, Feb 10, 2021 at 6:14 AM Daniel Latypov wrote: > > > > > > From: Uriel Guajardo > > > > > > Add a kunit_fail_current_test() function to

[PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-02-22 Thread Daniel Latypov
ext4 Also add in such a fragment for kunit itself so we can give that as an example more close to home (and thus less likely to be accidentally broken). [1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/ Signed-off-by: Daniel Latypov --- lib/kunit/.kunitconfig | 3

Re: [PATCH] ext4: add .kunitconfig fragment to enable ext4-specific tests

2021-02-22 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 6:33 PM Theodore Ts'o wrote: > > On Tue, Feb 09, 2021 at 05:32:06PM -0800, Daniel Latypov wrote: > > > > After [2]: > > $ ./tools/testing/kunit.py run --kunitconfig=fs/ext4/.kunitconfig > > Any chance that in the future this might bec

Re: [PATCH] kunit: tool: Fix a python tuple typing error

2021-02-22 Thread Daniel Latypov
an error only mypy cares about, this fix does make the code more stylistically correct and should definitely go in. > > Fixes 97752c39bd ("kunit: kunit_tool: Allow .kunitconfig to disable config > items") > Signed-off-by: David Gow Reviewed-by: Daniel Latypov >

[PATCH] ext4: add .kunitconfig fragment to enable ext4-specific tests

2021-02-09 Thread Daniel Latypov
test.git/commit/?h=kunit&id=243180f5924ed27ea417db39feb7f9691777688e * 372/5556 directories isn't too much, but still not a small number: $ find -type f -name 'k*' | xargs dirname | sort -u | wc -l 372 Signed-off-by: Daniel Latypov --- fs/ext4/.kunitconfig | 3 +++ 1 file cha

Re: [PATCH] ext4: add .kunitconfig fragment to enable ext4-specific tests

2021-02-09 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 6:33 PM Theodore Ts'o wrote: > > On Tue, Feb 09, 2021 at 05:32:06PM -0800, Daniel Latypov wrote: > > > > After [2]: > > $ ./tools/testing/kunit.py run --kunitconfig=fs/ext4/.kunitconfig > > Any chance that in the future this might bec

[PATCH 0/3] kunit: support running subsets of test suites from

2021-02-03 Thread Daniel Latypov
details and discussion about future work. This patch series also includes a bugfix for a latent bug that can't be triggered right now but has worse consequences as a result of the changes needed to plumb in this suite name glob. [1] https://lore.kernel.org/linux-kselftest/2021020120551

[PATCH 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-03 Thread Daniel Latypov
lter_glob=list-kunit-test.*del* But at the moment, it's far easier to manually comment out test cases in test files as opposed to messing with sets of Kconfig entries to select specific suites. So even just doing this makes using kunit far less annoying. Signed-off-by: Daniel Latypov --- lib

[PATCH 2/3] kunit: tool: add support for filtering suites by glob

2021-02-03 Thread Daniel Latypov
which currently only supports filtering by suite name. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 21 - tools/testing/kunit/kunit_kernel.py | 4 +++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tool

[PATCH 3/3] kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-03 Thread Daniel Latypov
unit tool only calls run_kernel() at most once, so it's not possible to trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_kernel.py | 4 +++- 1 file

[PATCH] kunit: don't show `1 == 1` in failed assertion messages

2021-01-28 Thread Daniel Latypov
p suggests 100+ comparisons to an integer literal as the right hand side. Signed-off-by: Daniel Latypov --- lib/kunit/assert.c | 39 +-- 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c index 33acdaa28a7d

Re: [PATCH] kunit: make kunit_tool accept optional path to .kunitconfig fragment

2021-01-29 Thread Daniel Latypov
On Thu, Jan 28, 2021 at 10:33 PM David Gow wrote: > > On Sat, Jan 23, 2021 at 8:17 AM Daniel Latypov wrote: > > > > Currently running tests via KUnit tool means tweaking a .kunitconfig > > file, which you'd keep around locally and never commit. > > This chan

Re: [PATCH] kunit: don't show `1 == 1` in failed assertion messages

2021-01-29 Thread Daniel Latypov
On Thu, Jan 28, 2021 at 8:51 PM David Gow wrote: > > On Fri, Jan 29, 2021 at 10:26 AM Daniel Latypov wrote: > > > > Currently, given something (fairly dystopian) like > > > KUNIT_EXPECT_EQ(test, 2 + 2, 5) > > > > KUnit will prints a failure message lik

[PATCH v2] kunit: make kunit_tool accept optional path to .kunitconfig fragment

2021-02-01 Thread Daniel Latypov
ts live in the same directory as the code-under-test, so it feels more natural to allow the kunitconfig fragments to live anywhere. (Though, people could create a separate directory if wanted; this patch imposes no restrictions on the path). Signed-off-by: Daniel Latypov --- Changes since v1: cha

[PATCH v2 2/3] kunit: tool: add support for filtering suites by glob

2021-02-04 Thread Daniel Latypov
which currently only supports filtering by suite name. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 21 - tools/testing/kunit/kunit_kernel.py | 4 +++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tool

[PATCH v2 0/3] kunit: support running subsets of test suites from kunit.py

2021-02-04 Thread Daniel Latypov
> v2: Fix free of `suites` subarray in suite_set. Found by Dan Carpenter and kernel test robot. Daniel Latypov (3): kunit: add kunit.filter_glob cmdline option to filter suites kunit: tool: add support for filtering suites by glob kunit: tool: fix unintentional state

[PATCH v2 3/3] kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-04 Thread Daniel Latypov
unit tool only calls run_kernel() at most once, so it's not possible to trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_kernel.py | 4 +++- 1 file

[PATCH v2 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-04 Thread Daniel Latypov
lter_glob=list-kunit-test.*del* But at the moment, it's far easier to manually comment out test cases in test files as opposed to messing with sets of Kconfig entries to select specific suites. So even just doing this makes using kunit far less annoying. Signed-off-by: Daniel Latypov --- lib

Re: [PATCH 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-04 Thread Daniel Latypov
On Wed, Feb 3, 2021 at 11:13 PM Dan Carpenter wrote: > > Hi Daniel, > > url: > https://github.com/0day-ci/linux/commits/Daniel-Latypov/kunit-support-running-subsets-of-test-suites-from/20210204-074405 > base: 88bb507a74ea7d75fa49edd421eaa710a7d80598 > config: x86_64-ran

[PATCH v3 0/3] kunit: support running subsets of test suites from kunit.py

2021-02-04 Thread Daniel Latypov
> v2: Fix free of `suites` subarray in suite_set. Found by Dan Carpenter and kernel test robot. v2 -> v3: Add MODULE_PARM_DESC() for kunit.filter_glob. Daniel Latypov (3): kunit: add kunit.filter_glob cmdline option to filter suites kunit: tool: add support for filtering suites by glob

[PATCH v3 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-04 Thread Daniel Latypov
lter_glob=list-kunit-test.*del* But at the moment, it's far easier to manually comment out test cases in test files as opposed to messing with sets of Kconfig entries to select specific suites. So even just doing this makes using kunit far less annoying. Signed-off-by: Daniel Latypov Reviewed-

[PATCH v3 3/3] kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-04 Thread Daniel Latypov
unit tool only calls run_kernel() at most once, so it's not possible to trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/

[PATCH v3 2/3] kunit: tool: add support for filtering suites by glob

2021-02-04 Thread Daniel Latypov
which currently only supports filtering by suite name. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/kunit/kunit.py| 21 - tools/testing/kunit/kunit_kernel.py | 4 +++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/tools/

Re: [PATCH v2 1/2] kunit: support failure from dynamic analysis tools

2021-02-09 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 9:26 AM Alan Maguire wrote: > > On Fri, 5 Feb 2021, Daniel Latypov wrote: > > > From: Uriel Guajardo > > > > Add a kunit_fail_current_test() function to fail the currently running > > test, if any, with an error message. > > &

[PATCH v3 2/2] kunit: ubsan integration

2021-02-09 Thread Daniel Latypov
() as well, so there's some slight duplication, but it also ensures an error is recorded in the debugfs entry for the running KUnit test. Print a shorter version of the message to make it less spammy. Co-developed-by: Daniel Latypov Signed-off-by: Uriel Guajardo Signed-off-by: Daniel La

Re: [PATCH v2 1/2] kunit: support failure from dynamic analysis tools

2021-02-09 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 2:12 PM Alan Maguire wrote: > > On Tue, 9 Feb 2021, Daniel Latypov wrote: > > > On Tue, Feb 9, 2021 at 9:26 AM Alan Maguire wrote: > > > > > > On Fri, 5 Feb 2021, Daniel Latypov wrote: > > > > > > > From: Uriel G

[PATCH v3 0/2] kunit: fail tests on UBSAN errors

2021-02-09 Thread Daniel Latypov
v1 by Uriel is here: [1]. Since it's been a while, I've dropped the Reviewed-By's. It depended on commit 83c4e7a0363b ("KUnit: KASAN Integration") which hadn't been merged yet, so that caused some kerfuffle with applying them previously and the series was reverted. This revives the series but mak

[PATCH v3 1/2] kunit: support failure from dynamic analysis tools

2021-02-09 Thread Daniel Latypov
sage [15:19:34] not ok 1 - example_simple_test Co-developed-by: Daniel Latypov Signed-off-by: Uriel Guajardo Signed-off-by: Daniel Latypov --- include/kunit/test-bug.h | 30 ++ lib/kunit/test.c | 37 + 2 files changed, 63 insertions(+),

[PATCH v2 3/3] kunit: tool: move kunitconfig parsing into __init__, make it optional

2020-12-11 Thread Daniel Latypov
might not need/want to load the config. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 20 tools/testing/kunit/kunit_kernel.py | 25 + 2 files changed, 17 insertions(+), 28 deletions(-) diff --git a/tools/testing/kunit/kunit.py

[PATCH v2 1/3] kunit: tool: surface and address more typing issues

2020-12-11 Thread Daniel Latypov
quests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit.py| 14 - tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kun

[PATCH v2 2/3] kunit: tool: fix minor typing issue with None status

2020-12-11 Thread Daniel Latypov
f them is empty"). Also slightly simplify the code and add type annotations. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser

Re: [PATCH] kunit: Print test statistics on failure

2020-12-11 Thread Daniel Latypov
On Thu, Dec 10, 2020 at 11:23 PM David Gow wrote: > > When a number of tests fail, it can be useful to get higher-level > statistics of how many tests are failing (or how many parameters are > failing in parameterised tests), and in what cases or suites. This is > already done by some non-KUnit te

[PATCH] kunit: tool: simplify kconfig is_subset_of() logic

2020-12-08 Thread Daniel Latypov
Don't use an O(nm) algorithm* and make it more readable by using a dict. *Most obviously, it does a nested for-loop over the entire other config. A bit more subtle, it calls .entries(), which constructs a set from the list for _every_ outer iteration. Signed-off-by: Daniel Latypov ---

Re: [PATCH v2] lib: add basic KUnit test for lib/math

2021-01-19 Thread Daniel Latypov
On Mon, Jan 18, 2021 at 2:36 AM Andy Shevchenko wrote: > > On Thu, Jan 14, 2021 at 10:27:00AM -0800, Daniel Latypov wrote: > > Add basic test coverage for files that don't require any config options: > > * gcd.c > > * lcm.c > > * int_sqrt.c > > * reciproca

[PATCH v3] lib: add basic KUnit test for lib/math

2021-01-19 Thread Daniel Latypov
meterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy Signed-off-by: Daniel Latypov --- Changes since v2: mv math_test.c => math_kunit.c Changes since v1: * Rebase and rewrite to use the new par

[PATCH] KUnit: Docs: make start.rst example Kconfig follow style.rst

2021-01-19 Thread Daniel Latypov
The primary change is that we want to encourage people to respect KUNIT_ALL_TESTS to make it easy to run all the relevant tests for a given config. Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/start.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a

[PATCH v3 1/3] kunit: tool: surface and address more typing issues

2021-01-07 Thread Daniel Latypov
quests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit.py| 14 - tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kun

[PATCH v3 3/3] kunit: tool: move kunitconfig parsing into __init__, make it optional

2021-01-07 Thread Daniel Latypov
might not need/want to load the config. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 20 tools/testing/kunit/kunit_kernel.py | 25 + 2 files changed, 17 insertions(+), 28 deletions(-) diff --git a/tools/testing/kunit/kunit.py

[PATCH v3 2/3] kunit: tool: fix minor typing issue with None status

2021-01-07 Thread Daniel Latypov
f them is empty"). Also slightly simplify the code and add type annotations. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser

[PATCH v2] lib: add basic KUnit test for lib/math

2021-01-14 Thread Daniel Latypov
les in this dir * are written so adding new test cases to cover edge cases should be easy Signed-off-by: Daniel Latypov --- Changes since v1: * Rebase and rewrite to use the new parameterized testing support. * misc: fix overflow in literal and inline int_sqrt format string. * related: comm

[PATCH v4 1/3] kunit: tool: surface and address more typing issues

2021-01-14 Thread Daniel Latypov
quests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- Changes since v1: none here, reworked last patch. Changes since v2: rebased onto torvalds/master. Changes since v3:

[PATCH v4 3/3] kunit: tool: move kunitconfig parsing into __init__, make it optional

2021-01-14 Thread Daniel Latypov
might not need/want to load the config. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/kunit/kunit.py| 20 tools/testing/kunit/kunit_kernel.py | 25 + 2 files changed, 17 insertions(+), 28 deletions(-) diff --git

[PATCH v4 2/3] kunit: tool: fix minor typing issue with None status

2021-01-14 Thread Daniel Latypov
f them is empty"). Also slightly simplify the code and add type annotations. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser

[PATCH v6] lib: add basic KUnit test for lib/math

2021-04-16 Thread Daniel Latypov
nteresting, but they * provide short and simple examples of parameterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy * looking at code coverage, we hit all the branches in the .c files Signed-off-by:

Re: [PATCH v5] lib: add basic KUnit test for lib/math

2021-04-16 Thread Daniel Latypov
On Tue, Apr 13, 2021 at 5:33 PM Daniel Latypov wrote: > > On Mon, Apr 12, 2021 at 11:41 PM David Gow wrote: > > > > On Tue, Apr 13, 2021 at 3:07 AM Daniel Latypov wrote: > > > > > > Add basic test coverage for files that don't require any config options

[PATCH v2] kunit: add unit test for filtering suites by names

2021-04-16 Thread Daniel Latypov
includes a clever `kfree_at_end()` helper that makes this test easier to write than it otherwise would have been. Tested by running just the new tests using itself $ ./tools/testing/kunit/kunit.py run '*exec*' Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- v1 -> v2

Re: [PATCH] Documentation: kunit: Update kunit_tool page

2021-04-16 Thread Daniel Latypov
t; --alltests > > Note that this does overlap a little with the new running_tips page. I > don't think it's a problem having both: this page is supposed to be a > bit more of a reference, rather than a list of useful tips, so the fact > that they both describe the same fe

Re: [PATCH] Documentation: kunit: add tips for running KUnit

2021-04-12 Thread Daniel Latypov
of the world ~somewhere. But it did feel a bit strange to do it here, so I'm not against removing it. > > Otherwise, a few minor comments and nitpicks: > > -- David > > On Sat, Apr 10, 2021 at 2:01 AM Daniel Latypov wrote: > > > > This is long overdue. > > &

[PATCH v5] lib: add basic KUnit test for lib/math

2021-04-12 Thread Daniel Latypov
nteresting, but they * provide short and simple examples of parameterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy * looking at code coverage, we hit all the branches in the .c files Signed-off-by:

Re: [PATCH] Documentation: kunit: add tips for running KUnit

2021-04-12 Thread Daniel Latypov
On Mon, Apr 12, 2021 at 1:42 PM Brendan Higgins wrote: > > On Mon, Apr 12, 2021 at 10:27 AM Daniel Latypov wrote: > > > > hOn Fri, Apr 9, 2021 at 9:10 PM David Gow wrote: > > > > > > Thanks for writing this: it's good to have these things documen

[PATCH] kunit: add unit test for filtering suites by names

2021-04-12 Thread Daniel Latypov
includes a clever `kfree_at_end()` helper that makes this test easier to write than it otherwise would have been. Tested by running just the new tests using itself $ ./tools/testing/kunit/kunit.py run '*exec*' Signed-off-by: Daniel Latypov --- lib/kunit/executor.c | 26

[PATCH v2] Documentation: kunit: add tips for running KUnit

2021-04-12 Thread Daniel Latypov
ot using uml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhigg...@google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov --- D

Re: [PATCH v4 2/3] mm/slub, kunit: add a KUnit test for SLUB debugging functionality

2021-04-13 Thread Daniel Latypov
error whenever slab_bug() or slab_fix() is called or when > the count of pages is wrong. > > Signed-off-by: Oliver Glitta Acked-by: Daniel Latypov Looks good to me! My one minor suggestion: perhaps let's log a summary of the error or the func name in slab_add_kunit_errors(). >

Re: [PATCH] kunit: add unit test for filtering suites by names

2021-04-13 Thread Daniel Latypov
On Mon, Apr 12, 2021 at 10:00 PM David Gow wrote: > > On Tue, Apr 13, 2021 at 8:08 AM Daniel Latypov wrote: > > > > This adds unit tests for kunit_filter_subsuite() and > > kunit_filter_suites(). > > > > Note: what the executor means by "subsuite" is

Re: [PATCH v5] lib: add basic KUnit test for lib/math

2021-04-13 Thread Daniel Latypov
On Mon, Apr 12, 2021 at 11:41 PM David Gow wrote: > > On Tue, Apr 13, 2021 at 3:07 AM Daniel Latypov wrote: > > > > Add basic test coverage for files that don't require any config options: > > * part of math.h (what seem to be the most commonly used ma

[PATCH v3] Documentation: kunit: add tips for running KUnit

2021-04-13 Thread Daniel Latypov
ot using uml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhigg...@google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov --- v2 ->

Re: [PATCH v2] Documentation: dev-tools: Add Testing Overview

2021-04-14 Thread Daniel Latypov
it seemed worth at least having something. Hopefully > this can form the basis for more detailed documentation later. > > Signed-off-by: David Gow Reviewed-by: Daniel Latypov Looks good to me. Some minor typos and nits about wording here and there. > --- > Thanks, everyone, for t

Re: [PATCH v3] Documentation: kunit: add tips for running KUnit

2021-04-14 Thread Daniel Latypov
On Tue, Apr 13, 2021 at 9:22 PM David Gow wrote: > > On Wed, Apr 14, 2021 at 8:45 AM Daniel Latypov wrote: > > > > This is long overdue. > > > > There are several things that aren't nailed down (in-tree > > .kunitconfig's), or partially broken (GCOV

[PATCH v4] Documentation: kunit: add tips for running KUnit

2021-04-14 Thread Daniel Latypov
ot using uml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhigg...@google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov Reviewed

[PATCH] kunit: fix -Wunused-function warning for __kunit_fail_current_test

2021-04-06 Thread Daniel Latypov
When CONFIG_KUNIT is not enabled, __kunit_fail_current_test() an empty static function. But GCC complains about unused static functions, *unless* they're static inline. So add inline to make GCC happy. Signed-off-by: Daniel Latypov Fixes: 359a376081d4 ("kunit: support failure fr

Re: linux-next: build warning after merge of the kunit-next tree

2021-04-06 Thread Daniel Latypov
Thanks for the catch. Should be addressed by https://lore.kernel.org/linux-kselftest/20210406172901.1729216-1-dlaty...@google.com/ When I was testing the CONFIG_KUNIT=n case, I added it to a file that wasn't being compiled (CONFIG_UBSAN=y is not sufficient for lib/ubsan.c to be compiled...). On T

[PATCH] Documentation: kunit: add tips for using current->kunit_test

2021-04-06 Thread Daniel Latypov
mple of kunit_fail_current_test(). Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/tips.rst | 78 +- 1 file changed, 76 insertions(+), 2 deletions(-) diff --git a/Documentation/dev-tools/kunit/tips.rst b/Documentation/dev-tools/kunit/tips.rst index a6ca0af1

Re: [PATCH v3 1/2] kunit: add a KUnit test for SLUB debugging functionality

2021-04-01 Thread Daniel Latypov
On Thu, Apr 1, 2021 at 2:16 AM 'Marco Elver' via KUnit Development wrote: > > [Note, if you'd like me to see future versions, please Cc me, otherwise > it's unlikely I see it in time. Also add kunit-...@googlegroups.com if > perhaps a KUnit dev should have another look, too.] > > On Wed, Mar 31, 2

Re: [PATCH v4] lib: add basic KUnit test for lib/math

2021-04-09 Thread Daniel Latypov
On Fri, Apr 9, 2021 at 8:30 AM Andy Shevchenko wrote: > > On Thu, Apr 08, 2021 at 06:40:01PM -0700, Daniel Latypov wrote: > > Add basic test coverage for files that don't require any config options: > > * gcd.c > > * lcm.c > > * int_sqrt.c > > * reciproca

[PATCH] Documentation: kunit: add tips for running KUnit

2021-04-09 Thread Daniel Latypov
.@google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/index.rst | 1 + .../dev-tools/kunit/running_tips.rst | 278 ++ Docume

Re: [PATCH] Documentation: dev-tools: Add Testing Overview

2021-04-10 Thread Daniel Latypov
On Sat, Apr 10, 2021 at 12:05 AM David Gow wrote: > > The kernel now has a number of testing and debugging tools, and we've > seen a bit of confusion about what the differences between them are. > > Add a basic documentation outlining the testing tools, when to use each, > and how they interact. >

Re: [PATCH] kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals

2021-04-02 Thread Daniel Latypov
On Fri, Apr 2, 2021 at 10:47 AM Shuah Khan wrote: > > On 4/2/21 3:35 AM, Brendan Higgins wrote: > > On Fri, Feb 5, 2021 at 2:18 PM Daniel Latypov wrote: > >> > >> Before: > >>> Expected str == "world", but > >>> str == hell

Re: [PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-04-02 Thread Daniel Latypov
e can give that as an > >> example more close to home (and thus less likely to be accidentally > >> broken). > >> > >> [1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/ > >> > >> Signed-off-by: Daniel Latypov > > > > Reviewe

[PATCH v2] kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals

2021-04-02 Thread Daniel Latypov
CT_STREQ(test, "hello", "world") since we don't expect it to realistically happen in checked in tests. (If you really wanted a test to fail, KUNIT_FAIL("msg") exists) In that case, you'd get: > Expected "hello" == "world", but Signe

Re: [PATCH] kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals

2021-04-02 Thread Daniel Latypov
On Fri, Apr 2, 2021 at 12:19 PM Shuah Khan wrote: > > On 4/2/21 1:09 PM, Daniel Latypov wrote: > > On Fri, Apr 2, 2021 at 10:47 AM Shuah Khan > > wrote: > >> > >> On 4/2/21 3:35 AM, Brendan Higgins wrote: > >>> On Fri, Feb 5, 2021 at 2:1

[PATCH v5 0/2] kunit: fail tests on UBSAN errors

2021-04-02 Thread Daniel Latypov
v1 by Uriel is here: [1]. Since it's been a while, I've dropped the Reviewed-By's. It depended on commit 83c4e7a0363b ("KUnit: KASAN Integration") which hadn't been merged yet, so that caused some kerfuffle with applying them previously and the series was reverted. This revives the series but mak

[PATCH v5 1/2] kunit: support failure from dynamic analysis tools

2021-04-02 Thread Daniel Latypov
sage [15:19:34] not ok 1 - example_simple_test Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Guajardo Reviewed-by: Alan Maguire Reviewed-by: Brendan Higgins --- include/kunit/test-bug.h | 29 + lib/kunit/test.c | 39 ++

[PATCH v5 2/2] kunit: ubsan integration

2021-04-02 Thread Daniel Latypov
() as well, so there's some slight duplication, but it also ensures an error is recorded in the debugfs entry for the running KUnit test. Print a shorter version of the message to make it less spammy. Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Gua

Re: [PATCH v4 1/2] kunit: support failure from dynamic analysis tools

2021-04-02 Thread Daniel Latypov
On Fri, Apr 2, 2021 at 10:53 AM Shuah Khan wrote: > > On 4/2/21 2:55 AM, Brendan Higgins wrote: > > On Thu, Mar 11, 2021 at 7:23 AM Daniel Latypov wrote: > >> > >> From: Uriel Guajardo > >> > >> Add a kunit_fail_current_test() function to fail

[PATCH 2/5] kunit: tool: fix unit test so it can run from non-root dir

2020-11-30 Thread Daniel Latypov
ad. Fixes: 5578d008d9e0 ("kunit: tool: fix running kunit_tool from outside kernel tree") Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_tool_test.py | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/tools/testing/kunit/kunit_tool_test.py b

[PATCH 1/5] kunit: tool: fix unit test cleanup handling

2020-11-30 Thread Daniel Latypov
* Stop leaking file objects. * Use self.addCleanup() to ensure we call cleanup functions even if setUp() fails. * use mock.patch.stopall instead of more error-prone manual approach Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_tool_test.py | 14 ++ 1 file changed, 6

[PATCH 3/5] kunit: tool: stop using bare asserts in unit test

2020-11-30 Thread Daniel Latypov
t this way. * This is probably a misunderstanding from the docs which uses it since `mock.call` is in scope as `call`. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_tool_test.py | 50 +- 1 file changed, 24 insertions(+), 26 deletions(-) diff --git a/too

[PATCH 4/5] kunit: tool: use `with open()` in unit test

2020-11-30 Thread Daniel Latypov
The use of manual open() and .close() calls seems to be an attempt to keep the contents in scope. But Python doesn't restrict variables like that, so we can introduce new variables inside of a `with` and use them outside. Do so to make the code more Pythonic. Signed-off-by: Daniel La

[PATCH 5/5] minor: kunit: tool: s/get_absolute_path/test_data_path in unit test

2020-11-30 Thread Daniel Latypov
1. the name is a lie. It gives relative paths, e.g. if I run from the same dir as the test file, it gives './test_data/' 2. it's only used for generating paths to tools/testing/kunit/test_data/ So we can tersen things by making it less general. Signed-off-by: Daniel Latypov ---

Re: [PATCH] Documentation: kunit: provide guidance for testing many inputs

2020-11-23 Thread Daniel Latypov
On Mon, Nov 23, 2020 at 10:32 AM Brendan Higgins wrote: > > On Mon, Nov 2, 2020 at 1:37 PM Daniel Latypov wrote: > > > > usage.rst goes into a detailed about faking out classes, but currently > > lacks wording about how one might idiomatically test a range of inputs. &g

[PATCH v2] Documentation: kunit: provide guidance for testing many inputs

2020-11-23 Thread Daniel Latypov
rused at the moment, only appearing in 2 tests (both written by people involved in KUnit). [1] not even on https://www.kernel.org/doc/html/latest/dev-tools/kunit/api/test.html Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/usage.rst | 83 +++-- 1 file changed, 77 i

Re: [PATCH 2/3] kunit: tool: fix minor typing issue with None status

2020-12-04 Thread Daniel Latypov
On Thu, Dec 3, 2020 at 8:17 PM David Gow wrote: > > On Fri, Dec 4, 2020 at 3:41 AM Daniel Latypov wrote: > > > > This seems good to me, but I have a few questions, particularly around > the description. > > > The code to handle aggregating statuses didn't che

Re: [PATCH 3/3] kunit: tool: move kunitconfig parsing into __init__

2020-12-04 Thread Daniel Latypov
On Thu, Dec 3, 2020 at 7:57 PM David Gow wrote: > > On Fri, Dec 4, 2020 at 3:41 AM Daniel Latypov wrote: > > > > LinuxSourceTree will unceremoniously crash if the user doesn't call > > read_kunitconfig() first in a number of functions. > > This patch seems to p

Re: [PATCH 4/5] kunit: tool: use `with open()` in unit test

2020-12-01 Thread Daniel Latypov
On Mon, Nov 30, 2020 at 11:33 PM David Gow wrote: > > On Tue, Dec 1, 2020 at 7:33 AM Daniel Latypov wrote: > > > > The use of manual open() and .close() calls seems to be an attempt to > > keep the contents in scope. > > But Python doesn't restrict variables

Re: [PATCH 2/5] kunit: tool: fix unit test so it can run from non-root dir

2020-12-01 Thread Daniel Latypov
On Mon, Nov 30, 2020 at 11:33 PM David Gow wrote: > > On Tue, Dec 1, 2020 at 7:33 AM Daniel Latypov wrote: > > > > get_absolute_path() makes an attempt to allow for this. > > But that doesn't work as soon as os.chdir() gets called. > > Can we explain why

[PATCH v2 3/4] kunit: tool: use `with open()` in unit test

2020-12-02 Thread Daniel Latypov
The use of manual open() and .close() calls seems to be an attempt to keep the contents in scope. But Python doesn't restrict variables like that, so we can introduce new variables inside of a `with` and use them outside. Do so to make the code more Pythonic. Signed-off-by: Daniel La

[PATCH v2 1/4] kunit: tool: fix unit test cleanup handling

2020-12-02 Thread Daniel Latypov
* Stop leaking file objects. * Use self.addCleanup() to ensure we call cleanup functions even if setUp() fails. * use mock.patch.stopall instead of more error-prone manual approach Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_tool_test.py | 14 ++ 1 file changed, 6

[PATCH v2 2/4] kunit: tool: stop using bare asserts in unit test

2020-12-02 Thread Daniel Latypov
t this way. * This is probably a misunderstanding from the docs which uses it since `mock.call` is in scope as `call`. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_tool_test.py | 50 +- 1 file changed, 24 insertions(+), 26 deletions(-) diff --git a/too

[PATCH v2 4/4] minor: kunit: tool: fix unit test so it can run from non-root dir

2020-12-02 Thread Daniel Latypov
nel tree") Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_tool_test.py | 60 +++--- 1 file changed, 24 insertions(+), 36 deletions(-) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index cf160914bc55..1cd127b225a9 10

Re: [PATCH 2/5] kunit: tool: fix unit test so it can run from non-root dir

2020-12-02 Thread Daniel Latypov
On Tue, Dec 1, 2020 at 8:41 PM David Gow wrote: > > On Wed, Dec 2, 2020 at 3:00 AM Daniel Latypov wrote: > > > > On Mon, Nov 30, 2020 at 11:33 PM David Gow wrote: > > > > > > On Tue, Dec 1, 2020 at 7:33 AM Daniel Latypov wrote: > > > > > >

Re: [PATCH v2 1/4] kunit: tool: fix unit test cleanup handling

2020-12-02 Thread Daniel Latypov
On Wed, Dec 2, 2020 at 7:05 PM David Gow wrote: > > On Thu, Dec 3, 2020 at 3:09 AM Daniel Latypov wrote: > > > > * Stop leaking file objects. > > * Use self.addCleanup() to ensure we call cleanup functions even if > > setUp() fails. > > * use mock.patch.stopal

[PATCH 1/3] kunit: tool: surface and address more typing issues

2020-12-03 Thread Daniel Latypov
quests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 14 - tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kunit/kunit_json.py | 2

[PATCH 3/3] kunit: tool: move kunitconfig parsing into __init__

2020-12-03 Thread Daniel Latypov
forgotten and to reduce copy-paste. The https://github.com/google/pytype type-checker complained that _config wasn't initialized. With this, kunit_tool now type checks under both pytype and mypy. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 20

[PATCH 2/3] kunit: tool: fix minor typing issue with None status

2020-12-03 Thread Daniel Latypov
add type annotations. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 24954bbc9baf..97e070506c31 100644 --- a

[PATCH v4 2/2] kunit: ubsan integration

2021-03-11 Thread Daniel Latypov
() as well, so there's some slight duplication, but it also ensures an error is recorded in the debugfs entry for the running KUnit test. Print a shorter version of the message to make it less spammy. Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Gua

[PATCH v4 0/2] kunit: fail tests on UBSAN errors

2021-03-11 Thread Daniel Latypov
v1 by Uriel is here: [1]. Since it's been a while, I've dropped the Reviewed-By's. It depended on commit 83c4e7a0363b ("KUnit: KASAN Integration") which hadn't been merged yet, so that caused some kerfuffle with applying them previously and the series was reverted. This revives the series but mak

[PATCH v4 1/2] kunit: support failure from dynamic analysis tools

2021-03-11 Thread Daniel Latypov
sage [15:19:34] not ok 1 - example_simple_test Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Guajardo Reviewed-by: Alan Maguire --- include/kunit/test-bug.h | 30 ++ lib/kunit/test.c | 39 +++ 2

  1   2   >