Hi Stephen,
On Mon, 5 Oct 2020 at 15:35, Stephen Warren <swar...@wwwdotorg.org> wrote:
>
> On 10/5/20 1:51 PM, Simon Glass wrote:
> > Hi Stephen,
> >
> > On Mon, 5 Oct 2020 at 13:39, Stephen Warren <swar...@wwwdotorg.org> wrote:
> >>
> >> On 10/3/20 9:25 AM, Simon Glass wrote:
> >>> Add a new test_spl fixture to handle running SPL unit tests.
> >>
> >>> diff --git a/test/py/conftest.py b/test/py/conftest.py
> >>
> >>> @@ -317,10 +318,13 @@ def pytest_generate_tests(metafunc):
> >>> Returns:
> >>> Nothing.
> >>> """
> >>> -
> >>> + #print('name', metafunc.fixturenames)
> >>
> >> Revert that debug change?
> >
> > Will do.
> >
> >>
> >>> diff --git a/test/py/tests/test_spl.py b/test/py/tests/test_spl.py
> >>
> >>> + cons.restart_uboot_with_flags(['-u', ut_spl_subtest])
> >>
> >> How is that change reverted when the test runs, so that subsequent tests
> >> are run on the main U-Boot rather than this restarted U-Boot?
> >
> > Well actually at the moment it just continues into U-Boot. It will
> > mostly pass the tests, but probably not all of them.
>
> Looking at existing tests, there are quite a few that do this:
>
> try:
> test body which might do something nasty to U-Boot
> finally:
> u_boot_console.restart_uboot()
>
> ... so that might be applicable.
>
> >> It feels like it'd be better to start a separate top-level test run for
> >> this purpose, rather than swap out the U-Boot process in the middle of a
> >> test run.
> >
> > I was hoping that the fixture stuff would take care of that. How would
> > I do a separate top-level test run?
>
> That'd simply be just running test/py/test.py and passing it the
> relevant U-Boot binary/args rather than the main binary. I assume you'd
> want to pass relevant -k option to restrict the set of tests run to
> something relevant, and an appropriate --build-dir option to point at
> the binary.
I'm fiddling with this a bit. I do already have a separate run of the
SPL tests using the mechanism you describe here, in test/run :
# Run tests which require sandbox_spl
run_test "sandbox_spl" ./test/py/test.py --bd sandbox_spl --build \
-k 'test_ofplatdata or test_handoff or test_spl'
Also, the test_spl() collection function in this patch is used to read
out tests from spl/u-boot-spl and these tests are not present in
u-boot since they are only built for SPL (Makefile ensures this).
So yes I can add a try...finally thing, but in fact all that does is
make every SPL test be followed by a reboot into U-Boot proper, for no
purpose.
Could you please have another look at this?
Regards,
Simon
