On Wed, Mar 03, 2021 at 03:18:15PM -0500, Tom Rini wrote: > On Wed, Mar 03, 2021 at 02:37:51PM -0500, Tom Rini wrote: > > On Wed, Feb 03, 2021 at 05:44:06AM -0700, Simon Glass wrote: > > > > > At present U-Boot has two broad sets of tests in the C code: driver model > > > tests which do a lot of pre-/post-init and command tests which do not. > > > > > > This separation makes it slightly harder to write a test, since there are > > > two different test-state structures and different rules for running the > > > two different test types. At present these rules are determined by where > > > the test is (actually its prefix). > > > > > > All unit tests can be run from the command line with the 'ut' command. > > > Since SPL does not have commands, it currently calls the test runner > > > directly and offers no control of which tests are run. > > > > > > This seems like a good time to refactor the tests into a unified test > > > runner, allowing U-Boot proper and SPL to use the same path, perhaps with > > > some different conditions along the way. > > > > > > This series sets up a unified runner called ut_run_list(), which runs a > > > set of tests from a linker_list. Driver model tests are distinguished by > > > a new UT_TESTF_DM flag so that the necessary init and cleanup can still > > > be done. > > > > > > The runner is modified to support running SPL tests that are not solely > > > for driver model. An example test for FIT loading is added as a > > > demonstration. > > > > > > In addition, some documentation is added to explain how to write tests. > > > > > > This series is available at u-boot-dm/test-working > > > > > > Changes in v3: > > > - Add new patch to re-enable test_ofplatdata > > > - Reword the SPL tests section for clarity > > > - Use test_set_state() throughout test-main.c instead of direct assignment > > > - Update the pytest collector as well > > > > > > Changes in v2: > > > - Use correct rst format for 'Ad-hoc tests' section > > > - Expand docs on how each type of test is marked > > > - Put the docs in tests_sandbox since it is more related to sandbox > > > - Put in a mention of tests_sandbox in the main testing docs > > > - Add a note that SPL tests can in fact be run individualy > > > - Document how to run all C tests with 'ut all' > > > - Fix 'get list' typo > > > - Fix conditions so non-DM SPL tests are actually run > > > - Allow for prefix to be NULL, to match function comment > > > - Add new patches to cover running an SPL test > > > > There was a small problem applying one or two of these patches that I > > fixed up. But then when testing locally (Ubuntu 18.04, but virtualenv > > setup and populated with test/py/requirements.txt) like 200 tests failed > > to run for Sandbox. Please check this and that CI runs. > > Er, something has gone odd for me locally, please disregard.
Two problems! One, this series IS breaking tests for me. Fresh venv, ~260 tests fail to run. Two, I didn't notice a fail-to-build in my build+pytest script, so the binary from this really really failed with the current pytest scripts :) -- Tom
signature.asc
Description: PGP signature
