Move test/README to the 'Develop U-Boot' chapter of the HTML documentation.
Signed-off-by: Heinrich Schuchardt <xypron.g...@gmx.de> --- doc/develop/index.rst | 1 + doc/develop/testing.rst | 96 +++++++++++++++++++++++++++++++++++++++++ test/README | 96 ----------------------------------------- 3 files changed, 97 insertions(+), 96 deletions(-) create mode 100644 doc/develop/testing.rst delete mode 100644 test/README diff --git a/doc/develop/index.rst b/doc/develop/index.rst index b108df8e1b..ed2e73bf56 100644 --- a/doc/develop/index.rst +++ b/doc/develop/index.rst @@ -29,3 +29,4 @@ Testing :maxdepth: 1 coccinelle + testing diff --git a/doc/develop/testing.rst b/doc/develop/testing.rst new file mode 100644 index 0000000000..4bc9ca3a6a --- /dev/null +++ b/doc/develop/testing.rst @@ -0,0 +1,96 @@ +Testing in U-Boot +================= + +U-Boot has a large amount of code. This file describes how this code is +tested and what tests you should write when adding a new feature. + + +Running tests +------------- + +To run most tests on sandbox, type this: + + make check + +in the U-Boot directory. Note that only the pytest suite is run using this +command. + +Some tests take ages to run. To run just the quick ones, type this: + + make qcheck + + +Sandbox +------- +U-Boot can be built as a user-space application (e.g. for Linux). This +allows test to be executed without needing target hardware. The 'sandbox' +target provides this feature and it is widely used in tests. + + +Pytest Suite +------------ + +Many tests are available using the pytest suite, in test/py. This can run +either on sandbox or on real hardware. It relies on the U-Boot console to +inject test commands and check the result. It is slower to run than C code, +but provides the ability to unify lots of tests and summarise their results. + +You can run the tests on sandbox with: + + ./test/py/test.py --bd sandbox --build + +This will produce HTML output in build-sandbox/test-log.html + +See test/py/README.md for more information about the pytest suite. + + +tbot +---- + +Tbot provides a way to execute tests on target hardware. It is intended for +trying out both U-Boot and Linux (and potentially other software) on a +number of boards automatically. It can be used to create a continuous test +environment. See http://www.tbot.tools for more information. + + +Ad-hoc tests +------------ + +There are several ad-hoc tests which run outside the pytest environment: + + test/fs - File system test (shell script) + test/image - FIT and legacy image tests (shell script and Python) + test/stdint - A test that stdint.h can be used in U-Boot (shell script) + trace - Test for the tracing feature (shell script) + +TODO: Move these into pytest. + + +When to write tests +------------------- + +If you add code to U-Boot without a test you are taking a risk. Even if you +perform thorough manual testing at the time of submission, it may break when +future changes are made to U-Boot. It may even break when applied to mainline, +if other changes interact with it. A good mindset is that untested code +probably doesn't work and should be deleted. + +You can assume that the Pytest suite will be run before patches are accepted +to mainline, so this provides protection against future breakage. + +On the other hand there is quite a bit of code that is not covered with tests, +or is covered sparingly. So here are some suggestions: + +- If you are adding a new uclass, add a sandbox driver and a test that uses it +- If you are modifying code covered by an existing test, add a new test case + to cover your changes +- If the code you are modifying has not tests, consider writing one. Even a + very basic test is useful, and may be picked up and enhanced by others. It + is much easier to add onto a test - writing a new large test can seem + daunting to most contributors. + + +Future work +----------- + +Converting existing shell scripts into pytest tests. diff --git a/test/README b/test/README deleted file mode 100644 index 4bc9ca3a6a..0000000000 --- a/test/README +++ /dev/null @@ -1,96 +0,0 @@ -Testing in U-Boot -================= - -U-Boot has a large amount of code. This file describes how this code is -tested and what tests you should write when adding a new feature. - - -Running tests -------------- - -To run most tests on sandbox, type this: - - make check - -in the U-Boot directory. Note that only the pytest suite is run using this -command. - -Some tests take ages to run. To run just the quick ones, type this: - - make qcheck - - -Sandbox -------- -U-Boot can be built as a user-space application (e.g. for Linux). This -allows test to be executed without needing target hardware. The 'sandbox' -target provides this feature and it is widely used in tests. - - -Pytest Suite ------------- - -Many tests are available using the pytest suite, in test/py. This can run -either on sandbox or on real hardware. It relies on the U-Boot console to -inject test commands and check the result. It is slower to run than C code, -but provides the ability to unify lots of tests and summarise their results. - -You can run the tests on sandbox with: - - ./test/py/test.py --bd sandbox --build - -This will produce HTML output in build-sandbox/test-log.html - -See test/py/README.md for more information about the pytest suite. - - -tbot ----- - -Tbot provides a way to execute tests on target hardware. It is intended for -trying out both U-Boot and Linux (and potentially other software) on a -number of boards automatically. It can be used to create a continuous test -environment. See http://www.tbot.tools for more information. - - -Ad-hoc tests ------------- - -There are several ad-hoc tests which run outside the pytest environment: - - test/fs - File system test (shell script) - test/image - FIT and legacy image tests (shell script and Python) - test/stdint - A test that stdint.h can be used in U-Boot (shell script) - trace - Test for the tracing feature (shell script) - -TODO: Move these into pytest. - - -When to write tests -------------------- - -If you add code to U-Boot without a test you are taking a risk. Even if you -perform thorough manual testing at the time of submission, it may break when -future changes are made to U-Boot. It may even break when applied to mainline, -if other changes interact with it. A good mindset is that untested code -probably doesn't work and should be deleted. - -You can assume that the Pytest suite will be run before patches are accepted -to mainline, so this provides protection against future breakage. - -On the other hand there is quite a bit of code that is not covered with tests, -or is covered sparingly. So here are some suggestions: - -- If you are adding a new uclass, add a sandbox driver and a test that uses it -- If you are modifying code covered by an existing test, add a new test case - to cover your changes -- If the code you are modifying has not tests, consider writing one. Even a - very basic test is useful, and may be picked up and enhanced by others. It - is much easier to add onto a test - writing a new large test can seem - daunting to most contributors. - - -Future work ------------ - -Converting existing shell scripts into pytest tests. -- 2.29.2