Add an automated way to gather available capabilities of the tested hardware and skip test suites or cases which require capabilities that are not available.
This is done through two decorators: 1. The first marks a test suite method as test case. This populates the default attributes of each test case. 2. The seconds adds the required capabilities to a test suite or case, using the attributes from 1). Two types of capabilities are added: 1. NIC capabilities. These are gathered once DPDK has been built because we use testpmd for this. It's possible to add a function that will add configuration before assessing capabilities associated with the function. This is because some capabilities return different status with different configuration present. 2. The topology capability. Each test case is marked as requiring a default topology. The required topology of a test case (or a whole test suite) may be change with the second decorator. This is how it all works: 1. The required capabilities are first all gathered from all test suites and test cases. 2. The list of required capabilities is divided into supported and unsupported capabilities. In this step, the probing of hardware and/or anything else that needs to happen to gauge whether a capability is supported is done. 3. Each test suite and test case is then marked to be skipped if any of their required capabilities are not supported. v4: Rebased on next-dts. Adjusted the decorator for setting and reverting 9000 MTU to be parametrized. Added references to DPDK libraries and testpmd code to testpmd command parsing methods. Updated capability docstrings. Moved the capability decorator to NicCapability. Juraj Linkeš (11): dts: add the aenum dependency dts: add test case decorators dts: add mechanism to skip test cases or suites dts: add support for simpler topologies dts: add basic capability support dts: add NIC capability support dts: add NIC capabilities from show rxq info dts: add topology capability doc: add DTS capability doc sources dts: add Rx offload capabilities dts: add NIC capabilities from show port info .../framework.testbed_model.capability.rst | 6 + doc/api/dts/framework.testbed_model.rst | 2 + .../dts/framework.testbed_model.topology.rst | 6 + dts/framework/remote_session/testpmd_shell.py | 461 +++++++++++++++- dts/framework/runner.py | 155 +++--- dts/framework/test_result.py | 119 ++-- dts/framework/test_suite.py | 161 +++++- dts/framework/testbed_model/capability.py | 507 ++++++++++++++++++ dts/framework/testbed_model/node.py | 2 +- dts/framework/testbed_model/port.py | 4 +- dts/framework/testbed_model/topology.py | 127 +++++ dts/poetry.lock | 14 +- dts/pyproject.toml | 1 + dts/tests/TestSuite_hello_world.py | 10 +- dts/tests/TestSuite_os_udp.py | 3 +- dts/tests/TestSuite_pmd_buffer_scatter.py | 14 +- dts/tests/TestSuite_smoke_tests.py | 8 +- 17 files changed, 1447 insertions(+), 153 deletions(-) create mode 100644 doc/api/dts/framework.testbed_model.capability.rst create mode 100644 doc/api/dts/framework.testbed_model.topology.rst create mode 100644 dts/framework/testbed_model/capability.py create mode 100644 dts/framework/testbed_model/topology.py -- 2.43.0
