On Thu, Mar 12, 2026 at 2:08 PM Thomas Monjalon <[email protected]> wrote:
>
> 06/03/2026 17:40, Andrew Bailey:
> > Currently, a test case is decorated to signify whether to use Scapy or
> > TREX. This change allows test suites that do not use a traffic generator
> > to avoid the overhead of initializing either one.
> [...]
> > if (tt.test_type is TestCaseType.FUNCTIONAL and
> > self.func)
> > or (tt.test_type is TestCaseType.PERFORMANCE and
> > self.perf)
> > + or (tt.test_type is TestCaseType.CRYPTO and
> > self.crypto)
>
> That's a bit strange to read,
> because a crypto test can be functional or performance.
> I suppose we can re-discuss the classification of the tests.
> If the only need here is about the traffic generator,
In my opinion this is not quite true. There is another (and more
important) need associated with the test categories, which is allowing
users to run only a sub-category of tests according to their interest
or hardware setup. I.e. we can expect some people to show up and
desire only to run functional testcases (perhaps they have a minimal
hardware setup which is not good for performance, or they are only
working on resolving functional issues) and the test type categories
allow the users this flexibility with regards to what kinds of DTS
tests are run.
> we could make it "raw input" or something like that?
>
>
In any case I agree with Thomas and Andrew that the situation with
regards to test type classification/organization could use a review
and a tweak in the 26.07 release. Right now every single crypto_test
is a performance test (i.e. it runs a crypto workload, compares the
Gbps output against the Gbps baseline the user set in
tests_config.yaml for each testcast). But, some open questions for the
future:
1. Is there a need for users to select only functional crypto cases vs
only perf crypto cases? Or we can just leave all crypto cases in 1
bucket?
2. If the above is true, should we switch the testcase decorators over to:
a. ethdev_func
b. ethdev_perf
c. cryptodev_func
d. cryptodev_perf
and every testcase gets one of these decorators
OR
We apply multiple decorators to each testcase (i.e. add both
crypto_test and func_test to a testcase) and have the framework code
read the whole set of testcase type decorators and setup accordingly.
Thomas, is what we have right now in terms of testcase classification
okay for the 26.03 release, and we can tweak it in 26.07, or do you
want us to do something in the immediate term?
