12/03/2026 20:54, Patrick Robb: > On Thu, Mar 12, 2026 at 2:08 PM Thomas Monjalon <[email protected]> wrote: > > > > 06/03/2026 17:40, Andrew Bailey: > > > Currently, a test case is decorated to signify whether to use Scapy or > > > TREX. This change allows test suites that do not use a traffic generator > > > to avoid the overhead of initializing either one. > > [...] > > > if (tt.test_type is TestCaseType.FUNCTIONAL and > > > self.func) > > > or (tt.test_type is TestCaseType.PERFORMANCE and > > > self.perf) > > > + or (tt.test_type is TestCaseType.CRYPTO and > > > self.crypto) > > > > That's a bit strange to read, > > because a crypto test can be functional or performance. > > I suppose we can re-discuss the classification of the tests. > > If the only need here is about the traffic generator, > > In my opinion this is not quite true. There is another (and more > important) need associated with the test categories, which is allowing > users to run only a sub-category of tests according to their interest > or hardware setup. I.e. we can expect some people to show up and > desire only to run functional testcases (perhaps they have a minimal > hardware setup which is not good for performance, or they are only > working on resolving functional issues) and the test type categories > allow the users this flexibility with regards to what kinds of DTS > tests are run. > > > we could make it "raw input" or something like that? > > > > > > In any case I agree with Thomas and Andrew that the situation with > regards to test type classification/organization could use a review > and a tweak in the 26.07 release. Right now every single crypto_test > is a performance test (i.e. it runs a crypto workload, compares the > Gbps output against the Gbps baseline the user set in > tests_config.yaml for each testcast). But, some open questions for the > future: > > 1. Is there a need for users to select only functional crypto cases vs > only perf crypto cases? Or we can just leave all crypto cases in 1 > bucket? > 2. If the above is true, should we switch the testcase decorators over to: > > a. ethdev_func > b. ethdev_perf > c. cryptodev_func > d. cryptodev_perf > > and every testcase gets one of these decorators > > OR > > We apply multiple decorators to each testcase (i.e. add both > crypto_test and func_test to a testcase) and have the framework code > read the whole set of testcase type decorators and setup accordingly. > > Thomas, is what we have right now in terms of testcase classification > okay for the 26.03 release, and we can tweak it in 26.07, or do you > want us to do something in the immediate term?
There is no urgency. I'm just opening the discussion for future.
