Re: [Automated-testing] [RFC] Test catalog template

2024-10-18 Thread Cyril Hrubis
Hi! > Then there are other perspectives that crosses this. For example, many of the > LTP and > kselftests will just fail, but there is no accumulated knowledge on what the > result of > each test means. So understanding what is expected to pass/fail for each > platform is > a sort of dependance

Re: [Automated-testing] [RFC] Test catalog template

2024-10-17 Thread Cyril Hrubis
Hi! > > A few thousand tests to be more precise, and also the content tend to > > change between releases, be it test additions or removal and I do not > > think this level of changes is somehing that makes sense to be tracked > > in such database. > > > > It may be better to have more generic desc

Re: [Automated-testing] [RFC] Test catalog template

2024-10-16 Thread Donald Zickus
Hi Cyril, On Wed, Oct 16, 2024 at 9:11 AM Cyril Hrubis wrote: > > Hi! > > Just saying "LTP" is not granular enough. LTP has hundreds of individual > > test programs, and it would be useful to specify the individual tests > > from LTP that should be run per sub-system. > > A few thousand tests to

Re: [Automated-testing] [RFC] Test catalog template

2024-10-16 Thread Donald Zickus
Hi Tim, On Tue, Oct 15, 2024 at 12:01 PM Bird, Tim wrote: > > > -Original Message- > > From: automated-test...@lists.yoctoproject.org > > On Behalf Of Don Zickus > > Hi, > > > > At Linux Plumbers, a few dozen of us gathered together to discuss how > > to expose what tests subsystem main

Re: [Automated-testing] [RFC] Test catalog template

2024-10-16 Thread Cyril Hrubis
Hi! > Just saying "LTP" is not granular enough. LTP has hundreds of individual > test programs, and it would be useful to specify the individual tests > from LTP that should be run per sub-system. A few thousand tests to be more precise, and also the content tend to change between releases, be it

RE: [Automated-testing] [RFC] Test catalog template

2024-10-15 Thread Bird, Tim
> -Original Message- > From: automated-test...@lists.yoctoproject.org > On Behalf Of Don Zickus > Hi, > > At Linux Plumbers, a few dozen of us gathered together to discuss how > to expose what tests subsystem maintainers would like to run for every > patch submitted or when CI runs tests