Dear all, each arch, driver and app tested once would be enough I think. A matrix can help to identify test-gaps. Double testing is nice and triple testing is not of benefit any more.
The goal should be to have fast access to results with transparency. I fear starting to maintain a useless monster ;) Simon -- Hard- and Softwaredevelopment Consultant Geschäftsführung: Simon Filgis USt-IdNr.: DE305343278 ISO9001:2015 <https://activities.ingenieurbuero-filgis.de/certifications> On Wed, Feb 5, 2025 at 1:27 PM Alan C. Assis <acas...@gmail.com> wrote: > I think we should test the most complete/complex boards from each arch. It > will cover most of the issues that could after other boards. > > BR, > > Alan > > On Wed, Feb 5, 2025 at 4:23 AM raiden00pl <raiden0...@gmail.com> wrote: > > > As mentioned earlier, testing all boards is pointless, especially since > the > > project > > has very limited resources. Choosing a few boards that will allow us to > > test as many > > things as possible is the most optimal approach. > > > > But first we should determine what things we want to test, not what > boards. > > Knowing what things we want to test, we can design test cases and > possibly > > use what is already available. > > > > But e-mail and github don't seem to me to be a good tool for > brainstorming > > and ideas. Maybe Confluence pages would be better? I haven't used > > Confluence > > for a few years, I just hope it's not as slow as it used to be :) > > > > I can create a Confluence page and describe my ideas about testing, I > have > > some > > thoughts about NuttX testing written somewhere in my private notes. > Others > > can do > > the same. > > > > Email can be good for decision making and maybe gathering more feedback > > from > > the community, but it's a shitty tool for more complex work. Or I'm too > > young > > to use it comfortably :P > > > > wt., 4 lut 2025 o 12:03 Tomek CEDRO <to...@cedro.info> napisał(a): > > > > > On Tue, Feb 4, 2025 at 11:42 AM Luchian Mihai < > luchiann.mi...@gmail.com> > > > wrote: > > > > Hi! > > > > First thing, I'm fairly new to nuttx so I might be off subject but > here > > > is > > > > my hot take on this subject. > > > > > > Welcome and have fun Mihai! :-) > > > > > > > > > > NuttX is offering support for a lot of boards, more than what DRUNX > > > should > > > > require. > > > > > > > > Eg. stm32f3 family, offering support for all the boards would benefit > > the > > > > boards more than the NuttX codebase. > > > > These boards share 80% of the code but each in its own files, so most > > > > differences are due to lack of backporting. > > > > > > > > My suggestion is to take a single board for each mcu family as > > mandatory > > > > NuttX support, others as optional. > > > > I'm not saying to drop support, or offer less support for those, just > > to > > > > treat the mandatory as higher prio. > > > > For the moment we can choose what is mandatory, and at a later time, > > when > > > > DRUNX would be stable, move the optional ones to DRUNX repo (for > > > example). > > > > > > The idea is that everyone can test what they have at hand and then > > > gather the results, that should sum up to full board list one day :-) > > > Also different people will use different build hosts, different > > > compilers, etc, so even if the same board is tested in different build > > > environment that can also reveal potential issues to be fixed :-) > > > > > > Yes, for sure we need at least one board from each family for start, > > > then adding more boards, we all do release testing that way or > > > another, by hand or scripted, we now have to find a way to make it > > > distributed easy to setup fire-and-forget :-) > > > > > > > > > > TLDR: I think less is more, less "official" support, more "official" > > > > coverage. > > > > > > More means more. Less means less. Lets keep thing simple in this > > > inverted world where word have no meaning anymore :D :D :D > > > > > > Its a small project based on voluntary work with zero financial > > > support from big companies. We will define a testing architecture for > > > sure, but for now its a fresh concept and each one of us try different > > > area to create small building blocks that will give us the solution we > > > need, one day, hopefully ;-) > > > > > > Please go ahead Mihai and try your boards, start with one that you > > > know best, use PyTest to create build and runtime automation, create > > > "selftest" board config that will run a script with `uname -a; help; > > > ostest, coremark`, gather the logs, parse the results, see how > > > nuttx-dashboard works based on gist, see what problems we have, see > > > how can we solve them hands-on :-) Maybe there is something better > > > than PyTest. Maybe there are other ways. You try it out and share the > > > feedback :-) > > > > > > If this is too much, use smaller steps, think of it as automation tool > > > for release testing on different boards :-) > > > > > > Thank you and take care :-) > > > Tomek > > > > > > -- > > > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info > > > > > >
