Hi! I didn't say that `citest` should test "everything": of course, peripherals can't be tested (at least most of them), but it's the most basic test for every architecture/chip/board because it can run `ostest` and other test applications (`smp`, for instance). IMHO, it's the most important test that should run successfully before testing anything else. Also, it must provide the same results on QEMU and real hardware.
Peripherals can be tested directly on real hardware based on the existing defconfigs (I don't know if we should have a `hwtest` that tests everything. It makes sense, but it isn't usable for the end user). Em sex., 7 de fev. de 2025 às 08:32, raiden00pl <raiden0...@gmail.com> escreveu: > Support for peripherals in QEMU is very limited, and some peripherals > supported by QEMU are not available in new hardware. So the same > tests for qemu and real HW are just not possible for some targets. > > Also configurations with all peripherals/features enabled are not possible > for many chips - we are limited by available FLASH. > > pt., 7 lut 2025 o 12:02 Tiago Medicci Serrano <tiago.medi...@gmail.com> > napisał(a): > > > Hi! > > > > citest config: I suggest to keep citest and hardware test absolutely > > > separate. You want to be able to change one and not the other. Not the > > > same components will be tested. > > > > > > CI tests *include* HW testing... > > > > For now, we have the `citest` defconfig for some boards and they are > > intended to be built and run on QEMU (and, by interacting with `nsh` > > execute some test functions, like `ostest`). Extending this approach > would > > be the first step on HW test. If QEMU is available for some > > arch/chip/board, it can continue being an intermediary step (we call it > > *stage*, when applying it to CI), but as we've discussed previously, it's > > important to test on real hardware too: test the same defconfig on both > > QEMU and real HW (when both are available). It doesn't make sense to have > > separate defconfigs for QEMU and HW testing. The idea of using the QEMU > is > > simulating real hardware, so it's expected a defconfig to work on both! > > > > After extending the concept of the `citest` (which already exists!) to > test > > them on real hardware too, we can test other existing `defconfigs` on > real > > hardware. For example, testing `:wifi` defconfigs connecting to an AP. > > > > From my experience dealing with our internal CI: *do not create > unnecessary > > defconfigs*. Whenever it's required to change some configuration, do it > > for a test case specifically by using the `kconfig-merger` and > > `kconfig-tweak` functions when building the test case. We should only > > provide `defconfig`s that are usable by the users. Anything else should > be > > avoided. > > > > See, we want to test everything users use when experimenting NuttX. > > Unfortunately, users aren't that interested on `ostest` and other testing > > applications when developing their products. That's the reason why > `citest` > > defconfig exists. It's an exception! > > > > Best regards, > > > > Em sex., 7 de fev. de 2025 às 05:30, <michal.lyszc...@bofc.pl> escreveu: > > > > > On 2025-02-06 16:02:18, Matteo Golin wrote: > > > > I've got: > > > > - Raspberry Pi Pico > > > > - Raspberry Pi Pico W > > > > - XIAO RP2040 > > > > - XIAO SAMD21 > > > > - Raspberry Pi 4B > > > > > > > > And would also be willing to help test, provided it's easy enough for > > me > > > to > > > > automate. > > > Just looked into my lost and forgotten drawer. As of today I have: > > > > > > - nucleo wl55jc1 > > > - nucleo f091rc > > > - nucleo f303re > > > - nucleo f429zi > > > - stm32h735g > > > - stm32butterfly2 (stm32f107) > > > - mimxrt1050-evkb > > > - mimxrt1060-evkb > > > - mimxrt1062(teensy 4.2) > > > - rpi pico > > > - samc21 xplained > > > > > > Most of them have on board debugger. If not I also have few swd/jtag > > > debuggers > > > that I am not really using anymore. > > > > > > If you make this project so I can just put rasbian + your script to > > init.d > > > I can > > > donate these, rpi4, some USB hub and some space in my lab for testing. > > > Scripts > > > must not be running as root. I could also create some DMZ for boards > with > > > networking capability to test those as well. > > > > > > -- > > > > > > > > > .-----------------.-------------------.----------------------.-----------------. > > > | Michal Lyszczek | Embedded C, Linux | Company Address | .-. > > > opensource | > > > | +48 727 564 419 | Software Engineer | Akacjowa 10a; 55-330 | oo| > > > supporter | > > > | https://bofc.pl `----.--------------: Brzezinka Sredzka PL | /`'\ > > > & | > > > | GPG FF1EBFE7E3A974B1 | Bits of Code | NIP: 813 349 58 78 |(\_;/) > > > programer | > > > > > > > > > `----------------------^--------------^----------------------^-----------------' > > > > > >
