Hi Tom, On Thu, 22 Aug 2024 at 09:13, Tom Rini <tr...@konsulko.com> wrote: > > On Fri, Aug 16, 2024 at 05:53:59PM -0600, Simon Glass wrote: > [snip] > > > Some notes from a little bit of digging: There are almost 1000 sandbox > > > tests (17-20mins), but qemu_arm only runs 62 (2mins). With 'make > > > qcheck' it runs about 2500 tests in about 4 minutes. I just got 'make > > > pcheck' going again and that is a little faster (2.5 mins). Binman > > > tests run in parallel if you use 'pip install concurrencytest' > > > > While on this topic I forgot to mention that the sandbox tests (~800 > > of them) actually run in about 10 seconds. Try 'ut all' (although > > you'll need my test-fix series to stop it crashing). > > > > The reason pytest is so slow with sandbox is that it talks to U-Boot > > through a pty. Stephen Warren did something clever to stop it being > > horrible, but it is still very slow. > > Seeing about addressing this more would probably be very helpful then as > most of the runs (by user, I mean) happen via pytest and not directly > running sandbox I strongly suspect.
Yes, there is a long thread about performance[1] which makes for interesting reading these days. We do indeed have >1000 tests although it took more than 2 years. The Python approach is indeed quite slow, although half as slow as using the real 'expect' library. I do have one idea that might help, so I filed [2]. Regards, Simon [1] https://patchwork.ozlabs.org/project/uboot/patch/1447570381-1361-1-git-send-email-swar...@wwwdotorg.org/ [2] https://source.denx.de/u-boot/u-boot/-/issues/39
