On Fri, 2022-10-28 at 11:29 -0400, Sean Anderson wrote: > On 10/28/22 11:09, Richard Purdie wrote: > > On Wed, 2022-10-26 at 13:21 -0400, Sean Anderson wrote: > > > As noted in the cover letter, I ran > > > > > > oe-selftest -r fitimage.FitImageTests > > > > Ok, good. That at least means you were only running one class of tests. > > I was worried you were running all of them! > > > > > I also tried using -j$(nproc), but I saw no increase in parallelism > > > outside of the usual for bitbake. This was especially noticable for > > > do_rootfs, which is single-threaded. > > > > Sadly the parallelism works on a per test class basis so it wouldn't > > help in this case. There are only small marginal gains from running > > tests in individual build directories so we don't do that. > > I estimate it could have saved me 2-3 minutes every build, since it could > have parallelized the root filesystem stuff.
On an initial run, it could have also ended up building a lot of pieces in parallel needlessly so it is all a bit of a compromise. It might be worth looking into whether we can make that an option, off by default. > > > This is ommitted above, but I *had* to use -j1 in order to avoid > > > manually wiping out my existing build directory each time (and instead > > > ending up with dozens of pid-named directories). This is documented > > > nowhere, and I found it in some old IRC logs. > > > > Parallelism using differently named build directories is an > > implementation detail, not something which the -j option implies.I > > guess you were also using --keep-builddir > > Failing builds don't remove the test directory so you can inspect the build > output. As you might imagine, I had a lot of failing builds. I'm very familiar with that myself, yes. We did once used to reuse the build directory, that challenge is we have no idea what the user has done in there prior to the test so it potentially makes the test results potentially incorrect. > > > > We haven't really had anyone try and optimise the tests either, I'm > > > > sure there will be things in there which can help. Please don't let the > > > > speed put you off trying to improve things and extend our coverage! > > > > > > The poor speed of these self tests (and of everything related to the > > > yocto project in general) makes this project frustrating to contribute > > > to. It took me around 2 days to go from my prototype to this series, > > > most of which was spent waiting for tests to compile and losing whatever > > > train of thought I had. I probably went through perhaps 20 revisions. If > > > I was working on e.g. U-Boot, I could have made 20 revisions in 2 hours, > > > as it takes around 15 seconds to recompile it and run the full unit test > > > suite. > > > > > > On the topic of these specific tests, part of the problem is that > > > do_rootfs is a bottleneck which takes around 45-60s on my system. Every > > > test which modifies something in the rootfs incurs this overhead. > > > > For better or worse we've 'a few' more moving pieces than U-Boot. > > > > Building a root filesystem from packages is a non-trivial task, taking > > under a minute is in some ways pretty good already. The only other > > thing we could do is incremental rootfs construction where it would > > add/remove changed packages. I'd worry that the result may not always > > be equal to a build from scratch and it might cause weird and > > interesting reproducibility problems (particularly when you consider > > things like postinsts). > > > > I would love to improve our development "iteration" time but I'm > > struggling to see where we could get the speed gains from :(. Open to > > other ideas... > > We don't have to build a full root filesystem. All of these tests just want > e.g. an initramfs. An empty (or one file) filesystem would work just as well. > If you still want to boot, you can make a busybox filesystem. Could we update the test just to use an initramfs then? I'm definitely a fan of keeping the tests as simple as we can whilst still testing what we need to test. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172260): https://lists.openembedded.org/g/openembedded-core/message/172260 Mute This Topic: https://lists.openembedded.org/mt/94487631/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
