On 11/1/22 13:29, Richard Purdie wrote: > On Tue, 2022-11-01 at 12:14 -0400, Sean Anderson wrote: >> On 10/28/22 11:37, Richard Purdie wrote: >> > On Fri, 2022-10-28 at 11:29 -0400, Sean Anderson wrote: >> > > On 10/28/22 11:09, Richard Purdie wrote: >> > > > On Wed, 2022-10-26 at 13:21 -0400, Sean Anderson wrote: >> > > > > As noted in the cover letter, I ran >> > > > > >> > > > > oe-selftest -r fitimage.FitImageTests >> > > > >> > > > Ok, good. That at least means you were only running one class of tests. >> > > > I was worried you were running all of them! >> > > > >> > > > > I also tried using -j$(nproc), but I saw no increase in parallelism >> > > > > outside of the usual for bitbake. This was especially noticable for >> > > > > do_rootfs, which is single-threaded. >> > > > >> > > > Sadly the parallelism works on a per test class basis so it wouldn't >> > > > help in this case. There are only small marginal gains from running >> > > > tests in individual build directories so we don't do that. >> > > >> > > I estimate it could have saved me 2-3 minutes every build, since it could >> > > have parallelized the root filesystem stuff. >> > >> > On an initial run, it could have also ended up building a lot of pieces >> > in parallel needlessly so it is all a bit of a compromise. It might be >> > worth looking into whether we can make that an option, off by default. >> > >> > > > > This is ommitted above, but I *had* to use -j1 in order to avoid >> > > > > manually wiping out my existing build directory each time (and >> > > > > instead >> > > > > ending up with dozens of pid-named directories). This is documented >> > > > > nowhere, and I found it in some old IRC logs. >> > > > >> > > > Parallelism using differently named build directories is an >> > > > implementation detail, not something which the -j option implies.I >> > > > guess you were also using --keep-builddir >> > > >> > > Failing builds don't remove the test directory so you can inspect the >> > > build >> > > output. As you might imagine, I had a lot of failing builds. >> > >> > I'm very familiar with that myself, yes. >> > >> > We did once used to reuse the build directory, that challenge is we >> > have no idea what the user has done in there prior to the test so it >> > potentially makes the test results potentially incorrect. >> > >> > > > > > We haven't really had anyone try and optimise the tests either, I'm >> > > > > > sure there will be things in there which can help. Please don't >> > > > > > let the >> > > > > > speed put you off trying to improve things and extend our coverage! >> > > > > >> > > > > The poor speed of these self tests (and of everything related to the >> > > > > yocto project in general) makes this project frustrating to >> > > > > contribute >> > > > > to. It took me around 2 days to go from my prototype to this series, >> > > > > most of which was spent waiting for tests to compile and losing >> > > > > whatever >> > > > > train of thought I had. I probably went through perhaps 20 >> > > > > revisions. If >> > > > > I was working on e.g. U-Boot, I could have made 20 revisions in 2 >> > > > > hours, >> > > > > as it takes around 15 seconds to recompile it and run the full unit >> > > > > test >> > > > > suite. >> > > > > >> > > > > On the topic of these specific tests, part of the problem is that >> > > > > do_rootfs is a bottleneck which takes around 45-60s on my system. >> > > > > Every >> > > > > test which modifies something in the rootfs incurs this overhead. >> > > > >> > > > For better or worse we've 'a few' more moving pieces than U-Boot. >> > > > >> > > > Building a root filesystem from packages is a non-trivial task, taking >> > > > under a minute is in some ways pretty good already. The only other >> > > > thing we could do is incremental rootfs construction where it would >> > > > add/remove changed packages. I'd worry that the result may not always >> > > > be equal to a build from scratch and it might cause weird and >> > > > interesting reproducibility problems (particularly when you consider >> > > > things like postinsts). >> > > > >> > > > I would love to improve our development "iteration" time but I'm >> > > > struggling to see where we could get the speed gains from :(. Open to >> > > > other ideas... >> > > >> > > We don't have to build a full root filesystem. All of these tests just >> > > want >> > > e.g. an initramfs. An empty (or one file) filesystem would work just as >> > > well. >> > > If you still want to boot, you can make a busybox filesystem. >> > >> > Could we update the test just to use an initramfs then? >> > >> > I'm definitely a fan of keeping the tests as simple as we can whilst >> > still testing what we need to test. >> >> I can look into this, but I'd prefer to do it as a follow-up to this series. >> >> I'll probably send a v2 later this week a fleshed-out commit message for >> patch >> 5/6 (and with it possibly all those variables moved to a separate bbclass to >> make it easier for other classes to create signed FITs). > > The series did already merge so anything would be incremental > improvements at this point!
Huh... I really wish you guys sent thank-you messages when you merged something. Makes it a lot easier to keep track of which series still need work. --Sean
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172374): https://lists.openembedded.org/g/openembedded-core/message/172374 Mute This Topic: https://lists.openembedded.org/mt/94487631/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
