On Tue, 2022-11-01 at 13:40 -0400, Sean Anderson wrote: > On 11/1/22 13:29, Richard Purdie wrote: > > On Tue, 2022-11-01 at 12:14 -0400, Sean Anderson wrote: > > > On 10/28/22 11:37, Richard Purdie wrote: > > > > On Fri, 2022-10-28 at 11:29 -0400, Sean Anderson wrote: > > > > > On 10/28/22 11:09, Richard Purdie wrote: > > > > > > On Wed, 2022-10-26 at 13:21 -0400, Sean Anderson wrote: > > > > > > > As noted in the cover letter, I ran > > > > > > > > > > > > > > oe-selftest -r fitimage.FitImageTests > > > > > > > > > > > > Ok, good. That at least means you were only running one class of > > > > > > tests. > > > > > > I was worried you were running all of them! > > > > > > > > > > > > > I also tried using -j$(nproc), but I saw no increase in > > > > > > > parallelism > > > > > > > outside of the usual for bitbake. This was especially noticable > > > > > > > for > > > > > > > do_rootfs, which is single-threaded. > > > > > > > > > > > > Sadly the parallelism works on a per test class basis so it wouldn't > > > > > > help in this case. There are only small marginal gains from running > > > > > > tests in individual build directories so we don't do that. > > > > > > > > > > I estimate it could have saved me 2-3 minutes every build, since it > > > > > could > > > > > have parallelized the root filesystem stuff. > > > > > > > > On an initial run, it could have also ended up building a lot of pieces > > > > in parallel needlessly so it is all a bit of a compromise. It might be > > > > worth looking into whether we can make that an option, off by default. > > > > > > > > > > > This is ommitted above, but I *had* to use -j1 in order to avoid > > > > > > > manually wiping out my existing build directory each time (and > > > > > > > instead > > > > > > > ending up with dozens of pid-named directories). This is > > > > > > > documented > > > > > > > nowhere, and I found it in some old IRC logs. > > > > > > > > > > > > Parallelism using differently named build directories is an > > > > > > implementation detail, not something which the -j option implies.I > > > > > > guess you were also using --keep-builddir > > > > > > > > > > Failing builds don't remove the test directory so you can inspect the > > > > > build > > > > > output. As you might imagine, I had a lot of failing builds. > > > > > > > > I'm very familiar with that myself, yes. > > > > > > > > We did once used to reuse the build directory, that challenge is we > > > > have no idea what the user has done in there prior to the test so it > > > > potentially makes the test results potentially incorrect. > > > > > > > > > > > > We haven't really had anyone try and optimise the tests either, > > > > > > > > I'm > > > > > > > > sure there will be things in there which can help. Please don't > > > > > > > > let the > > > > > > > > speed put you off trying to improve things and extend our > > > > > > > > coverage! > > > > > > > > > > > > > > The poor speed of these self tests (and of everything related to > > > > > > > the > > > > > > > yocto project in general) makes this project frustrating to > > > > > > > contribute > > > > > > > to. It took me around 2 days to go from my prototype to this > > > > > > > series, > > > > > > > most of which was spent waiting for tests to compile and losing > > > > > > > whatever > > > > > > > train of thought I had. I probably went through perhaps 20 > > > > > > > revisions. If > > > > > > > I was working on e.g. U-Boot, I could have made 20 revisions in 2 > > > > > > > hours, > > > > > > > as it takes around 15 seconds to recompile it and run the full > > > > > > > unit test > > > > > > > suite. > > > > > > > > > > > > > > On the topic of these specific tests, part of the problem is that > > > > > > > do_rootfs is a bottleneck which takes around 45-60s on my system. > > > > > > > Every > > > > > > > test which modifies something in the rootfs incurs this overhead. > > > > > > > > > > > > For better or worse we've 'a few' more moving pieces than U-Boot. > > > > > > > > > > > > Building a root filesystem from packages is a non-trivial task, > > > > > > taking > > > > > > under a minute is in some ways pretty good already. The only other > > > > > > thing we could do is incremental rootfs construction where it would > > > > > > add/remove changed packages. I'd worry that the result may not > > > > > > always > > > > > > be equal to a build from scratch and it might cause weird and > > > > > > interesting reproducibility problems (particularly when you consider > > > > > > things like postinsts). > > > > > > > > > > > > I would love to improve our development "iteration" time but I'm > > > > > > struggling to see where we could get the speed gains from :(. Open > > > > > > to > > > > > > other ideas... > > > > > > > > > > We don't have to build a full root filesystem. All of these tests > > > > > just want > > > > > e.g. an initramfs. An empty (or one file) filesystem would work just > > > > > as well. > > > > > If you still want to boot, you can make a busybox filesystem. > > > > > > > > Could we update the test just to use an initramfs then? > > > > > > > > I'm definitely a fan of keeping the tests as simple as we can whilst > > > > still testing what we need to test. > > > > > > I can look into this, but I'd prefer to do it as a follow-up to this > > > series. > > > > > > I'll probably send a v2 later this week a fleshed-out commit message for > > > patch > > > 5/6 (and with it possibly all those variables moved to a separate bbclass > > > to > > > make it easier for other classes to create signed FITs). > > > > The series did already merge so anything would be incremental > > improvements at this point! > > Huh... > > I really wish you guys sent thank-you messages when you merged something. > Makes > it a lot easier to keep track of which series still need work.
That would result in a lot of messages on the mailing list and ends up being a lot of work for the maintainers as well. Looking at the repository is accurate and you can see exactly what merged and when... Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172376): https://lists.openembedded.org/g/openembedded-core/message/172376 Mute This Topic: https://lists.openembedded.org/mt/94487631/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
