On 11/1/22 13:44, Richard Purdie wrote: > On Tue, 2022-11-01 at 13:40 -0400, Sean Anderson wrote: >> On 11/1/22 13:29, Richard Purdie wrote: >> > On Tue, 2022-11-01 at 12:14 -0400, Sean Anderson wrote: >> > > On 10/28/22 11:37, Richard Purdie wrote: >> > > > On Fri, 2022-10-28 at 11:29 -0400, Sean Anderson wrote: >> > > > > On 10/28/22 11:09, Richard Purdie wrote: >> > > > > > On Wed, 2022-10-26 at 13:21 -0400, Sean Anderson wrote: >> > > > > > > As noted in the cover letter, I ran >> > > > > > > >> > > > > > > oe-selftest -r fitimage.FitImageTests >> > > > > > >> > > > > > Ok, good. That at least means you were only running one class of >> > > > > > tests. >> > > > > > I was worried you were running all of them! >> > > > > > >> > > > > > > I also tried using -j$(nproc), but I saw no increase in >> > > > > > > parallelism >> > > > > > > outside of the usual for bitbake. This was especially noticable >> > > > > > > for >> > > > > > > do_rootfs, which is single-threaded. >> > > > > > >> > > > > > Sadly the parallelism works on a per test class basis so it >> > > > > > wouldn't >> > > > > > help in this case. There are only small marginal gains from running >> > > > > > tests in individual build directories so we don't do that. >> > > > > >> > > > > I estimate it could have saved me 2-3 minutes every build, since it >> > > > > could >> > > > > have parallelized the root filesystem stuff. >> > > > >> > > > On an initial run, it could have also ended up building a lot of pieces >> > > > in parallel needlessly so it is all a bit of a compromise. It might be >> > > > worth looking into whether we can make that an option, off by default. >> > > > >> > > > > > > This is ommitted above, but I *had* to use -j1 in order to avoid >> > > > > > > manually wiping out my existing build directory each time (and >> > > > > > > instead >> > > > > > > ending up with dozens of pid-named directories). This is >> > > > > > > documented >> > > > > > > nowhere, and I found it in some old IRC logs. >> > > > > > >> > > > > > Parallelism using differently named build directories is an >> > > > > > implementation detail, not something which the -j option implies.I >> > > > > > guess you were also using --keep-builddir >> > > > > >> > > > > Failing builds don't remove the test directory so you can inspect >> > > > > the build >> > > > > output. As you might imagine, I had a lot of failing builds. >> > > > >> > > > I'm very familiar with that myself, yes. >> > > > >> > > > We did once used to reuse the build directory, that challenge is we >> > > > have no idea what the user has done in there prior to the test so it >> > > > potentially makes the test results potentially incorrect. >> > > > >> > > > > > > > We haven't really had anyone try and optimise the tests >> > > > > > > > either, I'm >> > > > > > > > sure there will be things in there which can help. Please >> > > > > > > > don't let the >> > > > > > > > speed put you off trying to improve things and extend our >> > > > > > > > coverage! >> > > > > > > >> > > > > > > The poor speed of these self tests (and of everything related to >> > > > > > > the >> > > > > > > yocto project in general) makes this project frustrating to >> > > > > > > contribute >> > > > > > > to. It took me around 2 days to go from my prototype to this >> > > > > > > series, >> > > > > > > most of which was spent waiting for tests to compile and losing >> > > > > > > whatever >> > > > > > > train of thought I had. I probably went through perhaps 20 >> > > > > > > revisions. If >> > > > > > > I was working on e.g. U-Boot, I could have made 20 revisions in >> > > > > > > 2 hours, >> > > > > > > as it takes around 15 seconds to recompile it and run the full >> > > > > > > unit test >> > > > > > > suite. >> > > > > > > >> > > > > > > On the topic of these specific tests, part of the problem is that >> > > > > > > do_rootfs is a bottleneck which takes around 45-60s on my >> > > > > > > system. Every >> > > > > > > test which modifies something in the rootfs incurs this overhead. >> > > > > > >> > > > > > For better or worse we've 'a few' more moving pieces than U-Boot. >> > > > > > >> > > > > > Building a root filesystem from packages is a non-trivial task, >> > > > > > taking >> > > > > > under a minute is in some ways pretty good already. The only other >> > > > > > thing we could do is incremental rootfs construction where it would >> > > > > > add/remove changed packages. I'd worry that the result may not >> > > > > > always >> > > > > > be equal to a build from scratch and it might cause weird and >> > > > > > interesting reproducibility problems (particularly when you >> > > > > > consider >> > > > > > things like postinsts). >> > > > > > >> > > > > > I would love to improve our development "iteration" time but I'm >> > > > > > struggling to see where we could get the speed gains from :(. Open >> > > > > > to >> > > > > > other ideas... >> > > > > >> > > > > We don't have to build a full root filesystem. All of these tests >> > > > > just want >> > > > > e.g. an initramfs. An empty (or one file) filesystem would work just >> > > > > as well. >> > > > > If you still want to boot, you can make a busybox filesystem. >> > > > >> > > > Could we update the test just to use an initramfs then? >> > > > >> > > > I'm definitely a fan of keeping the tests as simple as we can whilst >> > > > still testing what we need to test. >> > > >> > > I can look into this, but I'd prefer to do it as a follow-up to this >> > > series. >> > > >> > > I'll probably send a v2 later this week a fleshed-out commit message for >> > > patch >> > > 5/6 (and with it possibly all those variables moved to a separate >> > > bbclass to >> > > make it easier for other classes to create signed FITs). >> > >> > The series did already merge so anything would be incremental >> > improvements at this point! >> >> Huh... >> >> I really wish you guys sent thank-you messages when you merged something. >> Makes >> it a lot easier to keep track of which series still need work. > > That would result in a lot of messages on the mailing list and ends up > being a lot of work for the maintainers as well.
If you use a tool like b4 [1] it can automatically generate a summary thank-you message. As an example, [2]. This keeps the number of messages down and is not terribly burdensome for maintainers (since it fits into the usual patch workflow), You can set up something like pwbot [3], which integrates with patchwork and your repository. It produces messages like [4] automatically when it notices they have been merged into the repositiry, and marks them as "Accepted" in patchwork. This is even lower effort for your maintainers. --Sean [1] https://git.kernel.org/pub/scm/utils/b4/b4.git [2] https://lore.kernel.org/u-boot/164866411209.441601.3349958857188274518.b4...@gmail.com/ [3] https://korg.docs.kernel.org/patchwork/pwbot.html [4] https://lore.kernel.org/netdev/166702502193.25217.18098583636278445187.git-patchwork-not...@kernel.org/ > Looking at the repository is accurate and you can see exactly what > merged and when... I don't pull from upstream very often. Usually I only do it when upgrading releases, or when preparing a series. It's easy to miss when your patches get applied. However, since review is via email, I make sure to check that regularly so I don't miss anything. --Sean
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172377): https://lists.openembedded.org/g/openembedded-core/message/172377 Mute This Topic: https://lists.openembedded.org/mt/94487631/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
