On 01.08.23 15:40, Alexander Kanavin wrote: > In short, yes. Sstate files are first written into a random temporary > file, then renamed when writing is done. > DL_DIR items are guarded by lock files (look for lockfile in > bitbake/lib/bb/fetch2/__init__.py). > > Clearing out old items from sstate and download caches must be done > when no builds are running.
OK, awesome, thanks. I'll give it a try then. Thanks for pointing me towards fetch2/__init__.py too, I was only looking at fetch2/git.py previously. > > Alex > > On Tue, 1 Aug 2023 at 15:26, Daniel Klauer <daniel.kla...@gin.de> wrote: >> >> Hi, >> >> Is it safe to share DL_DIR and SSTATE_DIR between multiple Yocto builds that >> are running in parallel? Are all the accesses done by bitbake/Yocto atomic >> or protected by locking? This came up because I'm wondering whether it would >> be possible to safely share a single downloads mirror and sstate cache >> between multiple build agents (and not just read-only over a HTTP server, >> but writing too). >> >> The documentation contains some sentences like "You can safely share this >> directory between multiple builds on the same development machine" and "You >> can reuse the directory for multiple builds", but this does not seem to >> clarify whether parallel builds are allowed. I have found an older message >> [1], which suggests that SSTATE_DIR is safe to share, but that still leaves >> me wondering about DL_DIR. Presumably bitbake uses plain rename()/mv for >> tarballs, as for sstate, making them thread-safe. But what about Git repos - >> it's clear that Git itself uses locking, but bitbake may do more than just >> git operations. Does anyone have any more information on this, besides >> digging into the bitbake and bbclass sources? Is it "thread-safe"? A >> potentially related question: How does bitbake handle multiple do_fetch >> tasks for the same repository running in parallel (if you have multiple >> recipes using the same repo)? >> >> Thanks & kind regards, >> Daniel >> >> [1] https://www.yoctoproject.org/pipermail/yocto/2016-November/032873.html >> <https://www.yoctoproject.org/pipermail/yocto/2016-November/032873.html> >> >> >>
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60699): https://lists.yoctoproject.org/g/yocto/message/60699 Mute This Topic: https://lists.yoctoproject.org/mt/100483492/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
