Hi Alex, hi Richard
After some internal discussions, I would like to clarify my previous
answers on this topic.
* Usually there are two different workflows
- application developers: could use an SDK with a locked sstate-cache.
- Yocto/BSP developers: need an unlocked SDK. They change the recipes.
* A locked SDK
- can work with setscene from SSTATE_MIRRORS
- setscene does caching in the SSTATE_DIR (no issue about that)
- But network problems can occur during the initial build because
bitbake executes many independent setscene tasks. Opening so many
independent connections slows down the build, especially if the
server treats them as a denial of service attack.
- The denial of service problem is difficult to solve because each
setscene task runs in its own bibtake task. Reusing a connection to
download multiple sstate artifacts seems almost impossible.
This is much easier to solve with separate sstate download script.
* An unlocked SDK
- Tries to download the sstate cache for changed recipes and their
dependencies, which obviously can't work.
- The useless download requests slow down the build considerably and
cause a high load on the servers without any benefit.
- A script which gets a list of sstate artifacts from bitbake and then
does a upfront download works much better
+ The script runs only when the user calls it or the SDK gets boot-
strapped
+ The script uses a reasonable amount of parallel connections which
are re-used for more then one artifact download
* Idea for a smart lock/unlock implementation
- Form a user's perspective a locked vs. an unlocked SDK does not make
much sense. It makes more sense if the SDK would automatically
download the sstate-cache if it is expected to be available.
Lets think about an implementation (which allows to override the
logic) to switch from automatic to manual mode:
SSTATE_MIRRORS_ENABLED ?= "${is_sstate_mirror_available()}"
In our case the sstate mirror is expected to provide all artifacts
for tagged commits and for some git branches of the layer
repositories.
The sstate is obviousely not usable for a "dirty" git layer
repository. That's what the is_sstate_mirror_available function
could check to automatically enable and disable lazy downloads.
- If is_sstate_mirror_available() returns false, it should still be
possible to initiate a sstate-cache download manually.
* Terminology
- Older Yocto Releases:
+ eSDK means an installer which provides a different environment with
different tools
+ The eSDK was static, with a locked sstate cache
+ Was for one MACHINE, for one image...
- Newer Yocto Releases:
+ The bitbake environment offers all features of the eSDK installer. I
consider this as already implemented with meta-ide-support and
build-sysroots.
+ The term eSDK means a replicable bitbake environment. (The
documentation was recently changed in that sense)
+ The new SDK installer can be generated in different variants similar
to what was already supported by the eSDK
installer: https://docs.yoctoproject.org/sdk-manual/appendix-
customizing.html#customizing-the-extensible-sdk-standalone-
installer.
* The lightest variant is just a script that sets up the layers (git
clone or git checkout) and provides the build config. If
SSTATE_MIRRORS are configured lazy downloads will just work
otherwise bitbake will compile everything from scratch.
* The heaviest variant of the SDK installer includes the layers and
the sstate-cache. After installing it is_sstate_mirror_available()
evaluates to True.
* devtool ide
- Tries to bring this ideas further towards IDE configuration
- It supports two modes: No recipe mode is like the old eSDK
environment. The recipe mode goes beyond that. It is based on
devtool modify.
- Naming: I was thinking about "devtool ide" versus "devtool esdk"
+ Why ide? I want to set up my IDE to work with the Yocto SDK. And
that means, of course, that I have to bootstrap the SDK.
+ Could also be esdk: I want to bootstrap the eSDK and that should
also configure my IDE.
* Latest patch series is here:
- https://lists.openembedded.org/g/openembedded-core/message/189899
- docs: https://lists.yoctoproject.org/g/docs/message/4578
Adrian
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190170):
https://lists.openembedded.org/g/openembedded-core/message/190170
Mute This Topic: https://lists.openembedded.org/mt/101356420/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
