On 09.11.21 09:48, Robert Yang wrote:
The original value is very easy to cause do_packge error when cpu number is
larger, for example, 128 cores and 512G mem:
error: create archive failed: cpio: write failed - Cannot allocate memory"
Set the ZSTD_THREADS to half of the CPU number can avoid the error in my
testing.
Signed-off-by: Robert Yang <liezhi.y...@windriver.com>
---
meta/conf/bitbake.conf | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 71c1e52ad6..46ebf5113f 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -833,7 +833,7 @@ XZ_DEFAULTS ?= "--memlimit=${XZ_MEMLIMIT}
--threads=${XZ_THREADS}"
XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
# Default parallelism for zstd
-ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
+ZSTD_THREADS ?= "${@int(oe.utils.cpu_count(at_least=4)/2)}"
Then why not just limit it for the large setups you are referring to in
the example, for instance like
ZSTD_THREADS ?= "${@min(int(oe.utils.cpu_count(at_least=4)), <insert a
sane value of your choice>)}"
BTW this can be also done in your local.conf - as Alex already said,
there is simply no reason to make it slower for everyone, while also
half the threads of a 128 core machine could be a trouble some setup.
Last time I had this issue (with XZ back then) I used
"${@min(int(oe.utils.cpu_count(at_least=4)), 20}"
ZSTD_THREADS[vardepvalue] = "1"
# Limit the number of threads that OpenMP libraries will use. Otherwise they
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158001):
https://lists.openembedded.org/g/openembedded-core/message/158001
Mute This Topic: https://lists.openembedded.org/mt/86926962/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
