Hi,
during the task do_package_write_rpm I get the error "liblzma: Memory
allocation failed". It happens during packaging of binary RPM packages.
The root cause seems to be the host environment that is used in our
project. We run our builds on a big server with 32 cores and 256GB of
physical RAM but each user has a limit of virtual memory usage to 32GB
(ulimit -v). The packaging in rpm-native has been parallelized in the
commit
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta/recipes-devtools/rpm?id=84e0bb8d936f1b9094c9d5a92825e9d22e1bc7e3.
What seems to happen is that rpm-native put up 32 parallel tasks with
'#pragma omp', each task is using liblzma that also put up 32 tasks for
the compression work. The memory calculations in liblzma is based on the
amount of physical RAM but as the user is limited by 'ulimit -v' we get
into a OOM situation in liblzma.
Here is the code snippet from rpm-native/build/pack.c where it happens:
#pragma omp parallel
#pragma omp single
// re-declaring task variable is necessary, or older gcc versions will
produce code that segfaults
for (struct binaryPackageTaskData *task = tasks; task != NULL; task =
task->next) {
if (task != tasks)
#pragma omp task
{
task->result = packageBinary(spec, task->pkg, cookie, cheating,
&(task->filename), buildTime, buildHost);
rpmlog(RPMLOG_NOTICE, _("Finished binary package job, result %d, filename
%s\n"), task->result, task->filename);
}
}
Steps to reproduce is to set 'ulimit -v' in your shell to, for example,
1/8 of the amount of physical RAM and then build for example
glibc-locale. I have tested this with rocko. If the '#pragma omp'
statements in code snippet above is removed the problem is solved. But
that not good as the parallel processing speed up the process.
Is the host environment used here with restricted virtual memory
supported by Yocto? If it is, someone that have any suggestion for a
solution on this issue?
Best regards,
/Peter
--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
