I'd rather teach bitbake to abstain from starting new tasks when I/O or CPU gets tight.
Alex On Thu, 3 Dec 2020 at 18:48, Ross Burton <r...@burtonini.com> wrote: > Hi, > > Currently, BB_NUMBER_THREADS and PARALLEL_MAKE use the number of cores > available unless told otherwise. This was a good idea six years > ago[1] but some modern machines are moving to very large core counts. > > For example, 88 core dual Xeons are fairly common. A ThunderX2 has 256 > cores (2 sockets, 4 hyperthreads per physical core). The Ampere Altra > is dual socket 2*80=160 cores. > > At this level of parallelisation the sheer amount of I/O from the > unpack storm is quite excessive. As a strawman argument, I propose a > hard cap to the default BB_NUMBER_THREADS of -- and I'm literally > making up numbers here -- 32. Maybe 64. Comments? > > Cheers, > Ross > > [1] > http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=1529ef0504542145f2b81b2dba4bcc81d5dac96e > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#145246): https://lists.openembedded.org/g/openembedded-core/message/145246 Mute This Topic: https://lists.openembedded.org/mt/78690216/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
