Hi Takashi, I read that python can't execute multiple threads simultaneously, but that wouldn't be a problem if each python task is executed at its own interpreter instance (process). This last statement is what I don't really know, some experts on Yocto's internals may clarify that, maybe :)
Regards, 2016-06-30 22:29 GMT-03:00 Takashi Matsuzawa <tmatsuz...@uievolution.com>: > > Hello Yocto. > > Well, this may not be a very new topic, but I wonder if you have recent > suggetions. > So far I have tried the followings and see some improvement, making the build > time to days to hours. > > 1) All of the directories, DL_DIR, SSTATE_CACHE, TMPDIR, DEPLOY_DIR are on > SSD. > 2) .repo and working directory (where I synch my recipes) also on SSD. > 3) Set BB_NUMBER_PARSE_THREADS/PARALLEL_MAKE/BB_NUMBER_THREADS to something > like 20, 30, etc. > 4) Enable use of ICECC. > 5) Use pigz, pbzip2, pxz instead of gzip, bzip2 and xz. > > I can see with 3-5), I am benefited from my multicore PCs and see some > scaling effects (if I can add more PCs or CPU core.) > However I think there is i the build not paralleled or distributed easily. > > i) python tasks > They say python has slimitation of being single thread, and could be a bottle > neck since most of yocto build tasks are in python, if not native/cross > toolchain jobs. > > ii) zip / tar tasks > e.g. seeing rootfs taking forever, jobs not distributed to the other machines. > I understand it is writing files to one target tree and doing a bit tar/zip > for the files, being one big job though. > > I wonder if I can do anything with i) and ii) at least to make them processed > in parallel. > ICECC only can dispatch gcc jobs I believe. > Any previous attemps to do the similar to non-gcc jobs? > -- > _______________________________________________ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto -- "Do or do not. There is no try" Yoda Master -- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
