On Fri, Apr 13, 2012 at 07:51:51AM +0200, Martin Jansa wrote: > On Thu, Apr 12, 2012 at 04:37:00PM -0700, Flanagan, Elizabeth wrote: > > On Thu, Apr 12, 2012 at 7:12 AM, Darren Hart <dvh...@linux.intel.com> wrote: > > > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA1 > > > > > > > > > > > > On 04/12/2012 01:00 AM, Martin Jansa wrote: > > > > On Thu, Apr 12, 2012 at 01:05:00PM +0530, Joshua Immanuel wrote: > > > >> Darren, > > > >> > > > >> On Wed, 2012-04-11 at 17:30 -0700, Darren Hart wrote: > > > >>> I run on a beast with 12 cores, 48GB of RAM, OS and sources on > > > >>> a G2 Intel SSD, with two Seagate Barracudas in a RAID0 array > > > >>> for my /build partition. I run a headless Ubuntu 11.10 (x86_64) > > > >>> installation running the 3.0.0-16-server kernel. I can build > > > >>> core-image-minimal in < 30 minutes and core-image-sato in < 50 > > > >>> minutes from scratch. > > > > > > > > why not use so much RAM for WORKDIR in tmpfs? I bought 16GB just to > > > > be able to do my builds in tmpfs and keep only more permanent data > > > > on RAID. > > > > > > We've done some experiments with tmpfs, adding Beth on CC. If I recall > > > correctly, my RAID0 array with the mount options I specified > > > accomplishes much of what tmpfs does for me without the added setup. > > > > > > > This should be the case in general. For the most part, if you have a decent > > RAID setup (We're using RAID10 on the ab) with fast disks you should be > > able to hit tmpfs speed (or close to it). I've done some experiments with > > this and what I found was maybe a 5 minute difference, sometimes, from a > > clean build between tmpfs and RAID10. > > 5 minutes on very small image like core-image-minimal (30 min) is 1/6 of > that time :).. > > I have much bigger images and even bigger ipk feed, so to rebuild from > scratch takes about 24 hours for one architecture.. > > And my system is very slow compared to yours, I've found my measurement > of core-image-minimal-with-mtdutils around 95 mins > http://patchwork.openembedded.org/patch/17039/ > but this was with Phenom II X4 965, 4GB RAM, RAID0 (3 SATA2 disks) for > WORKDIR, RAID5 (the same 3 SATA2 disks) BUILDDIR (raid as mdraid), now > I have Bulldozer AMD FX(tm)-8120, 16GB RAM, still the same RAID0 but > different motherboard.. > > Problem with tmpfs is that no RAM is big enough to build whole feed in > one go, so I have to build in steps (e.g. bitbake gcc for all machines > with the same architecture, then cleanup WORKDIR and switch to another > arch, then bitbake small-image, bigger-image, qt4-x11-free, ...). > qt4-x11-free is able to eat 15GB tmpfs almost completely. > > > I discussed this during Yocto Developer Day. Let me boil it down a bit to > > explain some of what I did on the autobuilders. > > > > Caveat first though. I would avoid using autobuilder time as representative > > of prime yocto build time. The autobuilder hosts a lot of different > > services that sometimes impact build time and this can vary depending on > > what else is going on on the machine. > > > > There are four places, in general, where you want to look at optimizing > > outside of dependency issues. CPU, disk, memory, build process. What I > > found was that the most useful of these in getting the autobuilder time > > down was disk and build process. > > > > With disk, spreading it across the RAID saved us not only a bit of time, > > but also helped us avoid trashed disks. More disk thrash == higher failure > > rate. So far this year we've seen two disk failures that have resulted in > > almost zero autobuilder downtime. > > True for RAID10, but for WORKDIR itself RAID0 is cheeper and even higher > failure rate it's not big issue for WORKDIR.. just have to cleansstate > tasks which were in hit in the middle of build.. > > > The real time saver however ended up being maintaining sstate across build > > runs. Even with our sstate on nfs, we're still seeing a dramatic decrease > > in build time. > > > > I would be interested in seeing what times you get with tmpfs. I've done > > tmpfs builds before and have seen good results, but bang for the buck did > > end up being a RAID array. > > I'll check if core-image-minimal can be built with just 15GB tmpfs, > otherwise I would have to build it in 2 steps and the time wont be > precise.
It was enough with rm_work, so here are my results: The difference is much smaller then I've expected, but again those are very small images (next time I'll try to do just qt4 builds). Fastest is TMPDIR on tmpfs (BUILDDIR is not important - same times with BUILDDIR also in tmpfs and on SATA2 disk). raid0 is only about 4% slower single SATA2 disk is slowest but only a bit slower then raid5, but that could be caused by bug #2314 as I had to run build twice.. And all times were just from first successfull build, it could be different with avg time over 10 builds.. And all builds on AMD FX(tm)-8120 Eight-Core Processor 16G DDR3-1600 RAM standalone SATA2 disk ST31500341AS mdraid on 3 older SATA2 disks HDS728080PLA380 bitbake: commit 4219e2ea033232d95117211947b751bdb5efafd4 Author: Saul Wold <s...@linux.intel.com> Date: Tue Apr 10 17:57:15 2012 -0700 openembedded-core: commit 4396db54dba4afdb9f1099f4e386dc25c76f49fb Author: Richard Purdie <richard.pur...@linuxfoundation.org> Date: Sat Apr 14 23:42:16 2012 +0100 + fix for opkg-utils, so that package-index doesn't take ages to complete BUILDDIR = 1 SATA2 disk TMPDIR = tmpfs real 84m32.995s user 263m46.316s sys 48m26.376s BUILDDIR = tmpfs TMPDIR = tmpfs real 84m10.528s user 264m16.144s sys 50m21.853s BUILDDIR = raid5 TMPDIR = raid5 real 91m20.470s user 263m47.156s sys 52m23.400s BUILDDIR = raid0 TMPDIR = raid0 real 87m29.526s user 263m0.799s sys 51m37.242s BUILDDIR = 1 SATA2 disk TMPDIR = the same SATA2 disk Summary: 1 task failed: /OE/oe-core/openembedded-core/meta/recipes-core/eglibc/eglibc_2.15.bb, do_compile Summary: There was 1 ERROR message shown, returning a non-zero exit code. see https://bugzilla.yoctoproject.org/show_bug.cgi?id=2314 real 48m23.412s user 163m55.082s sys 23m26.990s + touch oe-core/tmp-eglibc/work/x86_64-oe-linux/eglibc-2.15-r6+svnr17386/eglibc-2_15/libc/Makerules + Summary: There were 6 WARNING messages shown. real 44m13.401s user 92m44.427s sys 27m38.347s = real 92m36.813s user 255m99.509s sys 51m05.337s -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
signature.asc
Description: Digital signature
_______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
