On 04/12/2012 03:43 PM, Chris Tapp wrote: > On 12 Apr 2012, at 15:34, Darren Hart wrote: >> >> >> On 04/12/2012 07:08 AM, Björn Stenberg wrote: >>> Darren Hart wrote: >>>> /dev/md0 /build ext4 >>>> noauto,noatime,nodiratime,commit=6000 >>> >>> A minor detail: 'nodiratime' is a subset of 'noatime', so there is no >>> need to specify both. >> >> Excellent, thanks for the tip. >> >>> >>>> I run on a beast with 12 cores, 48GB of RAM, OS and sources on a >>>> G2 Intel SSD, with two Seagate Barracudas in a RAID0 array for my >>>> /build partition. I run a headless Ubuntu 11.10 (x86_64) >>>> installation running the 3.0.0-16-server kernel. I can build >>>> core-image-minimal in < 30 minutes and core-image-sato in < 50 >>>> minutes from scratch. >>> >>> I'm guessing those are rather fast cores? >> >> They are: >> model name : Intel(R) Xeon(R) CPU X5680 @ 3.33GHz > > Nice, but well out of my budget - I've got to make do with what one of your > CPUs costs for the whole system ;-) > >> >>> I build on a different type >>> of beast: 64 cores at 2.1GHz and 128 GB ram. The OS is on a single >>> SSD and the build dir (and sources) is on a RAID0 array of Intel 520 >>> SSDs. Kernel is the same ubuntu 3.0.0-16-server as yours. >> >> Now that I think about it, my downloads are on the RAID0 array too. >> >> One thing that comes to mind is the parallel settings, BB_NUMBER_THREADS >> and PARALLEL_MAKE. I noticed a negative impact if I increased these >> beyond 12 and 14 respectively. I tested this with bb-matrix >> (scripts/contrib/bb-perf/bb-matrix.sh). The script is a bit fickle, but >> can provide useful results and killer 3D surface plots of build time >> with BB and PM on the axis. Can't seem to find a plot image at the >> moment for some reason... >> >>> >>> Yet for all the combined horsepower, I am unable to match your time >>> of 30 minutes for core-image-minimal. I clock in at around 37 minutes >>> for a qemux86-64 build with ipk output: >>> >>> ------ NOTE: Tasks Summary: Attempted 1363 tasks of which 290 didn't >>> need to be rerun and all succeeded. >>> >>> real 36m32.118s user 214m39.697s sys 108m49.152s ------ >>> >>> These numbers also show that my build is running less than 9x >>> realtime, indicating that 80% of my cores sit idle most of the time. >> >> Yup, that sounds about right. The build has a linear component to it, >> and anything above about 12 just doesn't help. In fact the added >> scheduling overhead seems to hurt. >> >>> This confirms what "ps xf" says during the builds: Only rarely is >>> bitbake running more than a handful tasks at once, even with >>> BB_NUMBER_THREADS at 64. And many of these tasks are in turn running >>> sequential loops on a single core. >>> >>> I'm hoping to find time soon to look deeper into this issue and >>> suggest remedies. It my distinct feeling that we should be able to >>> build significantly faster on powerful machines. >>> >> >> Reducing the dependency chains that result in the linear component of >> the build (forcing serialized execution) is one place we've focused, and >> could probably still use some attention. CC'ing RP as he's done a lot there. > > Current plan for a 'budget' system is: > > DX79TO motherboard, i7 3820, 16GB RAM, a pair of 60GB OCZ Vertex III's in > RAID-0 for downloads / build, SATA HD for OS (Ubuntu 11.10 x86_64). > > That'll give me a 2.7x boost just on CPU and the SSDs (and maybe some > over-clocking) will give some more. > > Not sure if SSDs in RAID-0 will give any boost, so I'll run some tests. > > Thanks to all for the comments in this thread.
Get back to us with times, and we'll build up a wiki page. > > Chris Tapp > > opensou...@keylevel.com > www.keylevel.com -- Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
