> -----Original Message----- > From: Khem Raj [mailto:raj.k...@gmail.com] > Sent: Thursday, June 16, 2011 9:29 AM > To: Xu, Dongxiao > Cc: Richard Purdie (richard.pur...@linuxfoundation.org); > yocto@yoctoproject.org > Subject: Re: [yocto] configure optimization feature update > > On Wed, Jun 15, 2011 at 5:57 PM, Xu, Dongxiao <dongxiao...@intel.com> > wrote: > > Hi Richard, > > > > Recently I was doing the "configure optimization" feature and collecting > > data > for it. > > > > The main logic of this feature is straight forward: > > > > 1. Use the diff file as autoreconf cache. (I use command: "diff -ruN > SOURCE-ORIG SOURCE", here "SOURCE-ORIG" is the source directory before > running autoreconf, while "SOURCE" is the directory after running autoreconf). > > 2. Add SRC_URI checksum for all patches of the source code. > > 3. Tag each autoreconf cache file with ${PN} and the SRC_URI checksum of > source code and all patches. > > 4. If the currently SRC_URI checksum matches the cached checksum, then we > can patch the cache instead of running "autoreconf" stage. > > > > The autoconf'ing is sort of arbitrary at the moment. Depending on what is > staged the results may vary. So some way of creating a cache is nice since it > can be used to verify if package rebuild happens with same configure variables > or not. Sometimes we have seen some packages are build differently first time > since many packages are not staged. > > > > I did some testings for sato build, the result is not as good as we > > expected: > > > > On a server build machine (Genuine Intel(R) CPU @ 2.40GHz, 2 sockets with 6 > core each and hyperthreading, thus 24 logical CPUs in all, 66G memory): > > > > w/o the optimization: > > real 83m40.963s > > user 496m58.550s > > sys 329m1.590s > > > > w/ the optimization: > > real 79m1.062s > > user 460m58.600s > > sys 347m42.120s > > > > It has about 5% performance gain. > > > > I also tested the patch on a desktop core-i7 machine (Intel(R) Core(TM) i7 > CPU 870 @ 2.93GHz, 4 core 8 logical CPU, 4G memory): > > > > w/o the optimization: > > real 105m25.436s > > user 372m48.040s > > sys 51m23.950s > > > > w/ the optimization: > > real 103m38.314s > > user 332m35.770s > > sys 49m4.520s > > > > It only has about 2% performance gain. > > > > The result is not encouraging. > > > > There are also some other things we need to take into consideration for this > feature: > > > > 1. If add this feature, the first build time should be longer than current > > since it > needs to build the autoreconf cache. > > 2. Maintainers needs to maintain the SRC_URI checksums not only for source > code, but also all its patches. For some recipes, it has more than 20 patches, > which needs assignable maintenance effort. > > Yeah thats definite pain. > > > 3. How to distribute the caches will be a problem. The total size of such > > cache > is about 900M (before compression) and 200M (after compression). Since the > size is not small, distributing it with Poky source code doesn't make sense. > On > another aspect, we can use something like "sstate". But since we already have > caches of sstate, I think it is not necessary for us to enable another similar > cache mechanism with little improvement. > > > > hmm this is a real problem and probably the perf killer. I wonder why the > sizes > are so big.
I diff the source files before and after autoreconf and using the generated patch as our autoreconf cache. For a certain source code, for example connman, such diff file will be as large as 3M, and for a sato build, we have about 300 recipes need autoreconf. Thanks, Dongxiao > > > Therefore my opinion is we may give up this feature. What's your comments > and suggestions? > > > > Thanks, > > Dongxiao > > > > _______________________________________________ > > yocto mailing list > > yocto@yoctoproject.org > > https://lists.yoctoproject.org/listinfo/yocto > > _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
