> -----Original Message----- > From: Jerin Jacob [mailto:jerin.ja...@caviumnetworks.com] > Sent: Tuesday, January 24, 2017 12:57 PM > To: Richardson, Bruce <bruce.richard...@intel.com> > Cc: Olivier Matz <olivier.m...@6wind.com>; dev@dpdk.org; Yigit, Ferruh > <ferruh.yi...@intel.com>; thomas.monja...@6wind.com > Subject: Re: [dpdk-dev] [PATCH] mk: optimize directory dependencies > > On Tue, Jan 24, 2017 at 12:15:06PM +0000, Bruce Richardson wrote: > > On Tue, Jan 24, 2017 at 05:10:15PM +0530, Jerin Jacob wrote: > > > On Mon, Jan 23, 2017 at 06:19:13PM +0100, Olivier Matz wrote: > > > > Before this patch, the management of dependencies between > > > > directories had several issues: > > > > > > > > - the generation of .depdirs, done at configuration is slow: it can > take > > > > more than one minute on some slow targets (usually ~10s on a > standard > > > > PC). > > > > > > > > - for instance, it is possible to expressed a dependency like: > > > > - app/foo depends on lib/librte_foo > > > > - and lib/librte_foo depends on app/bar > > > > But this won't work because the directories are traversed with a > > > > depth-first algorithm, so we have to choose between doing 'app' > before > > > > or after 'lib'. > > > > > > > > - the script depdirs-rule.sh is too complex. > > > > > > > > - we cannot use "make -d" for debug, because the output of make is > used for > > > > the generation of .depdirs. > > > > > > > > This patch moves the DEPDIRS-* variables in the upper Makefile, > > > > making the dependencies much easier to calculate. A DEPDIRS > > > > variable is still used to process library dependencies in LDLIBS. > > > > > > > > After this commit, "make config" is almost immediate. > > > > > > > > Signed-off-by: Olivier Matz <olivier.m...@6wind.com> > > > > > > Tested both approach on ThunderX. This patch looks better > > > > > > Tested-by: Jerin Jacob <jerin.ja...@caviumnetworks.com> > > > > > > ➜ [master]GB-2S [dpdk-master] $ time make config > > > T=arm64-thunderx-linuxapp-gcc Configuration done > > > > > > real 0m18.112s > > > user 0m2.810s > > > sys 0m0.660s > > > > > > ➜ [master]GB-2S [dpdk-master] $ pwclient git-am 19859 Applying patch > > > #19859 using 'git am' > > > Description: [dpdk-dev] mk: parallelize make config > > > Applying: mk: parallelize make config > > > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build ➜ [master]GB-2S > > > [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 8 > > > Configuration done > > > > > > real 0m2.812s > > > user 0m3.020s > > > sys 0m0.870s > > > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build ➜ [master]GB-2S > > > [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 16 > > > Configuration done > > > > > > real 0m1.748s > > > user 0m3.040s > > > sys 0m1.020s > > > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build ➜ [master]GB-2S > > > [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 32 > > > Configuration done > > > > > > real 0m1.422s > > > user 0m3.380s > > > sys 0m1.080s > > > > > > ➜ [master]GB-2S [dpdk-master] $ pwclient git-am 19918 Applying patch > > > #19918 using 'git am' > > > Description: [dpdk-dev] mk: optimize directory dependencies > > > Applying: mk: optimize directory dependencies > > > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build ➜ [master]GB-2S > > > [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc > > > Configuration done > > > > > > real 0m0.064s > > > user 0m0.000s > > > sys 0m0.000s > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build ➜ [master]GB-2S > > > [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 8 > > > Configuration done > > > > > > real 0m0.055s > > > user 0m0.000s > > > sys 0m0.000s > > > > > > > I agree that Olivier's patch is faster. However, I think I prefer > > having the library dependencies in the makefiles for the libs > > themselves rather than up a level. Given that we are only looking at > > ~2second of a difference here in your tests - assuming -j flag - what > > is the actual build time differences? My suspicion is that after > > Ferruh's simpler > > Without patch - 18sec > With patch -j1 - 18 sec > With patch -j2 - 9.2 sec > With patch -j4 - 4.9 sec > With patch -j8 - 2.8 sec > With patch -j16 - 1.7 sec > With patch -j32 - 1.4 sec > > > patch is applied, the config time becomes such a small part of the > > build, that the extra benefits from Oliviers work is not worth the > > extra complexity. > > The low-end embedded SoCs (SoC with 2 to 4 cores) will be get benefited > out of bring this extra complexity.My take is, if it is manageable > complexity then take the most optimized one.
Ok. Point taken for the lower-core count parts. Thomas - can at least one of these patches be merged into 17.02, since it will definitely help us developers? [If Olivier's is too big a change at this point in the cycle, can Ferruh's be taken as an intermediate fix for this release before merging Olivier's in 17.05?] Regards, /Bruce
