I think it’s ok to have it in Glibc/localedef not all distros have this version of util Linux yet and perhaps we should start creating a buildtools tarballs which is a prerequisite for building OE and collect these dependencies in there instead Of sticking them in unusual places
On Fri, Aug 16, 2019 at 7:59 PM Jason Wessel <jason.wes...@windriver.com> wrote: > On 8/16/19 4:50 PM, richard.pur...@linuxfoundation.org wrote: > > On Fri, 2019-08-16 at 14:29 -0700, Khem Raj wrote: > >> On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie > >> <richard.pur...@linuxfoundation.org> wrote: > >>> This was discussed on IRC. We really really want to avoid that as a > >>> dependency if we care about build performance. Its really easy to > >>> just > >>> add that in but the implications are significant :( > >>> > >> A captured version of this utility would also mean that we have to > >> keep this in sync for bug fixes and security fixes etc. which could > >> also be considered as expense, so how much do we lose if util-linux- > >> native is brought early into dependency chain? > > The tool is being used for a specific purpose at build time. Its not > > exposed on target and its use is extremely constrained to a specific > > set of files where its behaviour should be deterministic. > > > > As such, yes there are questions about bug fixes and about security but > > I don't think its going to be a serious burden. We'd need to carry this > > until such times as the host systems we support have hardlink which is > > probably a few years out but will arrive. > > > > The impact of having util-linux-native in the dependency graph at this > > point is significant. I say that as someone who has done a lot of work > > in trimming down the times of our bootstrap process. This is exactly > > the kind of dependency I've removed and avoided. > > > > glibc is special in that its the worst choke point on the dependency > > graph. Nothing to do with target binaries happens until it is ready and > > then gcc-runtime can build. Whilst the locales are split from glibc, > > they are still connected in the task graph for packaging purposes so > > this is important. > > > > Its been pointed out there are other places where util-linux-native has > > crept in, in particular python3-native. My preference would be to > > minimise these dependencies rather than use them as a way to justify > > adding it here too. > > > So what is the consensus here? Can we require the host system to have > hardlink? It is available as a stand alone binary on Ubuntu 14.04 and up > and on Fedora systems, but isn't installed by default, and we'd have to > check the version because the arguments sadly are slightly different. We > could certainly make util-linux a dependency, but that is why this patch > existed, in order to avoid the dependency. > > I did find a python version at > https://raw.githubusercontent.com/akaihola/hardlinkpy/master/hardlink.py. > I made a one line modification to it and it works, but it isn't nearly as > efficient. The proof is below, using the same locale-tree each time. It > is important to look at both the real and user time. The difference gets > more significant when you run it under pseudo, where we expect it to run > 100% of the time. It is roughly 10x slower to do this in python. > > > time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v > locale-tree > Directories: 982 > Objects: 7441 > Regular files: 6441 > Comparisons: 3331 > Linked: 3109 > Saved: 754675712 > > real 0m0.641s > user 0m0.120s > sys 0m0.518s > > time python3 hardlink.py -v 0 -c locale-tree > > > Hard linking Statistics: > Directories : 982 > Regular files : 6441 > Comparisons : 31884 > Hardlinked this run : 3109 > Total hardlinks : 3109 > Bytes saved this run : 744089843 (709.619 mebibytes) > Total bytes saved : 744089843 (709.619 mebibytes) > Total run time : 2.124422073364258 seconds > > real 0m2.180s > user 0m1.220s > sys 0m0.953s > > > ============Now lets run it under pseudo ================ > > time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v > locale-tree > Directories: 982 > Objects: 7441 > Regular files: 6441 > Comparisons: 3331 > Linked: 3109 > Saved: 754675712 > > real 0m2.505s > user 0m0.373s > sys 0m1.286s > > time python3 hardlink.py -v 0 -c locale-tree > > > Hard linking Statistics: > Directories : 982 > Regular files : 6441 > Comparisons : 31884 > Hardlinked this run : 3109 > Total hardlinks : 3109 > Bytes saved this run : 744089843 (709.619 mebibytes) > Total bytes saved : 744089843 (709.619 mebibytes) > Total run time : 12.971824884414673 seconds > > real 0m13.046s > user 0m3.633s > sys 0m5.192s > > >
-- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
