On Wed, Jun 07, 2017 at 02:23:04PM +0100, Luca Boccassi wrote: > On Wed, 2017-06-07 at 13:08 +0000, Van Haaren, Harry wrote: > > > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Bruce > > > Richardson > > > Sent: Wednesday, June 7, 2017 11:48 AM > > > To: dev@dpdk.org > > > Cc: Richardson, Bruce <bruce.richard...@intel.com> > > > Subject: [dpdk-dev] [RFC PATCH] replace DPDK config and build > > > system > > > > > > Hi all, > > > > > > following on from the pressing need to add support in DPDK for > > > detecting > > > and managing external dependencies, I undertook to see what options > > > we had. > > > However, unrelated to this, over time, I have become increasingly > > > frustrated by the complexity of the DPDK configuration and build > > > system. As > > > such, I feel that looking at some of the newer build tools that are > > > out > > > there might give us the additional functionality we want, along > > > with other > > > benefits. As such, I undertook a prototype using "meson"[1] for > > > configuration, which uses "ninja" as a backend for doing the actual > > > build. > > > > > > With these tools we get the following benefits (not a complete > > > list): > > > * support for detecting dependencies on the system > > > * support for detecting compiler features, including functions, > > > defines > > > * improved maintainability through a high-level language, which > > > gives > > > decent error messages including line numbers (!!) > > > * co-existence with the existing makefile system without making any > > > changes > > > to it > > > * faster builds using ninja - on my many-core system, the builds > > > seem > > > significantly faster than with our existing system. Especially in > > > the > > > nothing-has-changed case, builds with my prototype return > > > instantly, > > > compared to taking a few seconds to recursively check each > > > directory with > > > the current build system > > > * the ability to switch to using a standard "ninja" + "ninja > > > install" setup > > > * the chance to rework our existing build-config files, and > > > hopefully > > > pretty much remove them. > > > * pkg-config support. > > > * we get to move away from our bespoke build system > > > * dependencies in each lib can be moved back to being tracked in > > > the libs > > > files themselves, not up a level > > > > > > > > > Of course, it's not a panacea, but having spent hours on the > > > prototype thus > > > far, I find working with meson and ninja far more user-friendly > > > than > > > working on our makefiles, and again the build speed is a really > > > nice > > > improvment too. > > > > > > The prototype is incomplete, but it does build a reasonable number > > > of our > > > libraries, some unit tests, the i40e PMD and the testpmd binary, > > > and I have > > > successfully passed traffic using testpmd from the build. Some > > > things are > > > not fully correct, e.g. static builds aren't working right now, as > > > I haven't > > > correctly done all the dependency tracking, I think, and the cpu > > > flag > > > detection has issues. It also has only been tried on x86_64 linux, > > > on a > > > couple of systems, so YMMV. However, I feel it's a reasonable > > > enough start > > > point to show what we might be able to achieve. > > > > > > Please take the prototype and test it out. I think it's a better > > > alternative to trying to bolt on additional functionality to our > > > existing > > > config and build system. > > > > > > Test drive done; here are my experiences / thoughts: > > > > 1) Understanding the Meson build files is much easier for me than the > > current build-files. I'll admit that bash scripting is not my forte, > > with the caveat that if my basic bash scripting doesn't suffice, > > others in the community are probably in a similar position. > > > > 2) I see huge value in pkg-config integration - linking made easy. > > From a developer usability POV, a project should provide a pkg-config > > file, and configure itself based on the pkg-config files available on > > the system. > > > > 3) The speed of ninja is impressive - particularly in the small- > > amounts-of-work case. Try it on your own machine if you don't believe > > me :) > > > > 4) "build variants" are super easy, meson build && meson > > build_variant will create two directories, with .o files contained > > under it (same as one possible usage of the current system - just > > calling out that that feature remains). > > > > 5) Vim users, there is a "mesonic" plugin[1] which provides syntax > > highlighting for Meson files, the Meson Options file, and Meson > > integration to the Vim compiler hooks. Aka typing :make will cause > > Ninja to be invoked and compile. There are more fancy features, check > > the script page. > > > > 6) Ninja uninstall will reverse a previous installation, and returns > > a clean system. Nice for testing and integrating DPDK with > > applications that expect DPDK to be installed system-wide. > > > > > > Of course no build system is perfect - and I'd guess Meson also has a > > few gritty details that are not ideal - but to me this test-drive has > > shown promise. Opinions from non-developers may be of particular > > interest.. > > > > @DPDK Packagers, do you have experience with Meson? Does it integrate > > well with (your) existing infrastructure? Hope you don't mind me > > adding you to CC :) > > While I do not have personal experience with it, I know that recently > the Debian/Ubuntu tools have gained support for meson+ninja (debhelper > version 10.3), so it should not be a problem for myself and Christian. > > In fact, I personally think it's a big and very welcome improvement, so > thank you Bruce for this work! > > I will try to find time soon to test this patch in the context of > packaging the libraries. >
Great, thanks for taking a look at it. Please note that this does require meson 0.4 right now, because we do so much stuff with optimizing for instruction sets we really need the new "has_define()" function that comes only in meson 0.4. However, I anticipate that given the time it is likely to take to get anything like this into DPDK, meson 0.4 should be pretty widespread by then. :-) /Bruce
