Not quite a week ago I wrote about an idea to use XML files to store
the
declarative part of our makefiles: dependencies of libraries on source
files, which resources are to be created and so on. In the meantime
I have
found the time to do make (conduct?) an experiment. I am now able
to build
module sw from the XML files with the help of the ninja build
'system' [3].
Most of the work of converting the XML files into one single
build.ninja
file was done on one weekend. You can see the source code at [1] ([2]
contains everything zipped together).
The results are promising. It runs faster and the build.ninja
generator
looks more maintainable than our solenv/gbuild/... makefiles. But I am
certainly biased.
Before I give you some numbers, I should say that I have collected the
numbers totally unscientifically and it may be necessary to add some
missing
steps to the ninja build. To the best of my knowledge all C++ files
are
compiled, libraries linked, resource files built, XML files copied.
Only
the single sw.component file somehow escaped.
I ran my experiments on ani7 2.2GHz, 8GB notebook.
Complete build of a clean module:
gbuild about 9m30s (make -sr -j8)
ninja about 7m15s (ninja)
Cleaning up
gbuild about 40s (make clean)
ninja less then 1s (ninja -t clean)
rebuild after touching one single header (sw/inc/section.hxx)
gbuild about 1m10s (make -sr -j8)
ninja about 50s (ninja)
Building an already built module (nothing to do): depends very much on
whether the disk cache is warm or cold. Best times:
gbuild more than 3s (make -sr -j8)
ninja about 0.4s (ninja)
Why is ninja faster than make/gbuild?
- Make runs each recipe in its own shell (bash), ninja executes its
command
directly.
- Ninja understands the header dependencies created by gxx/clang and
msvc
and stores them in a compact format that can be read in very fast on
startup.
- I avoided some steps of build that are unnecessary in ninja
= Ninja creates directories for the targets it makes. Gbuild
creates them
explicitly.
= GBuild first creates empty dependency files and later, in a
second step,
fills them with the actual dependency information created by one of the
C/C++ compilers.
But, for me, these numbers are just a welcome side effect. More
important
to me is maintainability.
Ninja follows a very different approach from (GNU) make. Its lack
of even
simplest control structures such as if/then/else or foreach,
requires the
generation of the main makefile (by default that is called
build.ninja) by
program or script. This leads to my current approach:
- Use XML to represent the static data (C++ files, libraries, resource
files, XML files).
- Use a Perl script to translate the XML files into the build.ninja
file.
The best tool for each job (XML: data representation, Perl: data
processing). Instead of Perl we could use any language that is part
of our
current build requirements (Java, C/C++, Python (we would have to
compile
that first, though)). Look at the Perl files in [1] or [2]
(build/source/ninja/*pm) and compare them to solenv/gbuild/*mk and
see which
you can understand better.
I think this could be one way to set up a better maintainable build
system
that is even slightly faster then what we currently have.
