On Thursday, August 2, 2001 9:27, Scott A Crosby wrote:
> Hello.
>
> I read the ``Recursive Make Considered harmful'' paper[1] about a year ago.
Just an interjection...
I found a minor problem in depend.sh (or in gcc, depending on how you look at
it) with subdirectories. But I fixed it with more sed. See below.
> As I was recently faced with creating my first large codebaes, I needed a
> build system. I decided to do one the 'right way', so I could reuse it for
> future projects. I implemented the scheme described in the paper. Here,
> give my build system and my observations of it in use. Overall, it is very
> pleasant to use; it is error-free in parallel-make configurations, it does
> full dependency information automatically, and it always does the minimum
> work necessary during building.
I know, I like it too, and I was able to easily modify it to support things
such as moc headers.
> To satisfy GNU standards, it would need to be adapted and changed. I
> cannot do this work; I do not know all the subtle issues with Make, nor am
> I really interested in learning.
Where can I read about these standards you speak of?
> But I do offer this as a starting point for someone else.
>
> I am including the full buildsystem I use. This is all of my makefiles,
> but no source code.
>
> There are two issues dealing with subtleties of make semantics that I
> would appreciate help and/or suggestions in fixing. They are with
> program-generated code and with fragility during cleaning.
One problem I have is that I must manually remove the .d files before
deleting a .h file, or make'll barf. Another one is that vim thinks that
make's initial inability to find the .d files constitutes an error.
> If anyone has any suggestions for fixes to these, I'll be happy to test
> them and post up another tarball later.
>
> Finally, please keep me on the CC in these messages.
Okay.
> -- What I have
>
> I currently have about 30 .cpp and 40 .h files in 5 directories, and I've
> followed the advice given. I build 10 binaries and 2 libraries. If there
> is nothing to be done, it takes make .04 seconds to realize this. This
> includes 140 stat's, and opening 50 files for including. Admittiedly this
> isn't the largest project thats been done, but it is respectible in size.
Heh, I've never done anything so large...
> Dependencies are built fully automatically, in parallel and take
> 2.5 seconds.
I need to figure out that parallel build thingie.
> The toplevel makefile, which deals with 4 types of C++ source code (the
> different types use different options during compilation) is 168 lines.
> The subdirectory makefiles are 1-40 lines. Total size of all makefiles is
> 257 lines and 7kb.
I have one makefile for each of my whole projects. Every module
(subdirectory?) has a module.mk file which simply lists new files added (SRC
+= subdir/a.cc subdir/b.cc). I also have a toplevel project.mk that
specifies project-specific things so that I can use a generic makefile for
everything.
> As per the paper, make only processes the minimum amount is has to. I
> also have no worries or problems with parallel building.
>
> Finally, the makefile system is pretty simple. I do not deal with making
> sure that subdirectory makes get the right flags and options or get built
> in the right order. Nor, with one exception, do I have any shell scripts
> in the makefile. (I do have one ugly shell script responsible for getting
> and fixing-up dependency information.)
>
> Overall, its very sweet.
That's what I think to myself every time I :make.
> I am using
> GNU Make version 3.79.1, by Richard Stallman and Roland McGrath.
> Built for i586-pc-linux-gnu
>
> As I am doing this under linux, I have not been concerned about makefile
> portability to other systems. Thus, I may have used GNU extensions without
> intending so.
The main problem I've seen is that depend.sh is a *shell script*. I've never
managed (or tried...) to make those work in DOS so that my DJGPP-using
friends can take advantage of them.
> -- Problems or untested issues
>
> Overall, I've been extremely happy with it, though it is flawed in a
> couple of ways:
>
> First, 'make clean' builds dependencies before doing any cleaning. This is
> both slow, and fragile in the case where you rename a file, and make
> attempts to build dependencies on a file that no longer exists. Make
> breaks.
Oh, right, the .h thing.
> One workaround is '-k', or run 'make clean cleandep' before
> renaming or deleting files. A suggested fix for this would be appreciated!
Yeah, this is weird.
> Second, All subdirectory modules must be declared in the toplevel
> makefile, even recursive modules. No subdirectory makefile can add new
> directories into the search path.
# Include module descriptions
include $(patsubst %,%/module.mk,$(MODULES))
I'm sure it would be possible for a module.mk file to do the following:
MODULES += subdir/subdir/subdir
include subdur/subdir/subdir/module.mk
And after all that recursion, the toplevel makefile can do:
INCPATH += $(patsubst %,-I%,$(MODULES))
CFLAGS += -pipe $(LIBPATH) $(INCPATH)
But does inclusion work that way? I haven't tried it.
> This is probably not to severe,
> something like autoconf can build the full list for me.
Yeah, granted. ;^)
> Third, I cannot use '%' in any filename or path name. (see below for the
> reason why)
Who would want to?
> Currently, I am not seperating out the build directory from the run
> directory, though that could be done fairly easily in the toplevel build
> rules. My design is that make keeps its current directory always be the
> toplevel, so it uses full paths to all subfiles.
That's what I've seen in the RMCH system, but it can be overridden. I've
messed with the exact behavior of my makefile somewhat.
> I think the build system could be scaled to handle much larger
> professional jobs. For example, the case where people checking out a
> subset of a system and use symbolic links for *.o files. to avoid having
> to rebuild them unnecessarily. This could be done by limiting the
> directories checked at the toplevel and/or modifying my %.o : %.cpp rule
> to remove the destination (symbolic link or not)
>
> I have not tested it with program-built code. IE. the case where one
> program is run to generate code that is later compiled. (IE:
> './genCode interp.spec > interp.c') My problem is is with how and when do
> I create and include the dependency file on the generated code.
> ('interp.c') Suggestions/fixes welcome.
Oh, like moc headers? I made a (simple) system for that, and it works
alright. I just need to list everything a moc will be built for in MOCHDR.
> I cannot build only a single subdirectory/submodule. as-is. This can be
> emulated easily by having the subdirectory 'Foo/Makefile.dir' define a
> target 'subdir_Foo' that depends on the the targets that that subdirectory
> builds.
Add some sort of $(MODULES): target? Aw, I don't know. I'm no make expert.
> I have only not tested shared-library generation with libtool. I do not
> expect this to be an issue: just alter the compilation rules for *.lo.
>
> My project has only recently (last week or so) scaled to the point where I
> wanted static libraries to help me with building. But those seem to work.
Personally, I haven't messed with making my own libraries since I stopped
needing Borland. /me thinks back to smuggling a copy of djgpp across a
number of state lines, all on a stack of floppies. Yeah, I had to drive
pretty far to find Internet access.
> -- Bugs
>
> I've also found a few bugs given in the paper and make texinfo:
>
> First, dependency generation, the code given in the paper does not work if
> there are '/''s in filenames. This has to be fixed to work practically.
>
> $(patsubst %.c,%.d,$(filter %.c,$(SRC))) : %.d: %.c
> @echo "***** Doing dependencies for $<"
> # @echo "$(CC) -MM $(CPPFLAGS) $<"
> @set -e; b=`basename $* .c` ; d=`dirname $*` ; \
> $(CC) -MM $(CPPFLAGS) $< \
>
> | sed "s%\\($$b\\)\\.o[ :]*%$${d}/\\1.o $${d}/\\1.d : %g" >
> | $@; \
>
> [ -s $@ ] || rm -f $@
I'm not even going to try to read that...
> The origional bugs were:
> 1. The command-line sed pattern used a '/' as a seperator. Not good in
> the case of filenames (now paths) that may include a '/'. I've changed
> this to a '%', which means that '%' CANNOT be used in file names or
> directory names. Please suggest other characters.
@
Yet, I get away with using slashes. Odd.
> 2. 'gcc -MM' was futzing up filenames/pathnames in its output, if you
> gave it something with a path 'foo/bar.cpp' the automatically generated
> dependency file would make dependencies for 'bar.cpp' and lose the
> directory prefix: 'foo/'. This should be reported to GCC. I have not done
> this. Regardless, to deal with previous versions there would need to be an
> autoconf check to see if it was broken.
I fixed this problem by sprucing up my sed script. Want _my_ depend.sh?
> Or at least this is what I think is going on... I did this 4 months ago and
> have no idea how it works either. :)
>
> The bug I refer to:
>
> crosby@dragonlight:/tmp$ mkdir XXYYZZ
> crosby@dragonlight:/tmp$ touch XXYYZZ/bar.cpp
> crosby@dragonlight:/tmp$ gcc -MM XXYYZZ/bar.cpp
> bar.o: XXYYZZ/bar.cpp
>
> The paper[1] and the texinfo page for Make should be modified to include
> these corrections. Also, is this a bug in GCC that should be reported?
I tought it was probably a gcc bug, but an old one that might be depended on.
Maybe I should have reported it, but I guess I forgot to.
> -- The future
>
> But, if we fix these warts, maybe we can convince people to trash the more
> crazy build systems that do use recursive make?
It's terrible to watch five minutes of "nothing to be done for"'s.
> Its nice for things to 'just work', and in complete parallelization.
>
> So, what do you think?
I've been considering taking a large-scale project and redoing its build
system. If I were to succeed, I might even send my patches back. But they'd
probably be ignored because adopting them would require mass re-education.
So I gave up. Heh.
> Scott
>
>
> [1] http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html
By the way, I managed to get in touch with this guy a while back. He still
lives. ;^) I alerted him to the gcc/depend.sh bug, and he said thanks and
that he might merge my suggestions into the next version of the document.
But from his tone I doubt there will be one unless something happens to make
it popular.
--
Andy Goth | [EMAIL PROTECTED] | http://sevatech.com/~andy/
