On 08/22/2012 12:32 PM, Paolo Bonzini wrote:
> So I took a closer look at the whitelisting problem that was reported
> in GNU Smalltalk.
>
> The piece of code that was removed in Automake-NG is:
>
> foreach my $primary ('SOURCES', 'LIBADD', 'LDADD', 'LDFLAGS',
> 'DEPENDENCIES')
> {
> foreach my $var (variables $primary)
> {
> my $varname = $var->name;
> # A configure variable is always legitimate.
> next if exists $configure_vars{$varname};
>
> for my $cond ($var->conditions->conds)
> {
> $varname =~
> /^(?:EXTRA_)?(?:nobase_)?(?:dist_|nodist_)?(.*)_[[:alnum:]]+$/;
> msg_var ('syntax', $var, "variable '$varname' is defined but no"
> . " program or\nlibrary has '$1' as canonical name"
> . " (possible typo)")
> unless $var->rdef ($cond)->seen;
> }
> }
>
>
> Note the
>
> # A configure variable is always legitimate.
> next if exists $configure_vars{$varname};
>
> part in particular. Going back to the commit that introduced it, in
> 54ad2a6, the comment was even more expressive:
>
> # Note that a configure variable is always legitimate.
> # It is natural to name such variables after the
> # primary, so we explicitly allow it.
>
> Now this suggests that indeed the new behavior of Automake-NG _would_ be
> "naughty and annoying" (quoting from Stefano). Indeed there are cases
> where the new behavior would catch bugs, but are they enough overall?
>
>
> Generalizing the above, the point is that there _is_ a loss of information
> going from Automake runtime to GNU Make runtime.
>
Yes, unavoidable sometimes. Though not in this case: we might simply add
all the relevant AC_SUBST to AM_VARTYPOS_WHITELIST.
> Therefore, I don't see
> the point in moving the check from Automake runtime to GNU make runtime.
>
How would you diagnose a typo in here at Automake runtime?
bin_PROGRAMS = $(call user-func,args)
bin_PROGRAMS += $(if $(ON-CYGWIN),baz)
ifdef ON-CYGWIN
# Oops, this was meant to be baz_LDFLAGS!
bar_LDFLAGS = -lextra
endif
It's true that this currently (and sadly) wouldn't work anyway, but that
is due to the limitations inherited by mainline Automake, not to some
fundamental impossibility; so I'd like to be able to make it work in the
future (the same way we made TEST_EXTENSIONS completely processed at make
runtime). So we need to be prepared to the fact that a lot of information
will no longer be available at automake runtime.
For more information, refer to the commit that actually moved the
typo warnings from automake runtime to make runtime:
commit 67d61020ff23f630bfa47bf6eec5036ecad5de74
Author: Stefano Lattarini <stefano.lattar...@gmail.com>
AuthorDate: Tue Jun 5 13:49:04 2012 +0200
[ng] warns: typos in _SOURCES etc. reported at make runtime
For example, on an input:
bin_PROGRAMS = frob
forb_SOURCES = main.c
a proper warning should be given, since either 'forb' or 'frob' is likely a
typo. Mainline Automake gives the warning at automake runtime, while with
this change, the warnings will be moved at make runtime. This will allow
us to warn about variables like '_DEPENDENCIES', whose definitions can
be unavailable (or unanalyzable) at automake runtime (check about those
variables had to be disabled in commit 'v1.12-295-g9a5f837' of 2012-05-22,
"[ng] warns: don't report possible issues with '_DEPENDENCIES' variables",
to avoid spurious errors).
> The tasks of Automake are multiple:
>
> 1) gather information from configure.ac and merge it with Makefile.am
> files (options, subst-ed variables, etc.)
>
And this ability should be preserved in Automake-NG.
> 2) resolve conditionals
>
Ideally, once Automake-NG is powerful enough, Automake-time conditional
would disappear -- you should be able to use GNU make builtins like
'$(if ...)', 'ifeq' and 'ifdef' instead. That will be quite difficult
to do (something for Automake-NG 2.0 probably), but is worth keeping in
mind that we are working towards that goal, and acting accordingly from
right now.
> 3) gather primaries, and select the appropriate snippets into the "rules"
> section of Makefile.in (roughly speaking)
>
Ideally, once Automake-NG is "completed", this too should be done completely
at make runtime (refer to the sadly-quiescent-if-not-downright-dead Quagmire
project for examples: <http://tromey.com/blog/?p=401>).
> 4) apply transformations to the snippets
>
This too will disappear eventually, to be replaced with proper use of $(eval)
and user-defined functions. To get the gist of it, take a look at the
implementation of the parallel-tests harness in Automake-NG, in the file
'lib/am/parallel-tests.mk': that file is included *verbatim* in the generated
Makefile.in (well, almost: '##' comments still have to be stripped of course).
> 5) do semantic checks
>
Some of them (e.g., those involving only configure.ac) will be kept at
automake runtime; a lot will be moved at make runtime; some (I hope very
few) will just have to be dropped. An unavoidable price to pay IMO.
> For each of this, the amount of code that can be moved to GNU make runtime
> is different. For (1) it is obviously "zero",
>
Yep.
> and for (3) I suppose it is close to zero as well.
>
Disagree. I actually want it to be 100% eventually :-)
> (2) could probably be simplified somehow, with
> hardly any effect on the cost of actually building with GNU make.
>
True, but remember that the final objective is to completely get rid of
it, favouring the GNU make "equivalents" (ifeq, ifdef, $(if ...)).
> (4) can be simplified a lot, with some extra cost imposed on GNU make,
> but it should really be bearable. In fact, simplifying this complex, and
> little known, part of Automake is the main benefit I see in requiring
> Automake-NG.
>
I agree this is the best starting point, and the simplest part to tackle
(and most of my changes have in fact worked in that direction). But I
assure you this part of the Automake code base is actually one of the
simplest ans clearest ones! The changes done in this area are mostly
useful as a step toward the overhaul of (2) and (3), as well as a way
to give the user the power to use more GNU make features in its input
Makefile.am (e.g., using wildcards or user-defined functions in the
definition of stuff like $(TESTS) and $(info_TEXINFOS)).
> For (5), it _may_ be possible (using things like .VARIABLES) to move the
> checks to GNU make. But I don't really see the point.
>
This I should have explained above already.
> You want errors to be diagnosed as soon as possible, and as precisely
> as possible.
>
Yes -- as long as that is possible and doesn't impede further developments.
> Regarding quality, Perl can do the task better than Make,
>
But I envision a day when Automake-NG will be no more than a few hundred
lines of perl, mostly dedicated at tracing stuff in 'configure.ac', and
setting up few variable definitions accordingly. No more incomplete, messy,
half-unspecified parsing and tweaking of a subset of make syntax there!
Let GNU make parse GNU make input -- it's the only sane way ahead.
> just like it can
> do it better than M4. And Automake can also do it better than the generated
> Makefiles, because it has more information coming from tasks (1)
>
And this can be done, and will continue to be.
> and (2) in the list above.
>
Which I want to see disappeared some day, though (admittedly, that day is
still far away down the road, but I'm looking forward at it nonetheless).
> Regarding early diagnosis, I want "sh bootstrap.sh" to find all problems in
> my build system, *before* I embark into the expensive "./configure && make".
>
It's nice when it's possible of course, but if you want more "dynamicity" and
flexibility in your language (as I want in Automake-2.0), you'll have to forgo
some early warnings, sadly. Just think Python vs. C -- we are moving from C
towards python; we'll earn a great deal in power and flexibility, and loose
a good deal in static checking. Not a good thing, of course, but a necessary
trade-off.
> One of the reasons why the C compiler tries to warn about implicitly-defined
> functions, even though C89 does not require that, is that it diagnoses typos
> well before the final link.
>
> Thoughts?
>
Yes, see above ;-)
Thanks,
Stefano
