Hello Gary,
> The architecture of my current rewrite of gnulib bootstrap, moves all
> the top-level code into functions ... Then `bootstrap.conf' is sourced, where
> the nodes of that "shell-function-require-tree" can be overwritten
> with replacement functions to radically alter the behaviour of the
> script if necessary...
Sounds very open, powerful, and easy to use. Great!
------------------------------------
Regarding gnulib-cache.m4: I've now updated the documentation at
<http://www.gnu.org/software/gnulib/manual/html_node/Modified-imports.html>
<http://www.gnu.org/software/gnulib/manual/html_node/Simple-update.html>
<http://www.gnu.org/software/gnulib/manual/html_node/VCS-Issues.html>
and changed the --import option so that its arguments are a specification
and not an augmentation any more.
So, there are two ways in which 'bootstrap' can organize the specification
of the gnulib module names and other gnulib-tool arguments (--source-base,
--m4-base, ...):
A) The user specifies them in the file gnulib-cache.m4.
In this case:
- Neither the user nor the 'bootstrap' script must use the
--import option, because this would kill the contents of
gnulib-cache.m4. Instead the user must use --add-import,
--remove-import only, and the 'bootstrap' script must use
--update only.
- The gnulib-cache.m4 file is under version control.
B) The user specifies them in a file other than gnulib-cache.m4,
such as bootstrap.conf.
In this case:
- The user must not invoke gnulib-tool directly, not with
--import, --add-import, --remove-import, because these
options would leave bootstrap.conf in an inconsistent state
with the rest of the package, and not --update because this
is bootstrap's job.
- The 'bootstrap' script must use --import always. It can also
use --update if it detects that bootstrap.conf has not changed;
but there is no benefit in doing this because it will end up
doing the same as the complete --import command; no speedup.
- The gnulib-cache.m4 file is not under version control; it's
a mere "output" file.
I'll let you decide which approach your 'bootstrap' script shall support.
The advantage of A, compared to B, is:
- The user is able to use "gnulib-tool --add-import module" instead
of modifying a file.
The drawbacks of A, compared to B, is:
- The user has two different "input" files where he defines the
specification of the bootstrap process: bootstrap.conf and gnulib-cache.m4.
They use different syntaxes (shell variable assignments vs. m4 macro
invocations).
- The user can not add comments to gnulib-cache.m4, as they would be
erased the next time the file is written.
- Tool failure can accidentally overwrite gnulib-cache.m4.
> > The reason is to avoid mixing inputs and outputs, whereas your proposal
> > amounts to introducing circular dependencies.
>
> I don't think you can apply that argument to a cache file, which by its nature
> is both an input and an output.
In general you would have to worry. With gnulib-cache.m4 things will probably
be less severe: There will be no version control conflicts, because
gnulib-cache.m4 is stable: When it is rewritten after using itself as input,
it is unchanged.
> > It is *very* important to design things so that every file is either input
> > (maintained by the developers, manually) or output (generated by some
> > tools),
> > but not both. If you don't follow this principle,
> > - The file will be messed up and require manual correction when the
> > developer invoked the tools with wrong parametrizations, or when some
> > unintended failure happened.
>
> Reading a cache file with sensible fall-back options does not have this
> problem, and if some other tool messes up the cache, then it can easily
> be deleted at the penalty of having to recalculate its contents again from
> first principles. Really, it's just a cache
No. In case A above, where gnulib-cache.m4 is "input" and "output", tool
failure will erase the information it contains, with no way to recalculate it.
(The user will have a backup in version control, and may pray that this
backup is not too old.)
> [Really it's] a way to short circuit
> repeated identical calculations on consecutive runs.
You misunderstood gnulib-cache.m4: It is in no way a "short-circuit". Its
presence does not cause any computation, file creation or anything else to
be optimized away.
> > - Multiple developers working on one project will get confused about
> > changes that one developer made (unintentionally) and another developer
> > considers wrong.
>
> The project will need a policy to avoid this. core-utils, gettext and others
> have a policy of not checking in gnulib-cache.m4
> ... I don't see a problem with committing it, or not-
> committing it as long as the developers on each project reach a consensus.
As explained above, this is not a per-project policy. Either a project is
in case A, or it is case B. From that point on there is no choice any more.
> > - You will not be able to evolve the format of that file easily.
>
> Easily averted by recalculating the values that you wanted to extract from the
> cache-file, but which weren't available from the cache (because the cache is
> missing, or has changed formats).
If the values can be recalculated, then you're in case B, and gnulib-cache.m4
is mere "output", not "input"+"output".
> > 1. You must not use 'gnulib-tool --update'.
>
> Huh? That makes no sense to me!
I hope the doc makes it clear now?
> > 2. You even have to remove gnulib-cache.m4 before running
> > 'gnulib-tool --import ...'. Because when gnulib-cache.m4 is present,
> > 'gnulib-tool --import ...' will add the specified modules but not throw
> > away modules that are listed in gnulib-cache.m4 and not listed on the
> > command lines. In other words, if you don't remove gnulib-cache.m4,
> > you are never able to de-import modules.
>
> That is definitely a wrinkle.
It is fixed now, as of yesterday.
------------------------------------
> >> 6. AC_CONFIG_AUX_DIR detection
> >> ==============================
> >>
> >> Why another bunch of forks? Might as well get this data while running sed
> >> over
> >> configure the first time, by adding the following to
> >> `extract_package_values':
> >>
> >> /AC_CONFIG_AUX_DIR(/ {
> >> s|^.*AC_CONFIG_AUX_DIR([['"$tab"' ]*\([^])]*\).*$|config_aux_dir=\1|
> >> p
> >> }
> >
> > Yes, this does make sense. configure.ac is an input, bootstrap.conf is an
> > input;
> > they do not need to both specify the same thing.
>
> Better yet, I've written a shell function to extract these values from
> configure.ac
> using autom4te traces, which is much less fragile
Nope, this is not better. You will or may run into error messages such as
m4: aclocal.m4:851: Cannot open m4/absolute-header.m4: No such file or
directory
autom4te: m4 failed with exit status: 1
Recall that at the moment you need to know the config_aux_dir, you have an
inconsistent state of {configure.ac,aclocal.m4,m4/*} files. The purpose of
running 'bootstrap' is precisely to import the m4/* files and THEN to
regenerate a good aclocal.m4. Which means, at this stage you *must*not* do
anything that uses a potentially outdated aclocal.m4.
------------------------------------
> >> 10. slurp()
> >> ==========
> > ...
> > I think it has/had two purposes:
> > 1. It allowed Paul to use symbolic links or hard links to gnulib files,
> > at a time when gnulib-tool did not have options for symbolic links.
> > 2. It prevents some files from being edited. Again, this dated from before
> > it become gnulib-tool standard to mark generated files with "DO NOT
> > EDIT!"
>
> So, with a little work in existing gnulib bootstrap using projects, slurp can
> be removed?
I guess so, yes.
Bruno
