For some time the idea of having some kind of event queue notification
mechanism in dpkg has been floating about. For example, it would be
used to avoid running scrollkeeper-update dozens of times during an
upgrade and could simplify the emacs addon registration. Wichert and
I even had a good design conversation in a taxi some years ago but we
failed to make proper notes and write it up.
I hope to be able to implement such a feature at some point in the
nearish future. To this end I've written a draft specification which
you can find below. If you're interested, please read it and comment.
Note the wide crosspost of this message. I have set followups to go
to debian-dpkg, which is probably the right place for this discussion.
Regards,
Ian.
TRIGGERS
========
Introduction
------------
A dpkg trigger is a facility that allows events of interest to a
package to be recorded and aggregated, and processed later by that
package. This feature simplifies various registration and
system-update tasks and reduces duplication of processing.
(NB: Triggers are intended for events that occur during package
installation, not events that occur in general operation.)
Concepts
--------
Each trigger is named, and at any time zero or more packages may be
interested in it.
We currently envisage three kinds of triggers:
* Explicit triggers. These can be activated by any program
by running dpkg-trigger (at any time, but ideally from a maintainer
script).
* File triggers. These are activated automatically by dpkg
when a matching file is installed, upgraded or removed as part
of a package. They may also be explicitly activated by running
dpkg-trigger.
* Special triggers, which activate magic code in dpkg itself.
Currently none of these are defined.
Trigger names contain only printing 7-bit ascii characters (no
whitespace). Each trigger kind has a distinct subset of the trigger
name space so that the kind can be determined from the name. After we
run out of straightforward syntaxes, we will use <kind>:<details>.
When a trigger is activated, it becomes pending for every package
which is interested in the trigger at that time. Each package has a
list of zero or more pending triggers.
A package with pending triggers is not considered properly installed
until efforts to notify it of the trigger event have been successful.
The new state of having pending triggers is a dpkg package status of
`triggered', which lies somewhere between `config-failed' and
`installed'.
Details
-------
When one of the triggers in which a package is interested is
activated, the package goes from state `installed' to state
`triggered'; the list of pending triggers for each package is
recorded.
If no package is interested in a trigger then activations are ignored;
this is not an error.
To restore a package in state `triggered' to `installed', dpkg will
run the postinst script:
postinst triggered "<trigger-name> <trigger-name> ..."
This will be attempted for each package in state `triggered' at the
end of each dpkg run; so, normally, in the same dpkg run as the event
which made the package go to `triggered'. This leaves packages in
reasonable states by default.
If the `postinst triggered' run fails the package goes to
`config-failed', so that the trigger processing will not be attempted
again until explictly requested.
|
V
,------------.
| unpacked |
`------------'
|
|
(automatic)| ,----------.
| | config- |
| | failed |
| `----------'
| | ^
| | |
|,---<--' |
| (user |
postinst | request) |
"configure" | | ,-----------.
| | | triggered |
| | `-----------'
|`----->------'| | ^
| error | postinst | |
| | "triggered" | | trigger(s)
| | (automatic) | | activated
| | | |
| `-----<-----------'| |
| error | |
| | |
V V |
,------------------------------------------------.
| installed |
`------------------------------------------------'
Timing guarantees, races, etc.
------------------------------
Activating a trigger will not have any immediate effect, although
putative resulting status changes will show up in dpkg --status etc.
(Putative because the actual status changes may depend on the state of
trigger interests when dpkg processes the trigger activation into
the status database, rather than that when dpkg --status is run.)
A package is only guaranteed to become notified of a trigger
activation if it is continuously interested in the trigger during the
period from when the trigger is activatated until dpkg runs the
package postinst (either due to --configure --pending, or at the end
of the relevant run, as described above). Subsequent to activation
and before notification, the interested package will not be considered
in state `installed', so long as the package remains interested.
If the package is not in state `installed' then it not interested,
even if it declares some trigger interests; when a package is
deconfigured for any reason, its list of pending triggers is reset.
It is not defined in what order triggers will run. dpkg will make
some effort to minimise redundant work in the case where many packages
have postinst trigger processing activating another package's triggers
(for example, by processing triggers in fifo order during a single
dpkg run). Cycles in the triggering graph are prohibited and will
eventually, after some looping, be detected by dpkg and cause trigger
processing to fail; when this happens one of the packages involved
will be put in state `config-failed' so that the trigger loop will not
be reattempted.
Explicit triggers
-----------------
Explicit triggers have names with the same syntax as package names,
*but* should *not* normally be named identically to a package.
When choosing an explicit trigger name it is usually good to include a
relevant package name or some other useful identifier to help make the
trigger name unique. On the other hand, explicit triggers should
generally not be renamed just because the interested or activating
packages' names change.
Explicit trigger names form part of the interface between packages.
Therefore in case of wider use of any trigger the name and purpose
should be discussed in the usual way and documented in the appropriate
packaging guidelines (eg, in policy).
File triggers
-------------
File triggers have names of the form
/path/to/directory/or/file
and are activated when the specified filesystem object, or any object
under the specified subdirectory, is created, updated or deleted by
dpkg during package unpack or removal. The pathname must be absolute.
File triggers should not generally be used without mutual consent.
The use of a file trigger, and the name of the trigger used, should be
stated in policy, so that a package which creates a relevant file in a
maintainer script can activate the trigger explictly.
File triggers must definitely not be used as an escalation tool in
disagreements between different packages as to the desired contents of
the filesystem. Trigger activation due to a particular file should
not generally modify that file again.
Configuration files (whether dpkg-handled conffiles or not), or any
other files which are modifed at times other than package management,
should not be used for file triggers; dpkg triggers are not a general
mechanism for filesystem monitoring.
If there are or might be directory symlinks which result in packages
referring to files by different names, then to be sure of activation
all of the paths which might be included in packages should be listed.
The path specified by the interested package is matched against the
path included in the activating package, not against the truename of
the file as installed. Only textually identical filenames (or
filenames where the interest is a directory prefix of the installed
file) are guaranteed to match.
A file trigger is guaranteed to be activated before the file in
question is modified by dpkg; on the other hand, a file trigger might
be activated even though no file was actually modified.
Because of the restriction on trigger names, it is not possible to
declare a file trigger for a directory whose name contains
whitespace. Such a trigger should not be necessary.
Package declarations regarding triggers
---------------------------------------
A package declares its relationship to some trigger(s) by including a
`triggers' file in its control archive (ie, DEBIAN/triggers during
package creation). This file contains directives, one per line.
Leading and trailing whitespace is trimmed and lines starting with #
and empty lines will be ignored.
The trigger control directives currently supported are:
interest <trigger-name>
Specifies that the package is interested in the named trigger.
All triggers in which a package is interested must be listed using
this directive in the triggers control file.
Trigger names not matching any currently supported trigger kind
name syntax are ignored for `interest'. This will allow future
extensions (see TRANSITION PLAN, below).
activate <trigger-name>
Arranges that any change of the package's state will activate the
specified trigger. The trigger will be activated just before the
package state changes.
Unknown directives, or unknown trigger name syntaxes for `activate',
are an error which will prevent installation of the package. dpkg-deb
will be changed to to warn about unrecognised trigger names syntaxes
and unrecognised trigger control directives.
New command-line interfaces to dpkg tools
-----------------------------------------
dpkg will grow new options:
--suppress-triggers
Do not run any triggers in this run (activations will still be
recorded). If used with dpkg --configure <some package> or
--process-triggers <some package> then the named package
postinst will still be run even if only a triggers run is
needed.
--no-suppress-triggers
Cancels a previous --suppress-triggers.
--triggers
Processes only triggers. All pending triggers will be
processed. If package names are supplied only those packages'
triggers will be processed, exactly once each where necessary.
Use of --suppress-triggers, or an explicit list of packages with
--triggers, may leave packages in the improper `triggers' state. This
can be fixed later by running one of:
dpkg --configure --pending
dpkg --triggers
A trigger may be activated explicitly with:
dpkg-trigger [--verbose|--query] <name-of-trigger>
This can be used by maintainer scripts in complex and conditional
situations where the file triggers, or the declarative `activate'
triggers control file directive, are insufficiently rich. It can also
be used for testing and by system administrators (but note that the
triggers won't actually be run by dpkg-trigger - see `Timing...',
above).
The --verbose and --query options will show which packages were
interested and what the current activation state is, on stdout in
human- and machine-readable (untranslated) format. Without any
options there will be no output to stdout, and none to stderr unless
dpkg-trigger is unable to make a record of the trigger activation.
With --query no trigger is activated.
Unrecognised trigger name syntaxes are an error for dpkg-trigger.
NB that in the case of a file trigger the name of the trigger is
needed, not the name of a file which would match the trigger.
apt and aptitude
----------------
These must be taught about the new `triggered' state. Packages in
`triggered' should be treated roughly like those in `unpacked': the
remedy is to run dpkg --configure.
However, normally apt and aptitude will not see packages in
`triggered' since dpkg will generally attempt to run the triggers thus
leaving the package in `config-failed' or `installed'.
Note that automatic package management tools which call dpkg (like apt
and aptitude) should not attempt to configure individual packages in
state `triggers' with dpkg --triggers <package>... or dpkg
--suppress-triggers --configure <package>..., or similar approaches.
This would defeat dpkg's trigger cycle detection.
Error handling: bogus data found by trigger scripts
---------------------------------------------------
Where a trigger script finds bad data provided by a triggering
package, it should generally report to stderr the problem with the bad
data and exit nonzero, leaving the interested package in config-failed
and thus signalling the problem to the user.
Alternatively, in some situations it may be more desirable to allow
the interested package to be configured even though it can only
provide partial service. In this case clear information will have to
be given in appropriate places about the missing functionality, and a
record should be made of the cause of the errors. However, this
option is not normally recommended.
WORKED EXAMPLE - SCROLLKEEPER
=============================
Currently, every Gnome program which comes with some help installs the
help files in /usr/share/gnome/help and then in the postinst runs
scrollkeeper-update. scrollkeeper-update reads, parses and rewrites
some large xml files in /var/lib/scrollkeeper; currently this
occurs at every relevant package installation, upgrade or removal.
When triggers are available, this will work as follows:
* gnome-foobar will ship its `omf' file in /usr/share/omf as
normal, but will not contain any special machinery to invoke
scrollkeeper.
* scrollkeeper will in its triggers control file say:
interest /usr/share/omf
and in its postinst say:
scrollkeeper-update -q
dpkg will arrange that this is run once at the end of each run
where any documentation was updated.
Note that it is not necessary to execute this only on particular
postinst "$1" values; however, at the time of writing, scrollkeeper
does this:
if [ "$1" = "configure" ]; then
printf "Rebuilding the database. This may take some time.\n"
scrollkeeper-rebuilddb -q
fi
and to retain this behaviour, something along the following lines
would be sensible:
if [ "$1" = "configure" ]; then
printf "Rebuilding the database. This may take some time.\n"
scrollkeeper-rebuilddb -q
else
printf "Updating GNOME help database.\n"
scrollkeeper-update -q
fi
* dh_scrollkeeper will only adjust the DTD declarations and no longer
edit maintainer scripts.
Full implementation of the transition plan defined below, for
scrollkeeper, goes like this:
1. Update scrollkeeper:
- Add a `triggers' control archive file containing
interest /usr/share/omf
- Declare a versioned dependency on the triggers-supporting
dpkg.
- Make the postinst modifications as described above.
2. In gnome-policy chapter 2, `Use of scrollkeeper',
- delete the requirement that the package must depend on
scrollkeeper
- delete the requirement that the package must invoke
scrollkeeper in the postinst and postrm
- instead say:
OMF files should be installed under /usr/share/omf in the
usual way. A dpkg trigger is used to arrange to update the
scrollkeeper documentation index automatically and no special
care need be taken in packages which supply OMFs.
If an OMF file is placed, modified or removed other than as
an file installed in the ordinary way by dpkg, the dpkg file
trigger `/usr/share/omf' should be activated; see the dpkg
triggers specification for details.
Existing packages which Depend on scrollkeeper (>= 3.8)
because of dh_scrollkeeper or explicit calls to
scrollkeeper-update should be modified not to Depend on
scrollkeeper.
3. Update debhelper's dh_scrollkeeper not to edit maintainer
scripts. dh_scrollkeeper should also issue a warning if it finds
scrollkeeper (>= 3.8) in the Depends control file line.
4. Remove the spurious dependencies on scrollkeeper, at our leisure.
As a bonus, after this is complete it will be possible to remove
scrollkeeper while keeping all of the documentation-supplying
gnome packages installed.
5. If there are any packages which do by hand what dh_scrollkeeper
does, change them not to call scrollkeeper-update and change their
dependency to refer to the triggers-interested scrollkeeper
version, or delete the dependency entirely.
This not 100% in keeping with the full transition plan defined below:
if a new gnome package is used with an old scrollkeeper, there is some
possibility that the help will not properly be available. Likewise,
if a new gnome package is built with an old debhelper, its postinst
will call scrollkeeper-update but no dependency on scrollkeeper will
be present.
However, dh_scrollkeeper doesn't generate the scrollkeeper dependency
in the control file, which makes it excessively hard to get the
dependency up to date. (This was a mistake in the design of
dh_scrollkeeper.) The bad consequences of these missing dependencies
are less severe than the contortions which would be required to deal
with the problem.
TRANSITION PLAN
===============
>From old dpkg to new dpkg
-------------------------
The first time a trigger-supporting dpkg is run on any system, it will
activate all triggers in which anyone is interested, immediately.
These trigger activations will not be processed in the same dpkg run,
to avoid unexpectedly processing triggers while attempting an
unrelated operation. dpkg --configure --pending (and not other dpkg
operations) will run the triggers, and the dpkg postinst will warn the
user about the need to run it.
To use this correctly:
* Packages which are interested in triggers, or which want to
explicitly activate triggers, should Depend on the
triggers-supporting version of dpkg.
* Update instructions and tools should arrange to run
dpkg --configure --pending
after the install; this will process the pending triggers.
dpkg's prerm will check for attempts to downgrade while triggers are
pending and refuse. (Since the new dpkg would be installed but then
refuse to read the status file.) In case this is necessary a separate
tool will be provided which will:
* Put all packages whose status is `triggered' or `triggers-failed'
into state `config-failed'.
* Remove /var/lib/dpkg/triggers (to put the situation to that which
we would have seen if the trigger-supporting dpkg had never been
installed).
Higher-level programs
---------------------
The new dpkg will declare versioned Conflicts against apt and aptitude
and other critical package management tools which will be broken by
the new Status field value. Therefore, the new higher-level tools
will have to be deployed first.
The new dpkg will declare versioned Breaks against any known
noncritical package management tools which will be broken by the new
Status field value.
Transition hints for existing packages
--------------------------------------
When a central (consumer) package defines a directory where other leaf
(producer) packages may place files and/or directories, and currently
the producer packages are required to run an `update-consumer' script
in their postinst:
1. In the relevant policy, define a trigger name which is the name of
the directory where the individual files are placed by producer
packages.
2. Update the consumer package:
* Declare an interest in the trigger.
* Declare a versioned dependency on a triggers-supporting dpkg.
* In the postinst, arrange for the new `trigger' invocation to run
update-consumer. (The consumer package's postinst will alread
run update-consumer during configuration, and this should be
retained.)
3. Update the producer packages:
* In the postinst, remove the call to update-consumer
* Change the dependency on consumer to be versioned, specifying a
trigger-interested consumer.
4. After all producer packages have been updated according to step 3,
`update-consumer' has become an interface internal to the consumer
and need no longer be kept stable. If un-updated producers are
still of interest, incompatible changes to `update-consumer' imply
a versioned Breaks against the old producers.
(See also `Transition plan', below.)
If there are several consumer packages all of which are interested in
the features provided by producer packages, the current arrangements
usually involve an additional central switchboard package (eg,
emacsen-common). In this case:
1. Define the trigger name.
2. Update the switchboard to have any new functionality needed by the
consumers in step 3 (2nd bullet).
3. Update the consumer packages:
* Declare an interest in the trigger.
* In the postinst, arrange for the new `trigger' invocation to run
the compilation/registration process. This may involve scanning
for new or removed producers, and may involve new common
functionality from the switchboard (in which case a versioned
Depends is needed).
* The old interface allowing the switchboard to run
compilation/registration should be preserved, including
calls to the switchboard to register this consumer.
4. When all consumers have been updated, update the switchboard:
* Versioned dependency on triggers-supporting dpkg.
* Make the registration scripts called by producers be no-ops.
* Versioned Breaks, against the old (pre-step-3) consumers.
5. After the switchboard has been updated, producers can be updated:
* Remove the calls to the switchboard registration/compilation
functions.
* Change the dependency on the switchboard to a versioned one,
specifying the one which Breaks old consumers. Alternatively,
it may be the case that the switchboard is no longer needed (or
not needed for this producer), in which case the dependency on
the switchboard can be removed in favour of an appropriate
versioned Breaks (probably, identical to that in the new
switchboard).
6. After all the producers have been updated, the cruft in the
consumers can go away:
* Remove the calls to the switchboard's registration system.
* Versioned Breaks against old switchboards, or versioned Depends
on new switchboards, depending on whether the switchboard is
still needed for other common functionality.
7. After all of the producers and consumers have been updated, the
cruft in the switchboard can go away:
* Remove the switchboard's registration system (but not obviously
the common functionality from step 3, discussed above).
* Versioned Breaks against pre-step-6 consumers and pre-step-5
producers.
FUTURE DIRECTIONS
=================
The activation of a trigger does not record details of the activating
event. For example, file triggers do not inform the package of the
filename. In the future this might be added as an additional feature.
Error handling and the broken-package and broken-file triggers
--------------------------------------------------------------
Often trigger processing by will involve a central package
registering, compiling or generally parsing some data provided by a
leaf package.
If the central package finds problems with the leaf package data it is
usually more correct for the leaf package to be recorded as not
properly installed, than the central package.
There is not currently any way to do this. There are no plans to
implement an answer to this problem at this stage, but we imagine
something like:
A pair of special triggers, defined by dpkg:
broken-package:<package-name>
broken-file:<path-name>
dpkg is interested in these, and the effect is to put the package in
question from installed to a new state `broken' (which is like
`installed' from the point of view of maintainer script processing).
This feature probably depends on the ability to record some additional
information with a trigger activation, so that when a package is
declared broken in this way it's possible to tell who declared it
broken and why.
INTERNALS
=========
On-disk state
-------------
A single file /var/lib/dpkg/triggers/File lists all of the filename
trigger interests in the form
/path/to/directory/or/file package
For each explicit trigger in which any package is interested,
a file /var/lib/dpkg/triggers/<name-of-trigger> is a list of
the interested packages, one per line.
For each package which has pending triggers, the status
field contains an Pending-Triggers field which contains the
space-separated names of the pending triggers. This field
is present iff the package is in the `triggered' or
`triggers-failed' state. (But see below.)
During dpkg's running /var/lib/dpkg/triggers/Deferred is a list of
the trigger names which have been requested by dpkg-trigger but not
yet incorporated in the status file.
/var/lib/dpkg/triggers/Lock is the fcntl lockfile for the trigger
system. Processes hang onto this lock only briefly: dpkg-trigger
to add new activations, or dpkg to incorporate activations (and
perhaps when it update interests). Therefore this lock is always
acquired with F_GETLKW so as to serialise rather than fail on
contention.
Processing
----------
dpkg-trigger updates triggers/Deferred, and does not read or write the
status file or take out the dpkg status lock. dpkg (and dpkg-query)
reads triggers/Deferred after reading /var/lib/dpkg/status, and after
running a maintainer script. If the status database is opened for
writing then the data from Deferred is moved to updates as
Pending-Triggers entries and corresponding Status changes.
This means that the dpkg is only guaranteed to reincorporates pending
trigger information into the status file only 1. when a maintainer
script has finished, or 2. when dpkg starts up with a view to
performing some operation.
When a package is unpacked or removed, its triggers control file will
be parsed and /var/lib/dpkg/triggers/* updated accordingly.
The case where a triggers-supporting dpkg is run for the first time is
detected by the absence of /var/lib/dpkg/triggers/File. Each time the
triggers-supporting dpkg starts up without this file, it will set each
`installed' or `triggered' package to `installed' with a list of
triggers equal to its interests - but it will not create File at this
time.
When --configure --pending is run, indicating that the triggers system
should now be enabled, File and the per-trigger package lists in
/var/lib/dpkg/triggers/<trigger-name>, will be created from the
`triggers' control files of already-installed packages.
dpkg and/or dpkg-deb will be made to reject packages containing a
Pending-Triggers control file field, to prevent accidents.
Cycle detection
---------------
Trigger cycles are detected using the usual hare-and-tortoise
approach. Each time after dpkg runs a postinst for triggers, dpkg
records the set of pending triggers (ie, the set of activated
<package,triggername> tuples). If the hare set is a superset of the
tortoise set, a cycle has been found.
For guaranteed termination, it would be sufficient to declare a cycle
only when the two sets are identical, but because of the requirement
to make progress we can cut this short. Formally, there is supposed
to be a complete ordering of pending trigger sets satisfying the
condition that any set of pending triggers is (strictly) greater than
all its (strict) subsets. Trigger processing is supposed to
monotonically decrease the set in this ordering. (The set elements
are <package, trigger name> tuples.)
--
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
