On 05/01/2026 00:55, Wietse Venema via Postfix-users wrote:
John Fawcett via Postfix-users:
Hi Wietse
it seems like a sound approach. I was only wondering one thing: instead
of an on-demand conversion, would it be possible to do the conversion
when postfix is started and then do away with the need for having a
separate daemon that will run for ever after? I am assuming that it is
possible to identify all files needing conversion by reading the
existing configuration.
The problem is 'discovery'. If you want to convert databases (and
therefore, the Postfix configurations that use them) before Postfix
programs and daemons can access those databases, then you are asking
for the rejected approach to parse/modify/unparse Postfix configuration
files. That is not impossible, it is just more complex code.
Thanks Viktor and Wietse for the replies.
I wasn't thinking about modifying the config, "just" parsing it to get
the currently used db files.
But what I am understanding from your replies is that the big step is
exactly that one, parsing the config files, and discovery by use is easier.
However, if the test that is then applied is "but ONLY IF the legacy
name is legitimate (listed in $proxy_read_mapsor $proxy_write_maps)."
then isn't that the same complexity as reading the values of those
variables and converting up front? I realize I must be missing
something, but I just had to ask :-)
The proposed approach 'discovers' databases by the 'open' requests
that Postfix commands or daemons make. An unprivileged Postfix
daemon cannot re-index a database that must be owned by root.
The downside of the helper daemmon is that 'postmap' commands
that specify a legacy database can't access the helper daemon if
Postfix is not running. For that, some logic would have to be
duplicated in the hash/btree client code.
Or just have postmap output an error if invoked on non supported db types.
John
For cache files with no source file, since it is an optimization and
postfix can run fine even if they are removed, I would just start from
scratch with the new db type after logging an appropriate warning about
it, leaving any other migration needs to the sysadmin if they think it
is worth it.
Start from scratch is all that one can do on a system without
Berkeley DB support.
Wietse
On 04/01/2026 23:47, Wietse Venema via Postfix-users wrote:
The removal of Berkeley DB from Linux systems will affect a large
number of Postfix installations. Some (smaller) Linux distributions
have already removed Berkeley DB support, and RedHat will remove
it in Enterprise Linux version 10.
The goal: a Postfix configuration that uses 'hash' or 'btree' can
be migrated to a configuration that does not use Berkeley DB support.
This would require automatic transformation of configuration files
(and automatic database re-indexing), or emulation where a request
for an obsolete database type is automatically serviced with a
supported type.
I have considered, and rejected, automatic transformation of
configuration files. The problem is that database types (such as
hash or btree) and database pathnames may be the result of $name
expansion; to do a proper job one has to parse configuration files
like Postfix programs do, replace each legacy database type, unparse
the result, and then write an updated setting back to a Postfix
configuration file.
That leaves an implementation based on emulation; I opted for a
variant with on-demand database re-indexing in the background. This
does not require that configuration files are updated, though making
such updates would save a few CPU cycles later.
The basic idea:
- A Postfix daemon or command (called 'database client' below) wants
to open a database with a legacy name hash:/path/to/file or
btree:/path/to/file. The 'postmap' command could be such a client.
- The 'hash' and 'btree' database client implementation sends the
legacy name to a privileged helper daemon that replies with a
'new' database name (e.g., cdb:/path/to/file or lmdb:/path/to/file,
but ONLY IF the legacy name is legitimate (listed in $proxy_read_maps
or $proxy_write_maps). This requires that proxy_read_maps and
proxy_write_maps are left at their default settings.
- If /path/to/file exists (i.e. the table source data), but the
'.cdb' or '.lmdb' indexed file does not yet exist, the helper
daemon also runs 'postmap' or postalias' to create the '.cdb' or
'.lmdb' indexed file (unless the database client uses O_TRUNC or
O_CREAT) before replying to the database client. The helper daemon
must be privileged because many database files must be owned by
'root'. The 'postmap' and postalias' commands already already
drop 'root' privileges when indexing a file that is owned by a
non-root user.
- The database client receives the replacement database name from
the helper daemon and opens that database. The old hash and btree
'.db' files are not used.
Nitty-gritty details:
- This does not re-index cache files (verify, postscreen) because
there is no table source file. One would need Berkeley DB support
to list the entries in the obsolete file, and that is not possible
on systems without Berkeley DB support. But it might be feasible
if the migration is done on a system that still supports Berkeley
DB. I'll have to think about that. Cache files are populated by
Postfix daemons and they are only an optimization.
- The helper service will run 'postalias' if a database matches
$alias_maps or $alias_database, otherwise it will run 'postmap';
and it will specify a file type of $default_database_type if the
database type:/path/to/file are listed in $proxy_read_maps, or
$default_cache_db_type if the type:/path/to/file are listed in
$proxy_write_maps. The proxy_*_maps parameters are specified in
a file (main.cf or master.cf) that is writable only be root. This
guarantees that the helper daemon will not touch other files.
- The helper service will be single-threaded to avoid collisions
when it re-indexes a database file.
After a table is re-indexed, the helper daemon will occasionally log
a reminder that the table has been migrated. The user can then at
their convenience update Postfix configuration files and replace
the obsolete name (hash:/path/to/file or btree:/path/to/file) with
the new name (e.g., cdb:/path/to/file or lmdb:/path/to/file), and
they can delete the obsolete hash or btree '.db' file. But is is
not the end of the world if the user ignores the reminders. Postfix
will keep working just fine.
For maximal benefit the helper service should be accessible by local
programs that are not run by 'root' or 'postfix' users. For example,
Mailman3 wants to run 'postmap' commands after adding or removing
a mailing list. There may be other programs that also want to run
postmap or postalias commands.
BTW Mailman3 was updated 5 months ago to allow Postfix databases
other than 'hash' or 'regexp'. Postfix emulation may be needed until
Mailman3 has been adopted by distributions.
If we make the UNIX-domain helper socket world-accessible, then a
local process can attack the service and degrade Postfix performance,
but they cannot use the service to gain 'postfix' or 'root' privileges.
This should require very little new code; if we do not count the
boilerplater code, likely less than the size of this message. It
will definitely be in Postfix 3.11. If people think this will be
useful, then I will consider back porting.
Wietse
_______________________________________________
Postfix-users mailing list [email protected]
To unsubscribe send an email [email protected]
_______________________________________________
Postfix-users mailing list [email protected]
To unsubscribe send an email [email protected]
_______________________________________________
Postfix-users mailing list [email protected]
To unsubscribe send an email [email protected]
_______________________________________________
Postfix-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
