Thanks (Wietse) for your suggestion to switch to a LM database. Since
doing so, I have discovered that postmap -q will happily read from the
live LM verified sender database. I'd previously ruled this option out
as on my initial postmap trial using btree would hang if it was in use
by Postfix. Presumably this freeze is because postmap was never likely
to get the shared lock but waited regardless.
I incorporated the postmap -q bash command into a PERL test script and
SPAWN'd it as 'user=nobody:nogroup' though I can probably dispense with
nogroup now I'm no longer planning to change the group permissions of
the lmdb file to be 'nogroup'. The script has been fairly successful
with exceptions (next paragraph). I also admit to being pleasantly
surprised when I discovered postmap would run with 'nobody' perms as the
comments in 'verify.c' line 41 state that the verify service does not
talk to local users. If anyone can think of any consequences by using
postmap from a script, please let me know. Seeing as it was designed for
the job, there is no way could I do any better.
The exceptions :
I've been reviewing the logs that the test SPAWN script generated, and
for some reason messages from some bulk delivery domains including
googlegroups and ticketmaster (list non exhaustive) are not matched
though they exist in the database. I dumped the verified sender
database to a text file using postmap -s.
Eg :
Command line :
postmap -q
"londonstransport+bncbdzntlposqirbr426sgamgqeh7mv...@googlegroups.com"
lmdb:/var/lib/postfix/verified_senders_2021
returns NULL
The DB dump shows :
londonstransport+bncbdzntlposqirbr426sgamgqeh7mv...@googlegroups.com
0:0:1637682515:250 2.1.5 OK s8si901258edx.4 - gsmtp
I can provide more examples on request. If anyone has any idea why the
query isn't matching (I'm most likely doing something wrong), I'd be
grateful to know. I'm on Debian with postfix running version 3.1.15 so
well out of date.
Another thing that occurred to me (in direct opposition to my vanity
project which is to log those I allow to bypass the verify check against
those that would passed the check anyway), is that it is pretty
pointless adding certain groups to the database where there sender
address is unique and unlikely to ever be repeated.
Thanks and best wishes to you all,
Mick.
On 19/11/2021 22:11, M Champion wrote:
Hi Wietse,
On 19/11/2021 16:53, Wietse Venema wrote:
With hash and btree there is no guaranteed way to read a table after
it is updated, even when using locks. LMDB tables are better in
this respect, as long as you use the right locking procedure (man 5
lmdb_table) but you'd need to make the table group readable so that
your tool can lock and read it.
I didn't realise that btree wasn't good for sharing so I've switched
to LMDB format. Hopefully I can find a suitable Debian PERL module
(not CPAN) or get PHP to read the format. I have some reading and
experimenting to do. Thanks for the tips.
That leaves two problems.
The table format is an internal interface. I do make incompatible
changes internally, and that is OK because no-one is supported when
they rely on an internal interface. The format is described with a
comment in the verify.c source code.
Changes to the internal data format would only be a problem (for me)
once every 4 years as I only upgrade Postfix with Debian every other
Deb release (excluding security fixes). I build a new VPS each time as
'dist-upgrade' adds too many unwanted packages for my small VPS. I'm
about due a rebuild, meaning Debian 8 to 10 (Postfix v3.1.15 to v3.4).
While that's still 2 versions behind (soon to be 3) I'm looking
forward to trying out SNI. Ah the format is in the comments! I've
just had a look. So the first colon separated value of data is status.
Thanks.
Instead of hacking file permissions you could try to set up a
proxymap service over TCP on 127.0.0.1. But again the protocol is
an internal interface and breaking changes happen. I just noticed
that this protocol still needs to be changed into "server speaks
first" like the rest of Postfix. I have added that on the TODO list
for Postfix 3.7.
Yes I think you are right that the best way to query the database
would be by a local TCP service. As me for using stable protocol, well
sorry I wouldn't know where to start. I was more thinking about
keeping it simple. Having it reply with the status byte or -1 for no
match? I think that is about my limit sadly.
Thanks for your encouraging reply.
Best wishes,
Mick.
Reading the verify map (and other tables) would need a proxymap
like service with a stable protool. It should probably output JSON
format, like postqueue -j does.
Wietse
