Somewhere in the documentation there's some note about LMTPD not being
designed to be run in multiple instances at the same time.
In my experience that is not exactly true. It *CAN* create some unusual
but pretty innocent situations, caused by multiple transactions running
at the same time.
Your tables are pretty small, insertion time should be somewhere around
1 second but actually if the DB machine is busy it can get higher.
My previous mail was too vague about the headers.
Please check the header_cache_readonly setting in dbmail.conf. It should
be set to "yes" or your dbmail_header* will grow to an huge number or
rows slowing down everything.
Your dbmail_headername should be around 200 rows.
If that's not the case (and I think it will probably be not) I can give
you some hints on how to trim it down.
---
Andrea Brancatelli
Schema31 S.p.a.
Chief Technology Officier
ROMA - FI - PA
ITALY
Tel: +39.06.98.358.472
Cell: +39.331.2488468
Fax: +39.055.71.880.466
Società del Gruppo OVIDIO TECH S.R.L.
On 2018-03-01 04:15, rich carroll wrote:
> Andrea,
>
> My postfix lmtp usually has a delay of 2.2 (delays=0.01/0.01/0/2.2) but when
> it gets busy they can be higher, but they are never lower then 2.
>
> table rows
> dbmail_users 2246
> dbmail_header 135684624
> dbmail_phymessage 6315243
>
> I don't know if those are big numbers or not. It seems like I have alot of
> indexes. I upgraded from 2.2 this year. I could get a dump of the structure
> if that would help anyone. I don't see any triggers anywhere but I am not
> sure if I have looked in all the right spots.
>
> I am using xinetd so I got it working like this:
>
> service lmtp-xinetd
> {
> port = 9998
> disable = no
> socket_type = stream
> protocol = tcp
> wait = no
> user = root
> server = /usr/bin/timelimit
> server_args = -t120 -T120 /usr/local/sbin/dbmail-lmtpd -n
>
> }
>
> but shortly after it starting up I get 4 dbmail-lmtpd processes sitting at
> 100% (lmtp_destination_concurrency_limit = 4)
>
> root 36834 98.0 0.0 154388 12400 ? Rl 20:24 5:00
> /usr/local/sbin/dbmail-lmtpd -n
> root 37238 96.9 0.0 154572 13060 ? Rl 20:25 3:31
> /usr/local/sbin/dbmail-lmtpd -n
> root 37248 96.8 0.0 154388 12548 ? Rl 20:25 3:27
> /usr/local/sbin/dbmail-lmtpd -n
> root 37253 96.8 0.0 154388 12636 ? Rl 20:25 3:27
> /usr/local/sbin/dbmail-lmtpd -n
>
> I have switched it back to running 4 dbmail-lmtpd daemons running on ports 24
> 26 27 and 28 with with postfix feeding them to balance on port 9999 and
> balance switching between the 4 dbmail-lmtpd processes.
>
> /usr/bin/balance 9999 localhost:24 localhost:26 localhost:27 localhost:28
>
> And they do not have the same issues as running in -n mode.
>
> dbmail 38406 0.0 0.0 164252 14420 ? Sl 20:31 0:00
> /usr/local/sbin/dbmail-lmtpd -f /etc/dbmail/dbmail.conf
>
> dbmail 38416 0.0 0.0 165224 14996 ? Sl 20:31 0:00
> /usr/local/sbin/dbmail-lmtpd -f /etc/dbmail/dbmail2.conf -p
> /var/run/dbmail/dbmail-lmtpd2.pid
> dbmail 38426 0.0 0.0 162528 12452 ? Sl 20:31 0:00
> /usr/local/sbin/dbmail-lmtpd -f /etc/dbmail/dbmail3.conf -p
> /var/run/dbmail/dbmail-lmtpd3.pid
> dbmail 38436 0.0 0.0 162320 12376 ? Sl 20:31 0:00
> /usr/local/sbin/dbmail-lmtpd -f /etc/dbmail/dbmail4.conf -p
> /var/run/dbmail/dbmail-lmtpd4.pid
>
> However when I was running only 1 lmtpd I had all sorts of problems, too many
> files and maxing out db connections.. but for the last 5 days have been doing
> it this way with no issues. I have several 5 minute crons checking services
> and syslog for new errors and when it finds one I restart all mail services.
> which hasn't triggered since I moved to 4 dbmail-lmtpds. I will update after
> running it this way for a few more weeks. I still get an active queue when
> spammers get through or they send notifications to all the mailboxes but it
> chews through them pretty fast. I don't see a need for running more then 4
> but I bet 10 would work just as good.
>
> Mostly I am wanting to know if there is any reason not to run it this way.
> I'd also like more DB performance but I am not sure even where to begin with
> the db. I am afraid if I jack with indexes I may mess something up.
>
> Thanks for looking into this
> rich
>
> On Tue, Feb 27, 2018 at 11:40 AM, Andrea Brancatelli
> <abrancate...@schema31.it> wrote:
>
> Let's got in order:
>
> * Slow insert. It's not "normal". You'd better check what you have in the
> dbmail_headername tables, clean it up, and probabily disable the
> dbmail_headername auto-insert. Can you give a few information on the DB size
> as well? I'm not using pgsql but probably checking the indexes would help a
> lot.
> * LMTPD over INETD. There's a bug in the code (there is an open issue for
> that) that causes lmtpd to get stuck in an infinite loop when used thought
> inetd and the remote session get closed while receiving a mail. I don't think
> there will be a fix in a short time for that BUT you can overcome that with
> an utility called "timelimit". It just sits in the pipe of the commands and
> if a subsequent command runs for more than X seconds it kills it.
>
> In my inetd.conf I'm running:
>
> lmtpd stream tcp nowait/15 root /usr/local/bin/timelimit timelimit -t120
> -T120 /usr/local/sbin/dbmail-lmtpd -n
>
> And it works like a charm. It clearly doesn't fix the problem but at least it
> gives you a working situation.
>
> I can confirm that having multiple lmtpd dispatched by inetd is much faster
> than the daemon. Furthermore it helps you a lot with the DB Connection pool
> and avoid memory leaks.
>
> ---
>
> Andrea Brancatelli
> Schema31 S.p.a.
> Chief Technology Officier
>
> ROMA - FI - PA
> ITALY
> Tel: +39.06.98.358.472
> Cell: +39.331.2488468 [1]
> Fax: +39.055.71.880.466
> Società del Gruppo OVIDIO TECH S.R.L.
>
> On 2018-02-26 01:44, rich carroll wrote:
>
> Howdy folks,
>
> I'm running dbmail 3.2.3 on a postgresql db and am having slow inserts. About
> 2 seconds each. This is not really ok and it tends to get behind during peak
> usage or when someone emails every mailbox. I read that 2 seconds a message
> is not abnormal, can anyone confirm this?
>
> My solution is to run balance (https://www.inlab.de/balance.html [2]) to
> round robin postfix to 4 instances of dbmail-lmtp. I've been running it this
> way for a couple days and it looks like it is going to work. Does anyone else
> do it this way or see a reason for me not to run it this way?
>
> I did try to run dbmail-lmtp in stdin/stdout mode and call it from xinetd but
> the processes weren't ending like I expected and I ended up with a lot of
> them running and having file pointer errors eventually. Has anyone made it
> work with xinetd or inetd and can share the config with me?
>
> Not really related but my pg_dump backup is a little over twice the size of
> my db on disk. Is this normal or should I start worrying about that?
>
> Thanks
> Rich
>
> _______________________________________________
> DBmail mailing list
> DBmail@dbmail.org
> http://lists.nfg.nl/mailman/listinfo/dbmail [3]
>
> _______________________________________________
> DBmail mailing list
> DBmail@dbmail.org
> http://lists.nfg.nl/mailman/listinfo/dbmail [3]
--
Richard Carroll
richcarr...@gmail.com
785-288-1144
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://lists.nfg.nl/mailman/listinfo/dbmail
Links:
------
[1] tel:+39%20331%20248%208468
[2] https://www.inlab.de/balance.html
[3] http://lists.nfg.nl/mailman/listinfo/dbmail_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://lists.nfg.nl/mailman/listinfo/dbmail
