master_wakeup_timer_event

[natan]

Hi
I have very strong machine with load average: 2,22, 2,32, 2,19

and today i get

Jan 14 12:34:25 thebe postfix/master[4925]: warning:
master_wakeup_timer_event: service qmgr(public/qmgr): Resource
temporarily unavailable
Jan 14 12:39:25 thebe postfix/master[4925]: warning:
master_wakeup_timer_event: service qmgr(public/qmgr): Resource
temporarily unavailable

And i don't known where is a problem
--

Re: master_wakeup_timer_event

[Viktor Dukhovni]

On Fri, Jan 14, 2022 at 12:48:10PM +0100, natan wrote:

> Jan 14 12:34:25 thebe postfix/master[4925]: warning:
> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> temporarily unavailable
> Jan 14 12:39:25 thebe postfix/master[4925]: warning:
> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> temporarily unavailable

http://www.postfix.org/DEBUG_README.html#mail

$ postconf -nf
$ postconf -Mf

Perhaps qmgr is indirectly blocked on slow transport table lookups?

-- 
Viktor.

Re: master_wakeup_timer_event

[Wietse Venema]

natan:
> Hi
> I have very strong machine with load average: 2,22, 2,32, 2,19
> 
> and today i get
> 
> Jan 14 12:34:25 thebe postfix/master[4925]: warning:
> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> temporarily unavailable
> Jan 14 12:39:25 thebe postfix/master[4925]: warning:
> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> temporarily unavailable
> 
> And i don't known where is a problem

The Operating System Kernel is telling Postfix that it could not
connect to or write to the qmgr socket (typically, located at
/var/spool/postfix/public/qmgr).

Either Postfix has exceeded some per-process limit, or some Operating
System Kernel resource is exhausted.

Wietse

Re: How can I build a reliable distribution list?

[Wietse Venema]

raf:
> If the distribution itself is working, and the list of
> members doesn't change often, and it's only SPF that's
> getting in the way, perhaps the least disruptive
> solution is to add SRS to Postfix. The problem being
> that someone who sends to the list has SPF on their
> domain and a server who receives from the list bounces
> it because your server isn't allowed by the original
> sender's SPF.

NO, NO, NO. A properly operating mailing list overrides the SMTP
MAIL FROM address with its own address, so that bounces are sent
to the list admin, instead of individual list members who can't fix
the problem.

Wietse

Re: master_wakeup_timer_event

[natan]

W dniu 14.01.2022 o 14:54, Wietse Venema pisze:
> natan:
>> Hi
>> I have very strong machine with load average: 2,22, 2,32, 2,19
>>
>> and today i get
>>
>> Jan 14 12:34:25 thebe postfix/master[4925]: warning:
>> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
>> temporarily unavailable
>> Jan 14 12:39:25 thebe postfix/master[4925]: warning:
>> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
>> temporarily unavailable
>>
>> And i don't known where is a problem
> The Operating System Kernel is telling Postfix that it could not
> connect to or write to the qmgr socket (typically, located at
> /var/spool/postfix/public/qmgr).
>
> Either Postfix has exceeded some per-process limit, or some Operating
> System Kernel resource is exhausted.
>
>   Wietse
What I can realy do i systemctl ?
change to:

fs.file-max=13223142
net.ipv4.ip_local_port_range= 2048 65000
net.core.somaxconn = 2048




 6510 ?    Ss 0:50 /usr/lib/postfix/sbin/master

cat /proc/6510/limits
Limit Soft Limit   Hard Limit  
Units
Max cpu time  unlimited    unlimited   
seconds  
Max file size unlimited    unlimited   
bytes
Max data size unlimited    unlimited   
bytes
Max stack size    8388608  unlimited   
bytes
Max core file size    0    unlimited   
bytes
Max resident set  unlimited    unlimited   
bytes
Max processes 515277   515277  
processes
Max open files    4096 4096
files
Max locked memory 65536    65536   
bytes
Max address space unlimited    unlimited   
bytes
Max file locks    unlimited    unlimited   
locks
Max pending signals   515277   515277  
signals  
Max msgqueue size 819200   819200  
bytes
Max nice priority 0    0   
Max realtime priority 0    0   
Max realtime timeout  unlimited    unlimited   
us   

--

Re: master_wakeup_timer_event

[Wietse Venema]

natan:
> W dniu 14.01.2022 o?14:54, Wietse Venema pisze:
> > natan:
> >> Hi
> >> I have very strong machine with load average: 2,22, 2,32, 2,19
> >>
> >> and today i get
> >>
> >> Jan 14 12:34:25 thebe postfix/master[4925]: warning:
> >> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> >> temporarily unavailable
> >> Jan 14 12:39:25 thebe postfix/master[4925]: warning:
> >> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> >> temporarily unavailable
> >>
> >> And i don't known where is a problem
> > The Operating System Kernel is telling Postfix that it could not
> > connect to or write to the qmgr socket (typically, located at
> > /var/spool/postfix/public/qmgr).
> >
> > Either Postfix has exceeded some per-process limit, or some Operating
> > System Kernel resource is exhausted.


Do you know if the problem is a kernel limit or a per-process limit?
Does master have 4096 open files (including network sockets: ip,
unix-domain, etc.).

Wietse

Re: SASL questions

[Joe Acquisto-j4]

> On 2022-01-13 at 20:26:53 UTC-0500 (Thu, 13 Jan 2022 20:26:53 -0500)
> Joe Acquisto-j4 
> is rumored to have said:
> 
> [...]
>> Would it be valid to presume that an SMTP server that can be connected 
>> to,
>> securely, via Outlook, Thunderbird and the other common clients, can 
>> be
>> connected to via the postfix SASL stuff?
> 
> No. There are authentication mechanisms supported by interactive clients 
> that are not supported by Cyrus. The most important ones are OAUTHBEARER 
> and XOAUTH2, which require an out-of-band (web) interaction following 
> the OAuth2 protocol, typically to support 2FA methods that require a 
> live human interaction.
> 
>> Or is SASL/Cyrus an equine of
>> a different hue?
> 
> SASL is a broad framework used by many application protocols (SMTP, 
> IMAP, etc.) for authentication and each implementation is unique, but 
> hopefully they are interoperable when needed. As long as the relay isn't 
> requiring an authentication mechanism that is designed to exclude bots 
> (such as the those mentioned above) it should be possible to get Postfix 
> (using Cyrus) to authenticate to it.
> 
> 
> -- 
> Bill Cole
> b...@scconsult.com or billc...@apache.org 
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Not Currently Available For Hire

I guess this is going a bit astray, for some viewers anyway, but after 
repeated authentication failures, resorted to (or availed myself of)
SWAKS and still get authentication failures.  I did set swaks to 
echo the credentials in plaintext, to the screen and know they 
are correct.  I am unsure why they are "broken up" with "->" and 
<-" and wonder if that should provide a clue?  

Below is what is echoed to the screen.
~ # swaks
=== Trying mail.somehost.com:587...
=== Connected to mail.somehost.com.
<-  220 mail.somehost.com ESMTP Postfix
 -> EHLO auxilary
<-  250-forwardx.somehost.com
<-  250-PIPELINING
<-  250-SIZE 56789012
<-  250-VRFY
<-  250-ETRN
<-  250-STARTTLS
<-  250-AUTH PLAIN LOGIN
<-  250-AUTH=PLAIN LOGIN
<-  250-ENHANCEDSTATUSCODES
<-  250-8BITMIME
<-  250 DSN
 -> AUTH LOGIN
<-  334 bla-
 -> blah-user
<-  334 bla
 -> blah-password
<** 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6
*** No authentication type succeeded
 -> QUIT
<-  221 2.0.0 Bye
=== Connection closed with remote host.
 
joe a.

Re: master_wakeup_timer_event

[Wietse Venema]

Wietse Venema:
> natan:
> > W dniu 14.01.2022 o?14:54, Wietse Venema pisze:
> > > natan:
> > >> Hi
> > >> I have very strong machine with load average: 2,22, 2,32, 2,19
> > >>
> > >> and today i get
> > >>
> > >> Jan 14 12:34:25 thebe postfix/master[4925]: warning:
> > >> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> > >> temporarily unavailable
> > >> Jan 14 12:39:25 thebe postfix/master[4925]: warning:
> > >> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> > >> temporarily unavailable
> > >>
> > >> And i don't known where is a problem
> > > The Operating System Kernel is telling Postfix that it could not
> > > connect to or write to the qmgr socket (typically, located at
> > > /var/spool/postfix/public/qmgr).
> > >
> > > Either Postfix has exceeded some per-process limit, or some Operating
> > > System Kernel resource is exhausted.
> 
> 
> Do you know if the problem is a kernel limit or a per-process limit?
> Does master have 4096 open files (including network sockets: ip,
> unix-domain, etc.).

BTW that last one was a trick question: you need a huge number of
services in master.cf to exceed the 4096 limit. The master needs
three sockets for each service with type 'unix' in master.cf;
services with type 'inet' require two sockets plus one socket per
address in inet_interfaces.

Wietse

Re: master_wakeup_timer_event

[natan]

W dniu 14.01.2022 o 18:11, Wietse Venema pisze:
> Wietse Venema:
>> natan:
>>> W dniu 14.01.2022 o?14:54, Wietse Venema pisze:
 natan:
> Hi
> I have very strong machine with load average: 2,22, 2,32, 2,19
>
> and today i get
>
> Jan 14 12:34:25 thebe postfix/master[4925]: warning:
> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> temporarily unavailable
> Jan 14 12:39:25 thebe postfix/master[4925]: warning:
> master_wakeup_timer_event: service qmgr(public/qmgr): Resource
> temporarily unavailable
>
> And i don't known where is a problem
 The Operating System Kernel is telling Postfix that it could not
 connect to or write to the qmgr socket (typically, located at
 /var/spool/postfix/public/qmgr).

 Either Postfix has exceeded some per-process limit, or some Operating
 System Kernel resource is exhausted.
>>
>> Do you know if the problem is a kernel limit or a per-process limit?
>> Does master have 4096 open files (including network sockets: ip,
>> unix-domain, etc.).
> BTW that last one was a trick question: you need a huge number of
> services in master.cf to exceed the 4096 limit. The master needs
> three sockets for each service with type 'unix' in master.cf;
> services with type 'inet' require two sockets plus one socket per
> address in inet_interfaces.
>
>   Wietse
"Do you know if the problem is a kernel limit or a per-process limit?"

I realy dont known where is it the problem - and how diagnose this

I long think about kernel limit but ... no have idea
--

Re: SASL questions

[Bill Cole]

On 2022-01-14 at 12:10:23 UTC-0500 (Fri, 14 Jan 2022 12:10:23 -0500)
Joe Acquisto-j4 
is rumored to have said:
[...]
> I guess this is going a bit astray, for some viewers anyway, but after
> repeated authentication failures, resorted to (or availed myself of)
> SWAKS and still get authentication failures.  I did set swaks to
> echo the credentials in plaintext, to the screen and know they
> are correct.  I am unsure why they are "broken up" with "->" and
> <-" and wonder if that should provide a clue?

Lines with '<-' are coming from the server (Postfix,) '->' lines are sent by 
the client (SWAKS.)

> Below is what is echoed to the screen.
> ~ # swaks
> === Trying mail.somehost.com:587...
> === Connected to mail.somehost.com.
> <-  220 mail.somehost.com ESMTP Postfix
>  -> EHLO auxilary
> <-  250-forwardx.somehost.com
> <-  250-PIPELINING
> <-  250-SIZE 56789012
> <-  250-VRFY
> <-  250-ETRN
> <-  250-STARTTLS
> <-  250-AUTH PLAIN LOGIN
> <-  250-AUTH=PLAIN LOGIN
> <-  250-ENHANCEDSTATUSCODES
> <-  250-8BITMIME
> <-  250 DSN
>  -> AUTH LOGIN
> <-  334 bla-
>  -> blah-user
> <-  334 bla
>  -> blah-password
> <** 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6
> *** No authentication type succeeded
>  -> QUIT
> <-  221 2.0.0 Bye
> === Connection closed with remote host.

I do not see a "STARTTLS" there, so my guess would be that the server is 
configured to only allow authentication with encryption active. That would be 
entirely normal, as the server is only offering plaintext AUTH mechanisms 
(PLAIN and LOGIN) which should only be done over encrypted channels.




-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: SASL questions

[Bill Cole]

One addition:

The swaks command line you'll need to test properly will be something like this:

swaks -tls -t tar...@example.com  --auth-user joea --server mail.example.com:587




On 2022-01-14 at 12:33:18 UTC-0500 (Fri, 14 Jan 2022 12:33:18 -0500)
Bill Cole 
is rumored to have said:

> On 2022-01-14 at 12:10:23 UTC-0500 (Fri, 14 Jan 2022 12:10:23 -0500)
> Joe Acquisto-j4 
> is rumored to have said:
> [...]
>> I guess this is going a bit astray, for some viewers anyway, but after
>> repeated authentication failures, resorted to (or availed myself of)
>> SWAKS and still get authentication failures.  I did set swaks to
>> echo the credentials in plaintext, to the screen and know they
>> are correct.  I am unsure why they are "broken up" with "->" and
>> <-" and wonder if that should provide a clue?
>
> Lines with '<-' are coming from the server (Postfix,) '->' lines are sent by 
> the client (SWAKS.)
>
>> Below is what is echoed to the screen.
>> ~ # swaks
>> === Trying mail.somehost.com:587...
>> === Connected to mail.somehost.com.
>> <-  220 mail.somehost.com ESMTP Postfix
>>  -> EHLO auxilary
>> <-  250-forwardx.somehost.com
>> <-  250-PIPELINING
>> <-  250-SIZE 56789012
>> <-  250-VRFY
>> <-  250-ETRN
>> <-  250-STARTTLS
>> <-  250-AUTH PLAIN LOGIN
>> <-  250-AUTH=PLAIN LOGIN
>> <-  250-ENHANCEDSTATUSCODES
>> <-  250-8BITMIME
>> <-  250 DSN
>>  -> AUTH LOGIN
>> <-  334 bla-
>>  -> blah-user
>> <-  334 bla
>>  -> blah-password
>> <** 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6
>> *** No authentication type succeeded
>>  -> QUIT
>> <-  221 2.0.0 Bye
>> === Connection closed with remote host.
>
> I do not see a "STARTTLS" there, so my guess would be that the server is 
> configured to only allow authentication with encryption active. That would be 
> entirely normal, as the server is only offering plaintext AUTH mechanisms 
> (PLAIN and LOGIN) which should only be done over encrypted channels.
>
>
>
>
> -- 
> Bill Cole
> b...@scconsult.com or billc...@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Not Currently Available For Hire


-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: SASL questions

[Joe Acquisto-j4]

> One addition:
> 
> The swaks command line you'll need to test properly will be something like 
> this:
> 
> swaks -tls -t tar...@example.com  --auth-user joea --server 
> mail.example.com:587
> 

Thanks, that got me over that hump. Test email went through,

Now to translate this effort into fixing my postfix configuration.

joe a

INVALID MessageID reporting?

[Gomes, Rich]

Does anyone have a good way of reporting on this?
I see a great deal in the maillog with either an incorrect format (no @ symbol) 
or just completely blank ( message-id=<>).
We would like to be able to do the following:

Have a WARN message written to the log so we can report and investigate.

I have tried this, but it did not work:
Blocking messages with an empty message-id in postfix - Server 
Fault




Also, has anyone found a way to report on invalid id, showing both the message 
id and the From address?



Thanks in advance


Rich













Rich Gomes | Aramark | Senior Systems Administrator | Messaging and 
Collaboration Services
141 Longwater Drive
Norwell, MA  02061
P: 1 (781) 763-4508

Re: INVALID MessageID reporting?

[Wietse Venema]

Gomes, Rich:
> Does anyone have a good way of reporting on this?
> I see a great deal in the maillog with either an incorrect format (no @ 
> symbol) or just completely blank ( message-id=<>).

According to RFC,the  Message-ID header is optional, and therefore,
email without Message-ID MUST NOT BE BLOCKED.

> We would like to be able to do the following:
> 
> Have a WARN message written to the log so we can report and investigate.

Sloppy PCRE pattern for header_checks:

IF /^message-id:/
/^message-id:\s*<\S+@\S+>/ DUNNO
/./ WARN malformed Message-ID header
ENDIF

The warning logs the sender, recipient, etc.

warning: malformed Message-ID header; from= to= proto= ...

> I have tried this, but it did not work:
> Blocking messages with an empty message-id in postfix - Server 
> Fault

There is no pattern to match a header that does not exist.

Wietse

Re: master_wakeup_timer_event

[Wietse Venema]

natan:
Wietse:
> Do you know if the problem is a kernel limit or a per-process limit?
> Does master have 4096 open files (including network sockets: ip,
> unix-domain, etc.).

Wietse:
> BTW that last one was a trick question: you need a huge number of
> services in master.cf to exceed the 4096 limit. The master needs
> three sockets for each service with type 'unix' in master.cf;
> services with type 'inet' require two sockets plus one socket per
> address in inet_interfaces.

natan:
> "Do you know if the problem is a kernel limit or a per-process limit?"
> 
> I realy dont known where is it the problem - and how diagnose this
> 
> I long think about kernel limit but ... no have idea

Were you the person who has a Postfix process limit in the thousands?
If that is the case, then I suggest that you reduce the Postfix
process limit to half the number, do "postfix reload", wait for a
while, and keep reducing the limit to half its value until the
"resource temporarily unavailable" warnings go away. Also, make
arrangements for more (and more powerful) servers.

Wietse

Re: SASL questions

[Joe Acquisto-j4]

>>  One addition:
>> 
>> The swaks command line you'll need to test properly will be something like 
>> this:
>> 
>> swaks -tls -t tar...@example.com  --auth-user joea --server 
>> mail.example.com:587
>> 
> 
> Thanks, that got me over that hump. Test email went through,
> 
> Now to translate this effort into fixing my postfix configuration.
> 
> joe a

The old adage "read logs and be enlightened" (OK I made that up) seems to hold.

Eventually I emerged from my fog and found the bounce messages were via the 
provider 
IP port 25.  I thought that odd, since TLS seemed established.

Looking at /etc/postfix/transport(.db) found I had not used the form 
[some.host.com]:port.  Fixed that
and all seems to work.

Thanks again.

Re: How can I build a reliable distribution list?

[raf]

On Fri, Jan 14, 2022 at 08:57:31AM -0500, Wietse Venema  
wrote:

> raf:
> > If the distribution itself is working, and the list of
> > members doesn't change often, and it's only SPF that's
> > getting in the way, perhaps the least disruptive
> > solution is to add SRS to Postfix. The problem being
> > that someone who sends to the list has SPF on their
> > domain and a server who receives from the list bounces
> > it because your server isn't allowed by the original
> > sender's SPF.
> 
> NO, NO, NO. A properly operating mailing list overrides the SMTP
> MAIL FROM address with its own address, so that bounces are sent
> to the list admin, instead of individual list members who can't fix
> the problem.
> 
>   Wietse

Yes, but I didn't get the impression that they were
already using mailing list software. I was assuming
that it was just a virtual address that mapped to a
list of remote recipients. Then it would just be a case
of forwarding mail with SPF getting in the way. Adding
SRS would fix that without the need to migrate to
proper mailing list software. That's what I meant by
"less intrusive". But only if their list needs are
simple and fairly static. Otherwise, migrating to
proper mailing list software would be worthwhile.

cheers,
raf

Re: How can I build a reliable distribution list?

[Wietse Venema]

raf:
> If the distribution itself is working, and the list of
> members doesn't change often, and it's only SPF that's
> getting in the way, perhaps the least disruptive
> solution is to add SRS to Postfix. The problem being
> that someone who sends to the list has SPF on their
> domain and a server who receives from the list bounces
> it because your server isn't allowed by the original
> sender's SPF.

Wietse:
> NO, NO, NO. A properly operating mailing list overrides the SMTP
> MAIL FROM address with its own address, so that bounces are sent
> to the list admin, instead of individual list members who can't fix
> the problem.

raf:
> Yes, but I didn't get the impression that they were
> already using mailing list software.

What I wrote was regardless of whether one uses mailing list
management software.

http://www.postfix.org/aliases.5.html

With Postfix aliases(5), if mail is sent to an alias 'foo', and
there also is an alias 'owner-foo', then the enveloope sender address
will be set to owner-foo. This behavior already existed in Sendmail.

Wietse