Message-ID
Hi list, I'm trying to eliminate a problem with duplicate emails in alias expansion and I have seen that some (local generated and SMTP also) messages don't have the Message-ID header . I'm reading in in postconf that there is a configuration option always_add_missing_headers that does exactly that. If I enable this option do I get in any risk of braking anything in my existing setup ? The manual refers that " this may break DKIM signatures that cover non-existent headers" but I'm not sure if that refers to local generated messages or everything received my SMTP. I'm not currently using DKIM in my setup. Thanks for any info George
Re: Message-ID
On 27.01.18 11:16, CP wrote: I'm trying to eliminate a problem with duplicate emails in alias expansion and I have seen that some (local generated and SMTP also) messages don't have the Message-ID header . I'm reading in in postconf that there is a configuration option always_add_missing_headers that does exactly that. If I enable this option do I get in any risk of braking anything in my existing setup ? The manual refers that " this may break DKIM signatures that cover non-existent headers" but I'm not sure if that refers to local generated messages or everything received my SMTP. I'm not currently using DKIM in my setup. If you receive dkim-signed mail without message-id, but the dkim signature will claim it contains message-id too (does this happen?), and when you add it, the message will not pass the check (including your message-id changes the hash). this should not be a problem if you aren't resending the mail to outside, and can be avoided by checking the DKIM before message-id is added and never later. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Eagles may soar, but weasels don't get sucked into jet engines.
Re: Message-ID
On 01/27/2018 12:09 PM, Matus UHLAR - fantomas wrote: On 27.01.18 11:16, CP wrote: I'm trying to eliminate a problem with duplicate emails in alias expansion and I have seen that some (local generated and SMTP also) messages don't have the Message-ID header . I'm reading in in postconf that there is a configuration option always_add_missing_headers that does exactly that. If I enable this option do I get in any risk of braking anything in my existing setup ? The manual refers that " this may break DKIM signatures that cover non-existent headers" but I'm not sure if that refers to local generated messages or everything received my SMTP. I'm not currently using DKIM in my setup. If you receive dkim-signed mail without message-id, but the dkim signature will claim it contains message-id too (does this happen?), and when you add it, the message will not pass the check (including your message-id changes the hash). So far I have only seen messages without the message-id header , is there some way to check for what you're describing above in received message headers ? this should not be a problem if you aren't resending the mail to outside, and can be avoided by checking the DKIM before message-id is added and never later. Usually I don't resend the message but users might forward the message to an outside recipient, this is not the case I hope ? Checking DKIM is done by postfix ? any further info or reading about this ? Thanks In advance
Re: Message-ID
On 27.01.18 11:16, CP wrote: I'm trying to eliminate a problem with duplicate emails in alias expansion and I have seen that some (local generated and SMTP also) messages don't have the Message-ID header . I should add that such duplicity should not happen when using aliases. Also, it may happen that two people get different copies of the same mail (e.g. from this list). Are you sure your duplicities can't be fixed other way? I'm reading in in postconf that there is a configuration option always_add_missing_headers that does exactly that. If I enable this option do I get in any risk of braking anything in my existing setup ? The manual refers that " this may break DKIM signatures that cover non-existent headers" but I'm not sure if that refers to local generated messages or everything received my SMTP. I'm not currently using DKIM in my setup. On 01/27/2018 12:09 PM, Matus UHLAR - fantomas wrote: If you receive dkim-signed mail without message-id, but the dkim signature will claim it contains message-id too (does this happen?), and when you add it, the message will not pass the check (including your message-id changes the hash). On 27.01.18 13:21, CP wrote: So far I have only seen messages without the message-id header , is there some way to check for what you're describing above in received message headers ? this is from your mail: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; you see message-id there. if such DKIM header includes Message-Id: but message has none, DKIM breakage may happen when you add it. this should not be a problem if you aren't resending the mail to outside, and can be avoided by checking the DKIM before message-id is added and never later. Usually I don't resend the message but users might forward the message to an outside recipient, this is not the case I hope ? no. Checking DKIM is done by postfix ? any further info or reading about this ? as you configure, usually spam filter checks for it. if you are going to add message-id as postfix level you apparently need implement before that, DKIM milter probably. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. If Barbie is so popular, why do you have to buy her friends?
Re: Message-ID
On 01/27/2018 02:05 PM, Matus UHLAR - fantomas wrote: On 27.01.18 11:16, CP wrote: I'm trying to eliminate a problem with duplicate emails in alias expansion and I have seen that some (local generated and SMTP also) messages don't have the Message-ID header . I should add that such duplicity should not happen when using aliases. Also, it may happen that two people get different copies of the same mail (e.g. from this list). Are you sure your duplicities can't be fixed other way? In what way ? check alias expansion ? The thing is I have some groups say group1 group2 and group3 there may be users that are part of only one group but there are others that maybe in all groups . Until I implemented lmtp and duplicate supression from it (dovecot) a user who was in all 3 groups would receive the message 3 times (and if send to his own email address 4 !) The problem persists even with lmtp and messages that don't have Message-ID in header ( I guess this is the criteria for supressing duplicates) . How can I track down this ? Thanks for your time
Re: Message-ID
On 2018-01-27 17:24, CP wrote: > On 01/27/2018 02:05 PM, Matus UHLAR - fantomas wrote: On 27.01.18 11:16, CP wrote: > I'm trying to eliminate a problem with duplicate emails in alias > expansion and I have seen that some (local generated and SMTP also) > messages don't have the Message-ID header . >> >> I should add that such duplicity should not happen when using aliases. Also, >> it may happen that two people get different copies of the same mail >> (e.g. from this list). >> >> Are you sure your duplicities can't be fixed other way? > > In what way ? check alias expansion ? > > The thing is I have some groups say group1 group2 and group3 > > there may be users that are part of only one group but there are > others that maybe in all groups . > Until I implemented lmtp and duplicate supression from it (dovecot) a > user who was in all 3 groups > would receive the message 3 times (and if send to his own email > address 4 !) The problem persists > even with lmtp and messages that don't have Message-ID in header ( I > guess this is the criteria for supressing > duplicates) . > > How can I track down this ? > > > Thanks for your time Dovecot documentation says: "For more complex applications, the duplicate test can also use the content of a specific header field or other parts of the message." https://wiki.dovecot.org/Pigeonhole/Sieve/Extensions/Duplicate I couldn't find specific example but maybe dovecot list is a better place to ask. It might be possible to detect duplicates using different headers. Anyway just Message-ID might be not a perfect solution. If I understand your issue correctly I think that I have the same problem that the user is a part of an alias office@ and he gets duplicates of all mails send to him and copied to office@. So @office -> guy1@, @guy2. If the e-mail is addressed to guy1@, office@ then Guy1 gets two copies and Guy2 only one. Often this e-mail is sent in two different SMTP sessions so I don't see any possibility that Postfix can catch that. It's not a big deal so I have never looked into that but now I might. The problem with using solely Message-ID is that off-list replies will be matched with list-replies and I don't want that. Don't know if there are any other possible side-effects of basically discarding same Message-ID going to the same user in some finite timeframe. Karol -- Karol Augustin ka...@augustin.pl http://karolaugustin.pl/ +353 85 775 5312
Re: Message-ID
On 2018-01-27 19:30, Karol Augustin wrote: > On 2018-01-27 17:24, CP wrote: >> On 01/27/2018 02:05 PM, Matus UHLAR - fantomas wrote: > On 27.01.18 11:16, CP wrote: >> I'm trying to eliminate a problem with duplicate emails in alias >> expansion and I have seen that some (local generated and SMTP also) >> messages don't have the Message-ID header . >>> >>> I should add that such duplicity should not happen when using aliases. >>> Also, it may happen that two people get different copies of the same mail >>> (e.g. from this list). >>> >>> Are you sure your duplicities can't be fixed other way? >> >> In what way ? check alias expansion ? >> >> The thing is I have some groups say group1 group2 and group3 >> >> there may be users that are part of only one group but there are >> others that maybe in all groups . >> Until I implemented lmtp and duplicate supression from it (dovecot) a >> user who was in all 3 groups >> would receive the message 3 times (and if send to his own email >> address 4 !) The problem persists >> even with lmtp and messages that don't have Message-ID in header ( I >> guess this is the criteria for supressing >> duplicates) . >> >> How can I track down this ? >> >> >> Thanks for your time > > Dovecot documentation says: "For more complex applications, the > duplicate test can also use the content of a specific header field or > other parts of the message." > https://wiki.dovecot.org/Pigeonhole/Sieve/Extensions/Duplicate > > I couldn't find specific example but maybe dovecot list is a better > place to ask. It might be possible to detect duplicates using different > headers. Anyway just Message-ID might be not a perfect solution. > > If I understand your issue correctly I think that I have the same > problem that the user is a part of an alias office@ and he gets > duplicates of all mails send to him and copied to office@. > > So @office -> guy1@, @guy2. If the e-mail is addressed to guy1@, office@ > then Guy1 gets two copies and Guy2 only one. Often this e-mail is sent > in two different SMTP sessions so I don't see any possibility that > Postfix can catch that. > > It's not a big deal so I have never looked into that but now I might. > The problem with using solely Message-ID is that off-list replies will > be matched with list-replies and I don't want that. Don't know if there > are any other possible side-effects of basically discarding same > Message-ID going to the same user in some finite timeframe. > Also in the scenario I described adding Message-ID by postfix won't help you as I think it will add different message-ids if e-mail arrives in two SMTP sessions (like from gmail for example). -- Karol Augustin ka...@augustin.pl http://karolaugustin.pl/ +353 85 775 5312
Re: Request for feedback on SMTPD restrictions
On 1/26/2018 11:47 PM, Voytek wrote: > On Wed, January 24, 2018 3:55 am, Noel Jones wrote: > >> There is no simple regexp, but there is the fqrdns.pcre project. The >> project is a large hand-maintained list of dynamic hostnames with a goal of >> zero false positives. It's not perfect, but it's useful and safe for >> general use. >> >> https://github.com/stevejenkins/hardwarefreak.com-fqrdns.pcre > > within my current list, where should I add ? > > check_client_access hash:/etc/postfix/whitelist > check_reverse_client_hostname_access pcre:/etc/postfix/fqrdns.pcre > Generally, something like: [restrictions applied to your customers/users] [permit_mynetworks/sasl_authenticated] [reject_unauth_destination] [whitelists for client/sender/recipient exceptions] [local blacklist access lists] [policy services] [DNS blacklists] In some cases there are good reasons to do things differently, but the above suits many people. So generally, you can put it anywhere after reject_unauth_destination and after any whitelists. Just above the first reject_rbl_ is fine. -- Noel Jones > > smtpd_recipient_restrictions =. > reject_unknown_sender_domain, > reject_unknown_recipient_domain,. > reject_non_fqdn_sender, > reject_non_fqdn_recipient, > reject_unlisted_recipient, > permit_mynetworks, > check_sasl_access hash:/etc/postfix/sasl_access > permit_sasl_authenticated, > reject_unauth_destination, > check_recipient_access hash:/etc/postfix/recipient_no_checks, > check_recipient_access pcre:/etc/postfix/recipient_checks.pcre, > check_helo_access hash:/etc/postfix/helo_checks, > check_sender_access hash:/etc/postfix/sender_checks, > check_client_access hash:/etc/postfix/client_checks, > check_client_access pcre:/etc/postfix/client_checks.pcre, > reject_rbl_client zen.spamhaus.org, > reject_rhsbl_client dbl.spamhaus.org, > reject_rhsbl_sender dbl.spamhaus.org, > reject_rbl_client psbl.surriel.com, > reject_rbl_client ix.dnsbl.manitu.net, > reject_rbl_client bl.spamcop.net, > >
