Logfile condenser

[Dom Latter]

Helo, as one might say.

Wading through logfiles to find a particular email has been
taking up too much of my time recently [1].

I have written a logfile condenser that takes a postfix log and
generates a single line per email, with (for example) timestamp,
ham /spam, sender, recipient, spam score, size.

Thus, for example, a 25000 line logfile (about 4MB) is condensed
into 800 lines; grepping for " ham " pulls out 200 lines of
actual delivered emails.  It's reasonably fast (less than a second
to do the condensing on a modern-ish box).

I wrote my own because none of the existing logfile tools did
quite what I want.  The closest seemed to be this:
http://manpages.ubuntu.com/manpages/lucid/man1/postfix2dlf.1.html

The question is - is there any community interest in this tool?

As something hacked up for my own purposes it is most definitely
beta code.  But if there's interest I could tidy it up and
release it.

It will probably upset the purists for two reasons: it's written
in PHP [1] because that was going to be quickest for me; and it doesn't
use regexps, but rather a lot of substr()s to get the information
out.

If there's something better out there, please say.

[0] From the example stats - that's more than 100 lines of logfile
per "actual" email.
[1] It's still a command line tool.

RE: Logfile condenser

[Mark Scholten]

> -Original Message-
> From: owner-postfix-us...@postfix.org [mailto:owner-postfix-
> us...@postfix.org] On Behalf Of Dom Latter
> Sent: Tuesday, October 19, 2010 11:41 AM
> To: postfix-users@postfix.org
> Subject: Logfile condenser
> 
> Helo, as one might say.
> 
> Wading through logfiles to find a particular email has been
> taking up too much of my time recently [1].
> 
> I have written a logfile condenser that takes a postfix log and
> generates a single line per email, with (for example) timestamp,
> ham /spam, sender, recipient, spam score, size.
> 
> Thus, for example, a 25000 line logfile (about 4MB) is condensed
> into 800 lines; grepping for " ham " pulls out 200 lines of
> actual delivered emails.  It's reasonably fast (less than a second
> to do the condensing on a modern-ish box).
> 
> I wrote my own because none of the existing logfile tools did
> quite what I want.  The closest seemed to be this:
> http://manpages.ubuntu.com/manpages/lucid/man1/postfix2dlf.1.html
> 
> The question is - is there any community interest in this tool?
> 
> As something hacked up for my own purposes it is most definitely
> beta code.  But if there's interest I could tidy it up and
> release it.
>From here there is interest in it. I would edit it to include the
client/destination server and message ID, but for the rest it sounds great.
> 
> It will probably upset the purists for two reasons: it's written
> in PHP [1] because that was going to be quickest for me; and it doesn't
> use regexps, but rather a lot of substr()s to get the information
> out.
> 
> If there's something better out there, please say.
> 
> [0] From the example stats - that's more than 100 lines of logfile
> per "actual" email.
> [1] It's still a command line tool.

Re: Logfile condenser

[Dom Latter]

On 19/10/10 11:55, Mark Scholten wrote:


From here there is interest in it. I would edit it to include the


Encouraging!


client/destination server and message ID, but for the rest it sounds great.


Configurable as follows, e.g.:
$output = array ('time', 'meat', 'from', 'to', 'hits', 'size');

'meat' is one of ham / spam / flam.  'flam' is anything flagged
as BANNED.  Why 'flam'?  Not sure.  Maybe 'fat'?

Additional fields are msg-id, client (host / IP mail comes from),
relay (host / IP mail goes to), status (e.g. 'sent'), and possibly
'handle' which is the last ESMTP ID that the message has as it
goes through the server.

It may not scale well when faced with really huge logfiles but the
reasons for that probably deserve a separate thread.

Re: Resend emails from a Maildir

[Roberto Scattini]

On Mon, Oct 18, 2010 at 6:56 AM, Patric Falinder wrote:

> Hi,
>
> I'm not really sure if this has to do with Postifx so just tell me if I'm
> wrong.
>
> Is it possible to resend emails that are in a Maildir already?
> The reason for this is that a user changed server so I had to change the
> transport for that domain and there are something like 50 mails in the old
> Maildir that I need to "resend" to the new server.
>
> Is that possible?
>
>
> Thanks,
> -Patric
>


maybe imapsync?


-- 
Roberto Scattini
 ___ _
 ))_) __ )L __
((__)(('(( ((_)

Re: Fighting Backscatter

[Wietse Venema]

Steve Jenkins:
> Gotit. Thanks again for helping me out. I'm still learning.
> 
> So it seems I need to figure out how to stop the backscatter process at step
> 6 and NOT return the bounce to the original sender.

No. Solve the RIGHT problem. DO NOT forward SPAM.

Wietse

automatic add attachments in postfix

[David Touzeau]

Dear 

I would like to find a tool that be able to add attachment in each
outgoing mails, i was thinking about altermime but altermime add only
txt,html file.
I need to add vcf or pdf files.
Is anyone know if a such tool exists ?

best regards.

Re: Relay Access Denied using SASL

[Charles Marcus]

On 2010-10-19 1:42 AM, Yang Zhang wrote:
> smtpd_recipient_restrictions is specified a second time.
> 
> A depressing amount of time was spent debugging this.

For future reference, this is exactly why you should never go by what
you see in main.cf (nor should you post excerpts from it here), but what
you see in the output of postconf -n, which shows you exactly what
postfix is using, not what you think it is using.

Doing this avoids the problem you had, as well as if you had been
editing the wrong main.cf (happens a lot, especially for people new to
postfix)...

-- 

Best regards,

Charles

Re: Fighting Backscatter

[Charles Marcus]

On 2010-10-18 9:58 PM, Steve Jenkins wrote:
> The instructions at http://www.postfix.org/BACKSCATTER_README.html 
> seem to only address what to do if MY server is the one being
> forged. In the above example, it seems that procom.ca is being
> forged. How should I configure my Postfix installation so that I'm
> not sending the spam back to the innocent sender? Let me know if you
> need me to post my postconf -n again.

As has been told to you more than once, the correct solution is simple...

1. Stop forwarding spam, or

2. Do not forward *any* emails, period.

If you absolutely *must* allow your users the option to blindly forward
*all* incoming mail to another account, then you need to do one of two
things:

1. Disable *all* spam filtering on the target address, or

2. Implement some kind of additional spam filtering on accounts that
forward mail that refuses to forward any mail with a very *low* spam
threshhold.

If you cannot do one or the other reliably, then you will continue to
have the problem.

-- 

Best regards,

Charles

Re: Fighting Backscatter

[Ralf Hildebrandt]

* Charles Marcus :

> As has been told to you more than once, the correct solution is simple...
> 
> 1. Stop forwarding spam, or

As we all know that not really easily done. I might consider a mail
"ham" while other systems consider the mail to be "spam".

The first step must be to check HOW MANY mails are bouncing at all.

A big percentage? And then one needs to check the anti spam methods
one is using. And compare them to the checks the "real" recipient is
using.

Did the OP show "postconf -n" yet?

> 2. Do not forward *any* emails, period.

That's probably the simplest solution :)

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | http://www.charite.de

Re: Fighting Backscatter

[martijn.list]

>> 2. Do not forward *any* emails, period.
> 
> That's probably the simplest solution :)

Wouldn't using an owner alias be a solution? (see expand_owner_alias).

You can set the owner alias of the forward to some internal address.
This internal address will only be used to 'suck-up' the bounces of
forwarded messages.

Martijn

Re: Fighting Backscatter

[Ralf Hildebrandt]

* martijn.list :
> >> 2. Do not forward *any* emails, period.
> > 
> > That's probably the simplest solution :)
> 
> Wouldn't using an owner alias be a solution? (see expand_owner_alias).
> 
> You can set the owner alias of the forward to some internal address.
> This internal address will only be used to 'suck-up' the bounces of
> forwarded messages.

Interesting idea!

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | http://www.charite.de

Re: Fighting Backscatter

[martijn.list]

>> Wouldn't using an owner alias be a solution? (see expand_owner_alias).
>>
>> You can set the owner alias of the forward to some internal address.
>> This internal address will only be used to 'suck-up' the bounces of
>> forwarded messages.
> 
> Interesting idea!

It works for me. I forward a copy of every incoming email to my
BlackBerry and Gmail. Sometimes however, a spam/virus email with a zip
attachment makes it through the filters. Gmail however, blocks all email
containing zip files. Because I have used an owner alias for the
forward, the email is now bounced back to forw...@internal.tld which is
an internal account which is periodically pop'd for bounces.

Martijn

Re: Fighting Backscatter

[Noel Jones]

On 10/19/2010 7:00 AM, Ralf Hildebrandt wrote:

* martijn.list:

2. Do not forward *any* emails, period.


That's probably the simplest solution :)


Wouldn't using an owner alias be a solution? (see expand_owner_alias).

You can set the owner alias of the forward to some internal address.
This internal address will only be used to 'suck-up' the bounces of
forwarded messages.


Interesting idea!



If the owner-alias is truly monitored (likely an onerous 
task), this is a reasonable solution.  If the owner-alias is 
never checked or is forwarded to /dev/null then it's just 
another way to discard bounces and not acceptable.



  -- Noel Jones

smtpd_chat_query, dovecot sasl, AD, Samba4

[Trever L. Adams]

 Hello everyone,

I am hoping someone can help me with a problem.

I have a Samba4 AD domain. I have dovecot setup to do gssapi against
this AD. I have postfix configured to use dovecot sasl. It works great
for imap in windows and linux. In Windows, smtp doesn't work for gssapi,
but works for plain and login (which are configured along with gssapi in
dovecot). These are all Thunderbird.

With Windows I get the following:
postfix/smtpd[6364]: warning: CLIENT_FQDN[CLIENT_IP]: request longer
than 2048: AUTH GSSAPI gssapidata.

My query is this: does postfix truncate or otherwise mess up the data
when the above warning is given? If not, has anyone seen something
similar? Is it a bug in Thunderbird?

I imagine the reason Kerberos is larger in Windows than Linux is the PAC.

Some of you may have seen other posts elsewhere which state I have fixed
this and it was related to S4 not setting userPrincipalName correctly
for a service principal. This is what got Linux Thunderbird working, but
not Windows.

Any help anyone can offer would be GREATLY appreciated.

Thank you,
Trever Adams
-- 
"Black holes are where God divided by zero." -- Unknown




signature.asc
Description: OpenPGP digital signature

Re: Fighting Backscatter

[pf at alt-ctrl-del.org]

On 2010-10-18 9:58 PM, Steve Jenkins wrote:
The instructions at http://www.postfix.org/BACKSCATTER_README.html 
seem to only address what to do if MY server is the one being

forged. In the above example, it seems that procom.ca is being
forged. How should I configure my Postfix installation so that I'm
not sending the spam back to the innocent sender? Let me know if you
need me to post my postconf -n again.



"Charles Marcus" October 19, 2010 7:38 AM:

As has been told to you more than once, the correct solution is simple...

1. Stop forwarding spam, or

2. Do not forward *any* emails, period.



The OP can actually do #2. He probably doesn't need to forward any email at all.
gmail, yahoo, comcast all support POP3'ing third party email accounts, straight 
to the mailbox.

Automatic POP3 retrieval...
No forwarding, no bounces.

But I'm not sure about cox support for auto POP3 retrieval.

Re: smtpd_chat_query, dovecot sasl, AD, Samba4

[Wietse Venema]

Trever L. Adams:
> With Windows I get the following:
> postfix/smtpd[6364]: warning: CLIENT_FQDN[CLIENT_IP]: request longer
> than 2048: AUTH GSSAPI gssapidata.
> 
> My query is this: does postfix truncate or otherwise mess up the data
> when the above warning is given? If not, has anyone seen something
> similar? Is it a bug in Thunderbird?

Postfix enforces a very generous SMTP command line length limit of
2048 bytes. You're welcome to increase main.cf:line_length_limit
but I have never heard of a case where it was too small.

While the SMTP RFCs give limits on line lengths, and RFC 2554
overrides the length for MAIL FROM, RFC 2554 does not override
the length limit for other commands.

Wietse

Re: smtpd_chat_query, dovecot sasl, AD, Samba4

[Trever L. Adams]

 On 10/19/2010 08:10 AM, Wietse Venema wrote:
>
> Postfix enforces a very generous SMTP command line length limit of
> 2048 bytes. You're welcome to increase main.cf:line_length_limit
> but I have never heard of a case where it was too small.
>
> While the SMTP RFCs give limits on line lengths, and RFC 2554
> overrides the length for MAIL FROM, RFC 2554 does not override
> the length limit for other commands.
>
>   Wietse
Thank you very much for helping me out. It appears that Thunderbird for
Windows in an AD domain is sending more than 2048 for smtp kerberos. I
am going to try increasing the size and find out where it starts to work.

Thank you.
Trever
-- 
"The best we can hope for concerning the people at large is that they be
properly armed." -- Alexander Hamilton, The Federalist Papers at 184-188



signature.asc
Description: OpenPGP digital signature

Re: smtpd_chat_query, dovecot sasl, AD, Samba4

[Trever L. Adams]

 Ok, so it is documented for others.

If you are using dovecot sasl with postfix and are using Thunderbird in
Windows (part of an AD domain) and using smtp kerberos authentication,
make sure you have line_length_limit = 2176 in postfix's main.cf.

Thanks to Wietse for his help.

Trever
-- 
"The Master doesn't talk, he acts. When his work is done, the people
say, 'Amazing: we did it, all by ourselves!'" -- Lao-tzu



signature.asc
Description: OpenPGP digital signature

Re: Request for help with redesign of Postfix Configuration File ...

[Ben McGinnes]

On 19/10/10 8:04 AM, Ralf Hildebrandt wrote:
> * Christopher Koeber :
> 
>> inet_interfaces = all
> default
> 
>> mydomain = students.wesleyseminary.edu
>> myhostname = students.wesleyseminary.edu
> 
> I'd say myhostname = students.wesleyseminary.edu
> which implies mydomain = wesleyseminary.edu
> 
> Setting mydomain equal to myhostname strikes me as being odd.

Not if the students subdomain has different MX details to the main
domain, which appears to be the case here:

;; ANSWER SECTION:
wesleyseminary.edu. 43098   IN  MX  10 mail1.no-ip.com.
wesleyseminary.edu. 43098   IN  MX  15 mail2.no-ip.com.
wesleyseminary.edu. 43098   IN  MX  5 mail.wesleyseminary.edu.

;; ANSWER SECTION:
students.wesleyseminary.edu. 1674 INMX  5 students.wesleyseminary.edu.

Probably a good idea in any educational environment.


Regards,
Ben



signature.asc
Description: OpenPGP digital signature

Upgrade 2.5.4

[Linux Addict]

Hello, I am running postfix 2.5.4 and would like to upgrade it to latest
stable 2.7.0. What is the best way upgrade? Do a clean install and port the
settings to newer version? Any help is appreciated.

~LA

Re: Request for help with redesign of Postfix Configuration File ...

[Jeroen Geilman]

On 10/19/2010 06:12 PM, Ben McGinnes wrote:

Not if the students subdomain has different MX details to the main
domain, which appears to be the case here:

;; ANSWER SECTION:
wesleyseminary.edu. 43098   IN  MX  10 mail1.no-ip.com.
wesleyseminary.edu. 43098   IN  MX  15 mail2.no-ip.com.
wesleyseminary.edu. 43098   IN  MX  5 mail.wesleyseminary.edu.

;; ANSWER SECTION:
students.wesleyseminary.edu. 1674 INMX  5 students.wesleyseminary.edu.

Probably a good idea in any educational environment.

   


Yet totally superfluous; the A record suffices.

--
J.

Re: virtual_alias_domains

[Jeroen Geilman]

On 10/18/2010 10:56 PM, Jerrale G wrote:

On 10/18/2010 4:43 PM, Jeroen Geilman wrote:

On 10/18/2010 10:36 PM, Jerrale G wrote:

 On 10/18/2010 4:29 PM, The Doctor wrote:

REcently I have noted that virtual_alias_domains is growing.

Is their some way for main.cf to look a file up instead of
having to read a whole line?



You are limited to using mysql, ONE file, ldap, postgresql, or mssql 
for each virtual_*_* parameter or any parameter that looks up 
something.


Certainly not.

All *_maps parameters can be specified as many times as necessary, 
for instance:


virtual_alias_maps = hash:/etc/postfix/virtual, 
mysql:/etc/postfix/my_virtual, ldap:/etc/postfix/ldap_virtual


They're not called mapS for nothing.

virtual_alias_domains is not a map setting per se, as it does nothing 
with the RHS of the lookup - but it works the same way.


From the postconf(5) man page:

Specify a list of host or domain names, "/file/name" or "type:table 
" patterns, separated by 
commas and/or whitespace.


i.e. "a list of" any of the map types postfix supports.
--
J.
 


yes, but that concats and looks in multiple places and it should NOT 
exist in more than one place.


Because ?


You MIGHT be able to do:

virtual_alias_maps = hash:/etc/postfix/virtual, 
mysql:/etc/postfix/my_virtual, ldap:/etc/postfix/ldap_virtual, 
*hash:/etc/postfix/hasfiles/%s

*


Um, no.


* *
* *If there is no variable available that you want to use, then no


No ? What no ?


and I'm not saying you can use variables in the config.


You can, it's littered with them.

In dovecot you can use variables globally, with respect to some 
variables being exclusive to certain libexec's, and I'm only saying to 
try it.


I have no idea what that means.


--
J.

Re: automatic add attachments in postfix

[Jeroen Geilman]

On 10/19/2010 01:26 PM, David Touzeau wrote:

Dear

I would like to find a tool that be able to add attachment in each
outgoing mails, i was thinking about altermime but altermime add only
txt,html file.
I need to add vcf or pdf files.
Is anyone know if a such tool exists ?

   


A Mail Transport Agent or MTA is not the place to manipulate the 
contents of messages.

Postfix itself offers no support for this.

It's possible to send all mail through a process that adds an attachment 
and then re-sends the mail to the original recipients - but be advised 
that doing so will break, among other things,  S/MIME.



best regards.



   



--
J.

Re: Upgrade 2.5.4

[Victor Duchovni]

On Tue, Oct 19, 2010 at 12:17:23PM -0400, Linux Addict wrote:

> Hello, I am running postfix 2.5.4 and would like to upgrade it to latest
> stable 2.7.0. What is the best way upgrade? Do a clean install and port the
> settings to newer version? Any help is appreciated.

No. Do an upgrade. If installing from source:

Read the RELEASE_NOTES file for 2.6 and 2.7, then:

% make
% su root
# postfix stop
# make upgrade
# postfix start

If installing from a well constructed package:

Read the RELEASE_NOTES file for 2.6 and 2.7, then:

# postfix stop
# some-command-to-install-updated-package
# postfix start

In either case, save the updated main.cf and master.cf files that
are automatically upgraded as part of the install process.

If the package is not well constructed:

Read the RELEASE_NOTES file for 2.6 and 2.7, then:

# postfix stop

# mkdir -p /etc/postfix/cfsavedir
# cp /etc/postfix/main.cf /etc/postfix/master.cf \
/etc/postfix/cfsavedir/

# some-command-to-install-updated-poorly-constructed-package

# cp /etc/postfix/cfsavedir/main.cf /etc/postfix/cfsavedir/master.cf \
/etc/postfix/
# postfix set-permissions upgrade-configuration

# postfix start

A package is not well contstructed if it fails to preserve and upgrade
your existing main.cf and master.cf files.

-- 
Viktor.

Re: Logfile condenser

[Jeroen Geilman]

On 10/19/2010 12:37 PM, Dom Latter wrote:

On 19/10/10 11:55, Mark Scholten wrote:


From here there is interest in it. I would edit it to include the


Encouraging!

client/destination server and message ID, but for the rest it sounds 
great.


Configurable as follows, e.g.:
$output = array ('time', 'meat', 'from', 'to', 'hits', 'size');



How does it deal with address rewriting and alias expansion, which is 
the routing core of postfix ?


There's a good reason you need more than one line to log an email 
message - the envelope coming in isn't always the one going out.


I use awstats with the (provided) postfix logging method to get 
one-sided (i.e. one fact per line) stats, but the nature of mail is such 
that each message yields at least two relevant log lines: one coming in 
and one going out.
That unfortunately can't show me the correlation between incoming and 
outgoing mail, because you'd have to do heuristics on the queue-IDs to 
match them together.


If I had to name the log fields I want to see (and consider that these 
come from - possibly wildly - different log entries) it would be:


"timestamp - client IP [hostname] - orig_envelope_from > 
orig_envelope_to - Queue ID -  final_envelope_from > final_envelope_to - 
transport:nexthop - server IP [hostname] - delays"


And that's probably not even complete yet, I thought it up just now.


I guess we'd like to see a small sample :)

Can you share the script ?

--
J.

Re: smtpd_chat_query, dovecot sasl, AD, Samba4

[Victor Duchovni]

On Tue, Oct 19, 2010 at 10:10:32AM -0400, Wietse Venema wrote:

> > My query is this: does postfix truncate or otherwise mess up the data
> > when the above warning is given? If not, has anyone seen something
> > similar? Is it a bug in Thunderbird?
> 
> Postfix enforces a very generous SMTP command line length limit of
> 2048 bytes. You're welcome to increase main.cf:line_length_limit
> but I have never heard of a case where it was too small.
> 
> While the SMTP RFCs give limits on line lengths, and RFC 2554
> overrides the length for MAIL FROM, RFC 2554 does not override
> the length limit for other commands.

RFC 4954, which supercedes 2554, states in Section 4:

  Note that the AUTH command is still subject to the line length
  limitations defined in [SMTP].  If use of the initial response
  argument would cause the AUTH command to exceed this length,
  the client MUST NOT use the initial response parameter (and
  instead proceed as defined in Section 5.1 of [SASL]).

-- 
Viktor.

Re: smtpd_chat_query, dovecot sasl, AD, Samba4

[Victor Duchovni]

On Tue, Oct 19, 2010 at 08:37:10AM -0600, Trever L. Adams wrote:

>  Ok, so it is documented for others.
> 
> If you are using dovecot sasl with postfix and are using Thunderbird in
> Windows (part of an AD domain) and using smtp kerberos authentication,
> make sure you have line_length_limit = 2176 in postfix's main.cf.

This ad-hoc number is unlikely to be universal. The contents of a
GSSAPI token vary from site to site and user to user.

-- 
Viktor.

RE: What's done with malformed headers

[Murray S. Kucherawy]

> -Original Message-
> From: owner-postfix-us...@postfix.org 
> [mailto:owner-postfix-us...@postfix.org] On Behalf Of Wietse Venema
> Sent: Tuesday, September 28, 2010 9:52 PM
> To: Postfix users
> Subject: Re: What's done with malformed headers
> 
> The first line that is "not a header" is considered to be part of
> the message body.
> 
> Postfix may or may not prepend a blank line to this non-header
> line (it always, did but then it was found that it is better not
> to insert a blank line in malformed attachments. As of 20070112
> Postfix inserts the missing blank line only after the primary
> message header).

Does this go for a header field in which there's a space between the name and 
the colon as well?  For example:

MIME-Version : 1.0

Sendmail, for example, corrects this and accepts it without starting the body 
there.

I'm collecting enough of these results from open source and commercial MTAs 
alike that I'll probably seek to publish a "best current practices" about it.

Thanks,
-MSK

Re: automatic add attachments in postfix

[David Touzeau]

Le mardi 19 octobre 2010 à 19:33 +0200, Jeroen Geilman a écrit :

> On 10/19/2010 01:26 PM, David Touzeau wrote:
> > Dear
> >
> > I would like to find a tool that be able to add attachment in each
> > outgoing mails, i was thinking about altermime but altermime add only
> > txt,html file.
> > I need to add vcf or pdf files.
> > Is anyone know if a such tool exists ?
> >
> >
> 
> A Mail Transport Agent or MTA is not the place to manipulate the 
> contents of messages.
> Postfix itself offers no support for this.
> 
> It's possible to send all mail through a process that adds an attachment 
> and then re-sends the mail to the original recipients - but be advised 
> that doing so will break, among other things,  S/MIME.
> 
> > best regards.
> >
> >
> >
> >
> 
> 


Thanks for your advise... 

"It's possible to send all mail through a process" : I need to find
it...  

That should be cool to perform this task on a gateway if you cannot
handle back-end servers.

Re: What's done with malformed headers

[Wietse Venema]

Wietse:
> The first line that is "not a header" is considered to be part of
> the message body.
> 
> Postfix may or may not prepend a blank line to this non-header
> line (it always, did but then it was found that it is better not
> to insert a blank line in malformed attachments. As of 20070112
> Postfix inserts the missing blank line only after the primary
> message header).

Murray S. Kucherawy:
> Does this go for a header field in which there's a space between
> the name and the colon as well?  For example:
>
>   MIME-Version : 1.0

Postfix permits whitespace between header name and colon, as per
RFC 2822 Section 4.5 (see comment in global/is_header.c).

The Postfix MIME parser trims the space to simplify message handling
further down the line, even if it may break a digital signature
(see comment in mime_state.c:mime_state_update()).

The way that Postfix is built it will never send this obsolete
header form into a signing algorithm.

> Sendmail, for example, corrects this and accepts it without starting
> the body there.
> 
> I'm collecting enough of these results from open source and
> commercial MTAs alike that I'll probably seek to publish a "best
> current practices" about it.

Wietse

Re: What's done with malformed headers

[Victor Duchovni]

On Tue, Oct 19, 2010 at 11:40:10AM -0700, Murray S. Kucherawy wrote:

> Does this go for a header field in which there's a space between the name and 
> the colon as well?  For example:
> 
>   MIME-Version : 1.0

The *22 RFCs allow it, so Postfix does too.

Comments starting line 59 of src/global/is_header.c:

 * XXX RFC 2822 Section 4.5, Obsolete header fields: whitespace may
 * appear between header label and ":" (see: RFC 822, Section 3.4.2.).

> Sendmail, for example, corrects this and accepts it without starting
> the body there.

The Postfix MIME parser also strips the obsolete whitespace:

Comments starting line 866 of src/global/mime_state.c:

 * Normalize obsolete "name space colon" syntax to "name colon".
 * Things would be too confusing otherwise.

If MIME processing is disabled by the administrator, the above applies
only to the primary headers.

*Each* folded header-line may be up to 100k bytes in length. There is
no limit on the total length of all headers, other than the message
size limit.

-- 
Viktor.

Re: Logfile condenser

[Dom Latter]

[as subject]

On 19/10/10 19:42, Jeroen Geilman wrote:


How does it deal with address rewriting and alias expansion, which is
the routing core of postfix ?


Badly?  It's a site-specific script hacked up quickly for my own
purposes.


"timestamp - client IP [hostname] - orig_envelope_from >
orig_envelope_to - Queue ID - final_envelope_from > final_envelope_to -
transport:nexthop - server IP [hostname] - delays"


Here's roughly what I'm doing at the moment:

On the first incoming connection, store the time, ESMTP ID, and
client IP / hostname.

On the first postfix/cleanup, store the message ID.

On the second postfix/cleanup, use the message ID to find the
record, dump the original ESMTP ID and store the new one.

From the amavis record, get from, to, hits, size.

From the outgoing postfix/smtp that matches the ESMTP ID,
get sent status and outgoing relay.

For what I'm doing at the moment I could probably just use
those last two records; but I think I'm also interested in
getting a record of emails that never get as far as amavis.


Can you share the script ?


That's the idea, although it's embarrassingly "beta" at the moment.

Re: Resend emails from a Maildir

[Victor Duchovni]

On Mon, Oct 18, 2010 at 11:37:18PM +0200, mouss wrote:

> - get the MAIL FROM address from the "Return-Path" header. with this you 
> can do: sendmail -f $returnpath yourdestinationemail

Make that:

sendmail -i -f "$returnpath" -- "$destpath" < msgfile

The returnpath can have all kinds of interesting characters. If using
Perl, it is highly advisable to entirely bypass shell argument parsing:

$prog = "/usr/sbin/sendmail";
@rcpts = ( q{...@example.com} );
@args = qw(sendmail -i);
push(@args, "-f", "$envsender");
push(@args, "--", @rcpts);
system { $prog } @args;
if (($code = $?) ne 0) {
# handle errors
}

-- 
Viktor.

Re: Upgrade 2.5.4

[fake...@fakessh.eu]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Le 19.10.2010 19:42, Victor Duchovni a écrit :
> On Tue, Oct 19, 2010 at 12:17:23PM -0400, Linux Addict wrote:
> 
>> Hello, I am running postfix 2.5.4 and would like to upgrade it to latest
>> stable 2.7.0. What is the best way upgrade? Do a clean install and port the
>> settings to newer version? Any help is appreciated.
> 
> No. Do an upgrade. If installing from source:
> 
>   Read the RELEASE_NOTES file for 2.6 and 2.7, then:
> 
>   % make
>   % su root
>   # postfix stop
>   # make upgrade
>   # postfix start
> 
> If installing from a well constructed package:
> 
>   Read the RELEASE_NOTES file for 2.6 and 2.7, then:
> 
>   # postfix stop
>   # some-command-to-install-updated-package
>   # postfix start
> 
> In either case, save the updated main.cf and master.cf files that
> are automatically upgraded as part of the install process.
> 
> If the package is not well constructed:
> 
>   Read the RELEASE_NOTES file for 2.6 and 2.7, then:
> 
>   # postfix stop
> 
>   # mkdir -p /etc/postfix/cfsavedir
>   # cp /etc/postfix/main.cf /etc/postfix/master.cf \
>   /etc/postfix/cfsavedir/
> 
>   # some-command-to-install-updated-poorly-constructed-package
> 
>   # cp /etc/postfix/cfsavedir/main.cf /etc/postfix/cfsavedir/master.cf \
>   /etc/postfix/
>   # postfix set-permissions upgrade-configuration
> 
>   # postfix start
> 
> A package is not well contstructed if it fails to preserve and upgrade
> your existing main.cf and master.cf files.
> 


thanks Victor mouss must also appreciate
being a face to similar problem
and clear this particular response



- -- 
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x092164A7
gpg --keyserver pgp.mit.edu --recv-key 092164A7
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iD8DBQFMvfN4tXI/OwkhZKcRAuvVAJsGl2vJzkSBQqvkt6XwTPUnisByDACfR+wg
eob4NzUYEPLV3Qhadkv7cMk=
=CbXp
-END PGP SIGNATURE-


0x092164A7.asc
Description: application/pgp-keys


0x092164A7.asc.sig
Description: Binary data

Re: automatic add attachments in postfix

[Jeroen Geilman]

On 10/19/2010 08:54 PM, David Touzeau wrote:

Le mardi 19 octobre 2010 à 19:33 +0200, Jeroen Geilman a écrit :

On 10/19/2010 01:26 PM, David Touzeau wrote:
>  Dear
>
>  I would like to find a tool that be able to add attachment in each
>  outgoing mails, i was thinking about altermime but altermime add only
>  txt,html file.
>  I need to add vcf or pdf files.
>  Is anyone know if a such tool exists ?
>
>

A Mail Transport Agent or MTA is not the place to manipulate the
contents of messages.
Postfix itself offers no support for this.

It's possible to send all mail through a process that adds an attachment
and then re-sends the mail to the original recipients - but be advised
that doing so will break, among other things,  S/MIME.

>  best regards.
>
>
>
>


 


Thanks for your advise...

"It's possible to send all mail through a process" : I need to find it...

That should be cool to perform this task on a gateway if you cannot 
handle back-end servers.





Take a look at the Filter readme:

http://www.postfix.org/FILTER_README.html


Or, alternatively, the Milter readme:

http://www.postfix.org/MILTER_README.html

Or even the SMTP proxy readme:

http://www.postfix.org/SMTPD_PROXY_README.html

All of these can process mail before postfix delivers it.

--
J.

Re: Logfile condenser

[Jeroen Geilman]

On 10/19/2010 09:20 PM, Dom Latter wrote:

[as subject]

On 19/10/10 19:42, Jeroen Geilman wrote:


How does it deal with address rewriting and alias expansion, which is
the routing core of postfix ?


Badly?  It's a site-specific script hacked up quickly for my own
purposes.


"timestamp - client IP [hostname] - orig_envelope_from >
orig_envelope_to - Queue ID - final_envelope_from > final_envelope_to -
transport:nexthop - server IP [hostname] - delays"


Here's roughly what I'm doing at the moment:

On the first incoming connection, store the time, ESMTP ID, and
client IP / hostname.

On the first postfix/cleanup, store the message ID.

On the second postfix/cleanup, use the message ID to find the
record, dump the original ESMTP ID and store the new one.

From the amavis record, get from, to, hits, size.

From the outgoing postfix/smtp that matches the ESMTP ID,
get sent status and outgoing relay.

For what I'm doing at the moment I could probably just use
those last two records; but I think I'm also interested in
getting a record of emails that never get as far as amavis.


Can you share the script ?


That's the idea, although it's embarrassingly "beta" at the moment.


I am definitely interested, I wasn't baiting you.
This is a decidedly untrivial task, given the flexibility of logging.
We are currently evaluating Splunk to process logs, but I haven't gotten 
very far with it yet.


--
J.

Re: Logfile condenser

[Dom Latter]

On 19/10/10 21:53, Jeroen Geilman wrote:


I am definitely interested, I wasn't baiting you.
This is a decidedly untrivial task, given the flexibility of logging.


That's okay, I'm taking a very trivial approach - no idea
how it would behave when faced with more complicated logs.

I'll send it to you off-list.

Re: Logfile condenser

[mouss]

 Le 19/10/2010 11:41, Dom Latter a écrit :

Helo, as one might say.

Wading through logfiles to find a particular email has been
taking up too much of my time recently [1].

I have written a logfile condenser that takes a postfix log and
generates a single line per email, with (for example) timestamp,
ham /spam, sender, recipient, spam score, size.

Thus, for example, a 25000 line logfile (about 4MB) is condensed
into 800 lines; grepping for " ham " pulls out 200 lines of
actual delivered emails.  It's reasonably fast (less than a second
to do the condensing on a modern-ish box).

I wrote my own because none of the existing logfile tools did
quite what I want.  The closest seemed to be this:
http://manpages.ubuntu.com/manpages/lucid/man1/postfix2dlf.1.html

The question is - is there any community interest in this tool?

As something hacked up for my own purposes it is most definitely
beta code.  But if there's interest I could tidy it up and
release it.

It will probably upset the purists for two reasons: it's written
in PHP [1] because that was going to be quickest for me; and it doesn't
use regexps, but rather a lot of substr()s to get the information
out.

If there's something better out there, please say.

[0] From the example stats - that's more than 100 lines of logfile
per "actual" email.
[1] It's still a command line tool.


what I'd like to see is "requirements/needs". that is: what do people need?

once we get that, the implementation is trivial.

Re: automatic add attachments in postfix

[mouss]

 Le 19/10/2010 13:26, David Touzeau a écrit :

Dear

I would like to find a tool that be able to add attachment in each
outgoing mails, i was thinking about altermime but altermime add only
txt,html file.
I need to add vcf or pdf files.
Is anyone know if a such tool exists ?


hmm, are you sure? I thought altermime could add anything.
If it really does not, then you need to alter (;-p) it and submit a 
patch to the author.

Re: virtual_alias_domains

[mouss]

 Le 18/10/2010 22:56, Jerrale G a écrit :

On 10/18/2010 4:43 PM, Jeroen Geilman wrote:

On 10/18/2010 10:36 PM, Jerrale G wrote:

 On 10/18/2010 4:29 PM, The Doctor wrote:

REcently I have noted that virtual_alias_domains is growing.

Is their some way for main.cf to look a file up instead of
having to read a whole line?



You are limited to using mysql, ONE file, ldap, postgresql, or mssql 
for each virtual_*_* parameter or any parameter that looks up 
something.


Certainly not.

All *_maps parameters can be specified as many times as necessary, 
for instance:


virtual_alias_maps = hash:/etc/postfix/virtual, 
mysql:/etc/postfix/my_virtual, ldap:/etc/postfix/ldap_virtual


They're not called mapS for nothing.

virtual_alias_domains is not a map setting per se, as it does nothing 
with the RHS of the lookup - but it works the same way.


From the postconf(5) man page:

Specify a list of host or domain names, "/file/name" or "type:table 
" patterns, separated by 
commas and/or whitespace.


i.e. "a list of" any of the map types postfix supports.
--
J.


yes, but that concats and looks in multiple places and it should NOT 
exist in more than one place. You MIGHT be able to do:


virtual_alias_maps = hash:/etc/postfix/virtual, 
mysql:/etc/postfix/my_virtual, ldap:/etc/postfix/ldap_virtual, 
*hash:/etc/postfix/hasfiles/%s

*


dunno what you mean, but for the archives:
- '%s' has no specific meaning here. so the above is interepreted as a 
file named '%s'. which is really a bad idea.
- it is ok to use multiple maps. but using both mysql and ldap is at 
least funny...



* *If there is no variable available that you want to use, then no and 
I'm not saying you can use variables in the config.


do you refer to '%s'? This is not what we call a variable. other than 
that, a lot of people use variables in postfix. I mean things like


sql = proxy:mysql/etc/postfix/mysql
re = pcre:/etc/postfix/pcre
db = cdb:/etc/postfix/cdb

then

foo_maps =
${db}/foo
${sql}/foo
${re}/foo

or in master.cf:

submission ...
-o myhostname=${submission_hostname}
-o smtpd_client_restrictions=${submission_client_restrictions}
...
-o foo_option=${submission_foo_option}



In dovecot you can use variables globally, with respect to some 
variables being exclusive to certain libexec's, and I'm only saying to 
try it.




you can use -o options in master.cf, and you can also run postifx N 
times, each with its own main.cf/master.cf (on the same OS, inside 
jails, in VMs or on different physical hosts). with that, you can 
implement any combination of parameters. if an important config is 
missing, please say it.

outer mail sender question

[ren yufei]

Dear all,

I am a newbie in Postfix. I have successfully installed postfix in ubuntu 
refers 
the tutorials:
https://help.ubuntu.com/community/Postfix

now I could send mail internally, but could not send mail to the outside 
(gmail.com, yahoo.com, etc).

the Mail Delivery Status Report is as follows:
-
 The mail system

: unknown user: "xxx"

[-- Attachment #2: Delivery report --]
[-- Type: message/delivery-status, Encoding: 7bit, Size: 0.3K --]

Reporting-MTA: dns; xxx.cewit.stonybrook.edu
X-Postfix-Queue-ID: BA73BA070A
X-Postfix-Sender: rfc822; r...@xxx.cewit.stonybrook.edu
Arrival-Date: Tue, 19 Oct 2010 14:25:22 -0700 (PDT)

Final-Recipient: rfc822; x...@gmail.com
Action: failed
Status: 5.1.1
Diagnostic-Code: X-Postfix; unknown user: "xxx"
-


It seems the error msg was sent from your local host instead of gmail.com

I already tries to "Indirect delivery via the local delivery agent" refers 
to:http://www.postfix.org/MAILDROP_README.html.

BTW, my ubuntu is a vm, and i use port mapping (25 and 587) to connect to the 
out side.

Thank you for your help.

Output of "postconf -n":

alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
broken_sasl_auth_clients = yes
config_directory = /etc/postfix
home_mailbox = Mail/
inet_interfaces = all
inet_protocols = all
mailbox_command = /usr/bin/maildrop -d ${USER}
mailbox_size_limit = 0
mydestination = hash:/etc/postfix/mydomains
myhostname = xxx.cewit.stonybrook.edu
mynetworks = 192.168.1.0/24 127.0.0.0/8 [:::127.0.0.0]/104 [::1]/128
myorigin = /etc/mailname
readme_directory = no
recipient_delimiter = +
relay_domains = lists.xxx.cewit.stonybrook.edu
relayhost =
smtp_sasl_auth_enable = no
smtp_tls_note_starttls_offer = yes
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
smtpd_recipient_restrictions = 
permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination
smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain =
smtpd_sasl_security_options = noanonymous
smtpd_tls_CAfile = /etc/ssl/certs/cacert.pem
smtpd_tls_auth_only = no
smtpd_tls_cert_file = /etc/ssl/certs/smtpd.crt
smtpd_tls_key_file = /etc/ssl/private/smtpd.key
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_session_cache_timeout = 3600s
smtpd_use_tls = yes
tls_random_source = dev:/dev/urandom
transport_maps = hash:/etc/postfix/transport


  

Re: automatic add attachments in postfix

[David Touzeau]

Le mardi 19 octobre 2010 à 21:50 +0200, Jeroen Geilman a écrit :

> On 10/19/2010 08:54 PM, David Touzeau wrote: 
> 
> > Le mardi 19 octobre 2010 à 19:33 +0200, Jeroen Geilman a écrit : 
> > 
> > > On 10/19/2010 01:26 PM, David Touzeau wrote:
> > > > Dear
> > > >
> > > > I would like to find a tool that be able to add attachment in each
> > > > outgoing mails, i was thinking about altermime but altermime add only
> > > > txt,html file.
> > > > I need to add vcf or pdf files.
> > > > Is anyone know if a such tool exists ?
> > > >
> > > >
> > > 
> > > A Mail Transport Agent or MTA is not the place to manipulate the 
> > > contents of messages.
> > > Postfix itself offers no support for this.
> > > 
> > > It's possible to send all mail through a process that adds an attachment 
> > > and then re-sends the mail to the original recipients - but be advised 
> > > that doing so will break, among other things,  S/MIME.
> > > 
> > > > best regards.
> > > >
> > > >
> > > >
> > > >
> > > 
> > > 
> > > 
> > 
> > 
> > Thanks for your advise... 
> > 
> > "It's possible to send all mail through a process" : I need to find
> > it...  
> > 
> > That should be cool to perform this task on a gateway if you cannot
> > handle back-end servers.
> > 
> > 
> 
> 
> Take a look at the Filter readme:
> 
> http://www.postfix.org/FILTER_README.html
> 
> 
> Or, alternatively, the Milter readme:
> 
> http://www.postfix.org/MILTER_README.html
> 
> Or even the SMTP proxy readme:
> 
> http://www.postfix.org/SMTPD_PROXY_README.html
> 
> All of these can process mail before postfix delivers it.
> 
> 
> -- 
> J.



Thanks 
but i try to find the tool that unpack mail and repack it with specified
attached file... 

Re: outer mail sender question

[Jeroen Geilman]

On 10/19/2010 11:44 PM, ren yufei wrote:

Dear all,

I am a newbie in Postfix. I have successfully installed postfix in 
ubuntu refers the tutorials:

https://help.ubuntu.com/community/Postfix


Tutorials often fail to explain how it actually works. Don't depend on them.


now I could send mail internally, but could not send mail to the 
outside (gmail.com, yahoo.com, etc).


the Mail Delivery Status Report is as follows:
-
 The mail system

: unknown user: "xxx"


"Unknown user" is clear enough.



[-- Attachment #2: Delivery report --]
[-- Type: message/delivery-status, Encoding: 7bit, Size: 0.3K --]

Reporting-MTA: dns; xxx.cewit.stonybrook.edu
X-Postfix-Queue-ID: BA73BA070A
X-Postfix-Sender: rfc822; r...@xxx.cewit.stonybrook.edu
Arrival-Date: Tue, 19 Oct 2010 14:25:22 -0700 (PDT)

Final-Recipient: rfc822; x...@gmail.com
Action: failed
Status: 5.1.1
Diagnostic-Code: X-Postfix; unknown user: "xxx"
-



The logs of that message would provide more insight into what happened.

It seems the error msg was sent from your local host instead of 
gmail.com 


Is that in the DSN or are these your words ?



I already tries to "Indirect delivery via the local delivery agent" 
refers to:http://www.postfix.org/MAILDROP_README.html.


How postfix delivers local mail has little or nothing to do with remote 
MTAs.


If you can provide logs for such a message, it is fairly simple to 
determine what happens.


--
J.

Re: automatic add attachments in postfix

[Jeroen Geilman]

On 10/20/2010 12:29 AM, David Touzeau wrote:

Le mardi 19 octobre 2010 à 21:50 +0200, Jeroen Geilman a écrit :


Take a look at the Filter readme:

http://www.postfix.org/FILTER_README.html


Or, alternatively, the Milter readme:

http://www.postfix.org/MILTER_README.html

Or even the SMTP proxy readme:

http://www.postfix.org/SMTPD_PROXY_README.html

All of these can process mail before postfix delivers it.

--
J.
 



Thanks
but i try to find the tool that unpack mail and repack it with 
specified attached file...





"Unpacking" and "repacking" being - what ?

All of the above suggestions deliver the complete SMTP message to an 
external program.


What that program does with the message is up to you - as long as you 
re-inject valid SMTP mail back into postfix, it will deliver it.



--
J.

Re: outer mail sender question

[ren yufei]

Thank you.

Although the fail of sending email outside, there is no mail.err or mail.warn 
generated. Just mail.info and mail.log.

Info @ mail.info

Oct 19 16:07:44 ubuntu postfix/pickup[22916]: 9422DA070A: uid=1000 from=
Oct 19 16:07:44 ubuntu postfix/cleanup[23070]: 9422DA070A: message-id=<201010192
30744.9422da0...@hpdtl.cewit.stonybrook.edu>
Oct 19 16:07:44 ubuntu postfix/qmgr[22260]: 9422DA070A: 
from=, size=338, nrcpt=1 (queue active)
Oct 19 16:07:45 ubuntu postfix/local[23073]: 9422DA070A: to=, relay=local, delay=0.46, delays=0.19/0.05/0/0.22, dsn=5.1.1, status=bounced
(unknown user: "renyufei83")
Oct 19 16:07:45 ubuntu postfix/cleanup[23070]: 00F39A070C: message-id=<201010192
30745.00f39a0...@hpdtl.cewit.stonybrook.edu>
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 00F39A070C: from=<>, size=2205, nrcp
t=1 (queue active)
Oct 19 16:07:45 ubuntu postfix/bounce[23074]: 9422DA070A: sender non-delivery no
tification: 00F39A070C
Oct 19 16:07:45 ubuntu postfix/cleanup[23070]: 0D356A064D: message-id=<201010192
30745.0d356a0...@hpdtl.cewit.stonybrook.edu>
Oct 19 16:07:45 ubuntu postfix/bounce[23075]: 9422DA070A: sender delivery status
 notification: 0D356A064D
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 9422DA070A: removed
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 0D356A064D: from=<>, size=1974, nrcp
t=1 (queue active)
Oct 19 16:07:45 ubuntu postfix/local[23073]: 00F39A070C: 
to=, relay=local, delay=0.7, delays=0.01/0.02/0/0.67, dsn=2.0.0, status
=sent (delivered to command: /usr/bin/maildrop -d ${USER})
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 00F39A070C: removed
Oct 19 16:07:45 ubuntu postfix/local[23073]: 0D356A064D: 
to=, relay=local, delay=0.69, delays=0.1/0.56/0/0.03, dsn=2.0.0, status
=sent (delivered to command: /usr/bin/maildrop -d ${USER})
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 0D356A064D: removed

Info @ mail.log

Oct 19 16:07:44 ubuntu postfix/pickup[22916]: 9422DA070A: uid=1000 from=
Oct 19 16:07:44 ubuntu postfix/cleanup[23070]: 9422DA070A: message-id=<201010192
30744.9422da0...@hpdtl.cewit.stonybrook.edu>
Oct 19 16:07:44 ubuntu postfix/qmgr[22260]: 9422DA070A: 
from=, size=338, nrcpt=1 (queue active)
Oct 19 16:07:45 ubuntu postfix/local[23073]: 9422DA070A: to=, relay=local, delay=0.46, delays=0.19/0.05/0/0.22, dsn=5.1.1, status=bounced
(unknown user: "renyufei83")
Oct 19 16:07:45 ubuntu postfix/cleanup[23070]: 00F39A070C: message-id=<201010192
30745.00f39a0...@hpdtl.cewit.stonybrook.edu>
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 00F39A070C: from=<>, size=2205, nrcp
t=1 (queue active)
Oct 19 16:07:45 ubuntu postfix/bounce[23074]: 9422DA070A: sender non-delivery no
tification: 00F39A070C
Oct 19 16:07:45 ubuntu postfix/cleanup[23070]: 0D356A064D: message-id=<201010192
30745.0d356a0...@hpdtl.cewit.stonybrook.edu>
Oct 19 16:07:45 ubuntu postfix/bounce[23075]: 9422DA070A: sender delivery status
 notification: 0D356A064D
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 9422DA070A: removed
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 0D356A064D: from=<>, size=1974, nrcp
t=1 (queue active)
Oct 19 16:07:45 ubuntu postfix/local[23073]: 00F39A070C: 
to=, relay=local, delay=0.7, delays=0.01/0.02/0/0.67, dsn=2.0.0, status
=sent (delivered to command: /usr/bin/maildrop -d ${USER})
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 00F39A070C: removed
Oct 19 16:07:45 ubuntu postfix/local[23073]: 0D356A064D: 
to=, relay=local, delay=0.69, delays=0.1/0.56/0/0.03, dsn=2.0.0, status
=sent (delivered to command: /usr/bin/maildrop -d ${USER})
Oct 19 16:07:45 ubuntu postfix/qmgr[22260]: 0D356A064D: removed





From: Jeroen Geilman 
To: postfix-users@postfix.org
Sent: Tue, October 19, 2010 6:43:25 PM
Subject: Re: outer mail sender question

On 10/19/2010 11:44 PM, ren yufei wrote: 
Dear all,
>
>I am a newbie in Postfix. I have successfully installed postfix in ubuntu 
>refers 
>the tutorials:
>https://help.ubuntu.com/community/Postfix
>
Tutorials often fail to explain how it actually works. Don't depend on them.


>now I could send mail internally, but could not send mail to the outside 
>(gmail.com, yahoo.com, etc).
>
>the Mail Delivery Status Report is as follows:
>-
> The mail system
>
>: unknown user: "xxx"
>
"Unknown user" is clear enough.



>[-- Attachment #2: Delivery report --]
>[-- Type: message/delivery-status, Encoding: 7bit, Size: 0.3K --]
>
>Reporting-MTA: dns; xxx.cewit.stonybrook.edu
>X-Postfix-Queue-ID: BA73BA070A
>X-Postfix-Sender: rfc822; r...@xxx.cewit.stonybrook.edu
>Arrival-Date: Tue, 19 Oct 2010 14:25:22 -0700 (PDT)
>
>Final-Recipient: rfc822; x...@gmail.com
>Action: failed
>Status: 5.1.1
>Diagnostic-Code: X-Postfix; unknown user: "xxx"
>-
>
>
>
The logs of that message would provide more insight into what happened.


It seems the error msg was sent from your local host instead of gmail.com
>
Is that in the DSN or are these your words ?

--- This is my words.


>I already tries to "Indirect delivery via the local del

Re: outer mail sender question

[Wietse Venema]

ren yufei:
> Oct 19 16:07:45 ubuntu postfix/local[23073]: 9422DA070A: 
> to= m>, relay=local, delay=0.46, delays=0.19/0.05/0/0.22, dsn=5.1.1, 
> status=bounced
> (unknown user: "renyufei83")

You are sending mail for GMAIL.COM to the LOCAL delivery agent.
This usually means that by mistake you added gmail.com to
your local destination list:

mydestination = hash:/etc/postfix/mydomains

Remove gmail.com (and other remote destinations) from this file.

Wietse

Re: outer mail sender question

[ren yufei]

Thank you. It works.

But I got the new error message :

connect to alt4.gmail-smtp-in.l.google.com[74.125.95.27]:25: Connection refused.


I also tried yahoo.com and get the same error.




From: Wietse Venema 
To: ren yufei 
Cc: Jeroen Geilman ; postfix-users@postfix.org
Sent: Tue, October 19, 2010 7:56:24 PM
Subject: Re: outer mail sender question

ren yufei:
> Oct 19 16:07:45 ubuntu postfix/local[23073]: 9422DA070A: 
>to= m>, relay=local, delay=0.46, delays=0.19/0.05/0/0.22, dsn=5.1.1, 
status=bounced
> (unknown user: "renyufei83")

You are sending mail for GMAIL.COM to the LOCAL delivery agent.
This usually means that by mistake you added gmail.com to
your local destination list:

mydestination = hash:/etc/postfix/mydomains

Remove gmail.com (and other remote destinations) from this file.

Wietse



  

Re: outer mail sender question

[Wietse Venema]

ren yufei:
> Thank you. It works.
> 
> But I got the new error message :
> 
> connect to alt4.gmail-smtp-in.l.google.com[74.125.95.27]:25: Connection 
> refused.
> 
> I also tried yahoo.com and get the same error.

Many ISPs block direct mail to port 25 for security reasons.  If
that is the case for you, then you will have to send your mail to
a relayhost.

See, for example:

http://www.postfix.org/SOHO_README.html#client_sasl_enable

And perhaps:

http://www.postfix.org/SOHO_README.html#client_sasl_sender

Wietse

Re: outer mail sender question

[Jeroen Geilman]

On 10/20/2010 02:38 AM, Wietse Venema wrote:

ren yufei:
   

Thank you. It works.

But I got the new error message :

connect to alt4.gmail-smtp-in.l.google.com[74.125.95.27]:25: Connection refused.

I also tried yahoo.com and get the same error.
 

Many ISPs block direct mail to port 25 for security reasons.  If
that is the case for you, then you will have to send your mail to
a relayhost.

See, for example:

http://www.postfix.org/SOHO_README.html#client_sasl_enable

And perhaps:

http://www.postfix.org/SOHO_README.html#client_sasl_sender

Wietse
   


Also don't forget that Google actively cooperates with the Great Firewall.


--
J.

RE: Fighting Backscatter

[Steve Jenkins]

I will gladly solve the RIGHT problem. The fact that I'm here looking for
guidance should demonstrate that I'm looking to do exactly that.
Unfortunately, I can't simply put "DO NOT forward SPAM" in my main.cf and
have it work. ;) After reading through all the docs and various blog and
forum posts, and making my best efforts at incorporating what I've learned
into my configuration, it seems I'm still causing backscatter. That's
exactly why I'm posting on Postfix-users - because I need a little more
guidance than just "RTFM." :) So if anyone can help me with some SPECIFIC
steps to take, I'd be very appreciative.

I posted it initially, but here again is my postconf -n output:

alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
home_mailbox = Maildir/
html_directory = no
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
milter_default_action = accept
milter_protocol = 2
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain,
mail.$mydomain, www.$mydomain
mynetworks = 127.0.0.0/8
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases.postfix
non_smtpd_milters = inet:localhost:20209
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtp_tls_note_starttls_offer = yes
smtp_use_tls = yes
smtpd_data_restrictions = reject_unauth_pipelining, permit
smtpd_milters = inet:localhost:20209
smtpd_recipient_restrictions = permit_sasl_authenticated,
reject_unauth_destination, reject_unknown_recipient_domain,
reject_unknown_sender_domain, reject_non_fqdn_recipient,
reject_non_fqdn_sender, reject_invalid_hostname, permit_mynetworks, permit
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_local_domain =
smtpd_sasl_security_options = noanonymous
smtpd_sender_restrictions = permit_sasl_authenticated,  permit_mynetworks,
reject_unknown_sender_domain
smtpd_tls_CAfile = /etc/postfix/ssl/cacert.pem
smtpd_tls_auth_only = no
smtpd_tls_cert_file = /etc/postfix/ssl/smtpd.crt
smtpd_tls_key_file = /etc/postfix/ssl/smtpd.key
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
smtpd_use_tls = yes
tls_random_source = dev:/dev/urandom
unknown_local_recipient_reject_code = 550
virtual_alias_domains = familyname.com
virtual_alias_maps = hash:/etc/postfix/virtual

I've been experimenting with various smtp recipient and sender restrictions,
but clearly haven't got the right mix yet. Any specific guidance there, or
anywhere else, is much appreciated.

Thanks,

SteveJ

-Original Message-
From: Wietse Venema [mailto:wie...@porcupine.org] 
Sent: Tuesday, October 19, 2010 5:16 AM
To: Steve Jenkins
Cc: Postfix users
Subject: Re: Fighting Backscatter

Steve Jenkins:
> Gotit. Thanks again for helping me out. I'm still learning.
> 
> So it seems I need to figure out how to stop the backscatter process at
step
> 6 and NOT return the bounce to the original sender.

No. Solve the RIGHT problem. DO NOT forward SPAM.

Wietse

Re: Fighting Backscatter

[Jeroen Geilman]

On 10/20/2010 02:52 AM, Steve Jenkins wrote:

I will gladly solve the RIGHT problem. The fact that I'm here looking for
guidance should demonstrate that I'm looking to do exactly that.
Unfortunately, I can't simply put "DO NOT forward SPAM" in my main.cf and
have it work. ;) After reading through all the docs and various blog and
forum posts, and making my best efforts at incorporating what I've learned
into my configuration, it seems I'm still causing backscatter.


Don't accept mail you cannot deliver. Really, that's Numero Uno.
Proper sender and recipient verification - insofar as is feasible for 
your site - goes a long way to prevent that from happening.



  That's exactly why I'm posting on Postfix-users - because I need a little more
guidance than just "RTFM." :) So if anyone can help me with some SPECIFIC
steps to take, I'd be very appreciative.

I posted it initially, but here again is my postconf -n output:

   



smtpd_recipient_restrictions = permit_sasl_authenticated,
reject_unauth_destination, reject_unknown_recipient_domain,
reject_unknown_sender_domain, reject_non_fqdn_recipient,
reject_non_fqdn_sender, reject_invalid_hostname, permit_mynetworks, permit
   


You're missing some of the better spam prevention methods here, such as 
decent HELO checks, and an RBL or two.


I'd suggest at least adding reject_unknown_reverse_client_hostname in 
there, as well as (testing out) 
reject_[invalid|unknown|non_fqdn]_helo_hostname.


My personal server uses:

smtpd_recipient_restrictions 
=  permit_mynetworks,

permit_sasl_authenticated,

reject_unauth_destination,

reject_unknown_reverse_client_hostname,
warn_if_reject 
reject_non_fqdn_helo_hostname,
warn_if_reject 
reject_invalid_helo_hostname,
warn_if_reject 
reject_unknown_helo_hostname,

reject_unauth_pipelining,

reject_non_fqdn_sender,

reject_unknown_sender_domain,

reject_non_fqdn_recipient,

reject_unknown_recipient_domain,

check_helo_access hash:/etc/postfix/helo_access,

permit

helo_access contains permutations of my own IP and hostname(s), which I 
REJECT.


My zen RBL check is moved to postscreen, since I run a pre-2.8 build.


smtpd_sender_restrictions = permit_sasl_authenticated,  permit_mynetworks,
reject_unknown_sender_domain
   


Instead of specifying each restriction set by itself, put them all 
together under recipient_restrictions so you can follow along what happens.

It will also log more information.


virtual_alias_domains = familyname.com
virtual_alias_maps = hash:/etc/postfix/virtual
   


It would be mildly interesting to see what is in those files, since a 
virtual_alias_domain is potentially a wildcard recipient domain.



-Original Message-
From: Wietse Venema [mailto:wie...@porcupine.org]
Sent: Tuesday, October 19, 2010 5:16 AM
To: Steve Jenkins
Cc: Postfix users
Subject: Re: Fighting Backscatter

Steve Jenkins:
   


Oh, and please don't top-post.


--
J.

Re: outer mail sender question

[ren yufei]

Thank you and it works.^^





From: Wietse Venema 
To: Postfix users 
Sent: Tue, October 19, 2010 8:38:50 PM
Subject: Re: outer mail sender question

ren yufei:
> Thank you. It works.
> 
> But I got the new error message :
> 
> connect to alt4.gmail-smtp-in.l.google.com[74.125.95.27]:25: Connection 
>refused.
> 
> I also tried yahoo.com and get the same error.

Many ISPs block direct mail to port 25 for security reasons.  If
that is the case for you, then you will have to send your mail to
a relayhost.

See, for example:

http://www.postfix.org/SOHO_README.html#client_sasl_enable

And perhaps:

http://www.postfix.org/SOHO_README.html#client_sasl_sender

Wietse



  

RE: Fighting Backscatter

[Steve Jenkins]

Thanks for the reply. 

Forcing everyone to just check their mail with POP/IMAP was actually what I
was going to resort to if I couldn't figure out a way to do this with just
Postfix config settings. It seems that unless I want to manage a SPAM filter
on my server for everyone (which is not something I want to do) then that's
what I'm going to have to do.

So unless anyone sees anything glaring that I'm doing wrong from my postconf
-n, that's probably what I'm going to end up doing.

Thanks,

SteveJ

-Original Message-
From: owner-postfix-us...@postfix.org
[mailto:owner-postfix-us...@postfix.org] On Behalf Of pf at alt-ctrl-del.org
Sent: Tuesday, October 19, 2010 8:04 AM
To: postfix-users@postfix.org
Subject: Re: Fighting Backscatter


> On 2010-10-18 9:58 PM, Steve Jenkins wrote:
>> The instructions at http://www.postfix.org/BACKSCATTER_README.html 
>> seem to only address what to do if MY server is the one being
>> forged. In the above example, it seems that procom.ca is being
>> forged. How should I configure my Postfix installation so that I'm
>> not sending the spam back to the innocent sender? Let me know if you
>> need me to post my postconf -n again.
> 
"Charles Marcus" October 19, 2010 7:38 AM:
> As has been told to you more than once, the correct solution is simple...
> 
> 1. Stop forwarding spam, or
> 
> 2. Do not forward *any* emails, period.
> 

The OP can actually do #2. He probably doesn't need to forward any email at
all.
gmail, yahoo, comcast all support POP3'ing third party email accounts,
straight to the mailbox.

Automatic POP3 retrieval...
No forwarding, no bounces.

But I'm not sure about cox support for auto POP3 retrieval.

RE: Fighting Backscatter

[Terry Gilsenan]

From: owner-postfix-us...@postfix.org [owner-postfix-us...@postfix.org] On 
Behalf Of Steve Jenkins [st...@stevejenkins.com]
Sent: Wednesday, 20 October 2010 10:52 AM
To: Postfix users
Subject: RE: Fighting Backscatter

>I will gladly solve the RIGHT problem. The fact that I'm here looking for
>guidance should demonstrate that I'm looking to do exactly that.
>Unfortunately, I can't simply put "DO NOT forward SPAM" in my main.cf and
>have it work. ;) After reading through all the docs and various blog and
>forum posts, and making my best efforts at incorporating what I've learned
>into my configuration, it seems I'm still causing backscatter. That's
>exactly why I'm posting on Postfix-users - because I need a little more
>guidance than just "RTFM." :) So if anyone can help me with some SPECIFIC
>steps to take, I'd be very appreciative.

Steve, Backscatter is caused by a configuration that accepts all email and then 
bounces email it cannot deliver. This is where your configuration is faulty.

Only accept email that you can deliver! If you cannot deliver email for any 
reason you should be determining this within the SMTP transaction phase and 
responding to the sending MTA with the appropriate rejection code.

Any email that you do actually accept and for which your server tells the 
sending MTA "OK", you either need to deliver or if your filters are setup 
appropriately, quietly sink. (purists will say this should never happen, but 
pragmatists reallize that some content inspection testing cannot be done until 
the email has been fully rec'd)

If you have this sorted out then your backscatter problems will go away.

Rule of thumb: Start with a config that accepts nothing, then add exceptions 
for things that you want to accept email for, and nothing else.

RE: Fighting Backscatter

[Steve Jenkins]

THANK YOU Jeroen. J I really appreciate you taking the time to help me with
some specific steps I can try.

 

Here's the updated output of my postconf -n:

 

alias_database = hash:/etc/aliases

alias_maps = hash:/etc/aliases

broken_sasl_auth_clients = yes

command_directory = /usr/sbin

config_directory = /etc/postfix

daemon_directory = /usr/libexec/postfix

data_directory = /var/lib/postfix

debug_peer_level = 2

home_mailbox = Maildir/

html_directory = no

mailq_path = /usr/bin/mailq.postfix

manpage_directory = /usr/share/man

milter_default_action = accept

milter_protocol = 2

mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain,
mail.$mydomain, www.$mydomain

mynetworks = 127.0.0.0/8

myorigin = $mydomain

newaliases_path = /usr/bin/newaliases.postfix

non_smtpd_milters = inet:localhost:20209

sendmail_path = /usr/sbin/sendmail.postfix

setgid_group = postdrop

smtp_tls_note_starttls_offer = yes

smtp_use_tls = yes

smtpd_data_restrictions = reject_unauth_pipelining, permit

smtpd_milters = inet:localhost:20209

smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated,
reject_unauth_destination,  reject_unknown_reverse_client_hostname,
warn_if_reject reject_non_fqdn_helo_hostname,   warn_if_reject
reject_invalid_helo_hostname,  warn_if_reject reject_unknown_helo_hostname,
reject_unauth_pipelining,   reject_non_fqdn_sender,
reject_unknown_sender_domain, reject_non_fqdn_recipient,
reject_unknown_recipient_domain,reject_invalid_hostname,
permit

smtpd_sasl_auth_enable = yes

smtpd_sasl_authenticated_header = yes

smtpd_sasl_local_domain =

smtpd_sasl_security_options = noanonymous

smtpd_tls_CAfile = /etc/postfix/ssl/cacert.pem

smtpd_tls_auth_only = no

smtpd_tls_cert_file = /etc/postfix/ssl/smtpd.crt

smtpd_tls_key_file = /etc/postfix/ssl/smtpd.key

smtpd_tls_loglevel = 1

smtpd_tls_received_header = yes

smtpd_tls_session_cache_timeout = 3600s

smtpd_use_tls = yes

tls_random_source = dev:/dev/urandom

unknown_local_recipient_reject_code = 550

virtual_alias_domains = familyname.com

virtual_alias_maps = hash:/etc/postfix/virtual

 

The /etc/postfix/virtual is set up as follows. Every line in there is either
a local POP account or the destination forwarding address. I don't use any
catch-alls, and prefer that my server reject unknown local recipients (or in
this case, I should probably say "local").

 

Familyname.com #Family Domain for Mail

st...@familyname.comsteve

sis...@familyname.comsister

a...@familyname.com auntsaddr...@cox.net

d...@familyname.com   dadsaddr...@gmail.com

 

Like you, I'm also running a pre-2.8 build (2.6.5). I hadn't heard of
postscreen until just now, but I'll check it out.

 

Would you mind sharing (anonymized if you wish) some examples of
permutations of your IP and hostname(s) to reject from your helo_access
file? What types of permutations are classically used by spammers that I can
safely block without rejecting legitimate mail?

 

Thanks again,

 

Steve

 

 

From: owner-postfix-us...@postfix.org
[mailto:owner-postfix-us...@postfix.org] On Behalf Of Jeroen Geilman
Sent: Tuesday, October 19, 2010 7:10 PM
To: postfix-users@postfix.org
Subject: Re: Fighting Backscatter

 

On 10/20/2010 02:52 AM, Steve Jenkins wrote: 

I will gladly solve the RIGHT problem. The fact that I'm here looking for
guidance should demonstrate that I'm looking to do exactly that.
Unfortunately, I can't simply put "DO NOT forward SPAM" in my main.cf and
have it work. ;) After reading through all the docs and various blog and
forum posts, and making my best efforts at incorporating what I've learned
into my configuration, it seems I'm still causing backscatter.


Don't accept mail you cannot deliver. Really, that's Numero Uno.
Proper sender and recipient verification - insofar as is feasible for your
site - goes a long way to prevent that from happening.




 That's exactly why I'm posting on Postfix-users - because I need a little
more
guidance than just "RTFM." :) So if anyone can help me with some SPECIFIC
steps to take, I'd be very appreciative.
 
I posted it initially, but here again is my postconf -n output:
 
  





smtpd_recipient_restrictions = permit_sasl_authenticated,
reject_unauth_destination, reject_unknown_recipient_domain,
reject_unknown_sender_domain, reject_non_fqdn_recipient,
reject_non_fqdn_sender, reject_invalid_hostname, permit_mynetworks, permit
  


You're missing some of the better spam prevention methods here, such as
decent HELO checks, and an RBL or two.

I'd suggest at least adding reject_unknown_reverse_client_hostname in there,
as well as (testing out) reject_[invalid|unknown|non_fqdn]_helo_hostname.

My personal server uses:

smtpd_recipient_restrictions =  permit_mynetworks,
 
permit_sasl_authenticated,
 
reject_unauth_destination,
 
reject_unknown_reverse_client_hostname,

Re: Fighting Backscatter

[Jeroen Geilman]

On 10/20/2010 03:38 AM, Steve Jenkins wrote:


THANK YOU Jeroen. J I really appreciate you taking the time to help me 
with some specific steps I can try.




Well, let's say I can provide you with some pointers.
That doesn't absolve you of the responsibility to study the 
documentation thoroughly.



non_smtpd_milters = inet:localhost:20209

smtpd_milters = inet:localhost:20209


What are all these milters doing ?
Do you *know* ?
How can you use the same service for both smtp and non-smtp milters ?
Presumably, they don't take the same input format.

smtpd_recipient_restrictions = permit_mynetworks, 
permit_sasl_authenticated,  reject_unauth_destination,  
reject_unknown_reverse_client_hostname,   warn_if_reject 
reject_non_fqdn_helo_hostname,   warn_if_reject 
reject_invalid_helo_hostname,  warn_if_reject 
reject_unknown_helo_hostname,reject_unauth_pipelining,   
reject_non_fqdn_sender, reject_unknown_sender_domain, 
reject_non_fqdn_recipient,  
reject_unknown_recipient_domain,
reject_invalid_hostname,permit




Still missing a good RBL check; check out zen (www.spamhaus.org/zen)


virtual_alias_domains = familyname.com

virtual_alias_maps = hash:/etc/postfix/virtual

The /etc/postfix/virtual is set up as follows. Every line in there is 
either a local POP account or the destination forwarding address. I 
don't use any catch-alls, and prefer that my server reject unknown 
local recipients (or in this case, I should probably say "local").




No, since these are virtual aliases, postfix will reject any *virtual* 
recipients that don't appear here.

It makes no judgement on the RHS of the aliases.


Familyname.com #Family Domain for Mail

st...@familyname.com steve

sis...@familyname.com sister

a...@familyname.com  auntsaddr...@cox.net 



d...@familyname.com  dadsaddr...@gmail.com 



Like you, I'm also running a pre-2.8 build (2.6.5).



Um. pre-2.8 means I run a pre-release build of postfix 2.8 with the 
postscreen code patched in to it.


Postscreen doesn't work on earlier versions, and is still not finalized 
AFAIK.



I hadn't heard of postscreen until just now, but I'll check it out.



That would be why. Don't worry about it, you can do fine without.

Would you mind sharing (anonymized if you wish) some examples of 
permutations of your IP and hostname(s) to reject from your 
helo_access file? What types of permutations are classically used by 
spammers that I can safely block without rejecting legitimate mail?




Just list your literal IP and hostname(s) to start with.
Many spammers try to circumvent remote client restrictions that way.


*From:* owner-postfix-us...@postfix.org 
[mailto:owner-postfix-us...@postfix.org] *On Behalf Of *Jeroen Geilman

*Sent:* Tuesday, October 19, 2010 7:10 PM
*To:* postfix-users@postfix.org
*Subject:* Re: Fighting Backscatter


Oh, and please don't top-post.
J.



And you're still top-posting.



--
J.

RE: Fighting Backscatter

[Steve Jenkins]

Hi, Terry. Again, very helpful advice presented in a way I understand. :)
Thank you.

Based on Jeroen's advice, I've modified my main.cf file to restrict much
more of the undeliverable mail on the way IN. Just from watching my logfile
over the past few minutes, I'm seeing a LOT more rejections for "Domain not
found" and "cannot find your reverse hostname" as well as warnings for
"address not listed for" and "Helo command rejected: need fully-qualified
hostname." That's awesome! I'm assuming that after watching these warnings
for a while and being satisfied that these warnings are appearing only for
SPAM that I can turn off the warning and simply reject. What should I use as
a good indicator for when it's time to do that?

Like you, I also tend to be more practical than pragmatic, so even if it
causes a few sighs and finger wags, I'm open to quietly sinking mail that I
can't deliver. Any pointers on exactly how to do that?

Thanks again,

Steve

-Original Message-
From: Terry Gilsenan [mailto:terry.gilse...@interoil.com] 
Sent: Tuesday, October 19, 2010 7:27 PM
To: Steve Jenkins; Postfix users
Subject: RE: Fighting Backscatter

From: owner-postfix-us...@postfix.org [owner-postfix-us...@postfix.org] On
Behalf Of Steve Jenkins [st...@stevejenkins.com]
Sent: Wednesday, 20 October 2010 10:52 AM
To: Postfix users
Subject: RE: Fighting Backscatter

>I will gladly solve the RIGHT problem. The fact that I'm here looking for
>guidance should demonstrate that I'm looking to do exactly that.
>Unfortunately, I can't simply put "DO NOT forward SPAM" in my main.cf and
>have it work. ;) After reading through all the docs and various blog and
>forum posts, and making my best efforts at incorporating what I've learned
>into my configuration, it seems I'm still causing backscatter. That's
>exactly why I'm posting on Postfix-users - because I need a little more
>guidance than just "RTFM." :) So if anyone can help me with some SPECIFIC
>steps to take, I'd be very appreciative.

Steve, Backscatter is caused by a configuration that accepts all email and
then bounces email it cannot deliver. This is where your configuration is
faulty.

Only accept email that you can deliver! If you cannot deliver email for any
reason you should be determining this within the SMTP transaction phase and
responding to the sending MTA with the appropriate rejection code.

Any email that you do actually accept and for which your server tells the
sending MTA "OK", you either need to deliver or if your filters are setup
appropriately, quietly sink. (purists will say this should never happen, but
pragmatists reallize that some content inspection testing cannot be done
until the email has been fully rec'd)

If you have this sorted out then your backscatter problems will go away.

Rule of thumb: Start with a config that accepts nothing, then add exceptions
for things that you want to accept email for, and nothing else.

RE: Fighting Backscatter

[Terry Gilsenan]

From: owner-postfix-us...@postfix.org [owner-postfix-us...@postfix.org] On 
Behalf Of Steve Jenkins [st...@stevejenkins.com]
Sent: Wednesday, 20 October 2010 11:50 AM
To: Terry Gilsenan; Postfix users
Subject: RE: Fighting Backscatter

>Hi, Terry. Again, very helpful advice presented in a way I understand. :)
>Thank you.

>Based on Jeroen's advice, I've modified my main.cf file to restrict much
>more of the undeliverable mail on the way IN. Just from watching my logfile
>over the past few minutes, I'm seeing a LOT more rejections for "Domain not
>found" and "cannot find your reverse hostname" as well as warnings for
>"address not listed for" and "Helo command rejected: need fully-qualified
>hostname." That's awesome! I'm assuming that after watching these warnings
>for a while and being satisfied that these warnings are appearing only for
>SPAM that I can turn off the warning and simply reject. What should I use as
>a good indicator for when it's time to do that?

I have no idea what would work for you, I log everything and I have a legal 
requirement to do so.

>Like you, I also tend to be more practical than pragmatic, so even if it
>causes a few sighs and finger wags, I'm open to quietly sinking mail that I
>can't deliver. Any pointers on exactly how to do that?

Amavisd-new and spamassassin are great tools when configured correctly and 
DISCARD used as a final rule.

header_checks and body_checks allow the use of the DISCARD action.

Accept and then discard (silently delete) is perfectly valid if that is your 
decision as to the final disposition of rec'd email that fits the rules you 
have set. Ultimately you want to get the server config setup so that you dont 
even start the data phase of the SMTP transaction for most spam. Content filter 
should then be a last resort.

Regards,
T

RE: Fighting Backscatter

[Steve Jenkins]

Well, let's say I can provide you with some pointers.
That doesn't absolve you of the responsibility to study the documentation
thoroughly.



Thank you nonetheless. I was starting to get the impression that doing
anything other than telling people to read the documentation was verboten.
;) I'm not looking to just blindly type in config settings. I really want to
understand what it is I should be doing and then do it properly. 

 

What are all these milters doing ?
Do you *know* ?
How can you use the same service for both smtp and non-smtp milters ? 
Presumably, they don't take the same input format.



Those lines are in my main.cf for OpenDKIM (opendkim.org). I don't reject
incoming mail (yet) if it fails DKIM authentication, but I do sign all my
personal outgoing mail sent from this server. I'm not sure how to answer
"How can you use the same service for both smtp and non-smtp milters ?" but
I'll look into confirming whether that's set up properly.

 

Still missing a good RBL check; check out zen (www.spamhaus.org/zen)



I've added "reject_rbl_client zen.spamhaus.org" to my
smtpd_recipient_restrictions as the second-to-last value, right before
"permit."


No, since these are virtual aliases, postfix will reject any *virtual*
recipients that don't appear here. It makes no judgement on the RHS of the
aliases.



Yes. I want Postfix to reject any virtual recipients that don't appear here.
I was trying to be witty by saying they aren't "local" recipients (with
local in quotes) since I'm forwarding their mail somewhere else. But yes, I
understand that Postfix will reject if their address doesn't appear in this
file.

 

Thx,

 

SJ 

OT Gmail

[Julio Cesar Covolato]

 Hi!

I wonder know what system Gmail  use.  They use postfix as mta? I dont 
think so...


How to scale for milions acounts, how do they do it?

Thanks.

--
-
_Julio Cesar Covolato
   0v0
  /(_)\  F: 55-11-3129-3366
   ^ ^   PSI INTERNET
-

Re: OT Gmail

[Jay Bendon]

Everything is built inhouse and guarded very closely. U will not be able to
duplicate their product without significant financial investment or just use
their product.

On Oct 19, 2010 10:27 PM, "Julio Cesar Covolato"  wrote:

 Hi!

I wonder know what system Gmail  use.  They use postfix as mta? I dont think
so...

How to scale for milions acounts, how do they do it?

Thanks.

-- 
-
   _Julio Cesar Covolato
  0v0
 /(_)\  F: 55-11-3129-3366
  ^ ^   PSI INTERNET
-

Re: Fighting Backscatter

[Stan Hoeppner]

Jeroen Geilman put forth on 10/19/2010 8:09 PM:

> You're missing some of the better spam prevention methods here, such as
> decent HELO checks, and an RBL or two.
> 
> I'd suggest at least adding reject_unknown_reverse_client_hostname in
> there, as well as (testing out)
> reject_[invalid|unknown|non_fqdn]_helo_hostname.

This will probably be a big help to Steve.

smtpd_recipient_restrictions =
permit_mynetworks
permit_sasl_authenticated,
reject_unauth_destination
...
check_client_access pcre:/etc/postfix/fqrdns.pcre
...
reject_rbl_client zen.spamhaus.org
reject_rbl_client psbl.surriel.com
reject_rhsbl_client dbl.spamhaus.org
reject_rhsbl_sender dbl.spamhaus.org
reject_rhsbl_helo dbl.spamhaus.org
check_policy_service inet:127.0.0.1:6

http://www.hardwarefreak.com/fqrdns.pcre

This pcre rdns checker kills tons of bot spam from consumer IPs that
should not be sending direct smtp mail.  It picks up where the PBL
leaves off.  Zero FP rate.  As always, exclude it from your own
submission smtpd or it ighty well reject your own users.

The check_policy_service line is the postgrey daemon.  Doesn't stop
alot, maybe 5-10 per day of 1000, but it's a needed safety net.  Your
anti spam toolbox needs many tools to make all the tools effective as a
whole.  Merely adding fqrdns.pcre has put a big smile on the faces of
quite a few OPs.

As always, do a "postfix reload" after making changes to main.cf.

-- 
Stan

RE: Fighting Backscatter

[Steve Jenkins]

Jeroen said:

My personal server uses:

smtpd_recipient_restrictions =  permit_mynetworks,
 
permit_sasl_authenticated,
 
reject_unauth_destination,
 
reject_unknown_reverse_client_hostname,
warn_if_reject
reject_non_fqdn_helo_hostname,
warn_if_reject
reject_invalid_helo_hostname,
warn_if_reject
reject_unknown_helo_hostname,
 
reject_unauth_pipelining,
 
reject_non_fqdn_sender,
 
reject_unknown_sender_domain,
 
reject_non_fqdn_recipient,
 
reject_unknown_recipient_domain,
check_helo_access
hash:/etc/postfix/helo_access,
permit

Out of curiosity, does anyone see any drawbacks (such as possibly rejecting
valid mail) to adding "reject_invalid_hostname" to those
smtpd_recipient_restrictions? Also, some other reading I've been doing
suggest adding "smtpd_helo_required = yes" to the main.cf file. Is that
helpful/necessary, or would I already be covered there with the
"reject_invalid_helo_hostname" in the above recipient restrictions?

 

I've also read another suggestion to add "smtpd_sender_restrictions =
reject_unknown_address" to reject mail that doesn't have any return address.
I've moved all my checks to the recipient restrictions, so if I added this,
it would be my only sender restriction. Am I wrong in thinking this check is
superfluous because of the "reject_non_fqdn_sender" already in the above
recipient restrictions?

 

It's slowly starting to make more sense. Thanks again to those who have
given me helpful nudges.

 

Thanks,

 

Steve

RE: Fighting Backscatter

[Steve Jenkins]

Stan Hoeppner said:
>This will probably be a big help to Steve.

Thanks, Stan. That fqrdns.pcre file rocks. Is that something you created?
May I share the link with others?

I had already added the spamhaus DBL checks (after Jeroen nudged me toward
their Zen IP blocklist), but Surriel PSBL is new to me and I'll check that
out now. I also just Googled postgrey and will check that out as well.

Thanks again - your post WAS a big help. I appreciate it.

SJ