yet another stupid question on header rewriting?
Hi List, I need to rewrite incoming headers for one email address, how to do this? The main reason to do this is jira that accepts email only from one defined emails and I need to import mailinglist mails to that system. for example I need to rewrite centos mailing-list from header to from my@email address so that I can import mail to jira system .. yes, really stupid but way to do this but.. -- Eero
Correctness of the queue management
Dear Postfix Community,
after reading the documentation "Postfix Architecture Overview", I thought that
"qmgr" would handle operations between the different queues correctly and
despite any connection or performance issues.
Our company uses Mailman via Postfix on a regular basis for mailings with up to
50.000 recipients. We did not touch the standard configuration yet, except the
"relayhost" entry.
During our last mailing, an analysis of the log file showed, that the number of
messages sent thru postfix was not about 50.000 as expected, but near the
factor 10 of it. More than a hundred times, the following warnings appeared in
the log:
Jun 5 15:08:11 172 postfix/qmgr[31501]: warning: mail for 172.29.101.13 is
using up 2 of 2 active queue entries
Jun 5 15:08:11 172 postfix/qmgr[31501]: warning: you may need to increase the
main.cf smtp_destination_concurrency_limit from 20
Jun 5 15:08:11 172 postfix/qmgr[31501]: warning: please avoid flushing the
whole queue when you have
Jun 5 15:08:11 172 postfix/qmgr[31501]: warning: lots of deferred mail, that
is bad for performance
Jun 5 15:08:11 172 postfix/qmgr[31501]: warning: to turn off these warnings
specify: qmgr_clog_warn_time = 0
I guess that the relay host accepted the messages only with huge delays, so the
active queue filled up quickly. There were no log entries concerning the
deferred queue. In the log file (> 300 MiB), the delay values are increasing
during the mailing. It seems that the relay host was either overloaded or
misconfigured. Nevertheless, it was the first occurrence of any postfix problem
during our mailings and I still have no explanation, why a great number of
recipients got the same message multiple times.
Do you know about any behavior of the qmgr, that could explain the duplicate
transmission of messages?
main.cf:
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no
append_dot_mydomain = no
readme_directory = no
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
myhostname = xxx
smtpd_banner = $myhostname ESMTP
alias_maps= hash:/var/lib/mailman/data/aliases
alias_database = $alias_maps
mydomain = xxx
myorigin = /etc/mailname
mydestination = xxx
relayhost = xxx
mynetworks = 127.0.0.0/8 [:::127.0.0.0]/104 [::1]/128 xxx
mailbox_command =
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
message_size_limit = 104857600
Kindest regards,
Tobias Schwartz
German Bar Association
IT Department
--
Deutscher Anwaltverein (DAV) e. V.
Littenstraße 11, 10179 Berlin
Phone +49 30 726152-160, Fax -194
schwa...@anwaltverein.de,
www.anwaltverein.de
Re: yet another stupid question on header rewriting?
On 10 Jun 2014, at 13:02, Eero Volotinen wrote: > I need to rewrite incoming headers for one email address, how to do this? > > The main reason to do this is jira that accepts email only from one defined > emails > and I need to import mailinglist mails to that system. > > for example I need to rewrite centos mailing-list from header to from > my@email address so that I can import mail to jira system .. > > yes, really stupid but way to do this but.. If you are importing archived mail, just alter it outside Postfix, drop it in the pickup queue, and let it run its course? This would work fine for newly incoming messages, too. Pipe it to a script that does this for you, a script that will pipe it into JIRA for example, in the format it expects. Generally easier, and more reliable, than solving it at the header rewriting stage. Mvg, Joni
Re: Correctness of the queue management
Schwartz, Tobias (DAV): > Do you know about any behavior of the qmgr, that could explain the > duplicate transmission of messages? You report a problem, and therefore it is your responsibility to demonstrate that your claim is correct. To prove that your claim is correct, you need to show logfile records for one recipient that is delivered multiple times. The logfile records must be complete including date,, host, process name and so on. You should replace the recipient name with XXX, so that their email address will not end up on spammer lists. Wietse
Re: CIDR Whitelist ?
Ronald F. Guilmette: > > I really should have figured this out ages ago, but... > > Quite simply, there exits a small number of organizations that > run afoul of my various smtpd_recipient_restrictions and/or my > smtpd_helo_restrictions, but from which I need to be able to > receive mail anyway. (A small number of companies get snagged > on reject_unknown_helo_hostname due to having botched the HELO > strings on their outbound mail servers, and also, in the case of > Microsoft, they seem to have managed to get numerous of their > IPs listed on Spamcop.) > > So anyway, I just now added the following to my pre-existing > list of smtpd_recipient_restrictions: > > check_client_access cidr:/usr/local/etc/postfix/blacklists/cidr-whitelist > > where my cidr-whitelist file currently contains just: > > # Microsoft > 65.52.0.0/14 OK > > Of course, I placed this new check_client_access clause above all of > the other/pre-existing clauses in my smtpd_recipient_restrictions. Placing this before reject_unauth_destination will give them relay permissions (assuming that your system pre-dates smtpd_relay_restriction support). See: http://www.postfix.org/SMTPD_ACCESS_README.html#danger Wietse
Re: AW: Re: Correctness of the queue management
Schwartz, Tobias (DAV): > Dear Wietse, > > thank you for your reply! > You can find the log entries for a particular recipient below: > > Jun 5 15:25:18 172 postfix/smtpd[28500]: 9EE2B185BF0: client=unknown[::1] > Jun 5 15:25:18 172 postfix/cleanup[29770]: 9EE2B185BF0: > message-id=<1233AD723A027C47B6CDC1668C2AF06D60594C2F@xxx> > Jun 5 15:25:28 172 postfix/qmgr[31501]: 9EE2B185BF0: from=, size=35964, > nrcpt=1 (queue active) > Jun 5 15:51:12 172 postfix/smtp[31769]: 9EE2B185BF0: to=, > relay=xxx[xxx]:25, conn_use=148, delay=1554, delays=9.6/1543/0.01/1.4, > dsn=2.6.0, status=sent (250 2.6.0 message received) > Jun 5 15:51:12 172 postfix/qmgr[31501]: 9EE2B185BF0: removed The above receives and delivers ONE MESSAGE with queue ID 9EE2B185BF0. Postfix delivers the message EXACTLY ONCE. > Jun 5 16:25:26 172 postfix/smtpd[1149]: 7ED581862A6: client=unknown[::1] > Jun 5 16:25:26 172 postfix/cleanup[2274]: 7ED581862A6: > message-id=<1233AD723A027C47B6CDC1668C2AF06D60594C2F@xxx> > Jun 5 16:29:09 172 postfix/qmgr[31501]: 7ED581862A6: from=, size=37578, > nrcpt=1 (queue active) > Jun 5 16:55:09 172 postfix/smtp[4611]: 7ED581862A6: to=, > relay=xxx[xxx]:25, conn_use=89, delay=1783, delays=223/1558/0.01/1.5, > dsn=2.6.0, status=sent (250 2.6.0 message received) > Jun 5 16:55:09 172 postfix/qmgr[31501]: 7ED581862A6: removed The above receives and delivers ONE MESSAGE with queue ID 7ED581862A6. Postfix delivers the message EXACTLY ONCE. > Jun 5 17:23:22 172 postfix/smtpd[5764]: 614FC185577: client=unknown[::1] > Jun 5 17:23:22 172 postfix/cleanup[6995]: 614FC185577: > message-id=<1233AD723A027C47B6CDC1668C2AF06D60594C2F@xxx> > Jun 5 17:26:45 172 postfix/qmgr[31501]: 614FC185577: from=, size=37582, > nrcpt=1 (queue active) > Jun 5 17:49:31 172 postfix/smtp[9033]: 614FC185577: to=, > relay=xxx[xxx]:25, conn_use=69, delay=1569, delays=203/1364/0/1.3, dsn=2.6.0, > status=sent (250 2.6.0 message received) > Jun 5 17:49:31 172 postfix/qmgr[31501]: 614FC185577: removed The above receives and delivers ONE MESSAGE with queue ID 614FC185577. Postfix delivers the message EXACTLY ONCE. If there is any email duplication, the duplication happens before Postfix receives the email. Wietse
AW: Re: AW: Re: Correctness of the queue management
Dear Wietse, I apologize "accusing" Postfix. Looking at your answers, I think I have to investigate whether and why MailMan has put messages with the same message-id and same recipient multiple times to the smtp deamon. Kindest regards, Tobias -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von Wietse Venema Gesendet: Dienstag, 10. Juni 2014 16:23 An: Postfix users Betreff: Re: AW: Re: Correctness of the queue management Schwartz, Tobias (DAV): > Dear Wietse, > > thank you for your reply! > You can find the log entries for a particular recipient below: > > Jun 5 15:25:18 172 postfix/smtpd[28500]: 9EE2B185BF0: > client=unknown[::1] Jun 5 15:25:18 172 postfix/cleanup[29770]: > 9EE2B185BF0: message-id=<1233AD723A027C47B6CDC1668C2AF06D60594C2F@xxx> > Jun 5 15:25:28 172 postfix/qmgr[31501]: 9EE2B185BF0: from=, > size=35964, nrcpt=1 (queue active) Jun 5 15:51:12 172 > postfix/smtp[31769]: 9EE2B185BF0: to=, relay=xxx[xxx]:25, > conn_use=148, delay=1554, delays=9.6/1543/0.01/1.4, dsn=2.6.0, > status=sent (250 2.6.0 message received) Jun 5 15:51:12 172 > postfix/qmgr[31501]: 9EE2B185BF0: removed The above receives and delivers ONE MESSAGE with queue ID 9EE2B185BF0. Postfix delivers the message EXACTLY ONCE. > Jun 5 16:25:26 172 postfix/smtpd[1149]: 7ED581862A6: > client=unknown[::1] Jun 5 16:25:26 172 postfix/cleanup[2274]: > 7ED581862A6: message-id=<1233AD723A027C47B6CDC1668C2AF06D60594C2F@xxx> > Jun 5 16:29:09 172 postfix/qmgr[31501]: 7ED581862A6: from=, > size=37578, nrcpt=1 (queue active) Jun 5 16:55:09 172 > postfix/smtp[4611]: 7ED581862A6: to=, relay=xxx[xxx]:25, > conn_use=89, delay=1783, delays=223/1558/0.01/1.5, dsn=2.6.0, > status=sent (250 2.6.0 message received) Jun 5 16:55:09 172 > postfix/qmgr[31501]: 7ED581862A6: removed The above receives and delivers ONE MESSAGE with queue ID 7ED581862A6. Postfix delivers the message EXACTLY ONCE. > Jun 5 17:23:22 172 postfix/smtpd[5764]: 614FC185577: > client=unknown[::1] Jun 5 17:23:22 172 postfix/cleanup[6995]: > 614FC185577: message-id=<1233AD723A027C47B6CDC1668C2AF06D60594C2F@xxx> > Jun 5 17:26:45 172 postfix/qmgr[31501]: 614FC185577: from=, > size=37582, nrcpt=1 (queue active) Jun 5 17:49:31 172 > postfix/smtp[9033]: 614FC185577: to=, relay=xxx[xxx]:25, > conn_use=69, delay=1569, delays=203/1364/0/1.3, dsn=2.6.0, status=sent > (250 2.6.0 message received) Jun 5 17:49:31 172 postfix/qmgr[31501]: > 614FC185577: removed The above receives and delivers ONE MESSAGE with queue ID 614FC185577. Postfix delivers the message EXACTLY ONCE. If there is any email duplication, the duplication happens before Postfix receives the email. Wietse
Re: AW: Re: AW: Re: Correctness of the queue management
Schwartz, Tobias (DAV): > Dear Wietse, > > I apologize "accusing" Postfix. > Looking at your answers, I think I have to investigate whether and > why MailMan has put messages with the same message-id and same > recipient multiple times to the smtp deamon. When a mailing list has many recipients, MailMan must break up the list and deliver the same message content and message ID with up to 100 recipients at a time. The SMTP protocol requires that Postfix can receive 100 recipients per mail delivery request. The default Postfix limit is larger, with "smtpd_recipient_limit = 1000". So, MailMan is supposed to deliver the same content multiple times, it just isn't supposed to duplicate the recipients. Wietse
Re: How to block offering SASL auth to clients based on RBL
Peter schrieb: > On 06/08/2014 08:17 PM, Kai Krakow wrote: >> MX and Submission machine are the same postfix instance (and even the >> same worker process on port 25), it won't work. I'm planning to maybe >> change this in the future. But as with migrating all people to not submit >> on port 25 it is a long way to go. > > If you can't force your users off of port 25, then the next best thing > is to separate our your submission by IP address, if done correctly your > users will be able to stay on port 25, not have to change the hostname > (or any other settings in their MUA) and you will have separated out > submission from MX traffic and can treat the two with different configs. Yes, that is the plan. Separate submission, MX, and mailboxes from each other, while during the same process migrate people to use port 587 on the new submission machine which has its port 25 hopefully closed... We already started to migrate new customers to port 587. But it is still on the same machine (though with a little different config), thus there's also port 25 available used by other customers. And those silly autodetection of older MUAs sticks to port 25 unencrypted. :-( So even new customers who redo their installations on their own silently go back to port 25. Maybe I'll add some flag into our user tables to block port 25 auth for new users. At least modern software does it right and tries 587 first, even Google Mail does it right if you configure an outgoing SMTP account. Apparently people tend to love old software and only use what they know. BTW: In this context, what's the best approach to put mailboxes on a separate machine? Let the LDA drop mails into NFS mounts, or let postfix transport the mails via transport_map into a machine which hosts the LDA (dovecot in our case)? -- Replies to list only preferred.
Re: How to block offering SASL auth to clients based on RBL
On 6/10/2014 1:24 PM, Kai Krakow wrote: > And those silly autodetection of older MUAs sticks to port 25 unencrypted. So even new customers who redo > their installations on their own silently go back to port 25. So... why on earth are you allowing UNENCRYPTED AUTH at ALL, let alone on port 25?
Re: CIDR Whitelist ?
On 6/10/2014 1:24 AM, Michael Tokarev wrote: > 10.06.2014 05:02, Stan Hoeppner wrote: ... >> Yes. And if you have other separate smtpd_foo_restrictions sections you >> should move those restriction parameters under >> smtpd_recipient_restrictions as well. This will give you precise >> control over whitelisting and blacklisting order. > > I'm sorry to say that, but this is wrong. All smtpd_*_restrictions give > precise control over all the restrictions and their order, if you move "will give you precise control". Note "you", meaning the user, not Postfix. Having all restrictions in one place makes it easier, as in eyeballs on screen, to see what's going on. > it all to one stage it becomes clumsier. Also, moving stuff which should > be run at connect or hello time to recipient time is kinda wrong. Postfix performs delayed evaluation of restrictions by default so this is irrelevant. Evaluation order for the most part doesn't change, only the logical order in main.cf. > Such a suggestion - to move everything to recipient_restriction - can be > given to a novice postfix user (or by novice postfix user), who does not > understand smtp protocol stages and this smtpd_*_restrictions mechanics, > both of which are kinda trivial. Hammer or nail gun Michael? Everyone understands how the hammer works, but given the choice goes for the nail gun. It's faster, more precise, and saves your thumbs. :) Cheers, Stan
Re: CIDR Whitelist ?
On Tue, Jun 10, 2014 at 01:43:50PM -0500, Stan Hoeppner wrote: > > I'm sorry to say that, but this is wrong. All smtpd_*_restrictions give > > precise control over all the restrictions and their order, if you move > > "will give you precise control". Note "you", meaning the user, not > Postfix. Having all restrictions in one place makes it easier, as in > eyeballs on screen, to see what's going on. There is no single right answer. For many users a single linear list of conditions is simplest. For some users, where one wants a whitelist for one set of test to not short-circuit other tests, multiple lists are better. We should not be dogmatic about either approach. -- Viktor.
Re: CIDR Whitelist ?
10.06.2014 22:43, Stan Hoeppner wrote: > On 6/10/2014 1:24 AM, Michael Tokarev wrote: [] >> it all to one stage it becomes clumsier. Also, moving stuff which should >> be run at connect or hello time to recipient time is kinda wrong. > > Postfix performs delayed evaluation of restrictions by default so this > is irrelevant. Evaluation order for the most part doesn't change, only > the logical order in main.cf. It is irrelevant as long as you don't change the default. Which is okay to change too, ofcourse. Thanks, /mjt
Problem understanding check_client_access and tcp_table
Hello, Subject: Problem understanding check_client_access and tcp_table I have a problem understanding how to properly use check_client_access with an external tcp_table (daemon) in my main.cf i have put the following: smtpd_client_restrictions = check_client_access tcp:[127.0.0.1]:1 The lookup process itself works - but I was under the impression that my tcp_table "daemon" would receive a "get " lookup requests with the source ip address as argument... What I am receiving is lots of lookup "get" requests with argument "unknown" and nothing else. My guess is that the "unknown" string represents unresolvable reverse lookups - because I'm testing with loopback 127.0.0.x addresses But what I was expecting was to receive lookup "get" requests with the raw source ip address as argument - as "unknown" is not of much use for spam analysis. Did I misunderstand the check_client_access ? How do I get the raw source ip address to my tcp_table lookup "daemon" ? I'm using Postfix 2.11.1 on FreeBSD 10 Thanks in advance Kind regards Uffe -- View this message in context: http://postfix.1071664.n5.nabble.com/Problem-understanding-check-client-access-and-tcp-table-tp68481.html Sent from the Postfix Users mailing list archive at Nabble.com.
the after-queue simple filter -- I'm doing it wrong
RE: the after-queue, simple filter example
Just trying to figure out how it works.
I have a feeling I don't fully understand the full implications of
"after-queue".
>From master.cf:
# custom filter
filter unix- n n - 10 pipe
flags=Rq user=filter null_sender=
argv=/VOLUMES/gizmonics/work/afterqueue/simple -f ${sender} -- ${recipient}
The user is in
[root@clemi postfix]# grep -w filter /etc/passwd /etc/group
/etc/passwd:filter:x:506:506:smtp filter:/var/spool/filter/:/bin/bash
/etc/group:filter:x:506:
The user filter owns the directory
[root@clemi postfix]# ls -ld /var/spool/filter/
drwxr-xr-x 2 filter filter 4096 Jun 10 14:49 /var/spool/filter/
The user filter owns the script
[root@clemi postfix]# ls -l /VOLUMES/gizmonics/work/afterqueue/simple
-rwxr-xr-x 1 filter filter 814 Jun 10 14:46
/VOLUMES/gizmonics/work/afterqueue/simple
I've run that from the command line, ie:
/VOLUMES/gizmonics/work/afterqueue/simple -f gl -- b...@idiot.com <
/VOLUMES/gizmonics/work/afterqueue/dull
and it works when run like that. The simple script does what I expect, and the
mail
does get to bbb's mail file.
Here's my simple script, which is just the example script from the
FILTER_README.html page with only two or so lines changed, marked below:
#!/bin/bash
# Simple shell-based filter. It is meant to be invoked as follows:
# /path/to/script -f sender recipients...
# Localize these. The -G option does nothing before Postfix 2.3.
INSPECT_DIR=/var/spool/filter
SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
# Exit codes from
EX_TEMPFAIL=75
EX_UNAVAILABLE=69
# Clean up when done or when aborting.
trap "rm -f in.$$" 0 1 2 3 15
# Start processing.
cd $INSPECT_DIR || {
echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
cat >in.$$ || {
echo Cannot save mail to file; exit $EX_TEMPFAIL; }
# Specify your content filter here.
# filter smak.$$
$SENDMAIL "$@"
Re: the after-queue simple filter -- I'm doing it wrong
On 6/10/2014 3:05 PM, Jay G. Scott wrote:
>
> RE: the after-queue, simple filter example
>
> Just trying to figure out how it works.
> I have a feeling I don't fully understand the full implications of
> "after-queue".
>
> From master.cf:
> # custom filter
> filter unix- n n - 10 pipe
> flags=Rq user=filter null_sender=
> argv=/VOLUMES/gizmonics/work/afterqueue/simple -f ${sender} -- ${recipient}
>
> The user is in
> [root@clemi postfix]# grep -w filter /etc/passwd /etc/group
> /etc/passwd:filter:x:506:506:smtp filter:/var/spool/filter/:/bin/bash
> /etc/group:filter:x:506:
>
> The user filter owns the directory
> [root@clemi postfix]# ls -ld /var/spool/filter/
> drwxr-xr-x 2 filter filter 4096 Jun 10 14:49 /var/spool/filter/
>
> The user filter owns the script
> [root@clemi postfix]# ls -l /VOLUMES/gizmonics/work/afterqueue/simple
> -rwxr-xr-x 1 filter filter 814 Jun 10 14:46
> /VOLUMES/gizmonics/work/afterqueue/simple
>
> I've run that from the command line, ie:
>
> /VOLUMES/gizmonics/work/afterqueue/simple -f gl -- b...@idiot.com <
> /VOLUMES/gizmonics/work/afterqueue/dull
>
> and it works when run like that. The simple script does what I expect, and
> the mail
> does get to bbb's mail file.
>
> Here's my simple script, which is just the example script from the
> FILTER_README.html page with only two or so lines changed, marked below:
>
>
> #!/bin/bash
>
> # Simple shell-based filter. It is meant to be invoked as follows:
> # /path/to/script -f sender recipients...
>
> # Localize these. The -G option does nothing before Postfix 2.3.
> INSPECT_DIR=/var/spool/filter
> SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
>
> # Exit codes from
> EX_TEMPFAIL=75
> EX_UNAVAILABLE=69
>
> # Clean up when done or when aborting.
> trap "rm -f in.$$" 0 1 2 3 15
>
> # Start processing.
> cd $INSPECT_DIR || {
> echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
>
> cat >in.$$ || {
> echo Cannot save mail to file; exit $EX_TEMPFAIL; }
>
> # Specify your content filter here.
> # filter # echo Message content rejected; exit $EX_UNAVAILABLE; }
>
> ### this sed line is added, and the $SENDMAIL changed to read from smak.$$
> ### otherwise, it's completely stock.
> sed -e 's/http/hxxp/g' in.$$ > smak.$$
>
> $SENDMAIL "$@"
> exit $?
>
>
> ---
> I've changed master as shown above.
> I've done postfix reload.
>
> But all of my tests like so:
>
> mail -s "wogujwrg" b...@idiot.com < test.input.file
>
> deliver the test.input.file unmodified. Do I not understand what
> "after-queue" means? I've been assuming this would, sooner or later,
> send things to the filter. It looks to me like it's bypassing
> the filter.
>
> j.
>
So how is postfix supposed to know to use the filter? Did you set
content_filter in main.cf or are you using an access table the
responds with FILTER or what?
-- Noel Jones
Re: How to block offering SASL auth to clients based on RBL
Kai Krakow: > BTW: In this context, what's the best approach to put mailboxes on a > separate machine? Let the LDA drop mails into NFS mounts, or let postfix > transport the mails via transport_map into a machine which hosts the LDA > (dovecot in our case)? I recommend Dovecot via LMTP, but NFS would work, too, assuming one file per message. I can't say which approach would handle the most load. When I started work on Postfix, my dream configuration for inbound mail handling was: Internet -> N x Postfix server -> M x Mailbox server With N and M scaling up as needed. Wietse
Re: Problem understanding check_client_access and tcp_table
On 6/10/2014 2:59 PM, uffe wrote: > Hello, > > Subject: Problem understanding check_client_access and tcp_table > > I have a problem understanding how to properly use check_client_access with > an external tcp_table (daemon) > > in my main.cf i have put the following: > > smtpd_client_restrictions = check_client_access tcp:[127.0.0.1]:1 > > The lookup process itself works - but I was under the impression that my > tcp_table "daemon" would receive a "get " lookup requests with the source ip > address as argument... > > What I am receiving is lots of lookup "get" requests with argument "unknown" > and nothing else. > > My guess is that the "unknown" string represents unresolvable reverse > lookups - because I'm testing with loopback 127.0.0.x addresses > > But what I was expecting was to receive lookup "get" requests with the raw > source ip address as argument - as "unknown" is not of much use for spam > analysis. > > Did I misunderstand the check_client_access ? > > How do I get the raw source ip address to my tcp_table lookup "daemon" ? > > I'm using Postfix 2.11.1 on FreeBSD 10 > > Thanks in advance > > Kind regards Uffe check_client_access will test the hostname first, then if not found, the IP address. This behavior is independent of table lookup type. So your tcp table will need to respond to all hostname queries with 500 Not Found followed by a newline. This will allow the IP query to happen. -- Noel Jones
Re: Problem understanding check_client_access and tcp_table
uffe: > smtpd_client_restrictions = check_client_access tcp:[127.0.0.1]:1 > > But what I was expecting was to receive lookup "get" requests with the raw > source ip address as argument - as "unknown" is not of much use for spam > analysis. There will be no "get client-address" query when the "get client-name" answer is "reject", "defer", "permit", "ok", or something else that terminates the search. Wietse
Re: Problem understanding check_client_access and tcp_table
Thanks for your swift answers I'll try the recommendations immediately. English is not my native language - but it did not occour to me from reading that documentation for tcp_table lookups that returning 500 would be aproprialte - I would have believed that is would have stopped the while delivery "transaction". Rereading the docs as we speak makes it more clear - thx -- View this message in context: http://postfix.1071664.n5.nabble.com/Problem-understanding-check-client-access-and-tcp-table-tp68481p68487.html Sent from the Postfix Users mailing list archive at Nabble.com.
Re: the after-queue simple filter -- I'm doing it wrong
On Tue, Jun 10, 2014 at 03:37:34PM -0500, Noel Jones wrote:
> On 6/10/2014 3:05 PM, Jay G. Scott wrote:
> >
> > RE: the after-queue, simple filter example
> >
> > Just trying to figure out how it works.
> > I have a feeling I don't fully understand the full implications of
> > "after-queue".
> >
> > From master.cf:
> > # custom filter
> > filter unix- n n - 10 pipe
> > flags=Rq user=filter null_sender=
> > argv=/VOLUMES/gizmonics/work/afterqueue/simple -f ${sender} --
> > ${recipient}
> >
> > The user is in
> > [root@clemi postfix]# grep -w filter /etc/passwd /etc/group
> > /etc/passwd:filter:x:506:506:smtp filter:/var/spool/filter/:/bin/bash
> > /etc/group:filter:x:506:
> >
> > The user filter owns the directory
> > [root@clemi postfix]# ls -ld /var/spool/filter/
> > drwxr-xr-x 2 filter filter 4096 Jun 10 14:49 /var/spool/filter/
> >
> > The user filter owns the script
> > [root@clemi postfix]# ls -l /VOLUMES/gizmonics/work/afterqueue/simple
> > -rwxr-xr-x 1 filter filter 814 Jun 10 14:46
> > /VOLUMES/gizmonics/work/afterqueue/simple
> >
> > I've run that from the command line, ie:
> >
> > /VOLUMES/gizmonics/work/afterqueue/simple -f gl -- b...@idiot.com <
> > /VOLUMES/gizmonics/work/afterqueue/dull
> >
> > and it works when run like that. The simple script does what I expect, and
> > the mail
> > does get to bbb's mail file.
> >
> > Here's my simple script, which is just the example script from the
> > FILTER_README.html page with only two or so lines changed, marked below:
> >
> >
> > #!/bin/bash
> >
> > # Simple shell-based filter. It is meant to be invoked as follows:
> > # /path/to/script -f sender recipients...
> >
> > # Localize these. The -G option does nothing before Postfix 2.3.
> > INSPECT_DIR=/var/spool/filter
> > SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
> >
> > # Exit codes from
> > EX_TEMPFAIL=75
> > EX_UNAVAILABLE=69
> >
> > # Clean up when done or when aborting.
> > trap "rm -f in.$$" 0 1 2 3 15
> >
> > # Start processing.
> > cd $INSPECT_DIR || {
> > echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
> >
> > cat >in.$$ || {
> > echo Cannot save mail to file; exit $EX_TEMPFAIL; }
> >
> > # Specify your content filter here.
> > # filter > # echo Message content rejected; exit $EX_UNAVAILABLE; }
> >
> > ### this sed line is added, and the $SENDMAIL changed to read from smak.$$
> > ### otherwise, it's completely stock.
> > sed -e 's/http/hxxp/g' in.$$ > smak.$$
> >
> > $SENDMAIL "$@" >
> > exit $?
> >
> >
> > ---
> > I've changed master as shown above.
> > I've done postfix reload.
> >
> > But all of my tests like so:
> >
> > mail -s "wogujwrg" b...@idiot.com < test.input.file
> >
> > deliver the test.input.file unmodified. Do I not understand what
> > "after-queue" means? I've been assuming this would, sooner or later,
> > send things to the filter. It looks to me like it's bypassing
> > the filter.
> >
> > j.
> >
>
>
>
> So how is postfix supposed to know to use the filter? Did you set
> content_filter in main.cf or are you using an access table the
> responds with FILTER or what?
>
ahhh, I didn't know. The FILTER_README doesn't mention main.cf
in connection w/ the simple content filter, just the advanced.
I assumed the FILTER_README knew what it was doing.
but it did puzzle me, as my production/commercial filter
does require something in main.cf.
i will poke around w/ that.
j.
>
>
> -- Noel Jones
--
Jay Scott 512-835-3553g...@arlut.utexas.edu
Head of Sun Support, Sr. System Administrator
Applied Research Labs, Computer Science Div. S224
University of Texas at Austin
Re: the after-queue simple filter -- I'm doing it wrong
After trying various combinations of things in
main.cf and master.cf, I find that, using the script
below, if the mail reaches the filter script, the
resulting mail message bounces (too many hops)
but the filter has processed the input.
If I turn content_filter off in main.cf the
mail gets delivered unfiltered, no bounces, no hops.
main.cf:
content_filter = filter
got it filtered, but resulted in the hops bounce.
What am I missing?
j.
On Tue, Jun 10, 2014 at 03:37:34PM -0500, Noel Jones wrote:
> On 6/10/2014 3:05 PM, Jay G. Scott wrote:
> >
> > RE: the after-queue, simple filter example
> >
> > Just trying to figure out how it works.
> > I have a feeling I don't fully understand the full implications of
> > "after-queue".
> >
> > From master.cf:
> > # custom filter
> > filter unix- n n - 10 pipe
> > flags=Rq user=filter null_sender=
> > argv=/VOLUMES/gizmonics/work/afterqueue/simple -f ${sender} --
> > ${recipient}
> >
> > The user is in
> > [root@clemi postfix]# grep -w filter /etc/passwd /etc/group
> > /etc/passwd:filter:x:506:506:smtp filter:/var/spool/filter/:/bin/bash
> > /etc/group:filter:x:506:
> >
> > The user filter owns the directory
> > [root@clemi postfix]# ls -ld /var/spool/filter/
> > drwxr-xr-x 2 filter filter 4096 Jun 10 14:49 /var/spool/filter/
> >
> > The user filter owns the script
> > [root@clemi postfix]# ls -l /VOLUMES/gizmonics/work/afterqueue/simple
> > -rwxr-xr-x 1 filter filter 814 Jun 10 14:46
> > /VOLUMES/gizmonics/work/afterqueue/simple
> >
> > I've run that from the command line, ie:
> >
> > /VOLUMES/gizmonics/work/afterqueue/simple -f gl -- b...@idiot.com <
> > /VOLUMES/gizmonics/work/afterqueue/dull
> >
> > and it works when run like that. The simple script does what I expect, and
> > the mail
> > does get to bbb's mail file.
> >
> > Here's my simple script, which is just the example script from the
> > FILTER_README.html page with only two or so lines changed, marked below:
> >
> >
> > #!/bin/bash
> >
> > # Simple shell-based filter. It is meant to be invoked as follows:
> > # /path/to/script -f sender recipients...
> >
> > # Localize these. The -G option does nothing before Postfix 2.3.
> > INSPECT_DIR=/var/spool/filter
> > SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
> >
> > # Exit codes from
> > EX_TEMPFAIL=75
> > EX_UNAVAILABLE=69
> >
> > # Clean up when done or when aborting.
> > trap "rm -f in.$$" 0 1 2 3 15
> >
> > # Start processing.
> > cd $INSPECT_DIR || {
> > echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
> >
> > cat >in.$$ || {
> > echo Cannot save mail to file; exit $EX_TEMPFAIL; }
> >
> > # Specify your content filter here.
> > # filter > # echo Message content rejected; exit $EX_UNAVAILABLE; }
> >
> > ### this sed line is added, and the $SENDMAIL changed to read from smak.$$
> > ### otherwise, it's completely stock.
> > sed -e 's/http/hxxp/g' in.$$ > smak.$$
> >
> > $SENDMAIL "$@" >
> > exit $?
> >
> >
> > ---
> > I've changed master as shown above.
> > I've done postfix reload.
> >
> > But all of my tests like so:
> >
> > mail -s "wogujwrg" b...@idiot.com < test.input.file
> >
> > deliver the test.input.file unmodified. Do I not understand what
> > "after-queue" means? I've been assuming this would, sooner or later,
> > send things to the filter. It looks to me like it's bypassing
> > the filter.
> >
> > j.
> >
>
>
>
> So how is postfix supposed to know to use the filter? Did you set
> content_filter in main.cf or are you using an access table the
> responds with FILTER or what?
>
>
>
> -- Noel Jones
--
Jay Scott 512-835-3553g...@arlut.utexas.edu
Head of Sun Support, Sr. System Administrator
Applied Research Labs, Computer Science Div. S224
University of Texas at Austin
Re: Problem understanding check_client_access and tcp_table
Now I got it working - thanks Are tcp_table lookup "deamon" instances supposed to exit themselves - for every lookup - or can postfix reuse them "persistent" I'm having a hard time understanding the comment in the bugs section: The client does not hang up when the connection is idle for a long time (http://www.postfix.org/tcp_table.5.html) My tcp_table lookup "deamon" is currently running under daemontools (max concurrent 10 instances) and postfix seems to keep them hanging around - but sometimes it seems that the system gets stuck I'm going through my own code - but havent found anything yet - hence my question /Uffe -- View this message in context: http://postfix.1071664.n5.nabble.com/Problem-understanding-check-client-access-and-tcp-table-tp68481p68490.html Sent from the Postfix Users mailing list archive at Nabble.com.
Re: How to block offering SASL auth to clients based on RBL
On 6/10/2014 3:39 PM, Wietse Venema wrote: > Kai Krakow: >> BTW: In this context, what's the best approach to put mailboxes on a >> separate machine? Let the LDA drop mails into NFS mounts, or let postfix >> transport the mails via transport_map into a machine which hosts the LDA >> (dovecot in our case)? > > I recommend Dovecot via LMTP, but NFS would work, too, assuming one > file per message. I can't say which approach would handle the most > load. Dovecot's LMTP and LDA both perform index updates during delivery to the mailbox. They also enable Sieve. Dovecot's speedy performance is due in large part to its indexes. If you use the Postfix LDA to drop mail directly into maildir files, Dovecot will need to stat the new files to update its indexes, before responding to a LIST command. On a busy server this can be expensive, and responsiveness at the MUA may be sluggish. Thus I concur with Wietse. Use LMTP for performance, and to enable Sieve scripting. Cheers, Stan
Re: Problem understanding check_client_access and tcp_table
On 6/10/2014 6:32 PM, uffe wrote: > > Now I got it working - thanks > > Are tcp_table lookup "deamon" instances supposed to exit themselves - for > every lookup - or can postfix reuse them "persistent" > > I'm having a hard time understanding the comment in the bugs section: The > client does not hang up when the connection is idle for a long time > (http://www.postfix.org/tcp_table.5.html) > > My tcp_table lookup "deamon" is currently running under daemontools (max > concurrent 10 instances) and postfix seems to keep them hanging around - but > sometimes it seems that the system gets stuck > I'm going through my own code - but havent found anything yet - hence my > question > > /Uffe The tcp daemon should probably exit when the postfix client disconnects. This will happen periodically as postfix retires and recreates new smtpd processes. The max connections will need to be at least $default_process_limit (default 100). Each smtpd process will create its own connection to your table. I suppose you could use proxy:tcp: to limit the number of connections. -- Noel Jones
Re: the after-queue simple filter -- I'm doing it wrong
On 6/10/2014 5:45 PM, Jay G. Scott wrote: > > After trying various combinations of things in > main.cf and master.cf, I find that, using the script > below, if the mail reaches the filter script, the > resulting mail message bounces (too many hops) > but the filter has processed the input. > > If I turn content_filter off in main.cf the > mail gets delivered unfiltered, no bounces, no hops. > > main.cf: > > content_filter = filter > > got it filtered, but resulted in the hops bounce. > > What am I missing? > > j. Near the bottom of http://www.postfix.org/FILTER_README.html#simple_filter the document describes how to turn on filtering for mail arriving via SMTP, but leave it turned off as the main.cf default so mail won't loop. Of course this means you would need to submit mail via SMTP for testing; mail submitted via the sendmail(1) interface is not filtered. This is necessary since the "simple" filter resubmits the mail to postfix via the sendmail command. -- Noel Jones
Re: the after-queue simple filter -- I'm doing it wrong
On Tue, Jun 10, 2014 at 05:45:02PM -0500, Jay G. Scott wrote: > After trying various combinations of things in > main.cf and master.cf, I find that, using the script > below, if the mail reaches the filter script, the > resulting mail message bounces (too many hops) > but the filter has processed the input. > > If I turn content_filter off in main.cf the > mail gets delivered unfiltered, no bounces, no hops. > > main.cf: > > content_filter = filter > > got it filtered, but resulted in the hops bounce. > > What am I missing? Reading FILTER_README carefully, and disabling content_filter in master.cf for the pickup service as instructed: pickup fifo n - - 60 1 pickup -o content_filter= Simple filter resubmission via sendmail(1) must enter the queue at an injection interface that does not apply the content_filter again. -- Viktor.
