date:20140911

Re: Possible reasons for "lost connection after DATA"

2014-09-11 Thread Sean Durkin

Hello Wietse,

Am 10.09.2014 um 21:52 schrieb Wietse Venema:

> Slow performance is typical for TCP window scaling problems. Have
> you tried to turn it off in your kernel?

Yes, Viktor suggested that also and I tried it. It does not make a difference, 
the problem persists.

Regards,
Sean

Re: Possible reasons for "lost connection after DATA"

2014-09-11 Thread Wietse Venema

Sean Durkin:
> Hello Wietse,
> 
> Am 10.09.2014 um 21:52 schrieb Wietse Venema:
> 
> > Slow performance is typical for TCP window scaling problems. Have
> > you tried to turn it off in your kernel?
> 
> Yes, Viktor suggested that also and I tried it. It does not make
> a difference, the problem persists.

What is the distribution of DATA sizes before failure? In your
example I see numbers around 3kB, 9kB, 12kB.

Some failures are triggered by packet content, and may be replaced
only by replacing hardware that operates marginally. Does the problem
go away when you

- Replace the server (either the network card or the whole box)

- Replace the cable that connects the server to the network switch

- Replace the network switch that the server is plugged into.

- Replace the cable that connects the switch to the router

- Replace the router

- And so on...

If you think this is a stupid idea, then you haven't been around
long enough.

Wietse

Re: Positive DSN if delay_warning_time is reached?

2014-09-11 Thread A. Schulze

wietse:

This turned out to be easier than expected. Manpage fragment for
Postfix 2.12-20140907:

confirm_delay_cleared (default: no)
After sending a "your message is delayed" notification,
inform the sender when the delay clears up. This can result
in a sudden burst of notifications at the end of a prolonged
network outage, and is therefore disabled by default.

Adding this required only minor extension of existing infrastructure.
The feature is also available as a patch for older Postfix versions at
ftp://ftp.porcupine.org/mirrors/postfix-release/experimental/feature-patches/20140907-confirm_delay_cleared-patch

Hello,

thanks for the patch. I like to use it with postfix-2.11.1. At a first
view it works as expected.

But imagine th following scenario: client <-> relay <-> destination

- client send a message requesting DSN on success and failure.
Explicit not for delay.

- the relay has
delay_warning_time 1h
confirm_delay_cleared = yes

- the destination if offline

- the relay hold the message for two hours and do not send a "your
message is delayed" notification

- the destination go online

- the relay forward the message

- the destination send a "your messages is delivered"

But just here the relay also send a "your messages is delivered".
That's unexpected to me. I'm unsure if I like this.
If I read the documentation, it may be a bug because no delay warning
was sent.

Anyway: I checked, the relay do not send a success message if there
was no delay at all.
Also interesting to test: what happen if relay is greylisted and
deliver to a fallback_relay.

( that may have similar setting or not )

Andreas

Re: Possible reasons for "lost connection after DATA"

2014-09-11 Thread Sean Durkin

Hi Viktor,

Am 10.09.2014 um 23:03 schrieb Viktor Dukhovni:
> This trace has an insane level of debugging turned on, to the point
> that syslogd is overwhelmed and is losing messages.  PLEASE DISABLE
> ALL VERBOSE logging.  NO "-v" options in master.cf, NO debug_peer_list,
> ...
Yes, sorry, I cranked up the debug level, since normal logging looks like this:

Sep 11 09:43:31 mail postfix/smtpd[25170]: connect from 
mail18-21.srv2.de[193.169.181.21]
Sep 11 09:43:31 mail postfix/smtpd[25170]: 2C076C4026A: 
client=mail18-21.srv2.de[193.169.181.21]
Sep 11 09:46:33 mail postfix/smtpd[25170]: lost connection after DATA (33290 
bytes) from mail18-21.srv2.de[193.169.181.21]
Sep 11 09:46:33 mail postfix/smtpd[25170]: disconnect from 
mail18-21.srv2.de[193.169.181.21]
...
Sep 11 10:10:59 mail postfix/smtpd[25537]: connect from 
quattuorocto.psi.cust-cluster.com[195.140.187.48]
Sep 11 10:10:59 mail postfix/smtpd[25537]: 8736FC40A7D: 
client=quattuorocto.psi.cust-cluster.com[195.140.187.48]
Sep 11 10:36:44 mail postfix/smtpd[25537]: lost connection after DATA (36809 
bytes) from quattuorocto.psi.cust-cluster.com[195.140.187.48]
Sep 11 10:36:44 mail postfix/smtpd[25537]: disconnect from 
quattuorocto.psi.cust-cluster.com[195.140.187.48]
..
Sep 11 10:38:48 mail postfix/smtpd[25913]: connect from 
smtp-out-127-108.amazon.com[176.32.127.108]
Sep 11 10:38:49 mail postfix/smtpd[25913]: 2558DC40458: 
client=smtp-out-127-108.amazon.com[176.32.127.108]
Sep 11 10:41:01 mail postfix/smtpd[25913]: lost connection after DATA (17511 
bytes) from smtp-out-127-108.amazon.com[176.32.127.108]
Sep 11 10:41:01 mail postfix/smtpd[25913]: disconnect from 
smtp-out-127-108.amazon.com[176.32.127.108]

I didn't think that info alone was particularly useful...

> Please make sure that the /dev/log syslog socket is a "dgram" not
> a "stream" socket, and that mail logging is not synchronous.
Logging is not synchronous, the socket is a datagram socket (it has all been 
set up that way all along).

No change, still the same problem, see above.

Meanwhile, I've managed to record a tcpdump of such a failed session. What 
exactly am I looking for there?
I don't see anything out of the ordinary, except increasing delays between 
received packets from the external host, until the host sends a "RST".
It seems I simply do not receive any packets. The ones I get are immediately 
ACK'd, but then there's seconds and later minutes until the next one even 
arrives, until finally the remote host gives up and terminates the connection.

I'll try to get more dumps for comparison, including some from hosts that have 
no problems delivering.

There's no packet filtering or rate limiting on port 25, at least not on my 
machine. The hosting provider might have something there, I'd have to ask 
them...

Regards,
Sean

Re: pipemap, multiple results [patch]

2014-09-11 Thread Roel van Meer


Roel van Meer writes:

What I am actually trying to do is a lookup with a single key in two maps.   
Maybe stackmap or concatmap?


Now, if you specify two maps somewhere, and the first map returns a result,  
there is no lookup done in the second map.  With concatmap, both lookups  
would happen, and the combined result would be returned.


Largely based on the code of pipemap, I now have something that does this.
It goes by the name of joinmap.

/etc/postfix/a:
a   res-a
c   res-ca
d   res-d

/etc/postfix/b:
b   res-b
c   res-cb
d   res-d

# postmap -q a 'joinmap:!hash:/etc/postfix/a!hash:/etc/postfix/b'
res-a

# postmap -q b 'joinmap:!hash:/etc/postfix/a!hash:/etc/postfix/b'
res-b

# postmap -q c 'joinmap:!hash:/etc/postfix/a!hash:/etc/postfix/b'
res-ca,res-cb

# postmap -q d 'joinmap:!hash:/etc/postfix/a!hash:/etc/postfix/b'
res-d,res-d

# postmap -q e 'joinmap:!hash:/etc/postfix/a!hash:/etc/postfix/b'


So far it does what I need: I can now use the output of two ldap lookups in  
virtual_alias_maps.


The patch is just code though.  If you're interested, I can write some  
documentation to make it more complete.  And although it seems to work on my  
test server, it hasn't been tested very thoroughly yet..


Thanks again (it's always a pleasure to work with Postfix),

Roel
diff -ruN a/.indent.pro b/.indent.pro
--- a/.indent.pro   2014-07-18 01:09:15.0 +0200
+++ b/.indent.pro   2014-09-11 12:29:49.298253498 +0200
@@ -77,6 +77,7 @@
 -TDICT_DEBUG
 -TDICT_ENV
 -TDICT_FAIL
+-TDICT_JOIN
 -TDICT_HT
 -TDICT_LDAP
 -TDICT_LMDB
diff -ruN b/src/util/Makefile.in c/src/util/Makefile.in
--- b/src/util/Makefile.in  2014-07-21 01:21:23.0 +0200
+++ c/src/util/Makefile.in  2014-09-11 11:54:25.868238221 +0200
@@ -38,7 +38,7 @@
dict_fail.c msg_rate_delay.c dict_surrogate.c warn_stat.c \
dict_sockmap.c line_number.c recv_pass_attr.c pass_accept.c \
poll_fd.c timecmp.c slmdb.c dict_pipe.c dict_random.c \
-   valid_utf8_hostname.c midna.c
+   valid_utf8_hostname.c midna.c dict_join.c
 OBJS   = alldig.o allprint.o argv.o argv_split.o attr_clnt.o attr_print0.o \
attr_print64.o attr_print_plain.o attr_scan0.o attr_scan64.o \
attr_scan_plain.o auto_clnt.o base64_code.o basename.o binhash.o \
@@ -78,7 +78,7 @@
dict_fail.o msg_rate_delay.o dict_surrogate.o warn_stat.o \
dict_sockmap.o line_number.o recv_pass_attr.o pass_accept.o \
poll_fd.o timecmp.o $(NON_PLUGIN_MAP_OBJ) dict_pipe.o dict_random.o \
-   valid_utf8_hostname.o midna.o
+   valid_utf8_hostname.o midna.o dict_join.o
 # MAP_OBJ is for maps that may be dynamically loaded with dynamicmaps.cf.
 # When hard-linking these, makedefs sets NON_PLUGIN_MAP_OBJ=$(MAP_OBJ),
 # otherwise it sets the PLUGIN_* macros.
@@ -107,7 +107,7 @@
edit_file.h dict_cache.h dict_thash.h ip_match.h nbbio.h base32_code.h \
dict_fail.h warn_stat.h dict_sockmap.h line_number.h timecmp.h \
slmdb.h compat_va_copy.h dict_pipe.h dict_random.h \
-   valid_utf8_hostname.h midna.h
+   valid_utf8_hostname.h midna.h dict_join.h
 TESTSRC= fifo_open.c fifo_rdwr_bug.c fifo_rdonly_bug.c select_bug.c \
stream_test.c dup2_pass_on_exec.c
 DEFS   = -I. -D$(SYSTYPE)
@@ -1043,6 +1043,19 @@
 dict_ht.o: vbuf.h
 dict_ht.o: vstream.h
 dict_ht.o: vstring.h
+dict_join.o: argv.h
+dict_join.o: dict.h
+dict_join.o: dict_join.c
+dict_join.o: dict_join.h
+dict_join.o: htable.h
+dict_join.o: msg.h
+dict_join.o: myflock.h
+dict_join.o: mymalloc.h
+dict_join.o: stringops.h
+dict_join.o: sys_defs.h
+dict_join.o: vbuf.h
+dict_join.o: vstream.h
+dict_join.o: vstring.h
 dict_lmdb.o: argv.h
 dict_lmdb.o: dict.h
 dict_lmdb.o: dict_lmdb.c
@@ -1094,6 +1107,7 @@
 dict_open.o: dict_env.h
 dict_open.o: dict_fail.h
 dict_open.o: dict_ht.h
+dict_open.o: dict_join.h
 dict_open.o: dict_lmdb.h
 dict_open.o: dict_ni.h
 dict_open.o: dict_nis.h
diff -ruN b/src/util/dict_join.c c/src/util/dict_join.c
--- b/src/util/dict_join.c  1970-01-01 01:00:00.0 +0100
+++ c/src/util/dict_join.c  2014-09-11 12:02:19.538242235 +0200
@@ -0,0 +1,195 @@
+/*++
+/* NAME
+/* dict_join 3
+/* SUMMARY
+/* dictionary manager interface for pipelined tables
+/* SYNOPSIS
+/* #include 
+/*
+/* DICT*dict_join_open(name, open_flags, dict_flags)
+/* const char *name;
+/* int open_flags;
+/* int dict_flags;
+/* DESCRIPTION
+/* dict_join_open() opens a pipeline of one or more tables.
+/* Example: "\fBpipemap:\fI!type_1:name_1! ... !type_n:name_n\fR".
+/*
+/* Each "pipemap:" query is given to the first table.  Each
+/* lookup result becomes the query for the next table in the
+/* pipeline, and the last table produces the final result.
+/* When any table lookup produces no result, the pipeline
+/* produces no result.
+/*
+/* The first ASCII character after "pipemap:" will be used as
+/* the separator between the loo

Re: Possible reasons for "lost connection after DATA"

2014-09-11 Thread Viktor Dukhovni

On Thu, Sep 11, 2014 at 02:36:51PM +0200, Sean Durkin wrote:

> > PLEASE DISABLE
> > ALL VERBOSE logging.  NO "-v" options in master.cf, NO debug_peer_list,
>
> Yes, sorry, I cranked up the debug level, since normal logging looks like 
> this:
> 
> Sep 11 09:43:31 mail postfix/smtpd[25170]: connect from 
> mail18-21.srv2.de[193.169.181.21]
> Sep 11 09:43:31 mail postfix/smtpd[25170]: 2C076C4026A: 
> client=mail18-21.srv2.de[193.169.181.21]
> Sep 11 09:46:33 mail postfix/smtpd[25170]: lost connection after DATA (33290 
> bytes) from mail18-21.srv2.de[193.169.181.21]
> Sep 11 09:46:33 mail postfix/smtpd[25170]: disconnect from 
> mail18-21.srv2.de[193.169.181.21]

That's sufficient.  It shows you're likely not using TLS here, and
the time beetween message start and connection loss.  The number
of samples is rather small now.  I would expect the session duration
for each sending host to be essentially constant over multiple
deliveries (equal to the remote machine's TCP timeout).

Possibilities include a broken network interface somewhere or a
bad cable that corrupts IP or TCP packet headers given specific
input patterns.

If the problem is with the message payload, you could try enabling
inbound TLS, perhaps these sending servers support it.  Don't recall
whether you already have TLS.  If the problem is not with the
payload, then TLS won't make any difference (some hosts will still
fail even after TLS).

> I didn't think that info alone was particularly useful...

It is sufficient, and the verbose logs just add noise.

> Meanwhile, I've managed to record a tcpdump of such a failed
> session. What exactly am I looking for there?

Retransmissions from the sender of data with the same sequence
number...

Post "tcpdump" output (without packet content is fine), containing
packets from just a single failed session.

> There's no packet filtering or rate limiting on port 25, at least
> not on my machine. The hosting provider might have something there,
> I'd have to ask them...

They probably have middle boxes, which might be the cause of the
problem.

-- 
Viktor.

About mynetworks

2014-09-11 Thread Feel Zhou

Hello, my friend
This is tom, I'm sending my greeting from China.
My customer send mail via my postfix server
main.cf
mynetworks = hash:/etc/postfix/mynetworks_table
/etc/postfix/mynetworks_table
ip1 PERMIT
ip2 PERMIT
Today, one of my customer send mail via the new ip.
And this new ip not in the /etc/postfix/mynetworks_table
Can you help me to resolve it.
Thanks for your time.
Tom

Re: About mynetworks

2014-09-11 Thread Viktor Dukhovni

On Thu, Sep 11, 2014 at 09:11:01PM +0800, Feel Zhou wrote:

> My customer send mail via my postfix server
>
> main.cf:
> mynetworks = hash:/etc/postfix/mynetworks_table
>
> mynetworks_table:
> ip1 PERMIT
> ip2 PERMIT
>
> Today, one of my customer send mail via the new ip.
> And this new ip not in the /etc/postfix/mynetworks_table

Was the mail accepted or rejected?

> Can you help me to resolve it.

What did you want to happen?  Please follow instructions at:

http://www.postfix.org/DEBUG_README.html#mail

Post configuration and log information, and explain what it is that
you want to happen.

-- 
Viktor.

Re: Possible reasons for "lost connection after DATA"

2014-09-11 Thread Sean Durkin

Hi Wietse,

Am 11.09.2014 um 13:49 schrieb Wietse Venema:
> What is the distribution of DATA sizes before failure? In your
> example I see numbers around 3kB, 9kB, 12kB.

At the moment, I see these sizes:

- always exactly 17511 bytes from smtp-out-127-*.amazon.com (today, seems to be 
only 3 different hosts trying)
- always exactly 49116 bytes from *.psi.cust-cluster.com (I've seen about 60 
different hosts from there today)
- always exactly 33290 bytes from mail18-*.srv2.de (about a dozen different 
hosts)

It seems those are always the same 3 messages being re-tried constantly (when I 
look at them in the incoming queue folder, it's the same recipient and sender 
and the same message-ID, as far as I can tell). I have problems only with 
messages from these clusters, everything else seems unaffected (at least I 
haven't seen any "lost connection" messages from any other hosts as far as my 
logfiles go back).

Yesterday I had an additional message with exactly 17441 bytes on every try 
before failure from the Amazon-cluster. That one was finally delivered 
completely early this morning, and has since disappeared from the cycle.

FWIW, I have received a handful of messages from the Amazon-cluster that did 
not have any delays/problems yesterday and today, one of them even from one of 
the "problematic" hosts that can't deliver the other message.

> Some failures are triggered by packet content, and may be replaced
> only by replacing hardware that operates marginally. Does the problem
> go away when you
> 
> - Replace the server (either the network card or the whole box)
> 
> - Replace the cable that connects the server to the network switch
> 
> - Replace the network switch that the server is plugged into.
> 
> - Replace the cable that connects the switch to the router
> 
> - Replace the router
> 
> - And so on...
> 
> If you think this is a stupid idea, then you haven't been around
> long enough.

By no means do I think that's stupid. :)
I'm only doing this server stuff "for fun" in my spare time, but my real job is 
in microelectronics and hardware, so I've had my share of mysterious and 
seemingly unexplainable stuff (ISI, crosstalk, low-frequency jitter, ground 
bounce, ESD-induced phenomena, you know the drill...).

Problem is that this box is a rented root server in a data center somwhere, so 
I don't have access to the hardware to try any of that. I can contact support, 
but they of course charge you for everything they do, and as long as I haven't 
ruled out that the reason is just some stupid configuration mistake on my part 
(or a routing/filtering issue at my hosting provider, or Amazon, or...), I 
don't want to start replacing hardware, obviously...

Regards,
Sean

Re: Possible reasons for "lost connection after DATA"

2014-09-11 Thread Viktor Dukhovni

On Thu, Sep 11, 2014 at 03:25:57PM +0200, Sean Durkin wrote:

> I can contact support, but they of course charge you for
> everything they do, and as long as I haven't ruled out that the
> reason is just some stupid configuration mistake on my part (or a
> routing/filtering issue at my hosting provider, or Amazon, or...),
> I don't want to start replacing hardware, obviously...

The Postfix configuration has no impact on the TCP layer, beyond
optionally specifying the TCP window size.  Since it is the TCP
layer that fails, the problem is not related to the Postfix
configuration.

Your PCAP files should demonstrate repeated retransmission of data,
are the ACKs you're sending confirming receipt of packets that are
sent repeatedly?  In that case your ACKs are getting lost?   Is
there a sequence number gap in the data received from the server?
In that case the remote server's data is getting lost.  Does the
capture confirm that window scaling is not in use? ...

-- 
Viktor.

Re: Positive DSN if delay_warning_time is reached?

2014-09-11 Thread Wietse Venema

First, I think this is somewhat academic because many users will
be confused when they receive more than one notification for the
same email message, regardless of the content of that notification.

Historically, Postfix will send a "relayed" notification when the
sender requests "SUCCESS" notification but the remote SMTP server
does not announce DSN support. Otherwise, Postfix delegates the
responsibility for "SUCCESS" notification to the remote SMTP server.

Presently, we have a new feature to send a "relayed" (after delay)
notification when delayed mail leaves the local queue. If the user
requested "SUCCESS" notification, then I think that Postfix should
still delegate that responsibility to the remote SMTP server as it
has done historically.

I could remove the word "successfully" from the bounce daemon's
"success_template" text, but I doubt that it would improve most
user's experience.  The problem is primarily with too many
notifications, not their content.

Wietse

Re: Positive DSN if delay_warning_time is reached?

2014-09-11 Thread A. Schulze



wietse:


First, I think this is somewhat academic because many users will
be confused when they receive more than one notification for the
same email message, regardless of the content of that notification.

right. Users tend to not read such messages :-/


Presently, we have a new feature to send a "relayed" (after delay)
notification when delayed mail leaves the local queue. If the user
requested "SUCCESS" notification, then I think that Postfix should
still delegate that responsibility to the remote SMTP server as it
has done historically.



so do you think the feature is not finally finished? I find it definitive
useful in this existing version. But I think it could be optimised for
these there three cases:

1) sender didn't request "SUCCESS" notification. That is possible  
explicit by NOTIFY=DELAY,FAILURE

   or implicit by not specifying any NOTIFY= at all. ( legacy MUA mode )
   -> the relay deferring a message should send a "relayed" (after delay)
  notification

2) sender did request "SUCCESS" notification
   -> then the relay deferring a message should delegate that responsibility.
  No matter if the message was deferred or not.

3) sender did not request "DELAY" notification
   -> relay should not send a "relayed" (after delay)

Not easy to mix them into correct "if A then B code" ...

You name that a "new feature". I could imagine also a new
bounce type + separate text in bounce.cf

Andreas

Re: Possible reasons for "lost connection after DATA"

2014-09-11 Thread Wietse Venema

Sean Durkin:
> Hi Wietse,
> 
> Am 11.09.2014 um 13:49 schrieb Wietse Venema:
> > What is the distribution of DATA sizes before failure? In your
> > example I see numbers around 3kB, 9kB, 12kB.
> 
> At the moment, I see these sizes:
> 
> - always exactly 17511 bytes from smtp-out-127-*.amazon.com (today, seems to 
> be only 3 different hosts trying)
> - always exactly 49116 bytes from *.psi.cust-cluster.com (I've seen about 60 
> different hosts from there today)
> - always exactly 33290 bytes from mail18-*.srv2.de (about a dozen different 
> hosts)
> 
> It seems those are always the same 3 messages being re-tried
> constantly (when I look at them in the incoming queue folder, it's
> the same recipient and sender and the same message-ID, as far as
> I can tell). I have problems only with messages from these clusters,
> everything else seems unaffected (at least I haven't seen any "lost
> connection" messages from any other hosts as far as my logfiles
> go back).
> 
> Yesterday I had an additional message with exactly 17441 bytes on
> every try before failure from the Amazon-cluster. That one was
> finally delivered completely early this morning, and has since
> disappeared from the cycle.

That increases my suspicion of a data-dependent error - some marginal
cable/switch/router, perhaps some middle box with a memory bit error
that requires a power cycle to clear the problem. If the problem is 
caused by crosstalk defect, then only physical replacement will
solve it.

> Problem is that this box is a rented root server in a data center
> somwhere, so I don't have access to the hardware to try any of
> that. I can contact support, but they of course charge you for
> everything they do, and as long as I haven't ruled out that the
> reason is just some stupid configuration mistake on my part (or a
> routing/filtering issue at my hosting provider, or Amazon, or...),
> I don't want to start replacing hardware, obviously...

Try power cycling.

Wietse

Re: Positive DSN if delay_warning_time is reached?

2014-09-11 Thread Wietse Venema

A. Schulze:
> 
> wietse:
> 
> > First, I think this is somewhat academic because many users will
> > be confused when they receive more than one notification for the
> > same email message, regardless of the content of that notification.
> right. Users tend to not read such messages :-/
> 
> > Presently, we have a new feature to send a "relayed" (after delay)
> > notification when delayed mail leaves the local queue. If the user
> > requested "SUCCESS" notification, then I think that Postfix should
> > still delegate that responsibility to the remote SMTP server as it
> > has done historically.
> 
> 
> so do you think the feature is not finally finished?

Above, I described  the part of Postfix that works as intended. If
the user explicitly requests a SUCCESS notification, then Postfix
should keep delegating that to the remote SMTP server, regardless
of whether the new feature is in effect.

> 3) sender did not request "DELAY" notification
> -> relay should not send a "relayed" (after delay)

I looked at the code, and while the existing "delayed" notifications
can be suppressed with NOTIFY=FAILURE (i.e. without "DELAY"), it
looks like the new "relayed" (after delay) notifications ignore
that.  So that still needs to be fixed.

Wietse

45 matches

Mail list logo