Re: Why Postfix always complain "unexpected EOF" with my own tcp_table program?
On 28/07/13 08:27, Zhang Huangbin wrote:
> Dear all,
>
> I wrote a simple daemon service in Python, it's used in Postfix
> transport_maps like this:
>
> transport_maps = tcp:127.0.0.1:1234
>
> It always returns '200 my_transport\n' as described in Postfix manual page
> tcp_table(5), but Postfix always complains "unexpected EOF" like below:
>
> Jul 27 22:51:53 d7 postfix/trivial-rewrite[4260]: warning: read TCP map reply
> from 127.0.0.1:1234: unexpected EOF (Success)
>
> This Python daemon server uses 'asynchat' module, and return '200
> my_transport\n' with 'async_chat.push()' method like this:
>
> self.push('200 my_transport\n')
>
> Any idea why Postfix always complain "unexpected EOF"?
>
> Thanks for your time.
There may be something in your code that strips out \n before sending to
Postfix. Postfix gives this message
when the string it receives does not end in \n.
John
minor typo in log message
I noticed this minor typo in a log message in util/vstring_vstream.c in
function vstring_get_null_bound
I guess
if (bound <= 0)
msg_panic("vstring_get_nonl_bound: invalid bound %ld", (long)
bound);
should be
if (bound <= 0)
msg_panic("vstring_get_null_bound: invalid bound %ld", (long)
bound);
John
Reducing SPAM connections
Hi I am receiving a considerable amount of connection from SPAM hosts and I'm starting to wonder if this is normal ? Also, could setting up a spamd help decrease number of connections ? Thanks for any advice. Carsten S. ... ** Detail (10) * 109 Miscellaneous warnings -- 10 hostname host-46-241-156-54.orangearmenia.am does not resolve to address 46.241.156... 9 hostname 234-240-137-186.fibertel.com.ar does not resolve to address 186.137.240.23... 5 hostname 178.91.44.246.megaline.telecom.kz does not resolve to address 178.91.44.24... 5 hostname 88.249.200.82.dynamic.ttnet.com.tr does not resolve to address 88.249.200 5 hostname 92.47.187.169.megaline.telecom.kz does not resolve to address 92.47.187.16... 5 hostname Dynamic-IP-181540180245.cable.net.co does not resolve to address 181.54.18... 5 hostname Dynamic-IP-18687214135.cable.net.co does not resolve to address 186.87.214... 5 hostname customer-PUE-156-153.megared.net.mx does not resolve to address 189.192.15... 3 hostname 200-127-26-71.cab.prima.net.ar does not resolve to address 200.127.26.71: ... 3 hostname customer-GDL-140-83.megared.net.mx does not resolve to address 187.245.140... 2 hostname 53.red.141.225.212.user.ptvtelecom.com does not resolve to address 212.225... 2 hostname client-201.240.116.69.speedy.net.pe does not resolve to address 201.240.11... 2 hostname customer-CLN-223-16.megared.net.mx does not resolve to address 187.243.223... 2 hostname static-adsl190-250-72-98.une.net.co does not resolve to address 190.250.72... 1 hostname 1.116.166.190.f.sta.codetel.net.do does not resolve to address 190.166.116... 1 hostname 121-60-188-190.cab.prima.net.ar does not resolve to address 190.188.60.121... 1 hostname 126.26.166.190.f.sta.codetel.net.do does not resolve to address 190.166.26... 1 hostname 154.25.166.190.f.sta.codetel.net.do does not resolve to address 190.166.25... 1 hostname 155-60-194-190.cab.prima.net.ar does not resolve to address 190.194.60.155... 1 hostname 178.89.57.40.megaline.telecom.kz does not resolve to address 178.89.57.40:... 1 hostname 178.90.226.107.megaline.telecom.kz does not resolve to address 178.90.226 1 hostname 186.194-5-146.vtc-vegastelecom.com.br does not resolve to address 186.194 1 hostname 187-75-189-17.dsl.telesp.net.br does not resolve to address 187.75.189.17:... 1 hostname 188.subnet118-97-212.static.astinet.telkom.net.id does not resolve to addr... 1 hostname 190-172-131-240.speedy.com.ar does not resolve to address 190.172.131.240:... 1 hostname 190-174-238-122.speedy.com.ar does not resolve to address 190.174.238.122:... 1 hostname 192-211-52-68-customer-incero.com does not resolve to address 192.211.52.6... 1 hostname 202-253-194-190.cab.prima.net.ar does not resolve to address 190.194.253.2... 1 hostname 202.subnet118-97-168.static.astinet.telkom.net.id does not resolve to addr... 1 hostname 208-110-35-183.yourlink.ca does not resolve to address 208.110.35.183: hos... 1 hostname 235.subnet-103.23.101.host.unnes.ac.id does not resolve to address 103.23 1 hostname 254.226.185.190.cable.dyn.ridsa.com.ar does not resolve to address 190.185... 1 hostname 4-30-110-190-hou.servercraft.co does not resolve to address 4.30.110.190: ... 1 hostname 5-57-13-190.elcat.kg does not resolve to address 5.57.13.190: hostname nor... 1 hostname 62.56.143.58.satcom-systems.net does not resolve to address 62.56.143.58: ... 1 hostname 81.213.153.125.static.ttnet.com.tr does not resolve to address 81.213.153 1 hostname 88.250.201.15.static.ttnet.com.tr does not resolve to address 88.250.201.1... 1 hostname 95.57.252.178.megaline.telecom.kz does not resolve to address 95.57.252.17... 1 hostname 95.57.72.79.megaline.telecom.kz does not resolve to address 95.57.72.79: h... 1 hostname 95.58.40.170.megaline.telecom.kz does not resolve to address 95.58.40.170:... 1 hostname 95.59.123.227.megaline.telecom.kz does not resolve to address 95.59.123.22... 1 hostname ABTS-North-Static-155.86.160.122.airtelbroadband.in does not resolve to ad... 1 hostname CUST-114.215.102.5.018.net.il does not resolve to address 5.102.215.114: h... 1 hostname Dynamic-IP-18150087227.cable.net.co does not resolve to address 181.50.87 1 hostname Dynamic-IP-181530105149.cable.net.co does not re
Re: minor typo in log message
John Fawcett:
> I noticed this minor typo in a log message in util/vstring_vstream.c in
> function vstring_get_null_bound
>
> I guess
>
> if (bound <= 0)
> msg_panic("vstring_get_nonl_bound: invalid bound %ld", (long)
> bound);
>
> should be
>
> if (bound <= 0)
> msg_panic("vstring_get_null_bound: invalid bound %ld", (long)
> bound);
Thanks, you found a cut-and-paste error.
Wietse
Re: Why Postfix always complain "unexpected EOF" with my own tcp_table program?
Zhang Huangbin:
> Dear all,
>
> I wrote a simple daemon service in Python, it's used in Postfix
> transport_maps like this:
>
> transport_maps = tcp:127.0.0.1:1234
>
> It always returns '200 my_transport\n' as described in Postfix manual page
> tcp_table(5), but Postfix always complains "unexpected EOF" like below:
>
> Jul 27 22:51:53 d7 postfix/trivial-rewrite[4260]: warning: read TCP map reply
> from 127.0.0.1:1234: unexpected EOF (Success)
>
> This Python daemon server uses 'asynchat' module, and return '200
> my_transport\n' with 'async_chat.push()' method like this:
>
> self.push('200 my_transport\n')
>
> Any idea why Postfix always complain "unexpected EOF"?
1) Use a network sniffer to see what Python actually sends. You may
assume that your program sends \n, but Postfix does not receive \n.
2) Unrelated to this bug: closing the connection after one request
is inefficient.
Wietse
Re: Reducing SPAM connections
On Sun, Jul 28, 2013 at 02:07:46PM +0200, Carsten S. wrote: > I am receiving a considerable amount of connection from SPAM hosts > and I'm starting to wonder if this is normal ? For many years now, the vast majority of Internet mail traffic is spam and abuse. Yes, it's normal. > Also, could setting up a spamd help decrease number of connections ? Postfix has its own postscreen(8), which was influenced in design by spamd. Well worth the upgrade if you are not yet on 2.8+. http://www.postfix.org/POSTSCREEN_README.html http://rob0.nodns4.us/postscreen.html -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
Re: Why Postfix always complain "unexpected EOF" with my own tcp_table program?
On Jul 28, 2013 7:25 AM, "Wietse Venema" wrote:
>
> Zhang Huangbin:
> > Dear all,
> >
> > I wrote a simple daemon service in Python, it's used in Postfix
transport_maps like this:
> >
> > transport_maps = tcp:127.0.0.1:1234
> >
> > It always returns '200 my_transport\n' as described in Postfix manual
page tcp_table(5), but Postfix always complains "unexpected EOF" like below:
> >
> > Jul 27 22:51:53 d7 postfix/trivial-rewrite[4260]: warning: read TCP map
reply from 127.0.0.1:1234: unexpected EOF (Success)
> >
> > This Python daemon server uses 'asynchat' module, and return '200
my_transport\n' with 'async_chat.push()' method like this:
> >
> > self.push('200 my_transport\n')
> >
> > Any idea why Postfix always complain "unexpected EOF"?
>
> 1) Use a network sniffer to see what Python actually sends. You may
>assume that your program sends \n, but Postfix does not receive \n.
>
> 2) Unrelated to this bug: closing the connection after one request
>is inefficient.
But is it the fact that he closes the connection? Keep in mind that any
time a network connection is closed it sends EOF. Could it be that
Postfix wants to keep a constant connection?
Sam
> Wietse
Re: Why Postfix always complain "unexpected EOF" with my own tcp_table program?
On Sunday, July 28, 2013 at 8:24 PM, Wietse Venema wrote: > > 1) Use a network sniffer to see what Python actually sends. You may > assume that your program sends \n, but Postfix does not receive \n. Thanks Wietse, and John. I think this is the root cause, will try a network sniffer later. > 2) Unrelated to this bug: closing the connection after one request > is inefficient. My program closes the connection immediately.
Re: Why Postfix always complain "unexpected EOF" with my own tcp_table program?
Zhang Huangbin: > On Sunday, July 28, 2013 at 8:24 PM, Wietse Venema wrote: > > > 1) Use a network sniffer to see what Python actually sends. You may > > assume that your program sends \n, but Postfix does not receive \n. > > Thanks Wietse, and John. > I think this is the root cause, will try a network sniffer later. > > 2) Unrelated to this bug: closing the connection after one request > > is inefficient. > > My program closes the connection immediately. Caution: it is not a good idea to close a connection immediately after writing data to it. Some TCP/IP implementations may discard unsent data when a connection is closed. In Postfix, I avoid "write-close" bugs by keeping a connection open until the receiver closes it (with of course a time limit that forces the connection to be closed when things take too long). Wietse
smtpd_client_restrictions = reject_unauth_pipelining weirdness
Am I misunderstanding something here, that setting 'smtpd_client_restrictions = reject_unauth_pipelining' should reject a client that sends the EHLO, or HELO before the smtp banner? (http://www.postfix.org/postconf.5.html#reject_unauth_pipelining: 'Reject the request when the client sends SMTP commands ahead of time where it is not allowed, ...') In an effort to test this out (and following the example provided at http://www.postfix.org/postconf.5.html#smtpd_client_restrictions !): /etc/postfix/main.cf: smtpd_client_restrictions = sleep 1, reject_unauth_pipelining smtpd_delay_reject = no # ( echo -e 'helo f'; sleep 0.5; echo -e 'mail from:'; sleep 0.5; echo 'rcpt to:' ) | nc 127.0.0.1 25 220 xx.com ESMTP xx 250 xx.com 250 2.1.0 Ok 250 2.1.5 Ok # I've even tried this with 'smtpd_delay_reject = yes', with the same result. Now you'll notice that a sleep is inserted between each command - 'helo', 'mail', and 'rcpt'. What is interesting, though, is what happens when I take out the delay between the 'helo' and the 'mail': with smtpd_delay_reject = no : # ( echo -e 'helo f'; echo -e 'mail from:'; sleep 0.5; echo 'rcpt to:' ) | nc 127.0.0.1 25 220 xx.com ESMTP xx 250 xx.com 250 2.1.0 Ok 250 2.1.5 Ok # with smtpd_delay_reject = yes, however???: # ( echo -e 'helo f'; echo -e 'mail from:'; sleep 0.5; echo 'rcpt to:' ) | nc 127.0.0.1 25 220 xx.com ESMTP xx 250 xx.com 250 2.1.0 Ok 503 5.5.0 : Client host rejected: Improper use of SMTP command pipelining # (Note that I get the same results even if I take out the 'sleep 1' from smtpd_client_restrictions) And so it seems that I'm forced to come to the following conclusions: 1. 'smtpd_delay_reject = no' messes up 'smtpd_client_restrictions = reject_unauth_pipelining' somehow 2. early ehlo, or helo does not register as "unauth pipelining" 3. no delay between the 'helo' and 'mail' commands = command pipelining, but only if 'smtpd_delay_reject = yes'! I am running postfix 2.9.6. -jf -- He who settles on the idea of the intelligent man as a static entity only shows himself to be a fool. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." --Richard Stallman
Re: smtpd_client_restrictions = reject_unauth_pipelining weirdness
Jeffrey 'jf' Lim: > Am I misunderstanding something here, that setting > 'smtpd_client_restrictions = reject_unauth_pipelining' should reject a > client that sends the EHLO, or HELO before the smtp banner? > (http://www.postfix.org/postconf.5.html#reject_unauth_pipelining: > 'Reject the request when the client sends SMTP commands ahead of time > where it is not allowed, ...') Current reject_unauth_pipelining implementations reject clients that pipeline input that *follows* EHLO/HELO and later commands. They don't reject clients that talk before Postfix greets them. To reject clients that talk before Postfix greets them, use Postscreen's pregreet detection feature. Wietse
Re: smtpd_client_restrictions = reject_unauth_pipelining weirdness
On Mon, Jul 29, 2013 at 3:56 AM, Wietse Venema wrote: > Jeffrey 'jf' Lim: >> Am I misunderstanding something here, that setting >> 'smtpd_client_restrictions = reject_unauth_pipelining' should reject a >> client that sends the EHLO, or HELO before the smtp banner? >> (http://www.postfix.org/postconf.5.html#reject_unauth_pipelining: >> 'Reject the request when the client sends SMTP commands ahead of time >> where it is not allowed, ...') > > Current reject_unauth_pipelining implementations reject clients > that pipeline input that *follows* EHLO/HELO and later commands. > They don't reject clients that talk before Postfix greets them. > > To reject clients that talk before Postfix greets them, use > Postscreen's pregreet detection feature. > ok, thanks. How about my problems with setting 'smtpd_delay_reject = no'? It just seems that with it smtpd_delay_reject set to 'no', the rejection just isn't done (or detected?), for whatever reason. -jf
Re: smtpd_client_restrictions = reject_unauth_pipelining weirdness
Jeffrey 'jf' Lim: > On Mon, Jul 29, 2013 at 3:56 AM, Wietse Venema wrote: > > Jeffrey 'jf' Lim: > >> Am I misunderstanding something here, that setting > >> 'smtpd_client_restrictions = reject_unauth_pipelining' should reject a > >> client that sends the EHLO, or HELO before the smtp banner? > >> (http://www.postfix.org/postconf.5.html#reject_unauth_pipelining: > >> 'Reject the request when the client sends SMTP commands ahead of time > >> where it is not allowed, ...') > > > > Current reject_unauth_pipelining implementations reject clients > > that pipeline input that *follows* EHLO/HELO and later commands. > > They don't reject clients that talk before Postfix greets them. > > > > To reject clients that talk before Postfix greets them, use > > Postscreen's pregreet detection feature. > > > > ok, thanks. How about my problems with setting 'smtpd_delay_reject = > no'? It just seems that with it smtpd_delay_reject set to 'no', the > rejection just isn't done (or detected?), for whatever reason. Allow me to repeat my reply above: Current reject_unauth_pipelining implementations [...] don't reject clients that talk before Postfix greets them. To reject clients that talk before Postfix greets them, use Postscreen's pregreet detection feature. Wietse
Re: smtpd_client_restrictions = reject_unauth_pipelining weirdness
On Mon, Jul 29, 2013 at 4:13 AM, Wietse Venema wrote: > Jeffrey 'jf' Lim: >> On Mon, Jul 29, 2013 at 3:56 AM, Wietse Venema wrote: >> > Jeffrey 'jf' Lim: >> >> Am I misunderstanding something here, that setting >> >> 'smtpd_client_restrictions = reject_unauth_pipelining' should reject a >> >> client that sends the EHLO, or HELO before the smtp banner? >> >> (http://www.postfix.org/postconf.5.html#reject_unauth_pipelining: >> >> 'Reject the request when the client sends SMTP commands ahead of time >> >> where it is not allowed, ...') >> > >> > Current reject_unauth_pipelining implementations reject clients >> > that pipeline input that *follows* EHLO/HELO and later commands. >> > They don't reject clients that talk before Postfix greets them. >> > >> > To reject clients that talk before Postfix greets them, use >> > Postscreen's pregreet detection feature. >> > >> >> ok, thanks. How about my problems with setting 'smtpd_delay_reject = >> no'? It just seems that with it smtpd_delay_reject set to 'no', the >> rejection just isn't done (or detected?), for whatever reason. > > Allow me to repeat my reply above: > > Current reject_unauth_pipelining implementations [...] don't reject > clients that talk before Postfix greets them. > > To reject clients that talk before Postfix greets them, use > Postscreen's pregreet detection feature. > Yes, I got that. I also highlighted another question/issue I have in the 2nd part of my question, where the pipelining occurs *after* ehlo/helo. In that case, smtpd_delay_reject set to 'no' does not work. Should that be expected behaviour? -jf
Re: smtpd_client_restrictions = reject_unauth_pipelining weirdness
Jeffrey 'jf' Lim: > > Allow me to repeat my reply above: > > > > Current reject_unauth_pipelining implementations [...] don't reject > > clients that talk before Postfix greets them. > > > > To reject clients that talk before Postfix greets them, use > > Postscreen's pregreet detection feature. > > > > Yes, I got that. > > I also highlighted another question/issue I have in the 2nd part of my > question, where the pipelining occurs *after* ehlo/helo. In that case, > smtpd_delay_reject set to 'no' does not work. Should that be expected > behaviour? That's a bug. As of Postfix 2.6, reject_unauth_pipelining works only after the Postfix SMTP server has read input. I am currently too busy with real work to fix that. If you must block clients that talk too soon, use postscreen. It does a much better job, and it even has a trick to make buggy clients talk too soon. Wietse
Re: smtpd_client_restrictions = reject_unauth_pipelining weirdness
On Mon, Jul 29, 2013 at 4:51 AM, Wietse Venema wrote: > Jeffrey 'jf' Lim: >> > Allow me to repeat my reply above: >> > >> > Current reject_unauth_pipelining implementations [...] don't reject >> > clients that talk before Postfix greets them. >> > >> > To reject clients that talk before Postfix greets them, use >> > Postscreen's pregreet detection feature. >> > >> >> Yes, I got that. >> >> I also highlighted another question/issue I have in the 2nd part of my >> question, where the pipelining occurs *after* ehlo/helo. In that case, >> smtpd_delay_reject set to 'no' does not work. Should that be expected >> behaviour? > > That's a bug. As of Postfix 2.6, reject_unauth_pipelining works > only after the Postfix SMTP server has read input. I am currently > too busy with real work to fix that. > I see. Thanks for the confirmation! > If you must block clients that talk too soon, use postscreen. It > does a much better job, and it even has a trick to make buggy > clients talk too soon. > gotcha. thanks, -jf -- He who settles on the idea of the intelligent man as a static entity only shows himself to be a fool. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." --Richard Stallman
Re: Why Postfix always complain "unexpected EOF" with my own tcp_table program?
You can enable the debug mode of postfix smtp daemon ( http://www.postfix.org/DEBUG_README.html). Then you can see the request and response which postfix talked with your tcp_table script. PS: From the postfix source code, it seems postfix has not received any response from your tcp_table. However postfix will retry 10 times for such scenario (sleep 1 sec before each retry). 2013/7/28 Wietse Venema > Zhang Huangbin: > > On Sunday, July 28, 2013 at 8:24 PM, Wietse Venema wrote: > > > > > 1) Use a network sniffer to see what Python actually sends. You may > > > assume that your program sends \n, but Postfix does not receive \n. > > > > Thanks Wietse, and John. > > I think this is the root cause, will try a network sniffer later. > > > 2) Unrelated to this bug: closing the connection after one request > > > is inefficient. > > > > My program closes the connection immediately. > > Caution: it is not a good idea to close a connection immediately > after writing data to it. Some TCP/IP implementations may discard > unsent data when a connection is closed. > > In Postfix, I avoid "write-close" bugs by keeping a connection open > until the receiver closes it (with of course a time limit that > forces the connection to be closed when things take too long). > > Wietse >
