Matthias Schneider:
> I want to share my research about my milter tcp speed issue:
>
>
> I have turned on smtpd -v in master.cf to see the mss in the logfile:
>
> On my ubuntu 18.04 the loopback device has a mtu of 65536 which results
> in a "vstream_tweak_tcp: TCP_MAXSEG 21845 "
> This is slow and has the nagle timeout problem.
Why does the kernel return this nonsensical mss value which is
1/3 of the MTU? 40kB of packet the framing overhead for loopback?
> When I set the mtu to 65535 (or 65534) it still gets TCP_MAXSEG 21845
> (and is still slow).
>
>
> Once i do "ip link set dev lo mtu 21845" i will get vstream_tweak_tcp:
> TCP_MAXSEG 21793 and the performance is great (only 1 second for 100mb
> body).
>
>
> Installing the Milter on another machine (which is available throught
> the eno1 (1500mtu) interface i will get TCP_MAXSEG 1448 and still bad
> performance (43secs instead of 1.30m).
Are you on PPPoE or some other tunnel? I would expect mss=1460
for TCP/IP over ethernet.
> Since this is strange i prefer to switch to unixsocket.
Don't bother fighting with a hostile network stack :-)
Wietse
>
> I also researched where the delay happened in my milter: its while
> waiting for the 64k packet
> https://github.com/mschneider82/milter/blob/b4a6d7c78ac39b8a1d71b53b74bc08d95652d310/session.go#L101
>
>
> Best regards
>
> Matthias Schneider
>
>
>
> Am 19.08.19 um 20:05 schrieb Wietse Venema:
> > Matthias Schneider:
> >> Hi Wietse,
> >>
> >>
> >>> I suspect that the bottleneck is on the receiving side.
> >>>
> >>> - Maybe the loopback stack does not like the 65535 block size. Try
> >>> using an ethernet interface address instead.
> >> I tried switching from interface lo to eno1 - same issue.
> > What's the en01 MTU size? Postfix has code (in vstream_tweak.c
> > which is called by the Milter client and other Postfix code) to
> > automatically increase the VSTREAM buffer from 4096 to the maximal
> > segment size, to avoid Nagle delays, though that would not help if
> > the MTU is > 65535 (the Milter's 'packet' size).
> >
> > I would therefore not expect Nagle delays on en01.
> >
> >> I compared some tcpdumps with postfix as client and my milterclient
> >> (which is fast) as client.
> >>
> >> It seems that we run into nagle timeout (the ACK is always delayed).
> >> Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to
> >> set TCP_NODELAY on the milter connection in postfix?
> > You've got the source.
> >
> > Wietse
>
