Re: [9fans] fossil+venti performance question
I've enabled tcp, tcpwin and tcprxmt logs, but there isn't anything very interesting. tcpincoming s 127.0.0.1!53150/127.0.0.1!53150 d 127.0.0.1!17034/127.0.0.1!17034 v 4/4 Also, the issue is definitely related to the loopback. There is no problem when using an address on /dev/ether0. cpu% cat /net/tcp/3/local 192.168.0.100!43125 cpu% cat /net/tcp/3/remote 192.168.0.100!17034 cpu% cat /net/tcp/3/status Established qin 0 qout 0 rq 0.0 srtt 0 mdev 0 sst 1048560 cwin 396560 swin 1048560>>4 rwin 1048560>>4 qscale 4 timer.start 10 timer.count 10 rerecv 0 katimer.start 2400 katimer.count 2106 -- David du Colombier
Re: [9fans] fossil+venti performance question
On 8 May 2015 at 17:13, David du Colombier <0in...@gmail.com> wrote: > Also, the issue is definitely related to the loopback. > There is no problem when using an address on /dev/ether0. > oh. possibly the queue isn't big enough, given the window size. it's using qpass on a Queue with Qmsg and if the queue is full, Blocks will be discarded.
Re: [9fans] fossil+venti performance question
> oh. possibly the queue isn't big enough, given the window size. > it's using qpass on a Queue with Qmsg and if the queue is full, > Blocks will be discarded. I tried to increase the size of the queue, but no luck. -- David du Colombier
Re: [9fans] fossil+venti performance question
I've finally figured out the issue. The slowness issue only appears on the loopback, because it provides a 16384 MTU. There is an old bug in the Plan 9 TCP stack, were the TCP MSS doesn't take account the MTU for incoming connections. I originally fixed this issue in January 2015 for the Plan 9 port on Google Compute Engine. On GCE, there is an unusual 1460 MTU. The Plan 9 TCP stack defines a default 1460 MSS corresponding to a 1500 MTU. Then, the MSS is fixed according to the MTU for outgoing connections, but not incoming connections. On GCE, this issue leads to IP fragmentation, but GCE didn't handle IP fragmentation properly, so the connections were dropped. On the loopback medium, I suppose this is the opposite issue. Since the TCP stack didn't fix the MSS in the incoming connection, the programs sent multiple small 1500 bytes IP packets instead of large 16384 IP packets, but I don't know why it leads to such a slowdown. Here is the patch for the Plan 9 kernel: http://9legacy.org/9legacy/patch/9-tcp-mss.diff And Charles' 9k kernel: http://9legacy.org/9legacy/patch/9k-tcp-mss.diff -- David du Colombier
Re: [9fans] fossil+venti performance question
I confirm - my old performance is back. Thanks very much David. -Steve
Re: [9fans] fossil+venti performance question
On Fri, 08 May 2015 21:24:13 +0200 David du Colombier <0in...@gmail.com> wrote: > On the loopback medium, I suppose this is the opposite issue. > Since the TCP stack didn't fix the MSS in the incoming > connection, the programs sent multiple small 1500 bytes > IP packets instead of large 16384 IP packets, but I don't > know why it leads to such a slowdown. Looking at the first few bytes in each dir of the initial TCP handshake (with tcpdump) I see: 0x: 4500 0030 24da <= from plan9 to freebsd 0x: 4500 0030 d249 4000 <= from freebsd to plan9 Looks like FreeBSD always sets the DF (don't fragment) bit (0x40 in byte 6), while plan9 doesn't (byte 6 is 0x00). May be plan9 should set the DF (don't fragment) bit in the IP header and try to do path MTU discovery? Either by default or under some ctl option.
Re: [9fans] fossil+venti performance question
do we really need to initialize tcb->mss to tcpmtu() in procsyn()? as i see it, procsyn() is called only when tcb->state is Syn_sent, which only should happen for client connections doing a connect, in which case tcpsndsyn() would have initialized tcb->mss already no? -- cinap
Re: [9fans] fossil+venti performance question
> do we really need to initialize tcb->mss to tcpmtu() in procsyn()? > as i see it, procsyn() is called only when tcb->state is Syn_sent, > which only should happen for client connections doing a connect, in > which case tcpsndsyn() would have initialized tcb->mss already no? tcb->mss may still need to be adjusted at this point, as it is when /* our sending max segment size cannot be bigger than what he asked for */ so at worst this does no harm that I can see. Of course, I'm probably least qualified to pick these nits. Lucio.
