Re: ssh over WAN: TCP window too small

2015-08-28 Thread Chris Stankevitz
On 8/25/15 3:47 PM, Chris Stankevitz wrote: Can anyone explain my abysmally small TCP window? So I believe this is the story: 1. openssh limits the size of some outgoing buffer to 65KB 2. openssh/HPN tries to improve on this by increasing the size of the outgoing buffer to match getsockopt(S

Re: ssh over WAN: TCP window too small

2015-08-28 Thread Chris Stankevitz
On 8/27/15 6:14 AM, Kurt Lidl wrote: # tcp options for long-haul speedups kern.ipc.maxsockbuf=4194304 # (2 * default 2097152) net.inet.tcp.mssdflt=1448 # (default 576) net.inet.tcp.sendbuf_max=4194304# (2 * default 2097152) net.inet.tcp.recvbuf_max=4194304

Re: ssh over WAN: TCP window too small

2015-08-28 Thread Chris Stankevitz
On 8/26/15 5:46 PM, David DeSimone wrote: On 8/26/15 1:24 AM, John-Mark Gurney wrote: 94146 ssh 6.686140 CALL read(0x4,0x7fff6c70,0x4000) 94146 ssh 6.686154 GIO fd 4 read 4096 bytes [ read of stdin (/dev/zero) snipped) It would be interesting to know how long from th

Re: ssh over WAN: TCP window too small

2015-08-28 Thread Chris Stankevitz
John-Mark, I'm going to rearrange the conversation a bit in the quotes below: On 8/26/15 3:32 PM, John-Mark Gurney wrote: So, I looked at what getsockopt SO_RCVBUF returns, and it returns: case SO_RCVBUF: optval = so->so_rcv.sb_hiwat; Which is NOT S-

Re: ssh over WAN: TCP window too small

2015-08-27 Thread Kevin Oberman
On Tue, Aug 25, 2015 at 2:12 PM, Chris Stankevitz wrote: > On 8/25/15 4:11 PM, Bjoern A. Zeeb wrote: > >> >> On 25 Aug 2015, at 22:47 , Chris Stankevitz wrote: >>> >>> Can anyone recommend some tools/tricks to figure out what in FreeBSD >>> and/or >>> >> >> base SSH is limiting the send/recv buf

Re: ssh over WAN: TCP window too small

2015-08-27 Thread Kurt Lidl
Chris Stankevitz wrote: Hi, # cat /dev/urandom | ssh root at host 'cat > /dev/null' I use the above ssh command over a high-BDP WAN link (80 ms @ 100 Mbps). tcpdump shows I am TCP window limited to 64 KBytes (yielding 5 Mbps). iperf with default options gets the window opened to 500 KBytes

RE: ssh over WAN: TCP window too small

2015-08-26 Thread David DeSimone
On 8/26/15 1:24 AM, John-Mark Gurney wrote: > > 94146 ssh 6.686140 CALL read(0x4,0x7fff6c70,0x4000) > > 94146 ssh 6.686154 GIO fd 4 read 4096 bytes > >[ read of stdin (/dev/zero) snipped) > > It would be interesting to know how long from the read of stdin (and is > it reall

Re: ssh over WAN: TCP window too small

2015-08-26 Thread Jan Mikkelsen
Hi, > On 26 Aug 2015, at 08:47, Chris Stankevitz wrote: > > Hi, > > # cat /dev/urandom | ssh root@host 'cat > /dev/null' > > I use the above ssh command over a high-BDP WAN link (80 ms @ 100 Mbps). > tcpdump shows I am TCP window limited to 64 KBytes (yielding 5 Mbps). iperf > with default

Re: ssh over WAN: TCP window too small

2015-08-26 Thread John-Mark Gurney
Chris Stankevitz wrote this message on Wed, Aug 26, 2015 at 14:30 -0700: > Thanks again. I appreciate you teaching me "how to fish". I basically > spent all morning reading kdump output. No worries, glad you're learning... > On 8/26/15 1:24 AM, John-Mark Gurney wrote: > > It really looks like

Re: ssh over WAN: TCP window too small

2015-08-26 Thread Gary Palmer
On Wed, Aug 26, 2015 at 02:30:07PM -0700, Chris Stankevitz wrote: > > ktrace -i ssh ... > > Thank you, this is awesome. Is there a way for me to ktrace 'b' in this > example: `a | b | c`? I tried `ktrace a | b | c` and `ktrace test.sh` > (which included a|b|c) but neither seemed to work. I'm

Re: ssh over WAN: TCP window too small

2015-08-26 Thread Chris Stankevitz
John-Mark, Thanks again. I appreciate you teaching me "how to fish". I basically spent all morning reading kdump output. On 8/26/15 1:24 AM, John-Mark Gurney wrote: > It really looks like we should set TCPRcvBufPoll by default on > FreeBSD... According to /etc/ssh/sshd_config, TCPRcvBufPoll

Re: ssh over WAN: TCP window too small

2015-08-26 Thread Mark Martinec
Chris Stankevitz wrote: # cat /dev/urandom | ssh root@host 'cat > /dev/null' I use the above ssh command over a high-BDP WAN link (80 ms @ 100 Mbps). tcpdump shows I am TCP window limited to 64 KBytes (yielding 5 Mbps). iperf with default options gets the window opened to 500 KBytes (yieldin

Re: ssh over WAN: TCP window too small

2015-08-26 Thread John-Mark Gurney
Chris Stankevitz wrote this message on Tue, Aug 25, 2015 at 19:55 -0700: > John-Mark, > > Thank you for your reply. > > On 8/25/15 6:03 PM, John-Mark Gurney wrote: > > Chris Stankevitz wrote this message on Tue, Aug 25, 2015 at 15:47 -0700: > >> # cat /dev/urandom | ssh root@host 'cat > /dev/null

Re: ssh over WAN: TCP window too small

2015-08-25 Thread Chris Stankevitz
John-Mark, Thank you for your reply. On 8/25/15 6:03 PM, John-Mark Gurney wrote: Chris Stankevitz wrote this message on Tue, Aug 25, 2015 at 15:47 -0700: # cat /dev/urandom | ssh root@host 'cat > /dev/null' Don't use this for testing... use /dev/zero or some other device that can produce dat

Re: ssh over WAN: TCP window too small

2015-08-25 Thread John-Mark Gurney
Chris Stankevitz wrote this message on Tue, Aug 25, 2015 at 15:47 -0700: > # cat /dev/urandom | ssh root@host 'cat > /dev/null' Don't use this for testing... use /dev/zero or some other device that can produce data faster than this... > I use the above ssh command over a high-BDP WAN link (80 ms

Re: ssh over WAN: TCP window too small

2015-08-25 Thread Chris Stankevitz
On 8/25/15 4:11 PM, Bjoern A. Zeeb wrote: On 25 Aug 2015, at 22:47 , Chris Stankevitz wrote: Can anyone recommend some tools/tricks to figure out what in FreeBSD and/or >> base SSH is limiting the send/recv buffer and/or TCP window? if you have the memory, try these sysctls: kern.ipc.maxs

Re: ssh over WAN: TCP window too small

2015-08-25 Thread Bjoern A. Zeeb
> On 25 Aug 2015, at 22:47 , Chris Stankevitz wrote: > > Hi, > > # cat /dev/urandom | ssh root@host 'cat > /dev/null' > > I use the above ssh command over a high-BDP WAN link (80 ms @ 100 Mbps). > tcpdump shows I am TCP window limited to 64 KBytes (yielding 5 Mbps). iperf > with default op

ssh over WAN: TCP window too small

2015-08-25 Thread Chris Stankevitz
Hi, # cat /dev/urandom | ssh root@host 'cat > /dev/null' I use the above ssh command over a high-BDP WAN link (80 ms @ 100 Mbps). tcpdump shows I am TCP window limited to 64 KBytes (yielding 5 Mbps). iperf with default options gets the window opened to 500 KBytes (yielding 35 Mbps). Both