John-Mark,
I'm going to rearrange the conversation a bit in the quotes below:
On 8/26/15 3:32 PM, John-Mark Gurney wrote:
So, I looked at what getsockopt SO_RCVBUF returns, and it returns:
case SO_RCVBUF:
optval = so->so_rcv.sb_hiwat;
Which is NOT S-BMAX, so it looks like OpenSSH only ever gets 66052 bytes
for the buffer...
I believe that explains everything we are seeing. HPN developers
thought SO_RCVBUF would return S-BMAX but it instead returns S-HIWA.
If it's decided to base it's waiting for ack on SO_RCVBUF, then this
is probably where the issue is...
Try setting HPNBufferSize to something resonably large, like 1MB, or
above your bandwidth-delay product to see if this will make a difference..
Per:
Conditions: HPNBufferSize SET, TCPRcvBufPoll enabled, TCPRcvBuf NOT Set
Result: HPN Buffer Size = grows to HPNBufferSize
The buffer will grow up to the maximum size specified here.
Setting HPNBufferSize to 1 MB does not change S-BCNT which remains stuck
at ~60KB. Although I might not necessarily expect a change. From
README.hpn:
HPNBufferSize: This is the default buffer size the HPN functionality
uses when interacting with non-HPN SSH installations.
It would be interesting to know how long from the read of stdin (and is
it really reading stdin in 4k blocks? If so, that should be fixed)
to the write out the socket... Basicly, how long does it take to encrypt
the data...
Note in the above the blocking call to select at time 6.592065 that
takes ~0.1 second. This is the reason my connection is slow. The
content appears to be encrypted... I presume it's an application-level
ssh ack.
I posted the in-context kdump at:
https://www.stankevitz.com/ssh-ktrace.txt
stdin is read in 4k blocks... but each "cycle" reads 3 of these blocks.
I define as "cycle" as the period beginning and ending at a 0.1 second
select(2) sleep.
Looks like encryption of a 4KB block takes 10 microseconds, which means:
for every 0.1 seconds in select(2), .000030 seconds are encrypting.
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
Try increasing these and reporting back... the buf_auto should override
those, but this may be limiting it...
Okay. Now we're getting somewhere.
Increase sendspace on "sending client": no change
Increase recvspace on "sending client": no change
Increase sendspace on "receiving server": no change
Increase recvspace on "receiving server": sending client's S-BCNT
increases proportionally!
I'm going to try Kurts parameters now...
Chris
_______________________________________________
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
