W dniu 2010-11-16 16:14, Joel Sing pisze:
On Tuesday 16 November 2010, Robert Lewandowski wrote:
Hello,
PROBLEM: transfer speed is ONLY HALF if queue is defined in pf.conf
although queue is 950Mbit (1000Mbit-5%)
pf disabled: 768 Mbits/sec
pf enabled, queue 950Mbit: 337 Mbits/sec
ANALYSIS:
- OpenBSD 4.8 default intallation.
- Test made between OpenBSD 4.8 and Debian Linux.
(between two Debian systems speed is more than 900Mbit/s)
*********************************************************
LAN interface: Intel PRO/1000 PT Desktop Adapter (PCIe, model:
EXPI9300PTBLK)
DMESG: em0 at pci1 dev 0 function 0 "Intel PRO/1000 PT (82572EI)" rev
0x06: apic 1 int 16 (irq 5), address 00:1b:21:05:1f:39
*********************************************************
Default settings of TCP window size:
net.inet.tcp.recvspace=16384
net.inet.tcp.sendspace=16384
*********************************************************
1a) pf disabled
r...@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
------------------------------------------------------------
Client connecting to 10.0.0.6, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.8 port 27600 connected with 10.0.0.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 54.7 MBytes 459 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 1.0- 2.0 sec 54.7 MBytes 458 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 2.0- 3.0 sec 54.7 MBytes 459 Mbits/sec
1b) pf enabled, no queue
r...@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
------------------------------------------------------------
Client connecting to 10.0.0.6, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.8 port 46912 connected with 10.0.0.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 53.9 MBytes 452 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 1.0- 2.0 sec 52.6 MBytes 441 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 2.0- 3.0 sec 54.1 MBytes 454 Mbits/sec
1c) pf enabled, added queue to default pf.conf:
altq on em0 cbq bandwidth 1Gb queue { q_lan }
queue q_lan bandwidth 950Mb cbq (default)
r...@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
------------------------------------------------------------
Client connecting to 10.0.0.6, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.8 port 38266 connected with 10.0.0.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 33.9 MBytes 284 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 2.0- 3.0 sec 35.8 MBytes 300 Mbits/sec
*********************************************************
TCP window size changed to 131072.
net.inet.tcp.recvspace: 16384 -> 131072
net.inet.tcp.sendspace: 16384 -> 131072
*********************************************************
1a) pf disabled
r...@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
------------------------------------------------------------
Client connecting to 10.0.0.6, TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.8 port 32680 connected with 10.0.0.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 91.5 MBytes 768 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 1.0- 2.0 sec 92.1 MBytes 773 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 2.0- 3.0 sec 91.2 MBytes 765 Mbits/sec
1b) pf enabled, no queue
r...@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
------------------------------------------------------------
Client connecting to 10.0.0.6, TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.8 port 41092 connected with 10.0.0.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 80.5 MBytes 675 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 1.0- 2.0 sec 80.1 MBytes 672 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 2.0- 3.0 sec 80.2 MBytes 673 Mbits/sec
1c) pf enabled, added queue to default pf.conf:
altq on em0 cbq bandwidth 1Gb queue { q_lan }
queue q_lan bandwidth 950Mb cbq (default)
r...@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
------------------------------------------------------------
Client connecting to 10.0.0.6, TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.8 port 12499 connected with 10.0.0.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 40.1 MBytes 337 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 1.0- 2.0 sec 40.1 MBytes 336 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 2.0- 3.0 sec 40.0 MBytes 335 Mbits/sec
*********************************************************
any ideas, suggestions about this situation?
The default length for a queue is 50 packets - this only allows you to queue
around 75,000 bytes and the burstiness of TCP slow-start is likely to well
exceed this in your configuration (due to the BDP). I'd suggest increasing
the queue length - also run 'pfctl -vvs queue' or 'systat queue' and see
what's happening with regards to packets drops.
ok, a set qlimit to 200 and then 500, no difference
queue root_em0 on em0 bandwidth 1Gb priority 0 qlimit 500 cbq( wrr root
) {q_lan}
[ pkts: 858820 bytes: 1300055838 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/500 borrows: 0 suspends: 0 ]
[ measured: 29222.9 packets/s, 353.90Mb/s ]
queue q_lan on em0 bandwidth 950Mb qlimit 500 cbq( borrow default )
[ pkts: 858820 bytes: 1300055838 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/500 borrows: 0 suspends: 0 ]
[ measured: 29222.9 packets/s, 353.90Mb/s ]
best regards,
RLW
