This are the most recent stats. No advances so far. The system has
-Current right now.
Any help or feedback would be appreciated.
Hardware Configuration:
Dell PowerEdge R530 with 2 Intel(R) Xeon(R) E52695 CPU's, 18 cores per
cpu. Equipped with a Chelsio T-580-CR dual port in an 8x slot.
BIOS tweaks:
Hyperthreading (or Logical Processors) is turned off.
loader.conf
# Chelsio Modules
t4fw_cfg_load="YES"
t5fw_cfg_load="YES"
if_cxgbe_load="YES"
rc.conf
# Gateway Configuration
ifconfig_cxl0="inet 172.16.1.1/24"
ifconfig_cxl1="inet 172.16.2.1/24"
gateway_enable="YES"
Last Results:
packets errs idrops bytes packets errs bytes colls drops
2.7M 0 2.0M 1.4G 696k 0 368M 0 0
2.7M 0 2.0M 1.4G 686k 0 363M 0 0
2.6M 0 2.0M 1.4G 668k 0 353M 0 0
2.7M 0 2.0M 1.4G 661k 0 350M 0 0
2.8M 0 2.1M 1.5G 697k 0 369M 0 0
2.8M 0 2.1M 1.4G 684k 0 361M 0 0
2.7M 0 2.1M 1.4G 674k 0 356M 0 0
root@router1:~ # vmstat -i
interrupt total rate
irq9: acpi0 73 0
irq18: ehci0 ehci1 1155973 3
cpu0:timer 3551157 10
cpu29:timer 9303048 27
cpu9:timer 71693455 207
cpu16:timer 9798380 28
cpu18:timer 9287094 27
cpu26:timer 9342495 27
cpu20:timer 9145888 26
cpu8:timer 9791228 28
cpu22:timer 9288116 27
cpu35:timer 9376578 27
cpu30:timer 9396294 27
cpu23:timer 9248760 27
cpu10:timer 9756455 28
cpu25:timer 9300202 27
cpu27:timer 9227291 27
cpu14:timer 10083548 29
cpu28:timer 9325684 27
cpu11:timer 9906405 29
cpu34:timer 9419170 27
cpu31:timer 9392089 27
cpu33:timer 9350540 27
cpu15:timer 9804551 28
cpu32:timer 9413182 27
cpu19:timer 9231505 27
cpu12:timer 9813506 28
cpu13:timer 10872130 31
cpu4:timer 9920237 29
cpu2:timer 9786498 28
cpu3:timer 9896011 29
cpu5:timer 9890207 29
cpu6:timer 9737869 28
cpu7:timer 9790119 28
cpu1:timer 9847913 28
cpu21:timer 9192561 27
cpu24:timer 9300259 27
cpu17:timer 9786186 28
irq264: mfi0 151818 0
irq266: bge0 30466 0
irq272: t5nex0:evt 4 0
Total 402604945 1161
top -PHS
last pid: 18557; load averages: 2.58, 1.90, 0.95 up 4+00:39:54 18:30:46
231 processes: 40 running, 126 sleeping, 65 waiting
CPU 0: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 6: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% idle
CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 8: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 9: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 10: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 11: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 12: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 13: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 14: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 15: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 16: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 17: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 18: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 19: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 20: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 21: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 22: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 23: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 24: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 25: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 26: 0.0% user, 0.0% nice, 0.0% system, 59.6% interrupt, 40.4% idle
CPU 27: 0.0% user, 0.0% nice, 0.0% system, 96.3% interrupt, 3.7% idle
CPU 28: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 29: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 30: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 31: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 32: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 33: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 34: 0.0% user, 0.0% nice, 0.0% system, 100% interrupt, 0.0% idle
CPU 35: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
Mem: 15M Active, 224M Inact, 1544M Wired, 393M Buf, 29G Free
Swap: 3881M Total, 3881M Free
pmcstat -R sample.out -G - | head
@ CPU_CLK_UNHALTED_CORE [159 samples]
39.62% [63] acpi_cpu_idle_mwait @ /boot/kernel/kernel
100.0% [63] acpi_cpu_idle
100.0% [63] cpu_idle_acpi
100.0% [63] cpu_idle
100.0% [63] sched_idletd
100.0% [63] fork_exit
17.61% [28] cpu_idle @ /boot/kernel/kernel
root@router1:~ # pmcstat -R sample0.out -G - | head
@ CPU_CLK_UNHALTED_CORE [750 samples]
31.60% [237] acpi_cpu_idle_mwait @ /boot/kernel/kernel
100.0% [237] acpi_cpu_idle
100.0% [237] cpu_idle_acpi
100.0% [237] cpu_idle
100.0% [237] sched_idletd
100.0% [237] fork_exit
10.67% [80] cpu_idle @ /boot/kernel/kernel
On 03/01/17 13:46, Slawa Olhovchenkov wrote:
On Tue, Jan 03, 2017 at 12:35:42PM -0400, Jordan Caraballo wrote:
We recently tested a Dell R530 with a Chelsio T580 card, under FreeBSD 10.3,
11.0, -STABLE and -CURRENT, and Centos 7.
Based on our research, including netmap-fwd and with the routing improvements
project (https://wiki.freebsd.org/ProjectsRoutingProposal),
we hoped for packets-per-second (pps) in the 5+ million range, or even higher.
Based on prior testing (http://marc.info/?t=140604252400002&r=1&w=2), we
expected 3-4 Million to be easily obtainable.
Unfortunately, our current results top out at no more than 1.5 M (64 bytes
length packets) with FreeBSD, and
surprisingly around 3.2 M (128 bytes length packets) with Centos 7, and we are
at a loss as to why.
Server Description:
Dell PowerEdge R530 with 2 Intel(R) Xeon(R) E52695 CPU's, 18 cores per
cpu. Equipped with a Chelsio T-580-CR dual port in an 8x slot.
** Can this be a lack in support issue related to the R530's hardware? **
Any help appreciated!
What hardware configuration?
What BIOS setting?
What loader.conf/sysctl.conf setting?
What `vmstat -i`?
What `top -PHS`?
what
====
pmcstat -S CPU_CLK_UNHALTED_CORE -l 10 -O sample.out
pmcstat -R sample.out -G out.txt
pmcstat -c 0 -S CPU_CLK_UNHALTED_CORE -l 10 -O sample0.out
pmcstat -R sample0.out -G out0.txt
====
_______________________________________________
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
