Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-27 Thread John Jasen
On 03/24/2017 08:51 PM, Navdeep Parhar wrote: > On 03/24/2017 16:53, Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER > SCIENCE CORP] wrote: >> It looks like netmap is there; however, is there a way of figuring out >> if netmap is being used? > > If you're not running netmap-fwd or some other netmap

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-24 Thread Navdeep Parhar
On 03/24/2017 16:53, Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP] wrote: It looks like netmap is there; however, is there a way of figuring out if netmap is being used? If you're not running netmap-fwd or some other netmap application, it's not being used. You have just 1 txq/

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-24 Thread Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP]
It looks like netmap is there; however, is there a way of figuring out if netmap is being used? root@router1:~ # dmesg | grep cxl cxl0: on t5nex0 cxl0: Ethernet address: 00:07:43:2c:ac:50 cxl0: 16 txq, 8 rxq (NIC) vcxl0: on cxl0 vcxl0: netmap queues/slots: TX 2/1023, RX 2/1024 vcxl0: 1 txq, 1 rx

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-24 Thread Navdeep Parhar
On 03/24/2017 16:07, Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP] wrote: At the time of implementing the vcxl* interfaces we get very bad results. You're probably not using netmap with the vcxl interfaces, and the number of "normal" tx and rx queues is just 2 for these interfac

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-24 Thread Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP]
At the time of implementing the vcxl* interfaces we get very bad results. packets errs idrops bytespackets errs bytes colls drops 629k 4.5k 066M 629k 066M 0 0 701k 5.0k 074M 701k 074M 0 0

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-17 Thread Navdeep Parhar
On Fri, Mar 17, 2017 at 11:43:32PM -0400, John Jasen wrote: > On 03/17/2017 03:32 PM, Navdeep Parhar wrote: > > > On Fri, Mar 17, 2017 at 12:21 PM, John Jasen wrote: > >> Yes. > >> We were hopeful, initially, to be able to achieve higher packet > >> forwarding rates through either netmap-fwd or d

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-17 Thread John Jasen
On 03/17/2017 03:32 PM, Navdeep Parhar wrote: > On Fri, Mar 17, 2017 at 12:21 PM, John Jasen wrote: >> Yes. >> We were hopeful, initially, to be able to achieve higher packet >> forwarding rates through either netmap-fwd or due to enhancements based >> off https://wiki.freebsd.org/ProjectsRouting

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-17 Thread Slawa Olhovchenkov
On Fri, Mar 17, 2017 at 03:21:37PM -0400, John Jasen wrote: > On 03/17/2017 06:08 AM, Slawa Olhovchenkov wrote: > > > On Thu, Mar 16, 2017 at 03:50:42PM -0400, John Jasen wrote: > > > >> As a few points of note, partial resolution, and curiosity: > >> > >> Following down leads that 11-STABLE had

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-17 Thread Navdeep Parhar
On Fri, Mar 17, 2017 at 12:21 PM, John Jasen wrote: > On 03/17/2017 06:08 AM, Slawa Olhovchenkov wrote: > >> On Thu, Mar 16, 2017 at 03:50:42PM -0400, John Jasen wrote: >> >>> As a few points of note, partial resolution, and curiosity: >>> >>> Following down leads that 11-STABLE had tryforward imp

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-17 Thread John Jasen
On 03/17/2017 06:08 AM, Slawa Olhovchenkov wrote: > On Thu, Mar 16, 2017 at 03:50:42PM -0400, John Jasen wrote: > >> As a few points of note, partial resolution, and curiosity: >> >> Following down leads that 11-STABLE had tryforward improvements over >> 11-RELENG, I upgraded. The same tests (24 c

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-17 Thread Slawa Olhovchenkov
On Thu, Mar 16, 2017 at 03:50:42PM -0400, John Jasen wrote: > As a few points of note, partial resolution, and curiosity: > > Following down leads that 11-STABLE had tryforward improvements over > 11-RELENG, I upgraded. The same tests (24 client streams over UDP with > small packets), the system

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-16 Thread John Jasen
As a few points of note, partial resolution, and curiosity: Following down leads that 11-STABLE had tryforward improvements over 11-RELENG, I upgraded. The same tests (24 client streams over UDP with small packets), the system went from passing 1.7m pps to about 2.5m. Following indications from N

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-13 Thread Navdeep Parhar
On Mon, Mar 13, 2017 at 10:13 AM, John Jasen wrote: > On 03/13/2017 01:03 PM, Navdeep Parhar wrote: > >> On Sun, Mar 12, 2017 at 5:35 PM, John Jasen wrote: >>> UDP traffic. dmesg reports 16 txq, 8 rxq -- which is the default for >>> Chelsio. >>> >> I don't recall offhand, but UDP might be using 2

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-13 Thread John Jasen
The issue does not seem to be specific to Chelsio cards. The same tests with Mellanix cards using the mlx4 drivers exhibit similar behaviors and results. On 03/12/2017 06:13 PM, John Jasen wrote: > I think I am able to confirm Mr. Caraballo's findings. > > _

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-13 Thread John Jasen
On 03/13/2017 01:03 PM, Navdeep Parhar wrote: > On Sun, Mar 12, 2017 at 5:35 PM, John Jasen wrote: >> UDP traffic. dmesg reports 16 txq, 8 rxq -- which is the default for >> Chelsio. >> > I don't recall offhand, but UDP might be using 2-tuple hashing by > default and that might affect the distrib

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-13 Thread Navdeep Parhar
On Sun, Mar 12, 2017 at 5:35 PM, John Jasen wrote: > > UDP traffic. dmesg reports 16 txq, 8 rxq -- which is the default for > Chelsio. > I don't recall offhand, but UDP might be using 2-tuple hashing by default and that might affect the distribution of flows across queues. Are there senders gener

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-12 Thread John Jasen
n 03/12/2017 07:18 PM, Slawa Olhovchenkov wrote: > On Sun, Mar 12, 2017 at 06:13:46PM -0400, John Jasen wrote: > > what traffic you generated (TCP? UDP? ICMP? other?), what reported in > dmesg | grep txq ? UDP traffic. dmesg reports 16 txq, 8 rxq -- which is the default for Chelsio. ___

Re: bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-12 Thread Slawa Olhovchenkov
On Sun, Mar 12, 2017 at 06:13:46PM -0400, John Jasen wrote: > I think I am able to confirm Mr. Caraballo's findings. > > I pulled a Dell PowerEdge 720 out of production, and upgraded it to > 11-RELEASE-p8. > > Currently, as in the R530, it has a single Chelsio T5-580, but has two > v2 Intel E5-2

bad throughput performance on multiple systems: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-12 Thread John Jasen
I think I am able to confirm Mr. Caraballo's findings. I pulled a Dell PowerEdge 720 out of production, and upgraded it to 11-RELEASE-p8. Currently, as in the R530, it has a single Chelsio T5-580, but has two v2 Intel E5-26xx CPUs versus the newer ones in the R530. Both ports are configured for

Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-03-01 Thread Olivier Cochard-Labbé
On Tue, Feb 28, 2017 at 3:13 AM, Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP] wrote: > As a summarywe have a Dell R530 with a Chelsio T580 cardwith -CURRENT. > > In an attempt to reduce the time the system was taking to look for the > cpus; we changed the BIOS setting to let the s

Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-02-27 Thread Sepherosa Ziehau
Did you compile and installed GENERIC-NODEBUG kernel for the CURRENT test? On Tue, Feb 28, 2017 at 10:13 AM, Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP] wrote: > As a summarywe have a Dell R530 with a Chelsio T580 cardwith -CURRENT. > > In an attempt to reduce the time the syste

Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530

2017-02-27 Thread Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP]
As a summarywe have a Dell R530 with a Chelsio T580 cardwith -CURRENT. In an attempt to reduce the time the system was taking to look for the cpus; we changed the BIOS setting to let the system have 8 visible cores and tested cxl* and vcxl* chelsio interfaces. Scores are still way lower than what