Re: FreeBSD 10G forwarding performance @Intel

2012-07-18 Thread Alexander V. Chernikov
On 17.07.2012 17:21, Eugene Grosbein wrote: 17.07.2012 06:23, Konstantin Belousov пишет: I do not think that your 'per-cpu' counter are correct. The thread migration or rescheduling causes the fetch or update of the wrong per-cpu structure. This allows parallel updates with undefined consequenc

Re: FreeBSD 10G forwarding performance @Intel

2012-07-18 Thread Alexander V. Chernikov
On 17.07.2012 13:38, Alexander V. Chernikov wrote: On 17.07.2012 12:36, Alexander V. Chernikov wrote: On 17.07.2012 01:22, Luigi Rizzo wrote: On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote: On 06.07.2012 10:11, Luigi Rizzo wrote: On Thu, Jul 05, 2012 at 05:40:37PM +040

Re: FreeBSD 10G forwarding performance @Intel

2012-07-17 Thread Eugene Grosbein
17.07.2012 06:23, Konstantin Belousov пишет: > I do not think that your 'per-cpu' counter are correct. The thread > migration or rescheduling causes the fetch or update of the wrong > per-cpu structure. This allows parallel updates with undefined > consequences. >From practical point of view, I'l

Re: FreeBSD 10G forwarding performance @Intel

2012-07-17 Thread Konstantin Belousov
On Tue, Jul 17, 2012 at 02:33:24PM +0400, Alexander V. Chernikov wrote: > On 17.07.2012 03:23, Konstantin Belousov wrote: > >On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote: > >>On 06.07.2012 10:11, Luigi Rizzo wrote: > >>>On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V

Re: FreeBSD 10G forwarding performance @Intel

2012-07-17 Thread Alexander V. Chernikov
On 17.07.2012 03:23, Konstantin Belousov wrote: On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote: On 06.07.2012 10:11, Luigi Rizzo wrote: On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote: On 04.07.2012 19:48, Luigi Rizzo wrote: the thing discussed a

Re: FreeBSD 10G forwarding performance @Intel

2012-07-17 Thread Alexander V. Chernikov
On 17.07.2012 12:36, Alexander V. Chernikov wrote: On 17.07.2012 01:22, Luigi Rizzo wrote: On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote: On 06.07.2012 10:11, Luigi Rizzo wrote: On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote: On 04.07.2012 19:4

Re: FreeBSD 10G forwarding performance @Intel

2012-07-17 Thread Alexander V. Chernikov
On 17.07.2012 01:22, Luigi Rizzo wrote: On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote: On 06.07.2012 10:11, Luigi Rizzo wrote: On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote: On 04.07.2012 19:48, Luigi Rizzo wrote: well, it seems that the co

Re: FreeBSD 10G forwarding performance @Intel

2012-07-17 Thread Alexander V. Chernikov
On 17.07.2012 11:17, Hooman Fazaeli wrote: May be slightly off-topic, but do you have tested (or have plans to test ) with bidirectional traffic? Situation with bi-directional traffic is better (not sure how much). I'm intentionally not testing this case to discover rough cases (like contested

Re: FreeBSD 10G forwarding performance @Intel

2012-07-17 Thread Hooman Fazaeli
On 7/16/2012 10:13 PM, Alexander V. Chernikov wrote: Old kernel from previous letters, same setup: net.inet.ip.fw.enable=0 2.3 MPPS net.inet.ip.fw.update_counters=0 net.inet.ip.fw.enable=1 1.93MPPS net.inet.ip.fw.update_counters=1 1.74MPPS Kernel with ipfw pcpu counters: net.inet.ip.fw.enable=

Re: FreeBSD 10G forwarding performance @Intel

2012-07-16 Thread Konstantin Belousov
On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote: > On 06.07.2012 10:11, Luigi Rizzo wrote: > >On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote: > >>On 04.07.2012 19:48, Luigi Rizzo wrote: > >the thing discussed a few years ago (at least the one i took ou

Re: FreeBSD 10G forwarding performance @Intel

2012-07-16 Thread Luigi Rizzo
On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote: > On 06.07.2012 10:11, Luigi Rizzo wrote: > >On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote: > >>On 04.07.2012 19:48, Luigi Rizzo wrote: > >the thing discussed a few years ago (at least the one i took ou

Re: FreeBSD 10G forwarding performance @Intel

2012-07-16 Thread Alexander V. Chernikov
On 06.07.2012 10:11, Luigi Rizzo wrote: On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote: On 04.07.2012 19:48, Luigi Rizzo wrote: the thing discussed a few years ago (at least the one i took out of the discussion) was that the counter fields in rules should hold the index

Re: FreeBSD 10G forwarding performance @Intel

2012-07-05 Thread Luigi Rizzo
On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote: > On 04.07.2012 19:48, Luigi Rizzo wrote: ... > Traffic stats with most possible counters eliminated: > (there is a possibility in ixgbe code to update rx/tx packets once per > rx_process_limit (which is 100 by default)): > >

Re: FreeBSD 10G forwarding performance @Intel

2012-07-05 Thread Alexander V. Chernikov
On 04.07.2012 19:48, Luigi Rizzo wrote: On Wed, Jul 04, 2012 at 01:54:01PM +0400, Alexander V. Chernikov wrote: On 04.07.2012 13:12, Luigi Rizzo wrote: Alex, i am sure you are aware that in FreeBSD we have netmap too Yes, I'm aware of that :) which is probably a lot more usable than packetsh

Re: FreeBSD 10G forwarding performance @Intel

2012-07-04 Thread Alexander V. Chernikov
On 04.07.2012 23:37, Lev Serebryakov wrote: Hello, Alexander. You wrote 4 июля 2012 г., 12:46:09: AVC> http://shader.kaist.edu/packetshader/ (and links there) are good example AVC> of what is going on. But HOW?! GPU has very high "preparation" and data transfer cost, how it could be use

Re: FreeBSD 10G forwarding performance @Intel

2012-07-04 Thread Lev Serebryakov
Hello, Alexander. You wrote 4 июля 2012 г., 12:46:09: AVC> http://shader.kaist.edu/packetshader/ (and links there) are good example AVC> of what is going on. But HOW?! GPU has very high "preparation" and data transfer cost, how it could be used for such small packets of data, as 1.5-9K data

Re: FreeBSD 10G forwarding performance @Intel

2012-07-04 Thread Luigi Rizzo
On Wed, Jul 04, 2012 at 01:54:01PM +0400, Alexander V. Chernikov wrote: > On 04.07.2012 13:12, Luigi Rizzo wrote: > >Alex, > >i am sure you are aware that in FreeBSD we have netmap too > Yes, I'm aware of that :) > > >which is probably a lot more usable than packetshader > >(hw independent, includ

Re: FreeBSD 10G forwarding performance @Intel

2012-07-04 Thread Alexander V. Chernikov
On 04.07.2012 13:12, Luigi Rizzo wrote: Alex, i am sure you are aware that in FreeBSD we have netmap too Yes, I'm aware of that :) which is probably a lot more usable than packetshader (hw independent, included in the OS, also works on linux...) I'm actually not talking about usability and com

Re: FreeBSD 10G forwarding performance @Intel

2012-07-04 Thread Luigi Rizzo
On Wed, Jul 04, 2012 at 12:46:09PM +0400, Alexander V. Chernikov wrote: > On 04.07.2012 12:13, Doug Barton wrote: > >On 07/03/2012 23:29, Alexander V. Chernikov wrote: > >>On 04.07.2012 01:29, Doug Barton wrote: > >Just curious ... what's the MTU on your FreeBSD box, and the Linux box? > >> > >

Re: FreeBSD 10G forwarding performance @Intel

2012-07-04 Thread Alexander V. Chernikov
On 04.07.2012 12:13, Doug Barton wrote: On 07/03/2012 23:29, Alexander V. Chernikov wrote: On 04.07.2012 01:29, Doug Barton wrote: Just curious ... what's the MTU on your FreeBSD box, and the Linux box? In this particular setup - 1500. You're probably meaning type of mbufs which are allocated

Re: FreeBSD 10G forwarding performance @Intel

2012-07-04 Thread Doug Barton
On 07/03/2012 23:29, Alexander V. Chernikov wrote: > On 04.07.2012 01:29, Doug Barton wrote: Just curious ... what's the MTU on your FreeBSD box, and the Linux box? > > In this particular setup - 1500. You're probably meaning type of mbufs > which are allocated by ixgbe driver? 1500 for both?

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Alexander V. Chernikov
On 04.07.2012 01:29, Doug Barton wrote: On 07/03/2012 14:44, Luigi Rizzo wrote: On Tue, Jul 03, 2012 at 02:19:06PM -0700, Doug Barton wrote: Just curious ... what's the MTU on your FreeBSD box, and the Linux box? he is (correctly) using min-sized packets, and counting packets not bps. In thi

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Doug Barton
On 07/03/2012 14:44, Luigi Rizzo wrote: > On Tue, Jul 03, 2012 at 02:19:06PM -0700, Doug Barton wrote: >> Just curious ... what's the MTU on your FreeBSD box, and the Linux box? > > he is (correctly) using min-sized packets, and counting packets not bps. Yes, I know. That wasn't what I asked.

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Luigi Rizzo
On Tue, Jul 03, 2012 at 02:19:06PM -0700, Doug Barton wrote: > Just curious ... what's the MTU on your FreeBSD box, and the Linux box? he is (correctly) using min-sized packets, and counting packets not bps. cheers luigi ___ freebsd-net@freebsd.org mai

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Doug Barton
Just curious ... what's the MTU on your FreeBSD box, and the Linux box? (also, please don't cross-post to so many lists) :) Doug ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Luigi Rizzo
On Wed, Jul 04, 2012 at 12:31:56AM +0400, Alexander V. Chernikov wrote: > On 04.07.2012 00:27, Luigi Rizzo wrote: > >On Tue, Jul 03, 2012 at 09:37:38PM +0400, Alexander V. Chernikov wrote: > >... > >>Thanks, another good point. I forgot to merge this option from andre's > >>patch. > >> > >>Another

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Alexander V. Chernikov
On 04.07.2012 00:27, Luigi Rizzo wrote: On Tue, Jul 03, 2012 at 09:37:38PM +0400, Alexander V. Chernikov wrote: ... Thanks, another good point. I forgot to merge this option from andre's patch. Another 30-40-50kpps to win. not much gain though. What about the other IPSTAT_INC counters ? Well,

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Luigi Rizzo
On Tue, Jul 03, 2012 at 09:37:38PM +0400, Alexander V. Chernikov wrote: ... > Thanks, another good point. I forgot to merge this option from andre's > patch. > > Another 30-40-50kpps to win. not much gain though. What about the other IPSTAT_INC counters ? I think the IPSTAT_INC macros were intro

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Alexander V. Chernikov
On 03.07.2012 20:55, Luigi Rizzo wrote: On Tue, Jul 03, 2012 at 08:11:14PM +0400, Alexander V. Chernikov wrote: Hello list! I'm quite stuck with bad forwarding performance on many FreeBSD boxes doing firewalling. ... In most cases system can forward no more than 700 (or 1400) kpps which is qu

Re: FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Luigi Rizzo
On Tue, Jul 03, 2012 at 08:11:14PM +0400, Alexander V. Chernikov wrote: > Hello list! > > I'm quite stuck with bad forwarding performance on many FreeBSD boxes > doing firewalling. ... > In most cases system can forward no more than 700 (or 1400) kpps which > is quite a bad number (Linux does, s

FreeBSD 10G forwarding performance @Intel

2012-07-03 Thread Alexander V. Chernikov
Hello list! I'm quite stuck with bad forwarding performance on many FreeBSD boxes doing firewalling. Typical configuration is E5645 / E5675 @ Intel 82599 NIC. HT is turned off. (Configs and tunables below). I'm mostly concerned with unidirectional traffic flowing to single interface (e.g. us