Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-09-18 Thread Josef.Zahner1
Hi Navdeep Any plans to make a pull request for your patch that we have the feature as well in FreeBSD officially? We use your change in prod now, works like a charm… But I don’t see your modification here https://github.com/freebsd/freebsd-src/blob/main/sys/dev/cxgbe Without your patch, we ha

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-13 Thread Josef.Zahner1
It was not my intention to tune before testing. The point was, when we started testing, we have seen that RSS wasn’t working with the Intel E810 NIC out of the box just after the installation of FreeBSD (OPNsense). All load was only on CPU0, no load distribution at all. So we enabled the RSS sys

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-10 Thread Josef.Zahner1
Ok we are getting closer to a solution. But I’m a little bit confused, I’ll try to explain why. I’ve removed the following sysctl values, as you stated that they are not official FreeBSD values: - net.inet.rss.bits="2" - net.inet.rss.enabled="1" - net.isr.bindthreads="1" - net.isr.maxthreads="-1

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
Sorry for the spam, I do see the values with sysctl now. It seems that FreeBSD always loads the if_cxgbe.ko from /boot/kernel/if_cxgbe.ko. So what I’ve done is, I renamed the original file and copied the newly compiled if_cxgbe.ko from /boot/module to /boot/kernel. Is there a cleaner way to get

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
Again me, I tried to build the chelsio driver (if_cxgbe) on my machine. root@fw-94:/usr/src/sys/modules/cxgbe # make ; make install -> no errors so far and I see the new kernel module in /boot/modules: root@fw-94:/usr/src/sys/modules/cxgbe # ls -alh /boot/modules/ total 4584 drwxr-xr-x 2 root

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
Hi Navdeep Nevermind, I found the following command to apply your patch diff: root@fw-94:/sys/dev/cxgbe # patch < /root/chelsio_rss_patch.diff There was an inconsistency with “adapter.h”, your change on line 219 was in my case a few lines further below, so I decided to manually edit those two ch

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
Hi Navdeep Sorry if it is a dump question, but your link contains only a diff for your git commit. Where do I find the patch? I’ve executed the requested commands. The netstat command doesn’t show any drops during the flapping. You can see the number of packets goes down somewhere in the middl

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-29 Thread Josef.Zahner1
Can you tell me which netstat command you have in mind? I tried “netstat -Q”, it shows a few drops but not that much that it would explain the CARP drops. What I can tell you is, that especially CARP on the corresponding server is just sending out packets as long as it is the master box and CPU0

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-27 Thread Josef.Zahner1
We are familiar with «hw.cxgbe.rsrv_noflowq», but as you already stated it’s only for TX direction, it doesn’t help us at all. Our problem is in fact, that on FreeBSD only CPU0 seems to do the slow protocol (LACP, CARP,…) stuff and even though the other CPUs are completely idle, if CPU0 has 100

Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-26 Thread Josef.Zahner1
Hi guys I’m trying to do something similar like in the question here (https://lists.freebsd.org/pipermail/freebsd-net/2017-March/047378.html). We have Chelsio NICs (T580-LP-CR). Our goal is the following: * RSS Queue 0 -> only control plane traffic, eg. CARP (IP Protocol 112 – Multicast),