Re: IPsec performace - netisr hits %100

2021-05-06 Thread Mark Johnston
On Thu, May 06, 2021 at 05:07:30PM +0300, Özkan KIRIK wrote: > I wonder that if you received the flame graphs ? Yes, there's nothing obviously problematic there. > I also tested system with multiple if_ipsec interfaces using different > source-dst tunnel address. > By this way, system can utilize

Re: IPsec performace - netisr hits %100

2021-05-06 Thread Özkan KIRIK
I wonder that if you received the flame graphs ? I also tested system with multiple if_ipsec interfaces using different source-dst tunnel address. By this way, system can utilize all cpu cores. But for single if_ipsec interface, is there a way to speed up transfer ? Thanks! On Mon, May 3, 2021 a

Re: IPsec performace - netisr hits %100

2021-05-03 Thread Mark Johnston
On Sun, May 02, 2021 at 04:08:18PM +0300, Andrey V. Elsukov wrote: > 30.04.2021 23:32, Mark Johnston пишет: > > Second, netipsec unconditionally hands rx processing off to netisr > > threads for some reason, that's why changing the dispatch policy doesn't > > help. Maybe it's to help avoid running

Re: IPsec performace - netisr hits %100

2021-05-02 Thread Özkan KIRIK
Hello, The flamegraph is attached. # netstat -s ... ipsec: 0 inbound packets violated process security policy 0 inbound packets failed due to insufficient memory 0 invalid inbound packets 0 outbound packets violated process security policy 0 outbound packets

Re: IPsec performace - netisr hits %100

2021-05-02 Thread Andrey V. Elsukov
30.04.2021 23:11, Özkan KIRIK пишет: > Hello, > > I'm using FreeBSD stable/12 built world on 12 April 2021. > my setup is: > [freebsd host cc0] <> [cc1 - same freebsd, but jail] > > without IPsec, I can achieve easily to 20Gbps. (test was run with different > source IPs using multiple ipe

Re: IPsec performace - netisr hits %100

2021-05-02 Thread Andrey V. Elsukov
30.04.2021 23:32, Mark Johnston пишет: > Second, netipsec unconditionally hands rx processing off to netisr > threads for some reason, that's why changing the dispatch policy doesn't > help. Maybe it's to help avoid running out of kernel stack space or to > somehow avoid packet reordering in some

Re: IPsec performace - netisr hits %100

2021-05-01 Thread Özkan KIRIK
the previous flamegraph is captured while iperf client (jail) sends to iperf server. the attached flamegraph to this mail is captured while iperf configured full-duplex mode. Throughput is about up: 1.5 Gbps down: 1.5 Gbps total: 3 Gbps On Sat, May 1, 2021 at 11:57 PM Özkan KIRIK wrote: > Hello

Re: IPsec performace - netisr hits %100

2021-05-01 Thread Mark Johnston
On Sat, May 01, 2021 at 04:30:59PM +0300, Özkan KIRIK wrote: > This bug is related to CCR. @Navdeep Parhar , @John Baldwin > if you are interested to fix this bug related with CCR, I > can test if you provide patches. Test environment is explained in my first > email on this thread. > > @Mark Jo

Re: IPsec performace - netisr hits %100

2021-05-01 Thread Özkan KIRIK
This bug is related to CCR. @Navdeep Parhar , @John Baldwin if you are interested to fix this bug related with CCR, I can test if you provide patches. Test environment is explained in my first email on this thread. @Mark Johnston Now again on stable/13, - with aesni, without netipsec/ipsec_inpu

Re: IPsec performace - netisr hits %100

2021-05-01 Thread Özkan KIRIK
I've pulled the latest stable/13 branch and make buildworld. Same configuration and same usage, ping works between two sides, iperf can connect to server but no data transferred. throughput is around 10Kbps. I tried both with and without netipsec/ipsec_input.c patch, but no change. There is someth

Re: IPsec performace - netisr hits %100

2021-04-30 Thread Özkan KIRIK
Here is fresh results; a bit performance gain achieved. But strangely QAT is better now. - with CCR - 2.14 Gbps PID USERNAMEPRI NICE SIZERES STATEC TIMEWCPU COMMAND 15 root-16- 0B16K CPU7 7 0:11 100.00% [crypto returns 9] 11 root-92

Re: IPsec performace - netisr hits %100

2021-04-30 Thread Mark Johnston
On Sat, May 01, 2021 at 12:31:57AM +0300, Özkan KIRIK wrote: > Hello again, > > patch is applied, now netisr is not eating CPU. but performance drops > around 0.2Gbps according to previous kernel. > > I tried also both net.isr.maxthreads=1 and net.isr.maxthreads=4 . results > are same > > Result

Re: IPsec performace - netisr hits %100

2021-04-30 Thread Özkan KIRIK
Hello again, patch is applied, now netisr is not eating CPU. but performance drops around 0.2Gbps according to previous kernel. I tried also both net.isr.maxthreads=1 and net.isr.maxthreads=4 . results are same Results are: - with CCR - 1.8Gbps top: PID USERNAMEPRI NICE SIZERES STAT

Re: IPsec performace - netisr hits %100

2021-04-30 Thread Özkan KIRIK
Thank you Mark, I'm going to recompile kernel with this patch and share results. After a few days, I can install stable/13 on same hardware and then I'll repeat same tests. Thanks again, Cheers On Fri, Apr 30, 2021 at 11:32 PM Mark Johnston wrote: > On Fri, Apr 30, 2021 at 11:11:48PM +0300, Öz

Re: IPsec performace - netisr hits %100

2021-04-30 Thread Mark Johnston
On Fri, Apr 30, 2021 at 11:11:48PM +0300, Özkan KIRIK wrote: > Hello, > > I'm using FreeBSD stable/12 built world on 12 April 2021. > my setup is: > [freebsd host cc0] <> [cc1 - same freebsd, but jail] > > without IPsec, I can achieve easily to 20Gbps. (test was run with different > sourc

IPsec performace - netisr hits %100

2021-04-30 Thread Özkan KIRIK
Hello, I'm using FreeBSD stable/12 built world on 12 April 2021. my setup is: [freebsd host cc0] <> [cc1 - same freebsd, but jail] without IPsec, I can achieve easily to 20Gbps. (test was run with different source IPs using multiple iperf to scale across multiple queues) My hardware is Xe