I modified the script to produce this config (2 threads):
unix {
nodaemon
log /tmp/vpe.log
cli-listen localhost:5002
full-coredump
}
api-trace {
on
}
cpu {
main-core 19 corelist-workers 20,21
}
dpdk {
socket-mem 1024,1024
dev default {
num-rx-queues 1
}
dev 0000:88:00.1
dev 0000:88:00.0
enable-cryptodev dev 0000:86:01.0 dev 0000:86:01.1
uio-driver igb_uio
no-multi-seg
}
ip6 {
hash-buckets 2000000
heap-size 3G
}
I am sure that QAT is initialized with 32VFs (cat
/sys/bus/pci/devices/0000\:86\:00.0/sriov_numvfs show 32 on both machines), but:
DUT1 is showing:
vat# vat# DPDK Cryptodev support is disabled
DUT2 is showing:
vat# vat# worker crypto device id(type)
1 1(HW)
2 0(HW)
The setup for both DUTs is the same. Do you know what may be wrong?
In syslog I do see for both DUTs (lspci shows QAT initialized):
…
Mar 16 01:27:53 t3-sut1 kernel: [94934.751859] uio_pci_generic 0000:86:04.7:
enabling device (0000 -> 0002)
Mar 16 01:27:53 t3-sut1 kernel: [94934.751862] uio_pci_generic 0000:86:04.7: No
IRQ assigned to device: no support for interrupts?
…
Mar 16 01:27:58 t3-sut2 kernel: [94846.606256] uio_pci_generic 0000:86:04.7:
enabling device (0000 -> 0002)
Mar 16 01:27:58 t3-sut2 kernel: [94846.606259] uio_pci_generic 0000:86:04.7: No
IRQ assigned to device: no support for interrupts?
So no more crypto not found.
Any ideas?
Peter Mikus
Engineer – Software
Cisco Systems Limited
From: Sergio Gonzalez Monroy [mailto:sergio.gonzalez.mon...@intel.com]
Sent: Wednesday, March 15, 2017 5:11 PM
To: Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmi...@cisco.com>
Cc: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco)
<tifr...@cisco.com>; csit-...@lists.fd.io; vpp-dev <vpp-dev@lists.fd.io>;
Maciek Konstantynowicz (mkonstan) <mkons...@cisco.com>; Rybalchenko, Kirill
<kirill.rybalche...@intel.com>; Nicolau, Radu <radu.nico...@intel.com>
Subject: Re: IPsec Multi-Tunnel performance test suite failure
I reckon something like the following should do:
enable-cryptodev dev 0000:86:01.0 dev 0000:86:01.1
Sergio
On 15/03/2017 15:36, Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco)
wrote:
So something like:
enable-cryptodev dev 0000:86:01.0
enable-cryptodev dev 0000:86:02.0
?
Peter Mikus
Engineer – Software
Cisco Systems Limited
From: Sergio Gonzalez Monroy [mailto:sergio.gonzalez.mon...@intel.com]
Sent: Wednesday, March 15, 2017 4:14 PM
To: Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco)
<pmi...@cisco.com><mailto:pmi...@cisco.com>; Rybalchenko, Kirill
<kirill.rybalche...@intel.com><mailto:kirill.rybalche...@intel.com>; Nicolau,
Radu <radu.nico...@intel.com><mailto:radu.nico...@intel.com>
Cc: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco)
<tifr...@cisco.com><mailto:tifr...@cisco.com>;
csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; vpp-dev
<vpp-dev@lists.fd.io><mailto:vpp-dev@lists.fd.io>; Maciek Konstantynowicz
(mkonstan) <mkons...@cisco.com><mailto:mkons...@cisco.com>
Subject: Re: IPsec Multi-Tunnel performance test suite failure
My bad.
I thought the test was already using two QAT VFs. Each workers needs one QAT VF.
Sergio
On 15/03/2017 13:47, Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco)
wrote:
After I run CSIT with 2 physical cores and 2 worker-threads, the HW cryptodev
is not working:
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-master-all/1178/console
Testing is HW is there was successful and it was initialized.
Can you please take a look?
The only change I did was adding 1 more worker threads. Initialization remains
the same and Cryptodev HW was not recognized?
Peter Mikus
Engineer – Software
Cisco Systems Limited
From: Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco)
Sent: Monday, March 13, 2017 2:19 PM
To: 'Sergio Gonzalez Monroy'
<sergio.gonzalez.mon...@intel.com><mailto:sergio.gonzalez.mon...@intel.com>;
Maciek Konstantynowicz (mkonstan)
<mkons...@cisco.com><mailto:mkons...@cisco.com>; Rybalchenko, Kirill
<kirill.rybalche...@intel.com><mailto:kirill.rybalche...@intel.com>; Nicolau,
Radu <radu.nico...@intel.com><mailto:radu.nico...@intel.com>
Cc: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco)
<tifr...@cisco.com><mailto:tifr...@cisco.com>;
csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; vpp-dev
<vpp-dev@lists.fd.io><mailto:vpp-dev@lists.fd.io>
Subject: RE: IPsec Multi-Tunnel performance test suite failure
+ Looping @csit-dev @vpp-dev
I will add 2 workers/threads that is not a problem.
To avoid possible exploding of number of test, we should pick only the
representative one. Apart from implementation are there any other differences
between tunnel and interface mode?
Thanks.
Peter Mikus
Engineer – Software
Cisco Systems Limited
From: Sergio Gonzalez Monroy [mailto:sergio.gonzalez.mon...@intel.com]
Sent: Monday, March 13, 2017 9:58 AM
To: Maciek Konstantynowicz (mkonstan)
<mkons...@cisco.com<mailto:mkons...@cisco.com>>; Peter Mikus -X (pmikus -
PANTHEON TECHNOLOGIES at Cisco) <pmi...@cisco.com<mailto:pmi...@cisco.com>>;
Rybalchenko, Kirill
<kirill.rybalche...@intel.com<mailto:kirill.rybalche...@intel.com>>; Nicolau,
Radu <radu.nico...@intel.com<mailto:radu.nico...@intel.com>>
Cc: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco)
<tifr...@cisco.com<mailto:tifr...@cisco.com>>
Subject: Re: IPsec Multi-Tunnel performance test suite failure
First, thank you to all involved!
I reckon those numbers are in the expected range.
The current test is single thread with bidirectional traffic.
I would definitely like to see tests with 2 workers/threads, one worker for
each direction. One of the reasons is that we cannot saturate QAT with a single
worker (QAT should be able to do +40Gbps of encryption).
Would it make sense to have another set of tests with 2 workers or just update
the current tests to use 2 workers?
Regarding the difference between ipsec interface and tunnels (a.k.a. SPD), the
results are expected.
Basically, it is all about the SPD (Security Policy Database) implementation.
The "tunnels" tests use the SPD, whereas the ipsec interfaces do not.
The current SPD implementation in VPP follows the guidelines of the RFC, but
it does not scale.
The ipsec interfaces do not use the SPD at all and a single entry in the fib is
all we need to "select" the traffic to encrypt.
They effectively are different graph node paths, and even though both end up
taking the same amount of cycles (at least for decryption), the interfaces
scale much better.
Sergio
On 11/03/2017 18:36, Maciek Konstantynowicz (mkonstan) wrote:
Great Peter, thanks for this final push !
Sergio, team - are these the results you expect to see?
Why such a difference interfaces vs. tunnels at 1k scale?
aes-gcm interfaces
1 tunnel 1k tunnels
Mpps Gbps Mpps Gbps
64B 2.5 1.7 2.3 1.5
IMIX 2.4 7.1 2.1 6.4
1518B 2.4 28.9 2.1 26.0
cbc-sha1 interfaces
1 tunnel 1k tunnels
Mpps Gbps Mpps Gbps
64B 2.5 1.7 2.3 1.5
IMIX 2.4 7.2 2.1 6.4
1518B 2.4 29.2 2.1 29.2
aes-gcm tunnels
1 tunnel 1k tunnels
Mpps Gbps Mpps Gbps
64B 2.4 1.6 0.4 0.2
IMIX 2.4 7.0 0.3 1.0
1518B 2.2 27.8 0.4 4.3
cbc-sha1 tunnels
1 tunnel 1k tunnels
Mpps Gbps Mpps Gbps
64B 2.5 1.7 0.4 0.3
IMIX 2.4 7.2 0.3 1.0
1518B 2.4 29.2 0.4 5.0
-Maciek
On 11 Mar 2017, at 06:45, Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at
Cisco) <pmi...@cisco.com<mailto:pmi...@cisco.com>> wrote:
IPSEC is now working. PDR and NDR results are same and can be found there:
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-master-all/1156/console
Plots will be updated to display IPsecHW (seems like wrong xpath eval). I will
check on Monday.
So far I will run couple more iterations to see the results
@Maciek, I think it is about time to populate all TBs with QAT. Can we
coordinate?
Peter Mikus
Engineer – Software
Cisco Systems Limited
From: Nicolau, Radu [mailto:radu.nico...@intel.com]
Sent: Wednesday, March 08, 2017 2:07 PM
To: Gonzalez Monroy, Sergio
<sergio.gonzalez.mon...@intel.com<mailto:sergio.gonzalez.mon...@intel.com>>;
Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco)
<pmi...@cisco.com<mailto:pmi...@cisco.com>>; Rybalchenko, Kirill
<kirill.rybalche...@intel.com<mailto:kirill.rybalche...@intel.com>>
Cc: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco)
<tifr...@cisco.com<mailto:tifr...@cisco.com>>; Maciek Konstantynowicz
(mkonstan) <mkons...@cisco.com<mailto:mkons...@cisco.com>>
Subject: RE: IPsec Multi-Tunnel performance test suite failure
Hi,
I submitted a small patch to only bind QAT VFs. https://gerrit.fd.io/r/#/c/5671/
The downside is that the additional check will have to be updated for new
devices.
Regards,
Radu
_______________________________________________
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev
