Sergio,
I'm having partial success. I can get a single "vdev crypto_ccp" to be
recognized with a single worker core, but I'm not able to get multiple
devices.
In my /etc/vpp/startup.conf, I have the following:
...
cpu {
main-core 1
corelist-workers 2,6,10,14
}
dpdk {
socket-mem 2048,2048,2048,2048
dev default {
num-rx-queues 2
num-tx-queues 2
num-tx-desc 2048
num-rx-desc 2048
}
uio-driver uio-pci-generic
dev 0000:41:00.0
dev 0000:41:00.1
vdev crypto_ccp0,socket_id=0
vdev crypto_ccp1,socket_id=1
vdev crypto_ccp2,socket_id=2
vdev crypto_ccp3,socket_id=3
}
...
When I start VPP using this configuration, I see a message about "not enough
DPDK crypto resources,
default to OpenSSL":
Feb 1 13:52:38 cl41 vpp: /usr/bin/vpp[15173]: dpdk_config:1344: socket-mem
argument is deprecated
Feb 1 13:52:38 cl41 vpp: /usr/bin/vpp[15173]: dpdk: EAL init args: -c 4446 -n
4 --in-memory --vdev crypto_ccp0,socket_id=0 --vdev crypto_ccp1,socket_id=1
--vdev crypto_ccp2,socket_id=2 --vdev crypto_ccp3,socket_id=3 --file-prefix vpp
-w 0000:41:00.0 -w 0000:41:00.1 --master-lcore 1
Feb 1 13:52:38 cl41 /usr/bin/vpp[15173]: dpdk: EAL init args: -c 4446 -n 4
--in-memory --vdev crypto_ccp0,socket_id=0 --vdev crypto_ccp1,socket_id=1
--vdev crypto_ccp2,socket_id=2 --vdev crypto_ccp3,socket_id=3 --file-prefix vpp
-w 0000:41:00.0 -w 0000:41:00.1 --master-lcore 1
Feb 1 13:52:38 cl41 vnet[15173]: dpdk_ipsec_process:1014: not enough DPDK
crypto resources, default to OpenSSL
Feb 1 13:53:13 cl41 vnet[15173]: unix_signal_handler:159: received signal
SIGCONT, PC 0x7f9aa76687e0
Feb 1 13:53:13 cl41 systemd: Stopping Vector Packet Processing Process...
Feb 1 13:53:13 cl41 vnet[15173]: received SIGTERM, exiting...
Feb 1 13:53:13 cl41 vnet[15173]: unix_signal_handler:159: received signal
SIGCONT, PC 0x7f9aa76687e0
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1456
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1468
Feb 1 13:53:13 cl41 vpp: Queue 0 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 1 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 2 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 3 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 4 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1456
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1468
Feb 1 13:53:13 cl41 vpp: Queue 0 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 1 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 2 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 3 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 4 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1456
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1468
Feb 1 13:53:13 cl41 vpp: Queue 0 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 1 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 2 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 3 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 4 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1456
Feb 1 13:53:13 cl41 vpp: CCP : Detected CCP device with ID = 0x1468
Feb 1 13:53:13 cl41 vpp: Queue 0 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 1 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 2 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 3 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: Queue 4 can access 7 LSB regions of mask 254
Feb 1 13:53:13 cl41 vpp: CCP : Crypto device count = 4
Feb 1 13:53:13 cl41 systemd: Stopped Vector Packet Processing Process.
If I modify the startup.conf to only have one worker core ("corelist-workers
2"),
I don't get the message about "not enough DPDK crypto resources", and I see:
vpp# show dpdk crypto device
crypto_ccp0 crypto_ccp up
numa_node 0, max_queues 1
free_resources 0, used_resources 1
SYMMETRIC_CRYPTO, SYM_OPERATION_CHAINING, HW_ACCELERATED
Cipher: aes-cbc-128, aes-cbc-192, aes-cbc-256, aes-ctr-128, aes-ctr-192,
aes-ctr-256, aes-gcm-128, aes-gcm-192, aes-gcm-256
Auth:
vpp#
When I look at the source code in crypto_parse_capabilities(), I am concerned
about the lines
" dcm->enabled |= (alg->resources >= n_mains);" in the following
code:
static void
crypto_parse_capabilities (crypto_dev_t * dev,
const struct rte_cryptodev_capabilities *cap,
u32 n_mains)
{
dpdk_crypto_main_t *dcm = &dpdk_crypto_main;
crypto_alg_t *alg;
u8 len, inc;
for (; cap->op != RTE_CRYPTO_OP_TYPE_UNDEFINED; cap++)
{
/* A single capability maps to multiple cipher/auth algorithms */
switch (cap->sym.xform_type)
{
case RTE_CRYPTO_SYM_XFORM_AEAD:
case RTE_CRYPTO_SYM_XFORM_CIPHER:
inc = cap->sym.cipher.key_size.increment;
inc = inc ? inc : 1;
for (len = cap->sym.cipher.key_size.min;
len <= cap->sym.cipher.key_size.max; len += inc)
{
alg = cipher_cap_to_alg (cap, len);
if (!alg)
continue;
dev->cipher_support[cipher_alg_index (alg)] = 1;
alg->resources += vec_len (dev->free_resources);
/* At least enough resources to support one algo */
dcm->enabled |= (alg->resources >= n_mains);
}
break;
case RTE_CRYPTO_SYM_XFORM_AUTH:
inc = cap->sym.auth.digest_size.increment;
inc = inc ? inc : 1;
for (len = cap->sym.auth.digest_size.min;
len <= cap->sym.auth.digest_size.max; len += inc)
{
alg = auth_cap_to_alg (cap, len);
if (!alg)
continue;
dev->auth_support[auth_alg_index (alg)] = 1;
alg->resources += vec_len (dev->free_resources);
/* At least enough resources to support one algo */
dcm->enabled |= (alg->resources >= n_mains);
}
break;
default:
;
}
}
}
crypto_scan_devs() calls crypto_parse_capabilities() within a loop over
cryptodevs.
The code appears to be comparing the resources of each cryptodev to the total
number,
n_mains. Should the code be comparing the sum of resources (from all of the
cryptodevs)
to the total value, n_mains?
- Lee
From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Sergio
Gonzalez Monroy
Sent: Wednesday, January 30, 2019 1:59 PM
To: Roberts, Lee A. <lee.robe...@hpe.com>; manuel.alo...@cavium.com;
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Question about crypto dev queue pairs #vpp
Hi folks,
Give the following patch a try and let me know.
https://gerrit.fd.io/r/#/c/17184/
I have done some very basic testing with SW cryptodev with single queue pair
and all seems fine.
Let me know if there is any issue with it.
Regards,
Sergio
________________________________
From: Roberts, Lee A. <lee.robe...@hpe.com<mailto:lee.robe...@hpe.com>>
Sent: Tuesday, January 29, 2019 3:49 PM
To: Sergio Gonzalez Monroy;
manuel.alo...@cavium.com<mailto:manuel.alo...@cavium.com>;
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: RE: [vpp-dev] Question about crypto dev queue pairs #vpp
Sergio,
I encountered the same problem when attempting to enable the AMD CCP poll mode
driver
in VPP 18.10.
As mentioned earlier in this e-mail thread, with max_qp = 1, max_res_idx
becomes 65535
in the following statement:
max_res_idx = (dev->max_qp / 2) - 1;
I hadn't found the time to study/debug the code to understand whether this was a
VPP or DPDK issue. If you have a patch available in the next few days, I could
test it with the AMD CCP device.
- Lee Roberts
From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
[mailto:vpp-dev@lists.fd.io] On Behalf Of Sergio Gonzalez Monroy
Sent: Tuesday, January 29, 2019 2:18 AM
To: manuel.alo...@cavium.com<mailto:manuel.alo...@cavium.com>;
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Question about crypto dev queue pairs #vpp
Hi Manuel,
This is likely a mismatch in VPP side. I only tested it with QAT (2 qps per VF)
and SW cryptodevs (default 8 qps) at the time (over a year ago).
I only tested it with SW cryptodevs and QAT, that was the HW I had access to.
So like I mentioned before, if you do not want to rework the code to support 1
qp per resource, then a check for at least 2 qps per device is required to use
that device.
I could provide a patch to use 1 pq per resource over the next few days if you
are interested in it or could review if you decide to do the work.
Which device do you want to use?
Regards,
Sergio
________________________________
From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
<vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> on behalf of
manuel.alo...@cavium.com<mailto:manuel.alo...@cavium.com>
<manuel.alo...@cavium.com<mailto:manuel.alo...@cavium.com>>
Sent: Monday, January 28, 2019 4:15 PM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Question about crypto dev queue pairs #vpp
Hi Sergio,
thank you for the explanation, I see that there are 2 (or more qps). My concern
was due to dpdk, since there are a few device drivers exporting only one queue
pair for their crypto devices.
(I followed the code assuming one qps, based on a dpdk-18.11 exported value)
So I do not know where is the mismatching, vpp or dpdk?
BR,
Manuel
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12133): https://lists.fd.io/g/vpp-dev/message/12133
Mute This Topic: https://lists.fd.io/mt/29538345/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp&subid=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
