[AMD Official Use Only - General] Hi David,
Please find the below update - only one DPDK application can use ccp crypto engines (PCI bus allow/blocklist is not respected, right?), Yes, only one crypto device can be used in a DPDK application for the crypto operations. This is introduced from the patch crypto/ccp: convert driver from vdev to PCI. This is implemented as per community suggestion. - since only one crypto device is exposed, there is no way for the application to dedicate/decide how to distribute crypto operations over the different ccp crypto engines available on the system. When there is no ccp device passed from the application dpdk-test-crypto-perf, it tries to probe all CCP devices present on a system and only one device can be used. It seems this is bug in the patch implemented for crypto/ccp: convert driver from vdev to PCI and we are looking into this. Thanks Sunil -----Original Message----- From: David Marchand <david.march...@redhat.com> Sent: Thursday, November 3, 2022 6:39 PM To: Uttarwar, Sunil Prakashrao <sunilprakashrao.uttar...@amd.com> Cc: Yigit, Ferruh <ferruh.yi...@amd.com>; Akhil Goyal <gak...@marvell.com>; Namburu, Chandu-babu <cha...@amd.com>; Sebastian, Selwin <selwin.sebast...@amd.com>; dev <dev@dpdk.org>; Thomas Monjalon <tho...@monjalon.net> Subject: Re: [PATCH v2 0/4] crypto/ccp cleanup Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Hello, On Wed, Nov 2, 2022 at 2:54 PM Uttarwar, Sunil Prakashrao <sunilprakashrao.uttar...@amd.com> wrote: > > [AMD Official Use Only - General] > > Hi David, > > Please find below response. Not sure why dev@ was dropped. Adding it back. > > -----Original Message----- > From: David Marchand <david.march...@redhat.com> > Sent: Wednesday, November 2, 2022 6:18 PM > To: Uttarwar, Sunil Prakashrao <sunilprakashrao.uttar...@amd.com> > Cc: Yigit, Ferruh <ferruh.yi...@amd.com>; Akhil Goyal > <gak...@marvell.com>; Namburu, Chandu-babu <cha...@amd.com> > Subject: Re: [PATCH v2 0/4] crypto/ccp cleanup > > Caution: This message originated from an External Source. Use proper caution > when opening attachments, clicking links, or responding. > > > Hello, > > On Wed, Nov 2, 2022 at 11:26 AM Uttarwar, Sunil Prakashrao > <sunilprakashrao.uttar...@amd.com> wrote: > > As mentioned earlier, observing issues with "crypto/ccp: fix PCI probing" > > patch (Floating point exception). Please find the below backtrace . > > > > (gdb) r -l 0,4 -n 4 -- --ptest throughput --buffer-sz 64 --burst-sz > > 32 --total-ops 3000 --silent --devtype crypto_ccp --optype > > cipher-only --cipher-algo aes-cbc --cipher-op encrypt > > --cipher-key-sz 16 --cipher-iv-sz 16 Starting program: > > /home/cae/sunil/dpdk_main/dpdk/build/app/dpdk-test-crypto-perf -l 0,4 -n 4 > > -- --ptest throughput --buffer-sz 64 --burst-sz 32 --total-ops 3000 > > --silent --devtype crypto_ccp --optype cipher-only --cipher-algo aes-cbc > > --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16 [Thread debugging > > using libthread_db enabled] Using host libthread_db library > > "/lib/x86_64-linux-gnu/libthread_db.so.1". > > EAL: Detected CPU lcores: 24 > > EAL: Detected NUMA nodes: 2 > > EAL: Detected static linkage of DPDK [New Thread 0x7ffff6dc5400 (LWP > > 171350)] > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket [New Thread > > 0x7ffff65c4400 (LWP 171351)] > > EAL: Selected IOVA mode 'PA' > > EAL: VFIO support initialized > > [New Thread 0x7ffff5dc3400 (LWP 171352)] > > EAL: Probe PCI driver: crypto_ccp (1022:1456) device: 0000:04:00.2 > > (socket 0) > > PMD: Initialising 0000:04:00.2 on NUMA node 0 > > PMD: Max number of queue pairs = 8 > > PMD: Authentication offload to CCP > > CRYPTODEV: User specified device name = 0000:04:00.2 > > CRYPTODEV: Creating cryptodev 0000:04:00.2 > > CRYPTODEV: Initialisation parameters - name: 0000:04:00.2,socket id: > > 0, max queue pairs: 8 > > EAL: Probe PCI driver: crypto_ccp (1022:1468) device: 0000:05:00.1 > > (socket 0) > > PMD: CCP PMD already initialized > > EAL: Requested device 0000:05:00.1 cannot be used > > EAL: Probe PCI driver: crypto_ccp (1022:1456) device: 0000:41:00.2 > > (socket 1) > > PMD: CCP PMD already initialized > > EAL: Requested device 0000:41:00.2 cannot be used > > EAL: Probe PCI driver: crypto_ccp (1022:1468) device: 0000:42:00.1 > > (socket 1) > > PMD: CCP PMD already initialized > > EAL: Requested device 0000:42:00.1 cannot be used [New Thread > > 0x7ffff55c2400 (LWP 171353)] > > TELEMETRY: No legacy callbacks, legacy socket not created Allocated > > pool "sess_mp_0" on socket 0 > > > > Thread 4 "rte-worker-4" received signal SIGFPE, Arithmetic exception. > > [Switching to Thread 0x7ffff5dc3400 (LWP 171352)] 0x000055555767397a > > in ccp_pmd_enqueue_burst (queue_pair=0x17fefe940, ops=0x7ffff5dbe6e0, > > nb_ops=32) at ../drivers/crypto/ccp/rte_ccp_pmd.c:97 > > 97 cur_ops = nb_ops / cryptodev_cnt + > > (nb_ops)%cryptodev_cnt; > > (gdb) bt > > I have a hard time understanding the logic in this enqueue code... > > Is this driver exposing a single crypto device and will "balance" > crypto operations across all pci devices on the system? > > Driver is exposing a single crypto device as physical device and only one > device can be used by the driver for single instance for all operations. > Afaik, this is the only crypto driver that implements this. I see two issues with the approach. - only one DPDK application can use ccp crypto engines (PCI bus allow/blocklist is not respected, right?), - since only one crypto device is exposed, there is no way for the application to dedicate/decide how to distribute crypto operations over the different ccp crypto engines available on the system. -- David Marchand
