> -----Original Message-----
> From: Power, Ciara
> Sent: Wednesday 8 March 2023 17:31
> To: Volodymyr Fialko <vfia...@marvell.com>; dev@dpdk.org; Akhil Goyal
> <gak...@marvell.com>; Fan Zhang <fanzhang....@gmail.com>
> Cc: jer...@marvell.com; ano...@marvell.com; hemant.agra...@nxp.com; Ji,
> Kai <kai...@intel.com>
> Subject: RE: [PATCH v2] test: add cryptodev crosscheck suite
>
> Hi Volodymyr,
>
>
> > -----Original Message-----
> > From: Volodymyr Fialko <vfia...@marvell.com>
> > Sent: Thursday 9 February 2023 12:34
> > To: dev@dpdk.org; Akhil Goyal <gak...@marvell.com>; Fan Zhang
> > <fanzhang....@gmail.com>
> > Cc: jer...@marvell.com; ano...@marvell.com; hemant.agra...@nxp.com;
> > Ji, Kai <kai...@intel.com>; Power, Ciara <ciara.po...@intel.com>;
> > Volodymyr Fialko <vfia...@marvell.com>
> > Subject: [PATCH v2] test: add cryptodev crosscheck suite
> >
> > Add a validation test suite that helps in verifying that the output
> > generated by two different cryptodevs match for a wide range of input
> > parameter combinations.
> >
> > Crypto autotest performs a comprehensive testing of the cryptodev but
> > since it performs verification by comparing against known vectors, the
> > extend to which various parameters (like packet size) can be tested is
> > limited. This test suite attempts to simulate various cases by running
> > same test case on different cryptodevs and compares the output
> > generated. The test suite relies on capabilities to determine the
> > combinations of tests to be attempted.
> >
> > A typical use case would be to compare outputs generated from a
> > standard driver such as openSSL PMD and a new cryptodev PMD. This test
> > suite is to compliment the testing coverage that crypto autotest provides.
> >
> > Currently supported symmetric xforms(cipher, auth, aead) without
> chaining.
> >
> > Example command:
> > DPDK_TEST=cryptodev_crosscheck ./dpdk-test \
> > -a <cryptodev> --vdev "crypto_openssl"
> >
> > Signed-off-by: Volodymyr Fialko <vfia...@marvell.com>
> > ---
> > V2:
> > - Updated commit message.
> >
> <snip>
> > +
> > +static void
> > +capabilities_inspect(void)
> > +{
> > + struct rte_cryptodev_sym_capability_idx
> > cap_indexes[CRYPTO_ALGOS_LEN], *cap_idx;
> > + struct crypto_testsuite_params *ts_params = &testsuite_params;
> > + const struct rte_cryptodev_symmetric_capability *sym_capa;
> > + struct rte_cryptodev_symmetric_capability *common_capa;
> > + uint32_t algo, i, dev_id, caps_idx;
> > +
> > + caps_idx = 0;
> > + /* Create capability idx for known algorithms*/
> > + for (algo = 1; algo <= CRYPTO_AUTH_MAX_IDX; algo++) {
> > + cap_idx = &cap_indexes[caps_idx++];
> > + cap_idx->type = RTE_CRYPTO_SYM_XFORM_AUTH;
> > + cap_idx->algo.auth = algo;
> > + }
> > + for (algo = 1; algo <= CRYPTO_CIPHER_MAX_IDX; algo++) {
> > + cap_idx = &cap_indexes[caps_idx++];
> > + cap_idx->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
> > + cap_idx->algo.cipher = algo;
> > + }
> > + for (algo = 1; algo <= CRYPTO_AEAD_MAX_IDX; algo++) {
> > + cap_idx = &cap_indexes[caps_idx++];
> > + cap_idx->type = RTE_CRYPTO_SYM_XFORM_AEAD;
> > + cap_idx->algo.aead = algo;
> > + }
> > +
> > + for (caps_idx = 0; caps_idx < CRYPTO_ALGOS_LEN; caps_idx++) {
> > + /* Gather common capabilities */
> > + common_capa = &common_symm_capas[caps_idx];
> > + common_capa->xform_type =
> > RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED;
> > + for (i = 0; i < ts_params->valid_dev_count; i++) {
> > + dev_id = ts_params->valid_devs[i];
> > + sym_capa =
> > rte_cryptodev_sym_capability_get(dev_id,
> > + &cap_indexes[caps_idx]);
> > + if (sym_capa == NULL) {
> > + /* Capability not supported by one of devs,
> > mark and skip */
> > + goto next_algo;
> > + }
> > +
> > + if (common_capa->xform_type ==
> > RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED) {
> > + /* First time initialization, copy data, go to
> > next device */
> > + *common_capa = *sym_capa;
> > + continue;
> > + }
> [CP]
>
> This function - from what I understand after review, is looping through all
> algorithms in DPDK crypto, and then checking if each is supported by each
> device?
> Could we instead just take the capabilities list from one device as the
> starting
> point - That will be the max list of capabilities, they will only get knocked
> out
> if not supported on the 2nd/3rd device etc.
>
>
> <snip>
>
> Some trials I did with this patch showed some issues.
>
> 1. For QAT + AESNI_MB I get all skipped tests. I would have expected some
> common algs here.
> + ------------------------------------------------------- +
> + Test Suite Summary : Crosscheck Unit Test Suite +
> -----------------------------
> -------------------------- + + Algo AUTH 'null' : 0/0 passed, 0/0 skipped,
> 0/0
> failed, 0/0 unsupported + Algo AUTH 'aes-cbc-mac' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'aes-cmac' : 0/0 passed,
> 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'aes-gmac' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'aes-xcbc-
> mac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH
> 'kasumi-f9' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> AUTH 'md5' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> AUTH 'md5-hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo AUTH 'sha1' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo AUTH 'sha1-hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0
> unsupported + Algo AUTH 'sha2-224' : 0/0 passed, 0/0 skipped, 0/0 failed,
> 0/0 unsupported + Algo AUTH 'sha2-224-hmac' : 0/0 passed, 0/0 skipped, 0/0
> failed, 0/0 unsupported + Algo AUTH 'sha2-256' : 0/0 passed, 0/0 skipped,
> 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-256-hmac' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-384' : 0/0 passed,
> 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-384-hmac' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-512' :
> 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-512-
> hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH
> 'snow3g-uia2' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> AUTH 'zuc-eia3' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo
> AUTH 'sha3-224' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo AUTH 'sha3-224-hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0
> unsupported + Algo AUTH 'sha3-256' : 0/0 passed, 0/0 skipped, 0/0 failed,
> 0/0 unsupported + Algo AUTH 'sha3-256-hmac' : 0/0 passed, 0/0 skipped, 0/0
> failed, 0/0 unsupported + Algo AUTH 'sha3-384' : 0/0 passed, 0/0 skipped,
> 0/0 failed, 0/0 unsupported + Algo AUTH 'sha3-384-hmac' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha3-512' : 0/0 passed,
> 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha3-512-hmac' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER 'null' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER '3des-cbc' :
> 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER '3des-ctr'
> : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER '3des-
> ecb' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER
> 'aes-cbc' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> CIPHER 'aes-ctr' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo
> CIPHER 'aes-ecb' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo CIPHER 'aes-f8' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo CIPHER 'aes-xts' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported
> +
> Algo CIPHER 'arc4' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo CIPHER 'kasumi-f8' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0
> unsupported + Algo CIPHER 'snow3g-uea2' : 0/0 passed, 0/0 skipped, 0/0
> failed, 0/0 unsupported + Algo CIPHER 'zuc-eea3' : 0/0 passed, 0/0 skipped,
> 0/0 failed, 0/0 unsupported + Algo CIPHER 'des-cbc' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER 'aes-docsisbpi' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER 'des-
> docsisbpi' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AEAD
> 'aes-gcm' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AEAD
> 'chacha20-poly1305' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> ------------------------------------------------------- +
> + Sub Testsuites Total : 46
> + Sub Testsuites Skipped : 46
> + Sub Testsuites Passed : 0
> + Sub Testsuites Failed : 0
> + ------------------------------------------------------- +
> + Tests Total : 0
> + Tests Skipped : 0
> + Tests Executed : 0
> + Tests Unsupported: 0
> + Tests Passed : 0
> + Tests Failed : 0
> + ------------------------------------------------------- + Test Skipped
>
>
> 2. For 2 openssl devices I get 3 failures, which is curious as they are the
> same.
> Seems to be a space issue:
> USER1: Operation status 2
> USER1: No space for aad in single mbuf
> EAL: Test assert crosscheck_all_devices line 776 failed: Error occurred during
> processing
> EAL: Test assert crosscheck_with_profile_run line 841 failed: Error occurred
> during encryption
>
>
> Thanks,
> Ciara
[CP]
Comments above are based on V3 version of patch ... replied to the wrong email.
-Ciara
