Hi Abhinandan, Shijith,
Please see inline.
Thanks,
Anoob
> -----Original Message-----
> From: Shijith Thotton <sthot...@marvell.com>
> Sent: Tuesday, September 14, 2021 10:07 AM
> To: Gujjar, Abhinandan S <abhinandan.guj...@intel.com>; dev@dpdk.org
> Cc: Jerin Jacob Kollanukkaran <jer...@marvell.com>; Anoob Joseph
> <ano...@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavat...@marvell.com>; Akhil Goyal <gak...@marvell.com>; Ray
> Kinsella <m...@ashroe.eu>; Ankur Dwivedi <adwiv...@marvell.com>
> Subject: RE: [PATCH v3] eventdev: update crypto adapter metadata
> structures
>
> >> >>
> >> >> >> In crypto adapter metadata, reserved bytes in request info
> >> >> >> structure is a space holder for response info. It enforces an
> >> >> >> order of operation if the structures are updated using memcpy
> >> >> >> to avoid overwriting response info. It is logical to move the
> >> >> >> reserved space out of request info. It also solves the ordering
> >> >> >> issue
> >> mentioned before.
> >> >> >I would like to understand what kind of ordering issue you have
> >> >> >faced with the current approach. Could you please give an
> >> >> >example/sequence
> >> >> and explain?
> >> >> >
> >> >>
> >> >> I have seen this issue with crypto adapter autotest (#n215).
> >> >>
> >> >> Example:
> >> >> rte_memcpy(&m_data.response_info, &response_info,
> >> >> sizeof(response_info)); rte_memcpy(&m_data.request_info,
> >> >> &request_info, sizeof(request_info));
> >> >>
> >> >> Here response info is getting overwritten by request info.
> >> >> Above lines can reordered to fix the issue, but can be ignored
> >> >> with this
> >> patch.
> >> >There is a reason for designing the metadata in this way.
> >> >Right now, sizeof (union rte_event_crypto_metadata) is 16 bytes.
> >> >So, the session based case needs just 16 bytes to store the data.
> >> >Whereas, for sessionless case each crypto_ops requires another 16
> bytes.
> >> >
> >> >By changing the struct in the following way you are doubling the
> >> >memory requirement.
> >> >With the changes, for sessionless case, each crypto op requires 32
> >> >bytes of space instead of 16 bytes and the mempool will be bigger.
> >> >This will have the perf impact too!
> >> >
> >> >You can just copy the individual members(cdev_id & queue_pair_id)
> >> >after the response_info.
> >> >OR You have a better way?
> >> >
> >>
> >> I missed the second word of event structure. It adds an extra 8
> >> bytes, which is not required.
> >I guess you meant not adding, it is overlapping with request information.
> >> Let me know, what you think of below change to metadata structure.
> >>
> >> struct rte_event_crypto_metadata {
> >> uint64_t response_info; // 8 bytes
> >With this, it lags clarity saying it is an event information.
> >> struct rte_event_crypto_request request_info; // 8 bytes };
> >>
> >> Total structure size is 16 bytes.
> >>
> >> Response info can be set like below in test app:
> >> m_data.response_info = response_info.event;
> >>
> >> The main aim of this patch is to decouple response info from request info.
> >The decoupling was required as it was doing memcpy.
> >If you copy the individual members of request info(after response), you
> >don't require it.
>
> With this change, the application will be free of such constraints.
[Anoob] Why don't we make it like,
struct rte_event_crypto_metadata {
uint64_t ev_w0_template;
/**< Template of the first word of ``struct rte_event``
(rte_event.event) to be set in the events generated by crypto adapter in both
RTE_EVENT_CRYPTO_ADAPTER_OP_NEW and
* RTE_EVENT_CRYPTO_ADAPTER_OP_FORWARD modes.
*/
struct rte_event_crypto_request request_info;
};
This way, the usage would become more obvious and we will not have additional
space reserved as well.
>
> >>
> >> >>
> >> >> >>
> >> >> >> This patch removes the reserve field from request info and
> >> >> >> makes event crypto metadata type to structure from union to
> >> >> >> make space for response info.
> >> >> >>
> >> >> >> App and drivers are updated as per metadata change.
> >> >> >>
> >> >> >> Signed-off-by: Shijith Thotton <sthot...@marvell.com>
> >> >> >> Acked-by: Anoob Joseph <ano...@marvell.com>
> >> >> >> ---
> >> >> >> v3:
> >> >> >> * Updated ABI section of release notes.
> >> >> >>
> >> >> >> v2:
> >> >> >> * Updated deprecation notice.
> >> >> >>
> >> >> >> v1:
> >> >> >> * Rebased.
> >> >> >>
> >> >> >> app/test/test_event_crypto_adapter.c | 14
> >> >> >> +++++++-------
> >> >> >> doc/guides/rel_notes/deprecation.rst | 6 ------
> >> >> >> doc/guides/rel_notes/release_21_11.rst | 2 ++
> >> >> >> drivers/crypto/octeontx/otx_cryptodev_ops.c | 8 ++++----
> >> >> >> drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 4 ++--
> >> >> >> .../event/octeontx2/otx2_evdev_crypto_adptr_tx.h | 4 ++--
> >> >> >> lib/eventdev/rte_event_crypto_adapter.c | 8 ++++----
> >> >> >> lib/eventdev/rte_event_crypto_adapter.h | 15
> >> >> >> +++++---------
> -
> >> >> >> 8 files changed, 26 insertions(+), 35 deletions(-)
> >> >> >>
> >> >> >> diff --git a/app/test/test_event_crypto_adapter.c
> >> >> >> b/app/test/test_event_crypto_adapter.c
> >> >> >> index 3ad20921e2..0d73694d3a 100644
> >> >> >> --- a/app/test/test_event_crypto_adapter.c
> >> >> >> +++ b/app/test/test_event_crypto_adapter.c
> >> >> >> @@ -168,7 +168,7 @@ test_op_forward_mode(uint8_t
> session_less)
> >> {
> >> >> >> struct rte_crypto_sym_xform cipher_xform;
> >> >> >> struct rte_cryptodev_sym_session *sess;
> >> >> >> - union rte_event_crypto_metadata m_data;
> >> >> >> + struct rte_event_crypto_metadata m_data;
> >> >> >> struct rte_crypto_sym_op *sym_op;
> >> >> >> struct rte_crypto_op *op;
> >> >> >> struct rte_mbuf *m;
> >> >> >> @@ -368,7 +368,7 @@ test_op_new_mode(uint8_t session_less) {
> >> >> >> struct rte_crypto_sym_xform cipher_xform;
> >> >> >> struct rte_cryptodev_sym_session *sess;
> >> >> >> - union rte_event_crypto_metadata m_data;
> >> >> >> + struct rte_event_crypto_metadata m_data;
> >> >> >> struct rte_crypto_sym_op *sym_op;
> >> >> >> struct rte_crypto_op *op;
> >> >> >> struct rte_mbuf *m;
> >> >> >> @@ -406,7 +406,7 @@ test_op_new_mode(uint8_t session_less)
> >> >> >> if (cap &
> >> >> >> RTE_EVENT_CRYPTO_ADAPTER_CAP_SESSION_PRIVATE_DATA) {
> >> >> >> /* Fill in private user data information */
> >> >> >> rte_memcpy(&m_data.response_info,
> >> >> &response_info,
> >> >> >> - sizeof(m_data));
> >> >> >> + sizeof(response_info));
> >> >> >>
> rte_cryptodev_sym_session_set_user_data(sess,
> >> >> >> &m_data,
> sizeof(m_data));
> >> >> >> }
> >> >> >> @@ -426,7 +426,7 @@ test_op_new_mode(uint8_t session_less)
> >> >> >> op->private_data_offset = len;
> >> >> >> /* Fill in private data information */
> >> >> >> rte_memcpy(&m_data.response_info,
> &response_info,
> >> >> >> - sizeof(m_data));
> >> >> >> + sizeof(response_info));
> >> >> >> rte_memcpy((uint8_t *)op + len, &m_data,
> >sizeof(m_data));
> >> >> >> }
> >> >> >>
> >> >> >> @@ -519,7 +519,7 @@ configure_cryptodev(void)
> >> >> >> DEFAULT_NUM_XFORMS *
> >> >> >> sizeof(struct rte_crypto_sym_xform) +
> >> >> >> MAXIMUM_IV_LENGTH +
> >> >> >> - sizeof(union rte_event_crypto_metadata),
> >> >> >> + sizeof(struct rte_event_crypto_metadata),
> >> >> >> rte_socket_id());
> >> >> >> if (params.op_mpool == NULL) {
> >> >> >> RTE_LOG(ERR, USER1, "Can't create
> >CRYPTO_OP_POOL\n");
> >> >> @@ -549,12
> >> >> >> +549,12 @@ configure_cryptodev(void)
> >> >> >> * to include the session headers & private data
> >> >> >> */
> >> >> >> session_size =
> >> >> >> rte_cryptodev_sym_get_private_session_size(TEST_CDEV_ID);
> >> >> >> - session_size += sizeof(union rte_event_crypto_metadata);
> >> >> >> + session_size += sizeof(struct rte_event_crypto_metadata);
> >> >> >>
> >> >> >> params.session_mpool =
> >rte_cryptodev_sym_session_pool_create(
> >> >> >> "CRYPTO_ADAPTER_SESSION_MP",
> >> >> >> MAX_NB_SESSIONS, 0, 0,
> >> >> >> - sizeof(union rte_event_crypto_metadata),
> >> >> >> + sizeof(struct rte_event_crypto_metadata),
> >> >> >> SOCKET_ID_ANY);
> >> >> >> TEST_ASSERT_NOT_NULL(params.session_mpool,
> >> >> >> "session mempool allocation failed\n"); diff --
> git
> >> >> >> a/doc/guides/rel_notes/deprecation.rst
> >> >> >> b/doc/guides/rel_notes/deprecation.rst
> >> >> >> index 76a4abfd6b..58ee95c020 100644
> >> >> >> --- a/doc/guides/rel_notes/deprecation.rst
> >> >> >> +++ b/doc/guides/rel_notes/deprecation.rst
> >> >> >> @@ -266,12 +266,6 @@ Deprecation Notices
> >> >> >> values to the function
> >> >> >> ``rte_event_eth_rx_adapter_queue_add``
> >> using
> >> >> >> the structure ``rte_event_eth_rx_adapter_queue_add``.
> >> >> >>
> >> >> >> -* eventdev: Reserved bytes of ``rte_event_crypto_request`` is
> >> >> >> a space holder
> >> >> >> - for ``response_info``. Both should be decoupled for better
> >> >> >> clarity.
> >> >> >> - New space for ``response_info`` can be made by changing
> >> >> >> - ``rte_event_crypto_metadata`` type to structure from union.
> >> >> >> - This change is targeted for DPDK 21.11.
> >> >> >> -
> >> >> >> * metrics: The function ``rte_metrics_init`` will have a non-void
> return
> >> >> >> in order to notify errors instead of calling ``rte_exit``.
> >> >> >>
> >> >> >> diff --git a/doc/guides/rel_notes/release_21_11.rst
> >> >> >> b/doc/guides/rel_notes/release_21_11.rst
> >> >> >> index d707a554ef..ab76d5dd55 100644
> >> >> >> --- a/doc/guides/rel_notes/release_21_11.rst
> >> >> >> +++ b/doc/guides/rel_notes/release_21_11.rst
> >> >> >> @@ -100,6 +100,8 @@ ABI Changes
> >> >> >> Also, make sure to start the actual text at the margin.
> >> >> >>
> >> >>
> =======================================================
> >> >> >>
> >> >> >> +* eventdev: Modified type of ``union
> >> >> >> +rte_event_crypto_metadata`` to struct and
> >> >> >> + removed reserved bytes from ``struct
> rte_event_crypto_request``.
> >> >> >>
> >> >> >> Known Issues
> >> >> >> ------------
> >> >> >> diff --git a/drivers/crypto/octeontx/otx_cryptodev_ops.c
> >> >> >> b/drivers/crypto/octeontx/otx_cryptodev_ops.c
> >> >> >> index eac6796cfb..c51be63146 100644
> >> >> >> --- a/drivers/crypto/octeontx/otx_cryptodev_ops.c
> >> >> >> +++ b/drivers/crypto/octeontx/otx_cryptodev_ops.c
> >> >> >> @@ -710,17 +710,17 @@ submit_request_to_sso(struct ssows
> *ws,
> >> >> >> uintptr_t req,
> >> >> >> ssovf_store_pair(add_work, req,
> >> >> >> ws->grps[rsp_info->queue_id]); }
> >> >> >>
> >> >> >> -static inline union rte_event_crypto_metadata *
> >> >> >> +static inline struct rte_event_crypto_metadata *
> >> >> >> get_event_crypto_mdata(struct rte_crypto_op *op) {
> >> >> >> - union rte_event_crypto_metadata *ec_mdata;
> >> >> >> + struct rte_event_crypto_metadata *ec_mdata;
> >> >> >>
> >> >> >> if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION)
> >> >> >> ec_mdata =
> rte_cryptodev_sym_session_get_user_data(
> >> >> >> op->sym-
> >>session);
> >> >> >> else if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS &&
> >> >> >> op->private_data_offset)
> >> >> >> - ec_mdata = (union rte_event_crypto_metadata *)
> >> >> >> + ec_mdata = (struct rte_event_crypto_metadata *)
> >> >> >> ((uint8_t *)op + op->private_data_offset);
> >> >> >> else
> >> >> >> return NULL;
> >> >> >> @@ -731,7 +731,7 @@ get_event_crypto_mdata(struct
> rte_crypto_op
> >> >> *op)
> >> >> >> uint16_t __rte_hot otx_crypto_adapter_enqueue(void *port,
> >> >> >> struct rte_crypto_op *op) {
> >> >> >> - union rte_event_crypto_metadata *ec_mdata;
> >> >> >> + struct rte_event_crypto_metadata *ec_mdata;
> >> >> >> struct cpt_instance *instance;
> >> >> >> struct cpt_request_info *req;
> >> >> >> struct rte_event *rsp_info;
> >> >> >> diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c
> >> >> >> b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c
> >> >> >> index 42100154cd..952d1352f4 100644
> >> >> >> --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c
> >> >> >> +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c
> >> >> >> @@ -453,7 +453,7 @@ otx2_ca_enqueue_req(const struct
> >> otx2_cpt_qp
> >> >> *qp,
> >> >> >> struct rte_crypto_op *op,
> >> >> >> uint64_t cpt_inst_w7)
> >> >> >> {
> >> >> >> - union rte_event_crypto_metadata *m_data;
> >> >> >> + struct rte_event_crypto_metadata *m_data;
> >> >> >> union cpt_inst_s inst;
> >> >> >> uint64_t lmt_status;
> >> >> >>
> >> >> >> @@ -468,7 +468,7 @@ otx2_ca_enqueue_req(const struct
> >> otx2_cpt_qp
> >> >> *qp,
> >> >> >> }
> >> >> >> } else if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS
> &&
> >> >> >> op->private_data_offset) {
> >> >> >> - m_data = (union rte_event_crypto_metadata *)
> >> >> >> + m_data = (struct rte_event_crypto_metadata *)
> >> >> >> ((uint8_t *)op +
> >> >> >> op->private_data_offset);
> >> >> >> } else {
> >> >> >> diff --git
> >> >> >> a/drivers/event/octeontx2/otx2_evdev_crypto_adptr_tx.h
> >> >> >> b/drivers/event/octeontx2/otx2_evdev_crypto_adptr_tx.h
> >> >> >> index ecf7eb9f56..458e8306d7 100644
> >> >> >> --- a/drivers/event/octeontx2/otx2_evdev_crypto_adptr_tx.h
> >> >> >> +++ b/drivers/event/octeontx2/otx2_evdev_crypto_adptr_tx.h
> >> >> >> @@ -16,7 +16,7 @@
> >> >> >> static inline uint16_t
> >> >> >> otx2_ca_enq(uintptr_t tag_op, const struct rte_event *ev) {
> >> >> >> - union rte_event_crypto_metadata *m_data;
> >> >> >> + struct rte_event_crypto_metadata *m_data;
> >> >> >> struct rte_crypto_op *crypto_op;
> >> >> >> struct rte_cryptodev *cdev;
> >> >> >> struct otx2_cpt_qp *qp;
> >> >> >> @@ -37,7 +37,7 @@ otx2_ca_enq(uintptr_t tag_op, const struct
> >> >> >> rte_event
> >> >> >> *ev)
> >> >> >> qp_id = m_data->request_info.queue_pair_id;
> >> >> >> } else if (crypto_op->sess_type ==
> RTE_CRYPTO_OP_SESSIONLESS
> >> >> &&
> >> >> >> crypto_op->private_data_offset) {
> >> >> >> - m_data = (union rte_event_crypto_metadata *)
> >> >> >> + m_data = (struct rte_event_crypto_metadata *)
> >> >> >> ((uint8_t *)crypto_op +
> >> >> >> crypto_op->private_data_offset);
> >> >> >> cdev_id = m_data->request_info.cdev_id; diff --git
> >> >> >> a/lib/eventdev/rte_event_crypto_adapter.c
> >> >> >> b/lib/eventdev/rte_event_crypto_adapter.c
> >> >> >> index e1d38d383d..6977391ae9 100644
> >> >> >> --- a/lib/eventdev/rte_event_crypto_adapter.c
> >> >> >> +++ b/lib/eventdev/rte_event_crypto_adapter.c
> >> >> >> @@ -333,7 +333,7 @@ eca_enq_to_cryptodev(struct
> >> >> >> rte_event_crypto_adapter *adapter,
> >> >> >> struct rte_event *ev, unsigned int cnt) {
> >> >> >> struct rte_event_crypto_adapter_stats *stats = &adapter-
> >> >> >> >crypto_stats;
> >> >> >> - union rte_event_crypto_metadata *m_data = NULL;
> >> >> >> + struct rte_event_crypto_metadata *m_data = NULL;
> >> >> >> struct crypto_queue_pair_info *qp_info = NULL;
> >> >> >> struct rte_crypto_op *crypto_op;
> >> >> >> unsigned int i, n;
> >> >> >> @@ -371,7 +371,7 @@ eca_enq_to_cryptodev(struct
> >> >> >> rte_event_crypto_adapter *adapter,
> >> >> >> len++;
> >> >> >> } else if (crypto_op->sess_type ==
> >> >> RTE_CRYPTO_OP_SESSIONLESS &&
> >> >> >> crypto_op->private_data_offset) {
> >> >> >> - m_data = (union
> rte_event_crypto_metadata *)
> >> >> >> + m_data = (struct
> rte_event_crypto_metadata *)
> >> >> >> ((uint8_t *)crypto_op +
> >> >> >> crypto_op-
> >private_data_offset);
> >> >> >> cdev_id = m_data->request_info.cdev_id;
> @@ -
> >> >> >> 504,7 +504,7 @@ eca_ops_enqueue_burst(struct
> >> >> rte_event_crypto_adapter
> >> >> >> *adapter,
> >> >> >> struct rte_crypto_op **ops, uint16_t num) {
> >> >> >> struct rte_event_crypto_adapter_stats *stats = &adapter-
> >> >> >> >crypto_stats;
> >> >> >> - union rte_event_crypto_metadata *m_data = NULL;
> >> >> >> + struct rte_event_crypto_metadata *m_data = NULL;
> >> >> >> uint8_t event_dev_id = adapter->eventdev_id;
> >> >> >> uint8_t event_port_id = adapter->event_port_id;
> >> >> >> struct rte_event events[BATCH_SIZE]; @@ -523,7 +523,7 @@
> >> >> >> eca_ops_enqueue_burst(struct rte_event_crypto_adapter
> *adapter,
> >> >> >> ops[i]->sym->session);
> >> >> >> } else if (ops[i]->sess_type ==
> >> RTE_CRYPTO_OP_SESSIONLESS &&
> >> >> >> ops[i]->private_data_offset) {
> >> >> >> - m_data = (union
> rte_event_crypto_metadata *)
> >> >> >> + m_data = (struct
> rte_event_crypto_metadata *)
> >> >> >> ((uint8_t *)ops[i] +
> >> >> >> ops[i]->private_data_offset);
> >> >> >> }
> >> >> >> diff --git a/lib/eventdev/rte_event_crypto_adapter.h
> >> >> >> b/lib/eventdev/rte_event_crypto_adapter.h
> >> >> >> index f8c6cca87c..3c24d9d9df 100644
> >> >> >> --- a/lib/eventdev/rte_event_crypto_adapter.h
> >> >> >> +++ b/lib/eventdev/rte_event_crypto_adapter.h
> >> >> >> @@ -200,11 +200,6 @@ enum rte_event_crypto_adapter_mode {
> >> >> >> * provide event request information to the adapter.
> >> >> >> */
> >> >> >> struct rte_event_crypto_request {
> >> >> >> - uint8_t resv[8];
> >> >> >> - /**< Overlaps with first 8 bytes of struct rte_event
> >> >> >> - * that encode the response event information. Application
> >> >> >> - * is expected to fill in struct rte_event response_info.
> >> >> >> - */
> >> >> >> uint16_t cdev_id;
> >> >> >> /**< cryptodev ID to be used */
> >> >> >> uint16_t queue_pair_id;
> >> >> >> @@ -223,16 +218,16 @@ struct rte_event_crypto_request {
> >> >> >> * operation. If the transfer is done by SW, event response
> >> information
> >> >> >> * will be used by the adapter.
> >> >> >> */
> >> >> >> -union rte_event_crypto_metadata {
> >> >> >> - struct rte_event_crypto_request request_info;
> >> >> >> - /**< Request information to be filled in by application
> >> >> >> - * for RTE_EVENT_CRYPTO_ADAPTER_OP_FORWARD mode.
> >> >> >> - */
> >> >> >> +struct rte_event_crypto_metadata {
> >> >> >> struct rte_event response_info;
> >> >> >> /**< Response information to be filled in by application
> >> >> >> * for RTE_EVENT_CRYPTO_ADAPTER_OP_NEW and
> >> >> >> * RTE_EVENT_CRYPTO_ADAPTER_OP_FORWARD mode.
> >> >> >> */
> >> >> >> + struct rte_event_crypto_request request_info;
> >> >> >> + /**< Request information to be filled in by application
> >> >> >> + * for RTE_EVENT_CRYPTO_ADAPTER_OP_FORWARD mode.
> >> >> >> + */
> >> >> >> };
> >> >> >>
> >> >> >> /**
> >> >> >> --
> >> >> >> 2.25.1
