On 5/5/2018 7:59 PM, Shahaf Shuler wrote: > Hi Ferruh, Dai, >> Subject: [dpdk-dev] [PATCH v7] ethdev: check Rx/Tx offloads >> >> This patch check if a input requested offloading is valid or not. >> Any reuqested offloading must be supported in the device capabilities. >> Any offloading is disabled by default if it is not set in the parameter >> dev_conf->[rt]xmode.offloads to rte_eth_dev_configure( ) and [rt]x_conf- >>> offloads to rte_eth_[rt]x_queue_setup( ). >> From application, a pure per-port offloading can't be enabled on any queue if >> it hasn't been enabled in rte_eth_dev_configure( ). >> If any offloading is enabled in rte_eth_dev_configure( ) by application, it >> is >> enabled on all queues no matter whether it is per-queue or per-port type >> and no matter whether it is set or cleared in [rt]x_conf->offloads to >> rte_eth_[rt]x_queue_setup( ). >> The underlying PMD must be aware that the requested offloadings to PMD >> specific queue_setup( ) function only carries those offloadings only enabled >> for the queue but not enabled in rte_eth_dev_configure( ) and they are >> certain per-queue type. >> >> This patch can make above such checking in a common way in rte_ethdev >> layer to avoid same checking in underlying PMD. >> >> Signed-off-by: Wei Dai <wei....@intel.com> >> Signed-off-by: Ferruh Yigit <ferruh.yi...@intel.com> >> >> --- >> v7: >> Give the maximum freedom for upper application, only minimal checking is >> performed in ethdev layer. >> Only requested specific pure per-queue offloadings are input to underlying >> PMD. >> >> v6: >> No need enable an offload in queue_setup( ) if it has already been enabled >> in dev_configure( ) >> >> v5: >> keep offload settings sent to PMD same as those from application >> >> v4: >> fix a wrong description in git log message. >> >> v3: >> rework according to dicision of offloading API in community >> >> v2: >> add offloads checking in rte_eth_dev_configure( ). >> check if a requested offloading is supported. >> --- >> lib/librte_ethdev/rte_ethdev.c | 150 >> +++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 150 insertions(+) >> >> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c >> index e560524..0ad05eb 100644 >> --- a/lib/librte_ethdev/rte_ethdev.c >> +++ b/lib/librte_ethdev/rte_ethdev.c >> @@ -1139,6 +1139,28 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t >> nb_rx_q, uint16_t nb_tx_q, >> ETHER_MAX_LEN; >> } >> >> + /* Any requested offloading must be within its device capabilities */ >> + if ((local_conf.rxmode.offloads & dev_info.rx_offload_capa) != >> + local_conf.rxmode.offloads) { >> + RTE_PMD_DEBUG_TRACE("ethdev port_id=%d requested Rx >> offloads " >> + "0x%" PRIx64 " doesn't match Rx offloads " >> + "capabilities 0x%" PRIx64 "\n", >> + port_id, >> + local_conf.rxmode.offloads, >> + dev_info.rx_offload_capa); >> + return -EINVAL; > > While I am OK with such behavior, we should be more careful not to get into > the same issue as in [1]. > There are PMD which don't report the capabilities correctly however do expect > to have the offload configured. > > All I am saying it is worth a check and cautious decision if it is right to > include this one w/o prior application notice and at such late RC of the > release.
This is valid concern. I think this is better than [1] which was less clear than this check but yes still a concern. > >> + } >> + if ((local_conf.txmode.offloads & dev_info.tx_offload_capa) != >> + local_conf.txmode.offloads) { >> + RTE_PMD_DEBUG_TRACE("ethdev port_id=%d requested Tx >> offloads " >> + "0x%" PRIx64 " doesn't match Tx offloads " >> + "capabilities 0x%" PRIx64 "\n", >> + port_id, >> + local_conf.txmode.offloads, >> + dev_info.tx_offload_capa); >> + return -EINVAL; >> + } >> + >> /* Check that device supports requested rss hash functions. */ >> if ((dev_info.flow_type_rss_offloads | >> dev_conf->rx_adv_conf.rss_conf.rss_hf) != @@ -1414,6 +1436,8 >> @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id, >> struct rte_eth_dev_info dev_info; >> struct rte_eth_rxconf local_conf; >> void **rxq; >> + uint64_t pure_port_offload_capa; >> + uint64_t only_enabled_for_queue; >> >> RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); >> >> @@ -1504,6 +1528,68 @@ rte_eth_rx_queue_setup(uint16_t port_id, >> uint16_t rx_queue_id, >> &local_conf.offloads); >> } >> >> + /* >> + * The requested offloadings by application for this queue >> + * can be per-queue type or per-port type. and >> + * they must be within the device offloading capabilities. >> + */ >> + if ((local_conf.offloads & dev_info.rx_offload_capa) != >> + local_conf.offloads) { >> + RTE_PMD_DEBUG_TRACE("Ethdev port_id=%d >> rx_queue_id=%d " >> + "Requested offload 0x%" PRIx64 "doesn't " >> + "match per-queue capability 0x%" PRIx64 >> + " in %s\n", >> + port_id, >> + rx_queue_id, >> + local_conf.offloads, >> + dev_info.rx_queue_offload_capa, >> + __func__); >> + return -EINVAL; >> + } >> + >> + /* >> + * A pure per-port offloading can't be enabled for any queue >> + * if it hasn't been enabled in rte_eth_dev_configure( ). >> + * >> + * Following pure_port_offload_capa is the capabilities which >> + * can't be enabled on some queue while disabled on other queue. >> + * pure_port_offload_capa must be enabled or disabled on all >> + * queues at same time. >> + * >> + * Following only_enabled_for_queue is the offloadings which >> + * are enabled for this queue but hasn't been enabled in >> + * rte_eth_dev_configure( ). >> + */ >> + pure_port_offload_capa = dev_info.rx_offload_capa ^ >> + dev_info.rx_queue_offload_capa; >> + only_enabled_for_queue = (local_conf.offloads ^ >> + dev->data->dev_conf.rxmode.offloads) & >> local_conf.offloads; > > It looks like above logic could be a lot simpler. > > How about: > local_conf.offloads &= ~dev->data->dev_conf.rxmode.offloads; // keep only the > added offloads on top of the port ones > if ((local_conf.offloads & dev_info.rx_queue_offload_capa) != > local_conf.offloads) { //check if added offloads are part of the queue > offload capa > ERROR... +1 > > >> + if (only_enabled_for_queue & pure_port_offload_capa) { >> + RTE_PMD_DEBUG_TRACE("Ethdev port_id=%d >> rx_queue_id=%d, only " >> + "enabled offload 0x%" PRIx64 "for this " >> + "queue haven't been enabled in " >> + "dev_configure( ), they are within " >> + "pure per-port capabilities 0x%" PRIx64 > > Need to re-work this error message. The user doesn't know what are "pure > per-port capabilities" +1 > >> + " in %s\n", >> + port_id, >> + rx_queue_id, >> + only_enabled_for_queue, >> + pure_port_offload_capa, >> + __func__); >> + return -EINVAL; >> + } >> + >> + /* >> + * If an offloading has already been enabled in >> + * rte_eth_dev_configure(), it has been enabled on all queues, >> + * so there is no need to enable it in this queue again. >> + * The local_conf.offloads input to underlying PMD only carries >> + * those offloadings which are only enabled on this queue and >> + * not enabled on all queues. >> + * The underlying PMD must be aware of this point. >> + */ >> + local_conf.offloads &= ~dev->data->dev_conf.rxmode.offloads; >> + >> ret = (*dev->dev_ops->rx_queue_setup)(dev, rx_queue_id, >> nb_rx_desc, >> socket_id, &local_conf, mp); >> if (!ret) { >> @@ -1549,6 +1635,8 @@ rte_eth_tx_queue_setup(uint16_t port_id, >> uint16_t tx_queue_id, >> struct rte_eth_dev_info dev_info; >> struct rte_eth_txconf local_conf; >> void **txq; >> + uint64_t pure_port_offload_capa; >> + uint64_t only_enabled_for_queue; >> >> RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); >> >> @@ -1612,6 +1700,68 @@ rte_eth_tx_queue_setup(uint16_t port_id, >> uint16_t tx_queue_id, >> &local_conf.offloads); >> } >> >> + /* >> + * The requested offloadings by application for this queue >> + * can be per-queue type or per-port type. and >> + * they must be within the device offloading capabilities. >> + */ >> + if ((local_conf.offloads & dev_info.tx_offload_capa) != >> + local_conf.offloads) { >> + RTE_PMD_DEBUG_TRACE("Ethdev port_id=%d >> tx_queue_id=%d " >> + "Requested offload 0x%" PRIx64 "doesn't " >> + "match per-queue capability 0x%" PRIx64 >> + " in %s\n", >> + port_id, >> + tx_queue_id, >> + local_conf.offloads, >> + dev_info.tx_queue_offload_capa, >> + __func__); >> + return -EINVAL; >> + } >> + >> + /* >> + * A pure per-port offloading can't be enabled for any queue >> + * if it hasn't been enabled in rte_eth_dev_configure( ). >> + * >> + * Following pure_port_offload_capa is the capabilities which >> + * can't be enabled on some queue while disabled on other queue. >> + * pure_port_offload_capa must be enabled or disabled on all >> + * queues at same time. >> + * >> + * Following only_enabled_for_queue is the offloadings which >> + * are enabled for this queue but hasn't been enabled in >> + * rte_eth_dev_configure( ). >> + */ >> + pure_port_offload_capa = dev_info.tx_offload_capa ^ >> + dev_info.tx_queue_offload_capa; >> + only_enabled_for_queue = (local_conf.offloads ^ >> + dev->data->dev_conf.txmode.offloads) & >> local_conf.offloads; > > Same comments as in the Rx part. > >> + if (only_enabled_for_queue & pure_port_offload_capa) { >> + RTE_PMD_DEBUG_TRACE("Ethdev port_id=%d >> tx_queue_id=%d, only " >> + "enabled offload 0x%" PRIx64 "for this " >> + "queue haven't been enabled in " >> + "dev_configure( ), they are within " >> + "pure per-port capabilities 0x%" PRIx64 >> + " in %s\n", >> + port_id, >> + tx_queue_id, >> + only_enabled_for_queue, >> + pure_port_offload_capa, >> + __func__); >> + return -EINVAL; >> + } >> + >> + /* >> + * If an offloading has already been enabled in >> + * rte_eth_dev_configure(), it has been enabled on all queues, >> + * so there is no need to enable it in this queue again. >> + * The local_conf.offloads input to underlying PMD only carries >> + * those offloadings which are only enabled on this queue and >> + * not enabled on all queues. >> + * The underlying PMD must be aware of this point. >> + */ >> + local_conf.offloads &= ~dev->data->dev_conf.txmode.offloads; >> + >> return eth_err(port_id, (*dev->dev_ops->tx_queue_setup)(dev, >> tx_queue_id, nb_tx_desc, socket_id, &local_conf)); } >> -- >> 2.7.5 > > > As for Ferruh's comment >> >> PMDs needs to be updated for: >> 1- Remove existing offload verify checks >> 2- Update offload configure logic based on new values >> >> (1) can be part of this patch. But PMD maintainers should send update >> for (2) if a change required. >> >> cc'ed Shahaf, specially for (2) one. > > I think PMD maintainers can help with that. If it will be integrated enough > time before the release Mellanox PMDs can be converted by us. > Thanks. As far as I can see in v8 Wei is adding some code [2] to keep same input for the PMD to not break the logic in PMD. But later PMD can be updated for better support of new offload input to the PMD. [2] + uint64_t offloads = conf->offloads | + dev->data->dev_conf.rxmode.offloads; > > > > [1] > http://dpdk.org/dev/patchwork/patch/38645/ > >