Re: [PATCH] service: fix getting service lcore attributes
> From: Piotr Krzewinski
> Sent: Friday, December 20, 2024 2:29 PM
> To: Van Haaren, Harry
> Cc: dev@dpdk.org ; Krzewinski, Piotr
> ; mattias.ronnb...@ericsson.com
> ; sta...@dpdk.org
> Subject: [PATCH] service: fix getting service lcore attributes
>
> Perf test service_perf_autotest was failing after introduction
> of lcore variables. Fixed getting of idle_ and error_ service
> call statistics.
>
> Fixes: b24bbaedbba2 ("service: keep per-lcore state in lcore variable")
> Cc: mattias.ronnb...@ericsson.com
> Cc: sta...@dpdk.org
>
> Signed-off-by: Piotr Krzewinski
Thanks - verified that these two occurrences were the last two of such kind in
the file too!
Acked-by: Harry van Haaren
Side-note; seems CI flagged a warning, but not a real issue, artifact is empty
https://lab.dpdk.org/results/dashboard/patchsets/32181/
https://lab.dpdk.org/results/dashboard/testruns/logs/1492956/
> ---
> lib/eal/common/rte_service.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c
> index dad3150df9..1821746337 100644
> --- a/lib/eal/common/rte_service.c
> +++ b/lib/eal/common/rte_service.c
> @@ -879,7 +879,7 @@ lcore_attr_get_service_calls(uint32_t service_id,
> unsigned int lcore)
> static uint64_t
> lcore_attr_get_service_idle_calls(uint32_t service_id, unsigned int lcore)
> {
> - struct core_state *cs = &lcore_states[lcore];
> + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states);
>
> return
> rte_atomic_load_explicit(&cs->service_stats[service_id].idle_calls,
> rte_memory_order_relaxed);
> @@ -888,7 +888,7 @@ lcore_attr_get_service_idle_calls(uint32_t service_id,
> unsigned int lcore)
> static uint64_t
> lcore_attr_get_service_error_calls(uint32_t service_id, unsigned int lcore)
> {
> - struct core_state *cs = &lcore_states[lcore];
> + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states);
>
> return
> rte_atomic_load_explicit(&cs->service_stats[service_id].error_calls,
> rte_memory_order_relaxed);
> --
> 2.36.0
Re: [PATCH 1/1] vhost: fix a double fetch when dequeue offloading
On Thu, 19 Dec 2024 14:38:28 +0800
Yunjian Wang wrote:
> diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c
> index 69901ab3b5..5c40ae7069 100644
> --- a/lib/vhost/virtio_net.c
> +++ b/lib/vhost/virtio_net.c
> @@ -2914,10 +2914,12 @@ desc_to_mbuf(struct virtio_net *dev, struct
> vhost_virtqueue *vq,
>* in a contiguous virtual area.
>*/
> copy_vnet_hdr_from_desc(&tmp_hdr, buf_vec);
> - hdr = &tmp_hdr;
> } else {
> - hdr = (struct virtio_net_hdr
> *)((uintptr_t)buf_vec[0].buf_addr);
> + rte_memcpy((void *)(uintptr_t)&tmp_hdr,
> + (void *)(uintptr_t)buf_vec[0].buf_addr,
> + sizeof(struct virtio_net_hdr));
> }
> + hdr = &tmp_hdr;
Since this if block is just an optimization of the case where vnet header
is contiguous why not just always use copy_vnet_hdr_from_desc? and inline it?
Re: [PATCH v1 1/1] dts: add EAL sanity check suite
On Fri, 20 Dec 2024 15:49:10 + Paul Szczepanek wrote: > On 09/12/2024 21:54, Dean Marx wrote: > > Add a test suite to replace hello_world which simply > > starts and stops a testpmd session. The user can use > > this as a sanity check to verify their configuration. > > > > Signed-off-by: Dean Marx > > --- > > dts/tests/TestSuite_eal.py | 28 > > 1 file changed, 28 insertions(+) > > create mode 100644 dts/tests/TestSuite_eal.py > > > > diff --git a/dts/tests/TestSuite_eal.py b/dts/tests/TestSuite_eal.py > > new file mode 100644 > > index 00..35660178f0 > > --- /dev/null > > +++ b/dts/tests/TestSuite_eal.py > > @@ -0,0 +1,28 @@ > > +# SPDX-License-Identifier: BSD-3-Clause > > +# Copyright(c) 2024 University of New Hampshire > > + > > +"""DPDK EAL sanity check suite. Do not use the term "sanity check". https://inclusivenaming.org/word-lists/tier-2/sanity-check/
Re: [PATCH v1 1/1] dts: add EAL sanity check suite
On 09/12/2024 21:54, Dean Marx wrote: > Add a test suite to replace hello_world which simply > starts and stops a testpmd session. The user can use > this as a sanity check to verify their configuration. > > Signed-off-by: Dean Marx > --- > dts/tests/TestSuite_eal.py | 28 > 1 file changed, 28 insertions(+) > create mode 100644 dts/tests/TestSuite_eal.py > > diff --git a/dts/tests/TestSuite_eal.py b/dts/tests/TestSuite_eal.py > new file mode 100644 > index 00..35660178f0 > --- /dev/null > +++ b/dts/tests/TestSuite_eal.py > @@ -0,0 +1,28 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2024 University of New Hampshire > + > +"""DPDK EAL sanity check suite. > + > +Starts and stops a testpmd session to verify EAL parameters > +are properly configured. > +""" > + > +from framework.remote_session.testpmd_shell import TestPmdShell > +from framework.test_suite import TestSuite, func_test > + > + > +class TestEal(TestSuite): > +"""EAL test suite. One test case, which starts and stops a testpmd > session.""" > + > +@func_test > +def test_verify_eal(self) -> None: > +"""EAL sanity test. > + > +Steps: > +Start testpmd session and check status. > +Verify: > +The testpmd session is alive after starting. > +""" > +with TestPmdShell(node=self.sut_node) as testpmd: > +testpmd.start() > +self.verify(True, "True") Not sure the self.verify(True, "True") is needed. Reviewed-by: Paul Szczepanek
Re: [RFC 0/8] ioring: network driver
On Fri, 20 Dec 2024 14:34:27 + Konstantin Ananyev wrote: > > > From: Konstantin Ananyev [mailto:konstantin.anan...@huawei.com] > > > > > > > > > This is first draft of new simplified TAP device that uses > > > > > > the Linux kernel ioring API to provide a read/write ring > > > > > > with kernel. > > > > > > > > > > > > This is split from tap device because there are so many > > > > > > unnecessary things in existing tap, and supporting ioring is > > > > > > better without ifdefs etc. The default name of the tap > > > > > > device is different that other uses in DPDK but the driver > > > > > > tries to keep the same relevant devargs as before. > > > > > > > > > > > > This driver will only provide features that match what kernel > > > > > > does, so no flow support etc. The next version will add checksum > > > > > > and multi-segment packets. Some of the doc files may need update > > > > > > as well. > > > > > > > > > > Makes sense to me, though didn't properly look inside. > > > > > One thing - probably add a 'tap' into the name, > > > > > 'tap_ioiring' or so, otherwise 'ioring' is a bit too generic > > > > > and might be confusing. > > > > Konstantin is referring to the name of the driver and the source code file > > names, "net/ioring" -> "net/tap_ioring". > > Yep, that what I meant. My thoughts, are shorter name is better, and avoids confusion. There are already multiple drivers that create tap devices: tap and virtio_user. > > > > > > > > > > There are some userspaces that look for "e*" in name for some setups. > > > > Stephen is referring to the device name of an instantiated interface, e.g. > > "eth0". > > > > And yes, assuming devices named "e*" are Ethernet devices is a common hack > > in Linux applications. I've done it myself. :-) > > Ok... and why such practice should prevent us to name PMD itself in a way we > think is appropriate? > I am more leaning towards not having a default name at all. The policy should be done by Linux (udev) not DPDK. If user wants a name they can add it via devargs.
net/mlx5: wrong Rx/Tx descriptor limits when DevX is off
Hello,
I have run into a regression following an update to stable dpdk-24.11 with
a number of my Mellanox cx4/5/6 nics. This regression occurs with all nics
in my lab which have DevX disabled: using mstconfig utility, I can see the
flag UCTX_EN is not set.
Mainly, the issue is that the ports cannot be started, with the following
error logs in the journal:
Set nb_rxd=1 (asked=512) for port=0
Set nb_txd=1 (asked=512) for port=0
starting port 0
Initializing port 0 [7c:fe:90:65:e6:54]
port 0: ntfp1 (mlx5_pci)
nb_rxq=2 nb_txq=2
rxq0=c9 rxq1=c25
txq0=c9 txq1=c25
port 0: rx_scatter=0 tx_scatter=0 max_rx_frame=1526
mlx5_net: port 0 number of descriptors requested for Tx queue 0 must be
higher than MLX5_TX_COMP_THRESH, using 33 instead of 1
mlx5_net: port 0 increased number of descriptors in Tx queue 0 to the next
power of two (64)
mlx5_net: port 0 number of descriptors requested for Tx queue 1 must be
higher than MLX5_TX_COMP_THRESH, using 33 instead of 1
mlx5_net: port 0 increased number of descriptors in Tx queue 1 to the next
power of two (64)
mlx5_net: Port 0 Rx queue 0 CQ creation failure.
mlx5_net: port 0 Rx queue allocation failed: Cannot allocate memory
rte_eth_dev_start(port 0) failed, error=-12
Failed to start port 0, set link down
Failed to start port 0
Looking more precisely into the problem, it appears that the number of Rx
and Tx descriptors configured for my queues is 1. This happens because
mlx5_dev_infos_get() return a limit of 1 for both Rx and Tx, which is
unexpected. I identified this patch to be responsible for the regression:
4c3d7961d9002: net/mlx5: fix reported Rx/Tx descriptor limits
https://git.dpdk.org/dpdk/commit/?id=4c3d7961d9002bb715a8ee76bcf464d633316d4c
After doing some debugging, I noticed that hca_attr.log_max_wq_sz is never
configured. This should be done in mlx5_devx_cmd_query_hca_attr() which is
called in this bit of code:
https://git.dpdk.org/dpdk/tree/drivers/common/mlx5/mlx5_common.c#n681
/*
* When CTX is created by Verbs, query HCA attribute is unsupported.
* When CTX is imported, we cannot know if it is created by DevX or
* Verbs. So, we use query HCA attribute function to check it.
*/
if (cdev->config.devx || cdev->config.device_fd != MLX5_ARG_UNSET) {
/* Query HCA attributes. */
ret = mlx5_devx_cmd_query_hca_attr(cdev->ctx, &cdev->config.hca_attr);
if (ret) {
DRV_LOG(ERR, "Unable to read HCA caps in DevX mode.");
rte_errno = ENOTSUP;
goto error;
}
cdev->config.devx = 1;
}
DRV_LOG(DEBUG, "DevX is %ssupported.", cdev->config.devx ? "" : "NOT ");
I deduced that following the above patch, the correct value for maximum Rx
and Tx descriptors will only be set if DevX is enabled (see the if
condition on cdev->config.devx). If it is disabled, then maximum Rx and Tx
descriptors will be 1, which will make the ports fail to start. Perhaps we
should keep the previous default value (65535) if config.devx == 0 (DevX
off)? This could be done like this, for example:
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index 7708a0b80883..8ba3eb4a32de 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -359,10 +359,12 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct
rte_eth_dev_info *info)
info->flow_type_rss_offloads = ~MLX5_RSS_HF_MASK;
mlx5_set_default_params(dev, info);
mlx5_set_txlimit_params(dev, info);
- info->rx_desc_lim.nb_max =
- 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz;
- info->tx_desc_lim.nb_max =
- 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz;
+ if (priv->sh->cdev->config.devx) {
+ info->rx_desc_lim.nb_max =
+ 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz;
+ info->tx_desc_lim.nb_max =
+ 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz;
+ }
if (priv->sh->cdev->config.hca_attr.mem_rq_rmp &&
priv->obj_ops.rxq_obj_new == devx_obj_ops.rxq_obj_new)
info->dev_capa |= RTE_ETH_DEV_CAPA_RXQ_SHARE;
Thanks in advance for your help.
Regards,
Edwin Brossette.
Re: [PATCH v2 1/1] vhost: fix a double fetch when dequeue offloading
On Fri, 20 Dec 2024 11:49:55 +0800
Yunjian Wang wrote:
> The hdr->csum_start does two successive reads from user space to read a
> variable length data structure. The result overflow if the data structure
> changes between the two reads.
>
> To fix this, we can prevent double fetch issue by copying virtio_hdr to
> the temporary variable.
>
> Fixes: 4dc4e33ffa10 ("net/virtio: fix Rx checksum calculation")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Yunjian Wang
How about something like the following *untested*
diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c
index 69901ab3b5..c65cb639b2 100644
--- a/lib/vhost/virtio_net.c
+++ b/lib/vhost/virtio_net.c
@@ -2861,25 +2861,28 @@ vhost_dequeue_offload(struct virtio_net *dev, struct
virtio_net_hdr *hdr,
}
}
-static __rte_noinline void
+static inline int
copy_vnet_hdr_from_desc(struct virtio_net_hdr *hdr,
- struct buf_vector *buf_vec)
+ const struct buf_vector *buf_vec,
+ uint16_t nr_vec)
{
- uint64_t len;
- uint64_t remain = sizeof(struct virtio_net_hdr);
- uint64_t src;
- uint64_t dst = (uint64_t)(uintptr_t)hdr;
+ size_t remain = sizeof(struct virtio_net_hdr);
+ uint8_t *dst = (uint8_t *)hdr;
- while (remain) {
- len = RTE_MIN(remain, buf_vec->buf_len);
- src = buf_vec->buf_addr;
- rte_memcpy((void *)(uintptr_t)dst,
- (void *)(uintptr_t)src, len);
+ while (remain > 0) {
+ size_t len = RTE_MIN(remain, buf_vec->buf_len);
+ const void *src = (const void *)(uintptr_t)buf_vec->buf_addr;
+ if (unlikely(nr_vec == 0))
+ return -1;
+
+ memcpy(dst, src, len);
remain -= len;
dst += len;
buf_vec++;
+ --nr_vec;
}
+ return 0;
}
static __rte_always_inline int
@@ -2908,16 +2911,12 @@ desc_to_mbuf(struct virtio_net *dev, struct
vhost_virtqueue *vq,
*/
if (virtio_net_with_host_offload(dev)) {
- if (unlikely(buf_vec[0].buf_len < sizeof(struct
virtio_net_hdr))) {
- /*
-* No luck, the virtio-net header doesn't fit
-* in a contiguous virtual area.
-*/
- copy_vnet_hdr_from_desc(&tmp_hdr, buf_vec);
- hdr = &tmp_hdr;
- } else {
- hdr = (struct virtio_net_hdr
*)((uintptr_t)buf_vec[0].buf_addr);
- }
+ if (unlikely(copy_vnet_hdr_from_desc(&tmp_hdr, buf_vec, nr_vec)
!= 0))
+ return -1;
+
+ /* ensure that compiler does not delay copy */
+ rte_compiler_barrier();
+ hdr = &tmp_hdr;
}
for (vec_idx = 0; vec_idx < nr_vec; vec_idx++) {
@@ -3363,7 +3362,6 @@ virtio_dev_tx_batch_packed(struct virtio_net *dev,
{
uint16_t avail_idx = vq->last_avail_idx;
uint32_t buf_offset = sizeof(struct virtio_net_hdr_mrg_rxbuf);
- struct virtio_net_hdr *hdr;
uintptr_t desc_addrs[PACKED_BATCH_SIZE];
uint16_t ids[PACKED_BATCH_SIZE];
uint16_t i;
@@ -3382,8 +3380,12 @@ virtio_dev_tx_batch_packed(struct virtio_net *dev,
if (virtio_net_with_host_offload(dev)) {
vhost_for_each_try_unroll(i, 0, PACKED_BATCH_SIZE) {
- hdr = (struct virtio_net_hdr *)(desc_addrs[i]);
- vhost_dequeue_offload(dev, hdr, pkts[i],
legacy_ol_flags);
+ struct virtio_net_hdr hdr;
+
+ memcpy(&hdr, (void *)desc_addrs[i], sizeof(struct
virtio_net_hdr));
+ rte_compiler_barrier();
+
+ vhost_dequeue_offload(dev, &hdr, pkts[i],
legacy_ol_flags);
}
}
Re: [PATCH v2 5/5] vhost: use strlcpy instead of strncpy
On Thu, Dec 19, 2024 at 02:49:55PM -0800, Stephen Hemminger wrote: > Some tools such as gcc address sanitizer will complain if strncpy > is used to completely fill a string since it will not be null > terminated. Since the previous code forced as null at end, > use strlcpy() to get the same effect. > > Signed-off-by: Stephen Hemminger > --- > lib/vhost/socket.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c > index d29d15494c..f938189cc2 100644 > --- a/lib/vhost/socket.c > +++ b/lib/vhost/socket.c > @@ -359,8 +359,7 @@ create_unix_socket(struct vhost_user_socket *vsocket) > > memset(un, 0, sizeof(*un)); > un->sun_family = AF_UNIX; > - strncpy(un->sun_path, vsocket->path, sizeof(un->sun_path)); > - un->sun_path[sizeof(un->sun_path) - 1] = '\0'; > + strlcpy(un->sun_path, vsocket->path, sizeof(un->sun_path)); > Acked-by: Bruce Richardson
[RFC PATCH 1/2] dts: add scoping and shell registration to Node
Add a basic scoping mechanism to Nodes, to improve the control over
test suite led environmental changes. Moreover, keep a pool of active
shells based on scope, therefore allowing shells to register
themselves.
Signed-off-by: Luca Vizzarro
---
.../single_active_interactive_shell.py| 2 +
dts/framework/runner.py | 15 +-
dts/framework/testbed_model/capability.py | 35 ++--
dts/framework/testbed_model/node.py | 65 ++-
dts/framework/testbed_model/sut_node.py | 14 +-
dts/tests/TestSuite_blocklist.py | 16 +-
dts/tests/TestSuite_checksum_offload.py | 168 +-
dts/tests/TestSuite_dynamic_queue_conf.py | 52 +++---
dts/tests/TestSuite_l2fwd.py | 20 +--
dts/tests/TestSuite_mac_filter.py | 124 +++--
dts/tests/TestSuite_pmd_buffer_scatter.py | 26 ++-
dts/tests/TestSuite_smoke_tests.py| 4 +-
dts/tests/TestSuite_vlan.py | 42 ++---
13 files changed, 333 insertions(+), 250 deletions(-)
diff --git a/dts/framework/remote_session/single_active_interactive_shell.py
b/dts/framework/remote_session/single_active_interactive_shell.py
index c43c54e457..910af8f655 100644
--- a/dts/framework/remote_session/single_active_interactive_shell.py
+++ b/dts/framework/remote_session/single_active_interactive_shell.py
@@ -112,7 +112,9 @@ def __init__(
the name of the underlying node which it is running on.
**kwargs: Any additional arguments if any.
"""
+node.register_shell(self)
self._node = node
+
if name is None:
name = type(self).__name__
self._logger = get_dts_logger(f"{node.name}.{name}")
diff --git a/dts/framework/runner.py b/dts/framework/runner.py
index 510be1a870..fd3a934a9f 100644
--- a/dts/framework/runner.py
+++ b/dts/framework/runner.py
@@ -460,6 +460,10 @@ def _run_test_suite(
DtsStage.test_suite_setup, Path(SETTINGS.output_dir,
test_suite_name)
)
test_suite = test_suite_with_cases.test_suite_class(sut_node, tg_node,
topology)
+
+sut_node.enter_scope("suite")
+tg_node.enter_scope("suite")
+
try:
self._logger.info(f"Starting test suite setup: {test_suite_name}")
test_suite.set_up_suite()
@@ -479,7 +483,6 @@ def _run_test_suite(
try:
self._logger.set_stage(DtsStage.test_suite_teardown)
test_suite.tear_down_suite()
-sut_node.kill_cleanup_dpdk_apps()
test_suite_result.update_teardown(Result.PASS)
except Exception as e:
self._logger.exception(f"Test suite teardown ERROR:
{test_suite_name}")
@@ -488,6 +491,10 @@ def _run_test_suite(
"the next test suite may be affected."
)
test_suite_result.update_setup(Result.ERROR, e)
+
+sut_node.exit_scope()
+tg_node.exit_scope()
+
if len(test_suite_result.get_errors()) > 0 and
test_suite.is_blocking:
raise BlockingTestSuiteError(test_suite_name)
@@ -511,6 +518,9 @@ def _execute_test_suite(
"""
self._logger.set_stage(DtsStage.test_suite)
for test_case in test_cases:
+test_suite.sut_node.enter_scope("case")
+test_suite.tg_node.enter_scope("case")
+
test_case_name = test_case.__name__
test_case_result = test_suite_result.add_test_case(test_case_name)
all_attempts = SETTINGS.re_run + 1
@@ -531,6 +541,9 @@ def _execute_test_suite(
)
test_case_result.update_setup(Result.SKIP)
+test_suite.sut_node.exit_scope()
+test_suite.tg_node.exit_scope()
+
def _run_test_case(
self,
test_suite: TestSuite,
diff --git a/dts/framework/testbed_model/capability.py
b/dts/framework/testbed_model/capability.py
index 6a7a1f5b6c..e883f59d11 100644
--- a/dts/framework/testbed_model/capability.py
+++ b/dts/framework/testbed_model/capability.py
@@ -221,24 +221,23 @@ def get_supported_capabilities(
)
if cls.capabilities_to_check:
capabilities_to_check_map = cls._get_decorated_capabilities_map()
-with TestPmdShell(
-sut_node, privileged=True, disable_device_start=True
-) as testpmd_shell:
-for (
-conditional_capability_fn,
-capabilities,
-) in capabilities_to_check_map.items():
-supported_capabilities: set[NicCapability] = set()
-unsupported_capabilities: set[NicCapability] = set()
-capability_fn = cls._reduce_capabilities(
-capabilities, supported_capabilities,
unsupported_capabilities
-)
-if conditional_capability_fn:
-
[RFC PATCH 2/2] dts: revert back shell split
The InteractiveShell was previously renamed to SingleActiveInteractiveShell to represent a shell that can only be run once. The mechanism used to enforce this was a context manager, which turned out to be more constrictive on test suite development. Shell closure is now handled by the scoping mechanism, and an attribute is used to enforce the single active shell. Also the split has been reverted. Signed-off-by: Luca Vizzarro --- dts/framework/remote_session/dpdk_shell.py| 8 +- .../remote_session/interactive_shell.py | 262 +++-- .../single_active_interactive_shell.py| 268 -- dts/framework/remote_session/testpmd_shell.py | 4 +- dts/framework/testbed_model/capability.py | 2 +- dts/framework/testbed_model/node.py | 10 +- 6 files changed, 250 insertions(+), 304 deletions(-) delete mode 100644 dts/framework/remote_session/single_active_interactive_shell.py diff --git a/dts/framework/remote_session/dpdk_shell.py b/dts/framework/remote_session/dpdk_shell.py index c11d9ab81c..c37dcb2b62 100644 --- a/dts/framework/remote_session/dpdk_shell.py +++ b/dts/framework/remote_session/dpdk_shell.py @@ -8,10 +8,11 @@ from abc import ABC from pathlib import PurePath +from typing import ClassVar from framework.params.eal import EalParams -from framework.remote_session.single_active_interactive_shell import ( -SingleActiveInteractiveShell, +from framework.remote_session.interactive_shell import ( +InteractiveShell, ) from framework.settings import SETTINGS from framework.testbed_model.cpu import LogicalCoreCount, LogicalCoreList @@ -61,7 +62,7 @@ def compute_eal_params( return params -class DPDKShell(SingleActiveInteractiveShell, ABC): +class DPDKShell(InteractiveShell, ABC): """The base class for managing DPDK-based interactive shells. This class shouldn't be instantiated directly, but instead be extended. @@ -71,6 +72,7 @@ class DPDKShell(SingleActiveInteractiveShell, ABC): _node: SutNode _app_params: EalParams +_single_active_per_node: ClassVar[bool] = True def __init__( self, diff --git a/dts/framework/remote_session/interactive_shell.py b/dts/framework/remote_session/interactive_shell.py index 9ca285b604..a136419181 100644 --- a/dts/framework/remote_session/interactive_shell.py +++ b/dts/framework/remote_session/interactive_shell.py @@ -1,44 +1,256 @@ # SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2023 University of New Hampshire +# Copyright(c) 2024 University of New Hampshire # Copyright(c) 2024 Arm Limited -"""Interactive shell with manual stop/start functionality. +"""Common functionality for interactive shell handling. -Provides a class that doesn't require being started/stopped using a context manager and can instead -be started and stopped manually, or have the stopping process be handled at the time of garbage -collection. +The base class, :class:`InteractiveShell`, is meant to be extended by subclasses that +contain functionality specific to that shell type. These subclasses will often modify things like +the prompt to expect or the arguments to pass into the application, but still utilize +the same method for sending a command and collecting output. How this output is handled however +is often application specific. If an application needs elevated privileges to start it is expected +that the method for gaining those privileges is provided when initializing the class. + +The :option:`--timeout` command line argument and the :envvar:`DTS_TIMEOUT` +environment variable configure the timeout of getting the output from command execution. """ -import weakref +from abc import ABC +from pathlib import PurePath from typing import ClassVar -from .single_active_interactive_shell import SingleActiveInteractiveShell +from paramiko import Channel, channel + +from framework.exception import ( +InteractiveCommandExecutionError, +InteractiveSSHSessionDeadError, +InteractiveSSHTimeoutError, +InternalError, +) +from framework.logger import DTSLogger, get_dts_logger +from framework.params import Params +from framework.settings import SETTINGS +from framework.testbed_model.node import Node +from framework.utils import MultiInheritanceBaseClass + + +class InteractiveShell(MultiInheritanceBaseClass, ABC): +"""The base class for managing interactive shells. +This class shouldn't be instantiated directly, but instead be extended. It contains +methods for starting interactive shells as well as sending commands to these shells +and collecting input until reaching a certain prompt. All interactive applications +will use the same SSH connection, but each will create their own channel on that +session. -class InteractiveShell(SingleActiveInteractiveShell): -"""Adds manual start and stop functionality to interactive shells. +Interactive shells are started and stopped using a context manager. This allows for
[RFC PATCH 0/2] dts: add basic scope to improve shell handling
Hi there, To try to improve the ease of use of the framework for the test developer, I have been trying to come up with a decent solution to improve shell handling and consistency. At the moment we have 2 patterns to do this, which could be confusing to the user. It probably is that a good approach, which is what I am proposing, is to introduce a scoping mechanism in DTS. What this would mean is to associate any shells or modification to different scopes: global test suite and individual test cases. Here's an RFC for this, please have a look. Looking forward to your feedback! This could be massively improved, but it'd require a lot more changes. One idea that I think is worth pursuing is turning the execution into a FSM. Best, Luca Luca Vizzarro (2): dts: add scoping and shell registration to Node dts: revert back shell split dts/framework/remote_session/dpdk_shell.py| 8 +- .../remote_session/interactive_shell.py | 262 +++-- .../single_active_interactive_shell.py| 266 -- dts/framework/remote_session/testpmd_shell.py | 4 +- dts/framework/runner.py | 15 +- dts/framework/testbed_model/capability.py | 35 ++- dts/framework/testbed_model/node.py | 65 - dts/framework/testbed_model/sut_node.py | 14 +- dts/tests/TestSuite_blocklist.py | 16 +- dts/tests/TestSuite_checksum_offload.py | 168 +-- dts/tests/TestSuite_dynamic_queue_conf.py | 52 ++-- dts/tests/TestSuite_l2fwd.py | 20 +- dts/tests/TestSuite_mac_filter.py | 124 dts/tests/TestSuite_pmd_buffer_scatter.py | 26 +- dts/tests/TestSuite_smoke_tests.py| 4 +- dts/tests/TestSuite_vlan.py | 42 +-- 16 files changed, 575 insertions(+), 546 deletions(-) delete mode 100644 dts/framework/remote_session/single_active_interactive_shell.py -- 2.43.0
Re: [PATCH v4 13/15] net/zxdh: rss hash config/update, reta update/get
On Wed, 18 Dec 2024 17:26:00 +0800
Junlong Wang wrote:
> provided rss hash config/update, reta update/get ops.
>
> Signed-off-by: Junlong Wang
> ---
> doc/guides/nics/features/zxdh.ini | 3 +
> doc/guides/nics/zxdh.rst | 1 +
> drivers/net/zxdh/zxdh_ethdev.c | 52
> drivers/net/zxdh/zxdh_ethdev.h | 3 +
> drivers/net/zxdh/zxdh_ethdev_ops.c | 410 +
> drivers/net/zxdh/zxdh_ethdev_ops.h | 26 ++
> drivers/net/zxdh/zxdh_msg.h| 22 ++
> drivers/net/zxdh/zxdh_tables.c | 82 ++
> drivers/net/zxdh/zxdh_tables.h | 7 +
> 9 files changed, 606 insertions(+)
>
Some suggestions:
> +int
> +zxdh_dev_rss_reta_update(struct rte_eth_dev *dev,
> + struct rte_eth_rss_reta_entry64 *reta_conf,
> + uint16_t reta_size)
> +{
> + struct zxdh_hw *hw = dev->data->dev_private;
> + struct zxdh_msg_info msg = {0};
> + uint16_t old_reta[RTE_ETH_RSS_RETA_SIZE_256];
> + uint16_t idx;
> + uint16_t i;
> + uint16_t pos;
> + int ret;
> +
> + if (reta_size != RTE_ETH_RSS_RETA_SIZE_256) {
> + PMD_DRV_LOG(ERR, "reta_size is illegal(%u).reta_size should be
> 256", reta_size);
> + return -EINVAL;
> + }
> + if (!hw->rss_reta) {
> + hw->rss_reta = rte_zmalloc(NULL, RTE_ETH_RSS_RETA_SIZE_256 *
> sizeof(uint16_t), 4);
This could be rte_calloc()
...
> +int
> +zxdh_rss_table_set(uint16_t vport, struct zxdh_rss_reta *rss_reta)
> +{
> + struct zxdh_rss_to_vqid_table rss_vqid = {0};
> + union zxdh_virport_num vport_num = (union zxdh_virport_num)vport;
> + int ret = 0;
> +
> + for (uint16_t i = 0; i < RTE_ETH_RSS_RETA_SIZE_256 / 8; i++) {
> + for (uint16_t j = 0; j < 8; j++) {
> + #if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
> + if (j % 2 == 0)
> + rss_vqid.vqm_qid[j + 1] = rss_reta->reta[i * 8
> + j];
> + else
> + rss_vqid.vqm_qid[j - 1] = rss_reta->reta[i * 8
> + j];
> + #else
> + rss_vqid.vqm_qid[j] = rss_init->reta[i * 8 + j];
> + #endif
Please put #if in first column not indented.
Better yet, use rte_byteorder functions to elimnate #if code
pattern.
> + }
> +
> + #if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
> + rss_vqid.vqm_qid[1] |= 0x8000;
> + #else
> + rss_vqid.vqm_qid[0] |= 0x8000;
> + #endif
Re: [PATCH v4 06/15] net/zxdh: dev start/stop ops implementations
On Wed, 18 Dec 2024 17:25:53 +0800
Junlong Wang wrote:
> +static void
> +zxdh_notify_queue(struct zxdh_hw *hw, struct zxdh_virtqueue *vq)
> +{
> + uint32_t notify_data = 0;
> +
> + if (!zxdh_pci_with_feature(hw, ZXDH_F_NOTIFICATION_DATA)) {
> + rte_write16(vq->vq_queue_index, vq->notify_addr);
> + return;
> + }
> +
> + if (zxdh_pci_with_feature(hw, ZXDH_F_RING_PACKED)) {
> + notify_data = ((uint32_t)(!!(vq->vq_packed.cached_flags &
> +
> ZXDH_VRING_PACKED_DESC_F_AVAIL)) << 31) |
> + ((uint32_t)vq->vq_avail_idx <<
> 16) |
> + vq->vq_queue_index;
> + } else {
> + notify_data = ((uint32_t)vq->vq_avail_idx << 16) |
> vq->vq_queue_index;
> + }
> + PMD_DRV_LOG(DEBUG, "queue:%d notify_data 0x%x notify_addr 0x%p",
> + vq->vq_queue_index, notify_data,
> vq->notify_addr);
> + rte_write32(notify_data, vq->notify_addr);
> +}
Looks like the notify_data part could be simplified to:
notify_data = ((uint32_t)vq->vq_avail_idx << 16) | vq->vq_queue_index;
if (zxdh_pci_with_feature(hw, ZXDH_F_RING_PACKED) &&
(vq->vq_packed.cached_flags & ZXDH_VRING_PACKED_DESC_F_AVAIL)
notify_data |= RTE_BIT32(31);
Re: [PATCH v2 0/7] dts: add Ruff and docstring linting
Series-reviewed-by: Patrick Robb Tested-by: Patrick Robb Paul I will merge to next-dts now instead of waiting until after Winter holidays if that is okay with you. On Thu, Dec 12, 2024 at 9:02 AM Luca Vizzarro wrote: > v2: > - updated the doc page > > Luca Vizzarro (7): > dts: add Ruff as linter and formatter > dts: enable Ruff preview pydoclint rules > dts: resolve docstring linter errors > dts: apply Ruff formatting > dts: update dts-check-format to use Ruff > dts: remove old linters and formatters > dts: update linters in doc page > > devtools/dts-check-format.sh | 30 +-- > doc/guides/tools/dts.rst | 26 +-- > dts/framework/params/eal.py | 5 +- > dts/framework/remote_session/dpdk_shell.py| 1 - > dts/framework/remote_session/python_shell.py | 1 + > .../single_active_interactive_shell.py| 3 +- > dts/framework/runner.py | 14 +- > dts/framework/settings.py | 3 + > dts/framework/test_suite.py | 6 +- > dts/framework/testbed_model/capability.py | 13 +- > dts/framework/testbed_model/cpu.py| 21 +- > dts/framework/testbed_model/linux_session.py | 6 +- > dts/framework/testbed_model/node.py | 3 + > dts/framework/testbed_model/os_session.py | 3 +- > dts/framework/testbed_model/port.py | 1 - > dts/framework/testbed_model/posix_session.py | 16 +- > dts/framework/testbed_model/sut_node.py | 2 +- > dts/framework/testbed_model/topology.py | 6 + > .../traffic_generator/__init__.py | 3 + > .../testbed_model/traffic_generator/scapy.py | 7 +- > .../traffic_generator/traffic_generator.py| 3 +- > dts/framework/utils.py| 6 +- > dts/poetry.lock | 197 +++--- > dts/pyproject.toml| 40 ++-- > dts/tests/TestSuite_vlan.py | 22 +- > 25 files changed, 179 insertions(+), 259 deletions(-) > > -- > 2.43.0 > >
Re: [PATCH v4 15/15] net/zxdh: mtu update ops implementations
On Wed, 18 Dec 2024 17:26:02 +0800
Junlong Wang wrote:
> +int zxdh_dev_mtu_set(struct rte_eth_dev *dev, uint16_t new_mtu)
> +{
> + struct zxdh_hw *hw = dev->data->dev_private;
> + struct zxdh_panel_table panel = {0};
> + struct zxdh_port_attr_table vport_att = {0};
> + uint16_t vfid = zxdh_vport_to_vfid(hw->vport);
> + uint16_t max_mtu = 0;
> + int ret = 0;
useless initializations.
> +
> + max_mtu = ZXDH_MAX_RX_PKTLEN - RTE_ETHER_HDR_LEN - RTE_VLAN_HLEN -
> ZXDH_DL_NET_HDR_SIZE;
> + if (new_mtu < ZXDH_ETHER_MIN_MTU || new_mtu > max_mtu) {
> + PMD_DRV_LOG(ERR, "invalid mtu:%d, range[%d, %d]",
> + new_mtu, ZXDH_ETHER_MIN_MTU, max_mtu);
> + return -EINVAL;
> + }
These checks are redundant. See rte_ethdev.c::eth_dev_validate_mtu function.
It already checks the mtu against values returned from info_get.
> +
> + if (dev->data->mtu == new_mtu)
> + return 0;
This should be done in ethdev_set_mtu but does not look like that is checked.
Will look into fixing it there.
[PATCH] service: fix getting service lcore attributes
Perf test service_perf_autotest was failing after introduction
of lcore variables. Fixed getting of idle_ and error_ service
call statistics.
Fixes: b24bbaedbba2 ("service: keep per-lcore state in lcore variable")
Cc: mattias.ronnb...@ericsson.com
Cc: sta...@dpdk.org
Signed-off-by: Piotr Krzewinski
---
lib/eal/common/rte_service.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c
index dad3150df9..1821746337 100644
--- a/lib/eal/common/rte_service.c
+++ b/lib/eal/common/rte_service.c
@@ -879,7 +879,7 @@ lcore_attr_get_service_calls(uint32_t service_id, unsigned
int lcore)
static uint64_t
lcore_attr_get_service_idle_calls(uint32_t service_id, unsigned int lcore)
{
- struct core_state *cs = &lcore_states[lcore];
+ struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states);
return
rte_atomic_load_explicit(&cs->service_stats[service_id].idle_calls,
rte_memory_order_relaxed);
@@ -888,7 +888,7 @@ lcore_attr_get_service_idle_calls(uint32_t service_id,
unsigned int lcore)
static uint64_t
lcore_attr_get_service_error_calls(uint32_t service_id, unsigned int lcore)
{
- struct core_state *cs = &lcore_states[lcore];
+ struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states);
return
rte_atomic_load_explicit(&cs->service_stats[service_id].error_calls,
rte_memory_order_relaxed);
--
2.36.0
RE: [RFC 0/8] ioring: network driver
> > From: Konstantin Ananyev [mailto:konstantin.anan...@huawei.com] > > > > > > > This is first draft of new simplified TAP device that uses > > > > > the Linux kernel ioring API to provide a read/write ring > > > > > with kernel. > > > > > > > > > > This is split from tap device because there are so many > > > > > unnecessary things in existing tap, and supporting ioring is > > > > > better without ifdefs etc. The default name of the tap > > > > > device is different that other uses in DPDK but the driver > > > > > tries to keep the same relevant devargs as before. > > > > > > > > > > This driver will only provide features that match what kernel > > > > > does, so no flow support etc. The next version will add checksum > > > > > and multi-segment packets. Some of the doc files may need update > > > > > as well. > > > > > > > > Makes sense to me, though didn't properly look inside. > > > > One thing - probably add a 'tap' into the name, > > > > 'tap_ioiring' or so, otherwise 'ioring' is a bit too generic > > > > and might be confusing. > > Konstantin is referring to the name of the driver and the source code file > names, "net/ioring" -> "net/tap_ioring". Yep, that what I meant. > > > > > > > There are some userspaces that look for "e*" in name for some setups. > > Stephen is referring to the device name of an instantiated interface, e.g. > "eth0". > > And yes, assuming devices named "e*" are Ethernet devices is a common hack in > Linux applications. I've done it myself. :-) Ok... and why such practice should prevent us to name PMD itself in a way we think is appropriate? > > > > Didn't get you here, pls try to re-phrase. > > > > > But names are totally arbitrary > >
[PATCH v4 02/24] net/_common_intel: provide common Tx entry structures
The Tx entry structures, both vector and scalar, are common across Intel
drivers, so provide a single definition to be used everywhere.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h| 27 +++
.../net/i40e/i40e_recycle_mbufs_vec_common.c | 2 +-
drivers/net/i40e/i40e_rxtx.c | 18 ++---
drivers/net/i40e/i40e_rxtx.h | 14 +++---
drivers/net/i40e/i40e_rxtx_vec_altivec.c | 2 +-
drivers/net/i40e/i40e_rxtx_vec_avx2.c | 2 +-
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 6 ++---
drivers/net/i40e/i40e_rxtx_vec_common.h | 4 +--
drivers/net/i40e/i40e_rxtx_vec_neon.c | 2 +-
drivers/net/i40e/i40e_rxtx_vec_sse.c | 2 +-
drivers/net/iavf/iavf_rxtx.c | 12 -
drivers/net/iavf/iavf_rxtx.h | 14 +++---
drivers/net/iavf/iavf_rxtx_vec_avx2.c | 2 +-
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 10 +++
drivers/net/iavf/iavf_rxtx_vec_common.h | 4 +--
drivers/net/iavf/iavf_rxtx_vec_sse.c | 2 +-
drivers/net/ice/ice_dcf_ethdev.c | 2 +-
drivers/net/ice/ice_rxtx.c| 16 +--
drivers/net/ice/ice_rxtx.h| 13 ++---
drivers/net/ice/ice_rxtx_vec_avx2.c | 2 +-
drivers/net/ice/ice_rxtx_vec_avx512.c | 6 ++---
drivers/net/ice/ice_rxtx_vec_common.h | 6 ++---
drivers/net/ice/ice_rxtx_vec_sse.c| 2 +-
.../ixgbe/ixgbe_recycle_mbufs_vec_common.c| 2 +-
drivers/net/ixgbe/ixgbe_rxtx.c| 16 +--
drivers/net/ixgbe/ixgbe_rxtx.h| 22 +++
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 8 +++---
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 2 +-
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c| 2 +-
29 files changed, 105 insertions(+), 117 deletions(-)
create mode 100644 drivers/net/_common_intel/tx.h
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
new file mode 100644
index 00..384352b9db
--- /dev/null
+++ b/drivers/net/_common_intel/tx.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Intel Corporation
+ */
+
+#ifndef _COMMON_INTEL_TX_H_
+#define _COMMON_INTEL_TX_H_
+
+#include
+#include
+
+/**
+ * Structure associated with each descriptor of the TX ring of a TX queue.
+ */
+struct ci_tx_entry {
+ struct rte_mbuf *mbuf; /* mbuf associated with TX desc, if any. */
+ uint16_t next_id; /* Index of next descriptor in ring. */
+ uint16_t last_id; /* Index of last scattered descriptor. */
+};
+
+/**
+ * Structure associated with each descriptor of the TX ring of a TX queue in
vector Tx.
+ */
+struct ci_tx_entry_vec {
+ struct rte_mbuf *mbuf; /* mbuf associated with TX desc, if any. */
+};
+
+#endif /* _COMMON_INTEL_TX_H_ */
diff --git a/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
b/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
index 14424c9921..260d238ce4 100644
--- a/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
+++ b/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
@@ -56,7 +56,7 @@ i40e_recycle_tx_mbufs_reuse_vec(void *tx_queue,
struct rte_eth_recycle_rxq_info *recycle_rxq_info)
{
struct i40e_tx_queue *txq = tx_queue;
- struct i40e_tx_entry *txep;
+ struct ci_tx_entry *txep;
struct rte_mbuf **rxep;
int i, n;
uint16_t nb_recycle_mbufs;
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 839c8a5442..2e1f07d2a1 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -378,7 +378,7 @@ i40e_build_ctob(uint32_t td_cmd,
static inline int
i40e_xmit_cleanup(struct i40e_tx_queue *txq)
{
- struct i40e_tx_entry *sw_ring = txq->sw_ring;
+ struct ci_tx_entry *sw_ring = txq->sw_ring;
volatile struct i40e_tx_desc *txd = txq->tx_ring;
uint16_t last_desc_cleaned = txq->last_desc_cleaned;
uint16_t nb_tx_desc = txq->nb_tx_desc;
@@ -1081,8 +1081,8 @@ uint16_t
i40e_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
{
struct i40e_tx_queue *txq;
- struct i40e_tx_entry *sw_ring;
- struct i40e_tx_entry *txe, *txn;
+ struct ci_tx_entry *sw_ring;
+ struct ci_tx_entry *txe, *txn;
volatile struct i40e_tx_desc *txd;
volatile struct i40e_tx_desc *txr;
struct rte_mbuf *tx_pkt;
@@ -1331,7 +1331,7 @@ i40e_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
static __rte_always_inline int
i40e_tx_free_bufs(struct i40e_tx_queue *txq)
{
- struct i40e_tx_entry *txep;
+ struct ci_tx_entry *txep;
uint16_t tx_rs_thresh = txq->tx_rs_thresh;
uint16_t i = 0, j = 0;
struct rte_mbuf *free[RTE_I40E_TX_MAX_FREE_BUF_SZ];
@@ -1418,7 +1418,7 @@ i40e_tx_fill_hw_ring(struct i40e_tx_queue *txq,
uint16_t nb_pkts)
[PATCH v4 01/24] net/_common_intel: add pkt reassembly fn for intel drivers
The code for reassembling a single, multi-mbuf packet from multiple
buffers received from the NIC is duplicated across many drivers. Rather
than having multiple copies of this function, we can create an
"_common_intel" directory to hold such functions and consolidate
multiple functions down to a single one for easier maintenance.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/rx.h| 79 +++
drivers/net/i40e/i40e_rxtx_vec_altivec.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_avx2.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_common.h | 64 +-
drivers/net/i40e/i40e_rxtx_vec_neon.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_sse.c | 4 +-
drivers/net/i40e/meson.build | 2 +-
drivers/net/iavf/iavf_rxtx_vec_avx2.c | 8 +--
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 8 +--
drivers/net/iavf/iavf_rxtx_vec_common.h | 65 +--
drivers/net/iavf/iavf_rxtx_vec_sse.c | 8 +--
drivers/net/iavf/meson.build | 2 +-
drivers/net/ice/ice_rxtx_vec_avx2.c | 4 +-
drivers/net/ice/ice_rxtx_vec_avx512.c | 8 +--
drivers/net/ice/ice_rxtx_vec_common.h | 66 +--
drivers/net/ice/ice_rxtx_vec_sse.c| 4 +-
drivers/net/ice/meson.build | 2 +-
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 63 +-
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 4 +-
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c| 4 +-
drivers/net/ixgbe/meson.build | 2 +-
22 files changed, 121 insertions(+), 292 deletions(-)
create mode 100644 drivers/net/_common_intel/rx.h
diff --git a/drivers/net/_common_intel/rx.h b/drivers/net/_common_intel/rx.h
new file mode 100644
index 00..5bd2fea7e3
--- /dev/null
+++ b/drivers/net/_common_intel/rx.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Intel Corporation
+ */
+
+#ifndef _COMMON_INTEL_RX_H_
+#define _COMMON_INTEL_RX_H_
+
+#include
+#include
+#include
+
+#define CI_RX_BURST 32
+
+static inline uint16_t
+ci_rx_reassemble_packets(struct rte_mbuf **rx_bufs, uint16_t nb_bufs, uint8_t
*split_flags,
+ struct rte_mbuf **pkt_first_seg, struct rte_mbuf **pkt_last_seg,
+ const uint8_t crc_len)
+{
+ struct rte_mbuf *pkts[CI_RX_BURST] = {0}; /*finished pkts*/
+ struct rte_mbuf *start = *pkt_first_seg;
+ struct rte_mbuf *end = *pkt_last_seg;
+ unsigned int pkt_idx, buf_idx;
+
+ for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
+ if (end) {
+ /* processing a split packet */
+ end->next = rx_bufs[buf_idx];
+ rx_bufs[buf_idx]->data_len += crc_len;
+
+ start->nb_segs++;
+ start->pkt_len += rx_bufs[buf_idx]->data_len;
+ end = end->next;
+
+ if (!split_flags[buf_idx]) {
+ /* it's the last packet of the set */
+ start->hash = end->hash;
+ start->vlan_tci = end->vlan_tci;
+ start->ol_flags = end->ol_flags;
+ /* we need to strip crc for the whole packet */
+ start->pkt_len -= crc_len;
+ if (end->data_len > crc_len) {
+ end->data_len -= crc_len;
+ } else {
+ /* free up last mbuf */
+ struct rte_mbuf *secondlast = start;
+
+ start->nb_segs--;
+ while (secondlast->next != end)
+ secondlast = secondlast->next;
+ secondlast->data_len -= (crc_len -
end->data_len);
+ secondlast->next = NULL;
+ rte_pktmbuf_free_seg(end);
+ }
+ pkts[pkt_idx++] = start;
+ start = NULL;
+ end = NULL;
+ }
+ } else {
+ /* not processing a split packet */
+ if (!split_flags[buf_idx]) {
+ /* not a split packet, save and skip */
+ pkts[pkt_idx++] = rx_bufs[buf_idx];
+ continue;
+ }
+ start = rx_bufs[buf_idx];
+ end = start;
+ rx_bufs[buf_idx]->data_len += crc_len;
+ rx_bufs[buf_idx]->pkt_len += crc_len;
+ }
+ }
+
+ /* save the partial packet for ne
[PATCH v4 00/24] Reduce code duplication across Intel NIC drivers
This RFC attempts to reduce the amount of code duplication across a number of Intel NIC drivers, specifically: ixgbe, i40e, iavf, and ice. The first patch extract a function from the Rx side, otherwise the majority of the changes are on the Tx side, leading to a converged Tx queue structure across the 4 drivers, and a large number of common functions. v3->v4: * Add patches 23 & 24 to set, to do a little more dedupliation on Rx side v2->v3: * Fix incorrect/unadjusted memset in patch 8, leading to incorrect threshold tracking in ixgbe. v1->v2: * Fix two additional checkpatch issues that were flagged. * Added in patch 21, which performs additional cleanup that is possible once all vector drivers use the same mbuf free/release process. [This brings the patchset to having over twice as many lines removed as added (1887 vs 930), and close to having a net removal of 1kloc] RFC->v1: * Moved the location of the common code from "common/intel_eth" to "net/_common_intel", and added only ".." to the driver include path so that the paths included "_common_intel" in them, to make it clear it's not driver-local headers. * Due to change in location, structure/fn prefix changes from "ieth" to "ci" for "common intel". * Removed the seeming-arbitrary split of vector and non-vector code - since much of the code taken from vector files was scalar code which was used by the vector drivers. * Split code into separate Rx and Tx files. * Fixed multiple checkpatch issues (but not all). * Attempted to improve name standardization, by using "_vec" as a common suffix for all vector-related fns and data. Previously, some names had "vec" in the middle, others had just "_v" suffix or full word "vector" as suffix. * Other minor changes... Bruce Richardson (24): net/_common_intel: add pkt reassembly fn for intel drivers net/_common_intel: provide common Tx entry structures net/_common_intel: add Tx mbuf ring replenish fn drivers/net: align Tx queue struct field names drivers/net: add prefix for driver-specific structs net/_common_intel: merge ice and i40e Tx queue struct net/iavf: use common Tx queue structure net/ixgbe: convert Tx queue context cache field to ptr net/ixgbe: use common Tx queue structure net/_common_intel: pack Tx queue structure net/_common_intel: add post-Tx buffer free function net/_common_intel: add Tx buffer free fn for AVX-512 net/iavf: use common Tx free fn for AVX-512 net/ice: move Tx queue mbuf cleanup fn to common net/i40e: use common Tx queue mbuf cleanup fn net/ixgbe: use common Tx queue mbuf cleanup fn net/iavf: use common Tx queue mbuf cleanup fn net/ice: use vector SW ring for all vector paths net/i40e: use vector SW ring for all vector paths net/iavf: use vector SW ring for all vector paths net/_common_intel: remove unneeded code net/ixgbe: use common Tx backlog entry fn net/_common_intel: create common mbuf initializer fn net/_common_intel: extract common Rx vector criteria drivers/net/_common_intel/rx.h| 112 drivers/net/_common_intel/tx.h| 249 ++ drivers/net/i40e/i40e_ethdev.c| 4 +- drivers/net/i40e/i40e_ethdev.h| 8 +- drivers/net/i40e/i40e_fdir.c | 10 +- .../net/i40e/i40e_recycle_mbufs_vec_common.c | 6 +- drivers/net/i40e/i40e_rxtx.c | 192 +- drivers/net/i40e/i40e_rxtx.h | 61 + drivers/net/i40e/i40e_rxtx_vec_altivec.c | 30 ++- drivers/net/i40e/i40e_rxtx_vec_avx2.c | 26 +- drivers/net/i40e/i40e_rxtx_vec_avx512.c | 144 +- drivers/net/i40e/i40e_rxtx_vec_common.h | 198 +- drivers/net/i40e/i40e_rxtx_vec_neon.c | 30 ++- drivers/net/i40e/i40e_rxtx_vec_sse.c | 30 ++- drivers/net/i40e/meson.build | 2 +- drivers/net/iavf/iavf.h | 2 +- drivers/net/iavf/iavf_ethdev.c| 4 +- drivers/net/iavf/iavf_rxtx.c | 195 +- drivers/net/iavf/iavf_rxtx.h | 62 + drivers/net/iavf/iavf_rxtx_vec_avx2.c | 47 ++-- drivers/net/iavf/iavf_rxtx_vec_avx512.c | 214 +++ drivers/net/iavf/iavf_rxtx_vec_common.h | 178 + drivers/net/iavf/iavf_rxtx_vec_neon.c | 3 +- drivers/net/iavf/iavf_rxtx_vec_sse.c | 59 ++--- drivers/net/iavf/iavf_vchnl.c | 8 +- drivers/net/iavf/meson.build | 2 +- drivers/net/ice/ice_dcf.c | 4 +- drivers/net/ice/ice_dcf_ethdev.c | 21 +- drivers/net/ice/ice_diagnose.c| 2 +- drivers/net/ice/ice_ethdev.c | 2 +- drivers/net/ice/ice_ethdev.h | 7 +- drivers/net/ice/ice_rxtx.c| 163 +--- drivers/net/ice/ice_rxtx.h| 52 +--- drivers/net/ice/ice_rxtx
[PATCH v4 03/24] net/_common_intel: add Tx mbuf ring replenish fn
Move the short function used to place mbufs on the SW Tx ring to common
code to avoid duplication.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h | 7 +++
drivers/net/i40e/i40e_rxtx_vec_altivec.c | 4 ++--
drivers/net/i40e/i40e_rxtx_vec_avx2.c| 4 ++--
drivers/net/i40e/i40e_rxtx_vec_common.h | 10 --
drivers/net/i40e/i40e_rxtx_vec_neon.c| 4 ++--
drivers/net/i40e/i40e_rxtx_vec_sse.c | 4 ++--
drivers/net/iavf/iavf_rxtx_vec_avx2.c| 4 ++--
drivers/net/iavf/iavf_rxtx_vec_common.h | 10 --
drivers/net/iavf/iavf_rxtx_vec_sse.c | 4 ++--
drivers/net/ice/ice_rxtx_vec_avx2.c | 4 ++--
drivers/net/ice/ice_rxtx_vec_common.h| 10 --
drivers/net/ice/ice_rxtx_vec_sse.c | 4 ++--
12 files changed, 23 insertions(+), 46 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index 384352b9db..5397007411 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -24,4 +24,11 @@ struct ci_tx_entry_vec {
struct rte_mbuf *mbuf; /* mbuf associated with TX desc, if any. */
};
+static __rte_always_inline void
+ci_tx_backlog_entry(struct ci_tx_entry *txep, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
+{
+ for (uint16_t i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
#endif /* _COMMON_INTEL_TX_H_ */
diff --git a/drivers/net/i40e/i40e_rxtx_vec_altivec.c
b/drivers/net/i40e/i40e_rxtx_vec_altivec.c
index ca1038eaa6..80f07a3e10 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_altivec.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_altivec.c
@@ -575,7 +575,7 @@ i40e_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry(txep, tx_pkts, n);
for (i = 0; i < n - 1; ++i, ++tx_pkts, ++txdp)
vtx1(txdp, *tx_pkts, flags);
@@ -592,7 +592,7 @@ i40e_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
txep = &txq->sw_ring[tx_id];
}
- tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry(txep, tx_pkts, nb_commit);
vtx(txdp, tx_pkts, nb_commit, flags);
diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx2.c
b/drivers/net/i40e/i40e_rxtx_vec_avx2.c
index e8441de759..b26bae4757 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_avx2.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_avx2.c
@@ -765,7 +765,7 @@ i40e_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry(txep, tx_pkts, n);
vtx(txdp, tx_pkts, n - 1, flags);
tx_pkts += (n - 1);
@@ -783,7 +783,7 @@ i40e_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
txep = &txq->sw_ring[tx_id];
}
- tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry(txep, tx_pkts, nb_commit);
vtx(txdp, tx_pkts, nb_commit, flags);
diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h
b/drivers/net/i40e/i40e_rxtx_vec_common.h
index 619fb89110..325e99c1a4 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_common.h
+++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
@@ -84,16 +84,6 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
return txq->tx_rs_thresh;
}
-static __rte_always_inline void
-tx_backlog_entry(struct ci_tx_entry *txep,
-struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
-{
- int i;
-
- for (i = 0; i < (int)nb_pkts; ++i)
- txep[i].mbuf = tx_pkts[i];
-}
-
static inline void
_i40e_rx_queue_release_mbufs_vec(struct i40e_rx_queue *rxq)
{
diff --git a/drivers/net/i40e/i40e_rxtx_vec_neon.c
b/drivers/net/i40e/i40e_rxtx_vec_neon.c
index 9b90a32e28..26bc345a0a 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_neon.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_neon.c
@@ -702,7 +702,7 @@ i40e_xmit_fixed_burst_vec(void *__rte_restrict tx_queue,
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry(txep, tx_pkts, n);
for (i = 0; i < n - 1; ++i, ++tx_pkts, ++txdp)
vtx1(txdp, *tx_pkts, flags);
@@ -719,7 +719,7 @@ i40e_xmit_fixed_burst_vec(void *__rte_restrict tx_queue,
txep = &txq->sw_ring[tx_id];
}
- tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry(txep, tx_pkts, nb_commit);
vtx(txdp, tx_pkts, nb_commit, flags);
diff --git a/drivers/net/i40e/i40e_rxtx_vec_sse.c
b/drivers/net/i40e/i40e_rxtx_vec_sse.c
index e1fa2ed543..ebc32b0d27 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_sse.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_sse.c
@
[PATCH v4 05/24] drivers/net: add prefix for driver-specific structs
In preparation for merging the Tx structs for multiple drivers into a
single struct, rename the driver-specific pointers in each struct to
have a prefix on it, to avoid conflicts.
Signed-off-by: Bruce Richardson
---
drivers/net/i40e/i40e_fdir.c | 6 +--
.../net/i40e/i40e_recycle_mbufs_vec_common.c | 2 +-
drivers/net/i40e/i40e_rxtx.c | 30 ++--
drivers/net/i40e/i40e_rxtx.h | 4 +-
drivers/net/i40e/i40e_rxtx_vec_altivec.c | 6 +--
drivers/net/i40e/i40e_rxtx_vec_avx2.c | 6 +--
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 8 ++--
drivers/net/i40e/i40e_rxtx_vec_common.h | 2 +-
drivers/net/i40e/i40e_rxtx_vec_neon.c | 6 +--
drivers/net/i40e/i40e_rxtx_vec_sse.c | 6 +--
drivers/net/iavf/iavf_rxtx.c | 24 +-
drivers/net/iavf/iavf_rxtx.h | 4 +-
drivers/net/iavf/iavf_rxtx_vec_avx2.c | 6 +--
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 14 +++---
drivers/net/iavf/iavf_rxtx_vec_common.h | 2 +-
drivers/net/iavf/iavf_rxtx_vec_sse.c | 6 +--
drivers/net/ice/ice_dcf_ethdev.c | 4 +-
drivers/net/ice/ice_rxtx.c| 48 +--
drivers/net/ice/ice_rxtx.h| 4 +-
drivers/net/ice/ice_rxtx_vec_avx2.c | 6 +--
drivers/net/ice/ice_rxtx_vec_avx512.c | 8 ++--
drivers/net/ice/ice_rxtx_vec_common.h | 4 +-
drivers/net/ice/ice_rxtx_vec_sse.c| 6 +--
.../ixgbe/ixgbe_recycle_mbufs_vec_common.c| 2 +-
drivers/net/ixgbe/ixgbe_rxtx.c| 22 -
drivers/net/ixgbe/ixgbe_rxtx.h| 2 +-
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 6 +--
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 6 +--
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c| 6 +--
29 files changed, 128 insertions(+), 128 deletions(-)
diff --git a/drivers/net/i40e/i40e_fdir.c b/drivers/net/i40e/i40e_fdir.c
index 47f79ecf11..c600167634 100644
--- a/drivers/net/i40e/i40e_fdir.c
+++ b/drivers/net/i40e/i40e_fdir.c
@@ -1383,7 +1383,7 @@ i40e_find_available_buffer(struct rte_eth_dev *dev)
volatile struct i40e_tx_desc *tmp_txdp;
tmp_tail = txq->tx_tail;
- tmp_txdp = &txq->tx_ring[tmp_tail + 1];
+ tmp_txdp = &txq->i40e_tx_ring[tmp_tail + 1];
do {
if ((tmp_txdp->cmd_type_offset_bsz &
@@ -1640,7 +1640,7 @@ i40e_flow_fdir_filter_programming(struct i40e_pf *pf,
PMD_DRV_LOG(INFO, "filling filter programming descriptor.");
fdirdp = (volatile struct i40e_filter_program_desc *)
- (&txq->tx_ring[txq->tx_tail]);
+ (&txq->i40e_tx_ring[txq->tx_tail]);
fdirdp->qindex_flex_ptype_vsi =
rte_cpu_to_le_32((fdir_action->rx_queue <<
@@ -1710,7 +1710,7 @@ i40e_flow_fdir_filter_programming(struct i40e_pf *pf,
fdirdp->fd_id = rte_cpu_to_le_32(filter->soft_id);
PMD_DRV_LOG(INFO, "filling transmit descriptor.");
- txdp = &txq->tx_ring[txq->tx_tail + 1];
+ txdp = &txq->i40e_tx_ring[txq->tx_tail + 1];
txdp->buffer_addr = rte_cpu_to_le_64(pf->fdir.dma_addr[txq->tx_tail >>
1]);
td_cmd = I40E_TX_DESC_CMD_EOP |
diff --git a/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
b/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
index 260d238ce4..8679e5c1fd 100644
--- a/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
+++ b/drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
@@ -75,7 +75,7 @@ i40e_recycle_tx_mbufs_reuse_vec(void *tx_queue,
return 0;
/* check DD bits on threshold descriptor */
- if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
+ if ((txq->i40e_tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=
rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))
return 0;
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index b0bb20fe9a..34ef931859 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -379,7 +379,7 @@ static inline int
i40e_xmit_cleanup(struct i40e_tx_queue *txq)
{
struct ci_tx_entry *sw_ring = txq->sw_ring;
- volatile struct i40e_tx_desc *txd = txq->tx_ring;
+ volatile struct i40e_tx_desc *txd = txq->i40e_tx_ring;
uint16_t last_desc_cleaned = txq->last_desc_cleaned;
uint16_t nb_tx_desc = txq->nb_tx_desc;
uint16_t desc_to_clean_to;
@@ -1103,7 +1103,7 @@ i40e_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
txq = tx_queue;
sw_ring = txq->sw_ring;
- txr = txq->tx_ring;
+ txr = txq->i40e_tx_ring;
tx_id = txq->tx_tail;
txe = &sw_ring[tx_id];
@@ -1338,7 +1338,7 @@ i40e_tx_free_bufs(struct i40e_tx_queue *
[PATCH v4 06/24] net/_common_intel: merge ice and i40e Tx queue struct
The queue structures of i40e and ice drivers are virtually identical, so
merge them into a common struct. This should allow easier function
merging in future using that common struct.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h| 55 +
drivers/net/i40e/i40e_ethdev.c| 4 +-
drivers/net/i40e/i40e_ethdev.h| 4 +-
drivers/net/i40e/i40e_fdir.c | 4 +-
.../net/i40e/i40e_recycle_mbufs_vec_common.c | 2 +-
drivers/net/i40e/i40e_rxtx.c | 58 +-
drivers/net/i40e/i40e_rxtx.h | 50 ++--
drivers/net/i40e/i40e_rxtx_vec_altivec.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_avx2.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 6 +-
drivers/net/i40e/i40e_rxtx_vec_common.h | 2 +-
drivers/net/i40e/i40e_rxtx_vec_neon.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_sse.c | 4 +-
drivers/net/ice/ice_dcf.c | 4 +-
drivers/net/ice/ice_dcf_ethdev.c | 10 ++--
drivers/net/ice/ice_diagnose.c| 2 +-
drivers/net/ice/ice_ethdev.c | 2 +-
drivers/net/ice/ice_ethdev.h | 4 +-
drivers/net/ice/ice_rxtx.c| 60 +--
drivers/net/ice/ice_rxtx.h| 41 +
drivers/net/ice/ice_rxtx_vec_avx2.c | 4 +-
drivers/net/ice/ice_rxtx_vec_avx512.c | 8 +--
drivers/net/ice/ice_rxtx_vec_common.h | 8 +--
drivers/net/ice/ice_rxtx_vec_sse.c| 6 +-
24 files changed, 165 insertions(+), 185 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index 5397007411..c965f5ee6c 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -8,6 +8,9 @@
#include
#include
+/* forward declaration of the common intel (ci) queue structure */
+struct ci_tx_queue;
+
/**
* Structure associated with each descriptor of the TX ring of a TX queue.
*/
@@ -24,6 +27,58 @@ struct ci_tx_entry_vec {
struct rte_mbuf *mbuf; /* mbuf associated with TX desc, if any. */
};
+typedef void (*ice_tx_release_mbufs_t)(struct ci_tx_queue *txq);
+
+struct ci_tx_queue {
+ union { /* TX ring virtual address */
+ volatile struct ice_tx_desc *ice_tx_ring;
+ volatile struct i40e_tx_desc *i40e_tx_ring;
+ };
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ struct ci_tx_entry *sw_ring; /* virtual address of SW ring */
+ rte_iova_t tx_ring_dma;/* TX ring DMA address */
+ uint16_t nb_tx_desc; /* number of TX descriptors */
+ uint16_t tx_tail; /* current value of tail register */
+ uint16_t nb_tx_used; /* number of TX desc used since RS bit set */
+ /* index to last TX descriptor to have been cleaned */
+ uint16_t last_desc_cleaned;
+ /* Total number of TX descriptors ready to be allocated. */
+ uint16_t nb_tx_free;
+ /* Start freeing TX buffers if there are less free descriptors than
+* this value.
+*/
+ uint16_t tx_free_thresh;
+ /* Number of TX descriptors to use before RS bit is set. */
+ uint16_t tx_rs_thresh;
+ uint8_t pthresh; /**< Prefetch threshold register. */
+ uint8_t hthresh; /**< Host threshold register. */
+ uint8_t wthresh; /**< Write-back threshold reg. */
+ uint16_t port_id; /* Device port identifier. */
+ uint16_t queue_id; /* TX queue index. */
+ uint16_t reg_idx;
+ uint64_t offloads;
+ uint16_t tx_next_dd;
+ uint16_t tx_next_rs;
+ uint64_t mbuf_errors;
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ bool q_set; /* indicate if tx queue has been configured */
+ union { /* the VSI this queue belongs to */
+ struct ice_vsi *ice_vsi;
+ struct i40e_vsi *i40e_vsi;
+ };
+ const struct rte_memzone *mz;
+
+ union {
+ struct { /* ICE driver specific values */
+ ice_tx_release_mbufs_t tx_rel_mbufs;
+ uint32_t q_teid; /* TX schedule node id. */
+ };
+ struct { /* I40E driver specific values */
+ uint8_t dcb_tc;
+ };
+ };
+};
+
static __rte_always_inline void
ci_tx_backlog_entry(struct ci_tx_entry *txep, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 30dcdc68a8..bf5560ccc8 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -3685,7 +3685,7 @@ i40e_dev_update_mbuf_stats(struct rte_eth_dev *ethdev,
struct i40e_mbuf_stats *mbuf_stats)
{
uint16_t idx;
- struct i40e_tx_queue *txq;
+ struct c
[PATCH v4 09/24] net/ixgbe: use common Tx queue structure
Merge in additional fields used by the ixgbe driver and then convert it
over to using the common Tx queue structure.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h| 14 +++-
drivers/net/ixgbe/ixgbe_ethdev.c | 4 +-
.../ixgbe/ixgbe_recycle_mbufs_vec_common.c| 2 +-
drivers/net/ixgbe/ixgbe_rxtx.c| 64 +--
drivers/net/ixgbe/ixgbe_rxtx.h| 56 ++--
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 26
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 14 ++--
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c| 14 ++--
8 files changed, 80 insertions(+), 114 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index c4a1a0c816..51ae3b051d 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -34,9 +34,13 @@ struct ci_tx_queue {
volatile struct i40e_tx_desc *i40e_tx_ring;
volatile struct iavf_tx_desc *iavf_tx_ring;
volatile struct ice_tx_desc *ice_tx_ring;
+ volatile union ixgbe_adv_tx_desc *ixgbe_tx_ring;
};
volatile uint8_t *qtx_tail; /* register address of tail */
- struct ci_tx_entry *sw_ring; /* virtual address of SW ring */
+ union {
+ struct ci_tx_entry *sw_ring; /* virtual address of SW ring */
+ struct ci_tx_entry_vec *sw_ring_vec;
+ };
rte_iova_t tx_ring_dma;/* TX ring DMA address */
uint16_t nb_tx_desc; /* number of TX descriptors */
uint16_t tx_tail; /* current value of tail register */
@@ -87,6 +91,14 @@ struct ci_tx_queue {
uint8_t tc;
bool use_ctx; /* with ctx info, each pkt needs two
descriptors */
};
+ struct { /* ixgbe specific values */
+ const struct ixgbe_txq_ops *ops;
+ struct ixgbe_advctx_info *ctx_cache;
+ uint32_t ctx_curr;
+#ifdef RTE_LIB_SECURITY
+ uint8_t using_ipsec; /**< indicates that IPsec TX
feature is in use */
+#endif
+ };
};
};
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 8bee97d191..5f18fbaad5 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1118,7 +1118,7 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void
*init_params __rte_unused)
* RX and TX function.
*/
if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
- struct ixgbe_tx_queue *txq;
+ struct ci_tx_queue *txq;
/* TX queue function in primary, set by last queue initialized
* Tx queue may not initialized by primary process
*/
@@ -1623,7 +1623,7 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
* RX function
*/
if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
- struct ixgbe_tx_queue *txq;
+ struct ci_tx_queue *txq;
/* TX queue function in primary, set by last queue initialized
* Tx queue may not initialized by primary process
*/
diff --git a/drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c
b/drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c
index a878db3150..3fd05ed5eb 100644
--- a/drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c
+++ b/drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c
@@ -51,7 +51,7 @@ uint16_t
ixgbe_recycle_tx_mbufs_reuse_vec(void *tx_queue,
struct rte_eth_recycle_rxq_info *recycle_rxq_info)
{
- struct ixgbe_tx_queue *txq = tx_queue;
+ struct ci_tx_queue *txq = tx_queue;
struct ci_tx_entry *txep;
struct rte_mbuf **rxep;
int i, n;
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
index 2ca26cd132..344ef85685 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx.c
@@ -98,7 +98,7 @@
* Return the total number of buffers freed.
*/
static __rte_always_inline int
-ixgbe_tx_free_bufs(struct ixgbe_tx_queue *txq)
+ixgbe_tx_free_bufs(struct ci_tx_queue *txq)
{
struct ci_tx_entry *txep;
uint32_t status;
@@ -195,7 +195,7 @@ tx1(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf
**pkts)
* Copy mbuf pointers to the S/W ring.
*/
static inline void
-ixgbe_tx_fill_hw_ring(struct ixgbe_tx_queue *txq, struct rte_mbuf **pkts,
+ixgbe_tx_fill_hw_ring(struct ci_tx_queue *txq, struct rte_mbuf **pkts,
uint16_t nb_pkts)
{
volatile union ixgbe_adv_tx_desc *txdp =
&txq->ixgbe_tx_ring[txq->tx_tail];
@@ -231,7 +231,7 @@ static inline uint16_t
tx_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
- struct ixgbe_tx_queue *txq = (struct ixgbe_tx_queue *)tx_queue;
+ struct ci_
[PATCH v4 07/24] net/iavf: use common Tx queue structure
Merge in the few additional fields used by iavf driver and convert it to
using the common Tx queue structure also.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h | 15 +++-
drivers/net/iavf/iavf.h | 2 +-
drivers/net/iavf/iavf_ethdev.c | 4 +-
drivers/net/iavf/iavf_rxtx.c| 42 ++---
drivers/net/iavf/iavf_rxtx.h| 49 +++--
drivers/net/iavf/iavf_rxtx_vec_avx2.c | 4 +-
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 14 +++
drivers/net/iavf/iavf_rxtx_vec_common.h | 8 ++--
drivers/net/iavf/iavf_rxtx_vec_sse.c| 8 ++--
drivers/net/iavf/iavf_vchnl.c | 6 +--
10 files changed, 62 insertions(+), 90 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index c965f5ee6c..c4a1a0c816 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -31,8 +31,9 @@ typedef void (*ice_tx_release_mbufs_t)(struct ci_tx_queue
*txq);
struct ci_tx_queue {
union { /* TX ring virtual address */
- volatile struct ice_tx_desc *ice_tx_ring;
volatile struct i40e_tx_desc *i40e_tx_ring;
+ volatile struct iavf_tx_desc *iavf_tx_ring;
+ volatile struct ice_tx_desc *ice_tx_ring;
};
volatile uint8_t *qtx_tail; /* register address of tail */
struct ci_tx_entry *sw_ring; /* virtual address of SW ring */
@@ -63,8 +64,9 @@ struct ci_tx_queue {
bool tx_deferred_start; /* don't start this queue in dev start */
bool q_set; /* indicate if tx queue has been configured */
union { /* the VSI this queue belongs to */
- struct ice_vsi *ice_vsi;
struct i40e_vsi *i40e_vsi;
+ struct iavf_vsi *iavf_vsi;
+ struct ice_vsi *ice_vsi;
};
const struct rte_memzone *mz;
@@ -76,6 +78,15 @@ struct ci_tx_queue {
struct { /* I40E driver specific values */
uint8_t dcb_tc;
};
+ struct { /* iavf driver specific values */
+ uint16_t ipsec_crypto_pkt_md_offset;
+ uint8_t rel_mbufs_type;
+#define IAVF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IAVF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+ uint8_t tc;
+ bool use_ctx; /* with ctx info, each pkt needs two
descriptors */
+ };
};
};
diff --git a/drivers/net/iavf/iavf.h b/drivers/net/iavf/iavf.h
index ad526c644c..956c60ef45 100644
--- a/drivers/net/iavf/iavf.h
+++ b/drivers/net/iavf/iavf.h
@@ -98,7 +98,7 @@
struct iavf_adapter;
struct iavf_rx_queue;
-struct iavf_tx_queue;
+struct ci_tx_queue;
struct iavf_ipsec_crypto_stats {
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 7f80cd6258..328c224c93 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -954,7 +954,7 @@ static int
iavf_start_queues(struct rte_eth_dev *dev)
{
struct iavf_rx_queue *rxq;
- struct iavf_tx_queue *txq;
+ struct ci_tx_queue *txq;
int i;
uint16_t nb_txq, nb_rxq;
@@ -1885,7 +1885,7 @@ iavf_dev_update_mbuf_stats(struct rte_eth_dev *ethdev,
struct iavf_mbuf_stats *mbuf_stats)
{
uint16_t idx;
- struct iavf_tx_queue *txq;
+ struct ci_tx_queue *txq;
for (idx = 0; idx < ethdev->data->nb_tx_queues; idx++) {
txq = ethdev->data->tx_queues[idx];
diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c
index 6eda91e76b..7e381b2a17 100644
--- a/drivers/net/iavf/iavf_rxtx.c
+++ b/drivers/net/iavf/iavf_rxtx.c
@@ -213,7 +213,7 @@ check_rx_vec_allow(struct iavf_rx_queue *rxq)
}
static inline bool
-check_tx_vec_allow(struct iavf_tx_queue *txq)
+check_tx_vec_allow(struct ci_tx_queue *txq)
{
if (!(txq->offloads & IAVF_TX_NO_VECTOR_FLAGS) &&
txq->tx_rs_thresh >= IAVF_VPMD_TX_MAX_BURST &&
@@ -282,7 +282,7 @@ reset_rx_queue(struct iavf_rx_queue *rxq)
}
static inline void
-reset_tx_queue(struct iavf_tx_queue *txq)
+reset_tx_queue(struct ci_tx_queue *txq)
{
struct ci_tx_entry *txe;
uint32_t i, size;
@@ -388,7 +388,7 @@ release_rxq_mbufs(struct iavf_rx_queue *rxq)
}
static inline void
-release_txq_mbufs(struct iavf_tx_queue *txq)
+release_txq_mbufs(struct ci_tx_queue *txq)
{
uint16_t i;
@@ -778,7 +778,7 @@ iavf_dev_tx_queue_setup(struct rte_eth_dev *dev,
struct iavf_info *vf =
IAVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
struct iavf_vsi *vsi = &vf->vsi;
- struct iavf_tx_queue *txq;
+ struct ci_tx_queue *txq;
const struct rte_memzone *mz;
uint32_t ring_size;
uint16_t tx_rs_thresh, tx_free_thresh;
@@ -814,7 +814,7 @@ i
[PATCH v4 04/24] drivers/net: align Tx queue struct field names
Across the various Intel drivers sometimes different names are given to
fields in the Tx queue structure which have the same function. Do some
renaming to align things better for future merging.
Signed-off-by: Bruce Richardson
---
drivers/net/i40e/i40e_rxtx.c| 6 +--
drivers/net/i40e/i40e_rxtx.h| 2 +-
drivers/net/iavf/iavf_rxtx.c| 60 -
drivers/net/iavf/iavf_rxtx.h| 14 +++---
drivers/net/iavf/iavf_rxtx_vec_avx2.c | 19
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 57 +++
drivers/net/iavf/iavf_rxtx_vec_common.h | 24 +-
drivers/net/iavf/iavf_rxtx_vec_sse.c| 18
drivers/net/iavf/iavf_vchnl.c | 2 +-
drivers/net/ixgbe/base/ixgbe_osdep.h| 2 +-
drivers/net/ixgbe/ixgbe_rxtx.c | 16 +++
drivers/net/ixgbe/ixgbe_rxtx.h | 6 +--
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 2 +-
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c | 2 +-
14 files changed, 116 insertions(+), 114 deletions(-)
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 2e1f07d2a1..b0bb20fe9a 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -2549,7 +2549,7 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev,
txq->vsi = vsi;
txq->tx_deferred_start = tx_conf->tx_deferred_start;
- txq->tx_ring_phys_addr = tz->iova;
+ txq->tx_ring_dma = tz->iova;
txq->tx_ring = (struct i40e_tx_desc *)tz->addr;
/* Allocate software ring */
@@ -2923,7 +2923,7 @@ i40e_tx_queue_init(struct i40e_tx_queue *txq)
/* clear the context structure first */
memset(&tx_ctx, 0, sizeof(tx_ctx));
tx_ctx.new_context = 1;
- tx_ctx.base = txq->tx_ring_phys_addr / I40E_QUEUE_BASE_ADDR_UNIT;
+ tx_ctx.base = txq->tx_ring_dma / I40E_QUEUE_BASE_ADDR_UNIT;
tx_ctx.qlen = txq->nb_tx_desc;
#ifdef RTE_LIBRTE_IEEE1588
@@ -3209,7 +3209,7 @@ i40e_fdir_setup_tx_resources(struct i40e_pf *pf)
txq->reg_idx = pf->fdir.fdir_vsi->base_queue;
txq->vsi = pf->fdir.fdir_vsi;
- txq->tx_ring_phys_addr = tz->iova;
+ txq->tx_ring_dma = tz->iova;
txq->tx_ring = (struct i40e_tx_desc *)tz->addr;
/*
diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
index 0f5d3cb0b7..f420c98687 100644
--- a/drivers/net/i40e/i40e_rxtx.h
+++ b/drivers/net/i40e/i40e_rxtx.h
@@ -129,7 +129,7 @@ struct i40e_rx_queue {
*/
struct i40e_tx_queue {
uint16_t nb_tx_desc; /**< number of TX descriptors */
- uint64_t tx_ring_phys_addr; /**< TX ring DMA address */
+ rte_iova_t tx_ring_dma; /**< TX ring DMA address */
volatile struct i40e_tx_desc *tx_ring; /**< TX ring virtual address */
struct ci_tx_entry *sw_ring; /**< virtual address of SW ring */
uint16_t tx_tail; /**< current value of tail register */
diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c
index e337f20073..adaaeb4625 100644
--- a/drivers/net/iavf/iavf_rxtx.c
+++ b/drivers/net/iavf/iavf_rxtx.c
@@ -216,8 +216,8 @@ static inline bool
check_tx_vec_allow(struct iavf_tx_queue *txq)
{
if (!(txq->offloads & IAVF_TX_NO_VECTOR_FLAGS) &&
- txq->rs_thresh >= IAVF_VPMD_TX_MAX_BURST &&
- txq->rs_thresh <= IAVF_VPMD_TX_MAX_FREE_BUF) {
+ txq->tx_rs_thresh >= IAVF_VPMD_TX_MAX_BURST &&
+ txq->tx_rs_thresh <= IAVF_VPMD_TX_MAX_FREE_BUF) {
PMD_INIT_LOG(DEBUG, "Vector tx can be enabled on this txq.");
return true;
}
@@ -309,13 +309,13 @@ reset_tx_queue(struct iavf_tx_queue *txq)
}
txq->tx_tail = 0;
- txq->nb_used = 0;
+ txq->nb_tx_used = 0;
txq->last_desc_cleaned = txq->nb_tx_desc - 1;
- txq->nb_free = txq->nb_tx_desc - 1;
+ txq->nb_tx_free = txq->nb_tx_desc - 1;
- txq->next_dd = txq->rs_thresh - 1;
- txq->next_rs = txq->rs_thresh - 1;
+ txq->tx_next_dd = txq->tx_rs_thresh - 1;
+ txq->tx_next_rs = txq->tx_rs_thresh - 1;
}
static int
@@ -845,8 +845,8 @@ iavf_dev_tx_queue_setup(struct rte_eth_dev *dev,
}
txq->nb_tx_desc = nb_desc;
- txq->rs_thresh = tx_rs_thresh;
- txq->free_thresh = tx_free_thresh;
+ txq->tx_rs_thresh = tx_rs_thresh;
+ txq->tx_free_thresh = tx_free_thresh;
txq->queue_id = queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
@@ -881,7 +881,7 @@ iavf_dev_tx_queue_setup(struct rte_eth_dev *dev,
rte_free(txq);
return -ENOMEM;
}
- txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring_dma = mz->iova;
txq->tx_ring = (struct iavf_tx_desc *)mz->addr;
txq->mz = mz;
@@ -2387,7 +2387,7 @@ iavf_xmit_cleanup(struct iavf_tx_queue *txq)
volatile struct iavf_tx_desc *txd = txq->tx_ring;
- desc_to_clean_to = (uint16_t)(last_desc_clea
[PATCH v4 08/24] net/ixgbe: convert Tx queue context cache field to ptr
Rather than having a two element array of context cache values inside
the Tx queue structure, convert it to a pointer to a cache at the end of
the structure. This makes future merging of the structure easier as we
don't need the "ixgbe_advctx_info" struct defined when defining a
combined queue structure.
Signed-off-by: Bruce Richardson
---
drivers/net/ixgbe/ixgbe_rxtx.c| 7 ---
drivers/net/ixgbe/ixgbe_rxtx.h| 4 ++--
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 3 +--
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
index f7ddbba1b6..2ca26cd132 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx.c
@@ -2522,8 +2522,7 @@ ixgbe_reset_tx_queue(struct ixgbe_tx_queue *txq)
txq->last_desc_cleaned = (uint16_t)(txq->nb_tx_desc - 1);
txq->nb_tx_free = (uint16_t)(txq->nb_tx_desc - 1);
txq->ctx_curr = 0;
- memset((void *)&txq->ctx_cache, 0,
- IXGBE_CTX_NUM * sizeof(struct ixgbe_advctx_info));
+ memset(txq->ctx_cache, 0, IXGBE_CTX_NUM * sizeof(struct
ixgbe_advctx_info));
}
static const struct ixgbe_txq_ops def_txq_ops = {
@@ -2741,10 +2740,12 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev,
}
/* First allocate the tx queue data structure */
- txq = rte_zmalloc_socket("ethdev TX queue", sizeof(struct
ixgbe_tx_queue),
+ txq = rte_zmalloc_socket("ethdev TX queue", sizeof(struct
ixgbe_tx_queue) +
+ sizeof(struct ixgbe_advctx_info) *
IXGBE_CTX_NUM,
RTE_CACHE_LINE_SIZE, socket_id);
if (txq == NULL)
return -ENOMEM;
+ txq->ctx_cache = RTE_PTR_ADD(txq, sizeof(struct ixgbe_tx_queue));
/*
* Allocate TX ring hardware descriptors. A memzone large enough to
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.h b/drivers/net/ixgbe/ixgbe_rxtx.h
index f6bae37cf3..847cacf7b5 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.h
+++ b/drivers/net/ixgbe/ixgbe_rxtx.h
@@ -215,8 +215,8 @@ struct ixgbe_tx_queue {
uint8_t wthresh; /**< Write-back threshold reg. */
uint64_t offloads; /**< Tx offload flags of RTE_ETH_TX_OFFLOAD_* */
uint32_tctx_curr; /**< Hardware context states. */
- /** Hardware context0 history. */
- struct ixgbe_advctx_info ctx_cache[IXGBE_CTX_NUM];
+ /** Hardware context history. */
+ struct ixgbe_advctx_info *ctx_cache;
const struct ixgbe_txq_ops *ops; /**< txq ops */
booltx_deferred_start; /**< not in global dev start. */
#ifdef RTE_LIB_SECURITY
diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
index cc51bf6eed..ec334b5f65 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
+++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
@@ -176,8 +176,7 @@ _ixgbe_reset_tx_queue_vec(struct ixgbe_tx_queue *txq)
txq->last_desc_cleaned = (uint16_t)(txq->nb_tx_desc - 1);
txq->nb_tx_free = (uint16_t)(txq->nb_tx_desc - 1);
txq->ctx_curr = 0;
- memset((void *)&txq->ctx_cache, 0,
- IXGBE_CTX_NUM * sizeof(struct ixgbe_advctx_info));
+ memset(txq->ctx_cache, 0, IXGBE_CTX_NUM * sizeof(struct
ixgbe_advctx_info));
}
static inline int
--
2.43.0
[PATCH v4 14/24] net/ice: move Tx queue mbuf cleanup fn to common
The functions to loop over the Tx queue and clean up all the mbufs on
it, e.g. for queue shutdown, is not device specific and so can move into
the common_intel headers. Only complication is ensuring that the
correct ring format, either minimal vector or full structure, is used.
Ice driver currently uses two functions and a function pointer to help
with this - though actually one of those functions uses a further check
inside it - so we can simplify this down to just one common function,
with a flag set in the appropriate place. This avoids checking for
AVX-512-specific functions, which were the only function using the
smaller struct in this driver.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h| 49 -
drivers/net/ice/ice_dcf_ethdev.c | 5 +--
drivers/net/ice/ice_ethdev.h | 3 +-
drivers/net/ice/ice_rxtx.c| 33 +
drivers/net/ice/ice_rxtx_vec_common.h | 51 ---
drivers/net/ice/ice_rxtx_vec_sse.c| 4 ---
6 files changed, 60 insertions(+), 85 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index 26aef528fa..1bf2a61b2f 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -65,6 +65,8 @@ struct ci_tx_queue {
rte_iova_t tx_ring_dma;/* TX ring DMA address */
bool tx_deferred_start; /* don't start this queue in dev start */
bool q_set; /* indicate if tx queue has been configured */
+ bool vector_tx; /* port is using vector TX */
+ bool vector_sw_ring;/* port is using vectorized SW ring
(ieth_tx_entry_vec) */
union { /* the VSI this queue belongs to */
struct i40e_vsi *i40e_vsi;
struct iavf_vsi *iavf_vsi;
@@ -74,7 +76,6 @@ struct ci_tx_queue {
union {
struct { /* ICE driver specific values */
- ice_tx_release_mbufs_t tx_rel_mbufs;
uint32_t q_teid; /* TX schedule node id. */
};
struct { /* I40E driver specific values */
@@ -270,4 +271,50 @@ ci_tx_free_bufs_vec(struct ci_tx_queue *txq,
ci_desc_done_fn desc_done, bool ctx
return txq->tx_rs_thresh;
}
+#define IETH_FREE_BUFS_LOOP(txq, swr, start) do { \
+ uint16_t i = start; \
+ if (txq->tx_tail < i) { \
+ for (; i < txq->nb_tx_desc; i++) { \
+ rte_pktmbuf_free_seg(swr[i].mbuf); \
+ swr[i].mbuf = NULL; \
+ } \
+ i = 0; \
+ } \
+ for (; i < txq->tx_tail; i++) { \
+ rte_pktmbuf_free_seg(swr[i].mbuf); \
+ swr[i].mbuf = NULL; \
+ } \
+} while (0)
+
+static inline void
+ci_txq_release_all_mbufs(struct ci_tx_queue *txq)
+{
+ if (unlikely(!txq || !txq->sw_ring))
+ return;
+
+ if (!txq->vector_tx) {
+ for (uint16_t i = 0; i < txq->nb_tx_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+ return;
+ }
+
+ /**
+* vPMD tx will not set sw_ring's mbuf to NULL after free,
+* so need to free remains more carefully.
+*/
+ const uint16_t start = txq->tx_next_dd - txq->tx_rs_thresh + 1;
+
+ if (txq->vector_sw_ring) {
+ struct ci_tx_entry_vec *swr = txq->sw_ring_vec;
+ IETH_FREE_BUFS_LOOP(txq, swr, start);
+ } else {
+ struct ci_tx_entry *swr = txq->sw_ring;
+ IETH_FREE_BUFS_LOOP(txq, swr, start);
+ }
+}
+
#endif /* _COMMON_INTEL_TX_H_ */
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index a0c065d78c..c20399cd84 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -24,6 +24,7 @@
#include "ice_generic_flow.h"
#include "ice_dcf_ethdev.h"
#include "ice_rxtx.h"
+#include "_common_intel/tx.h"
#define DCF_NUM_MACADDR_MAX 64
@@ -500,7 +501,7 @@ ice_dcf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
}
txq = dev->data->tx_queues[tx_queue_id];
- txq->tx_rel_mbufs(txq);
+ ci_txq_release_all_mbufs(txq);
reset_tx_queue(txq);
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
@@ -650,7 +651,7 @@ ice_dcf_stop_queues(struct rte_eth_dev *dev)
txq = dev->data->tx_queues[i];
if (!txq)
continue;
- txq->tx_rel_mbufs(txq);
+ ci_txq_release_all_mbufs(txq);
reset_tx_queue(txq);
dev->data->tx_queue_s
[PATCH v4 11/24] net/_common_intel: add post-Tx buffer free function
The actions taken for post-Tx buffer free for the SSE and AVX drivers
for i40e, iavf and ice drivers are all common, so centralize those in
common/intel_eth driver.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h | 71
drivers/net/i40e/i40e_rxtx_vec_common.h | 72 -
drivers/net/iavf/iavf_rxtx_vec_common.h | 61 -
drivers/net/ice/ice_rxtx_vec_common.h | 61 -
4 files changed, 98 insertions(+), 167 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index c372d2838b..a930309c05 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -7,6 +7,7 @@
#include
#include
+#include
/* forward declaration of the common intel (ci) queue structure */
struct ci_tx_queue;
@@ -107,4 +108,74 @@ ci_tx_backlog_entry(struct ci_tx_entry *txep, struct
rte_mbuf **tx_pkts, uint16_
txep[i].mbuf = tx_pkts[i];
}
+#define IETH_VPMD_TX_MAX_FREE_BUF 64
+
+typedef int (*ci_desc_done_fn)(struct ci_tx_queue *txq, uint16_t idx);
+
+static __rte_always_inline int
+ci_tx_free_bufs(struct ci_tx_queue *txq, ci_desc_done_fn desc_done)
+{
+ struct ci_tx_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[IETH_VPMD_TX_MAX_FREE_BUF];
+
+ /* check DD bits on threshold descriptor */
+ if (!desc_done(txq, txq->tx_next_dd))
+ return 0;
+
+ n = txq->tx_rs_thresh;
+
+/* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)];
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {
+ for (i = 0; i < n; i++) {
+ free[i] = txep[i].mbuf;
+ /* no need to reset txep[i].mbuf in vector path */
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, n);
+ goto done;
+ }
+
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+(void *)free,
+nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq->tx_rs_thresh);
+ txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq->tx_rs_thresh);
+ if (txq->tx_next_dd >= txq->nb_tx_desc)
+ txq->tx_next_dd = (uint16_t)(txq->tx_rs_thresh - 1);
+
+ return txq->tx_rs_thresh;
+}
+
#endif /* _COMMON_INTEL_TX_H_ */
diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h
b/drivers/net/i40e/i40e_rxtx_vec_common.h
index 57d6263ccf..907d32dd0b 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_common.h
+++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
@@ -16,72 +16,18 @@
#pragma GCC diagnostic ignored "-Wcast-qual"
#endif
+static inline int
+i40e_tx_desc_done(struct ci_tx_queue *txq, uint16_t idx)
+{
+ return (txq->i40e_tx_ring[idx].cmd_type_offset_bsz &
+ rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) ==
+ rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE);
+}
+
static __rte_always_inline int
i40e_tx_free_bufs(struct ci_tx_queue *txq)
{
- struct ci_tx_entry *txep;
- uint32_t n;
- uint32_t i;
- int nb_free = 0;
- struct rte_mbuf *m, *free[RTE_I40E_TX_MAX_FREE_BUF_SZ];
-
- /* check DD bits on threshold descriptor */
- if ((txq->i40e_tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
- rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=
- rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))
- return 0;
-
- n = txq->tx_rs_thresh;
-
-/* first buffer to free from S/W ring is at index
- * tx_next_dd - (tx_rs_thresh-1)
- */
- txep = &txq->
[PATCH v4 15/24] net/i40e: use common Tx queue mbuf cleanup fn
Update driver to be similar to the "ice" driver and use the common mbuf
ring cleanup code on shutdown of a Tx queue.
Signed-off-by: Bruce Richardson
---
drivers/net/i40e/i40e_ethdev.h | 4 +-
drivers/net/i40e/i40e_rxtx.c | 70 --
drivers/net/i40e/i40e_rxtx.h | 1 -
3 files changed, 9 insertions(+), 66 deletions(-)
diff --git a/drivers/net/i40e/i40e_ethdev.h b/drivers/net/i40e/i40e_ethdev.h
index d351193ed9..ccc8732d7d 100644
--- a/drivers/net/i40e/i40e_ethdev.h
+++ b/drivers/net/i40e/i40e_ethdev.h
@@ -1260,12 +1260,12 @@ struct i40e_adapter {
/* For RSS reta table update */
uint8_t rss_reta_updated;
-#ifdef RTE_ARCH_X86
+
+ /* used only on x86, zero on other architectures */
bool rx_use_avx2;
bool rx_use_avx512;
bool tx_use_avx2;
bool tx_use_avx512;
-#endif
};
/**
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 539b170266..b70919c5dc 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -1875,6 +1875,7 @@ i40e_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
int err;
struct ci_tx_queue *txq;
struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ const struct i40e_adapter *ad =
I40E_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
PMD_INIT_FUNC_TRACE();
@@ -1889,6 +1890,9 @@ i40e_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
PMD_DRV_LOG(WARNING, "TX queue %u is deferred start",
tx_queue_id);
+ txq->vector_tx = ad->tx_vec_allowed;
+ txq->vector_sw_ring = ad->tx_use_avx512;
+
/*
* tx_queue_id is queue id application refers to, while
* rxq->reg_idx is the real queue index.
@@ -1929,7 +1933,7 @@ i40e_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
return err;
}
- i40e_tx_queue_release_mbufs(txq);
+ ci_txq_release_all_mbufs(txq);
i40e_reset_tx_queue(txq);
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
@@ -2604,7 +2608,7 @@ i40e_tx_queue_release(void *txq)
return;
}
- i40e_tx_queue_release_mbufs(q);
+ ci_txq_release_all_mbufs(q);
rte_free(q->sw_ring);
rte_memzone_free(q->mz);
rte_free(q);
@@ -2701,66 +2705,6 @@ i40e_reset_rx_queue(struct i40e_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
-void
-i40e_tx_queue_release_mbufs(struct ci_tx_queue *txq)
-{
- struct rte_eth_dev *dev;
- uint16_t i;
-
- if (!txq || !txq->sw_ring) {
- PMD_DRV_LOG(DEBUG, "Pointer to txq or sw_ring is NULL");
- return;
- }
-
- dev = &rte_eth_devices[txq->port_id];
-
- /**
-* vPMD tx will not set sw_ring's mbuf to NULL after free,
-* so need to free remains more carefully.
-*/
-#ifdef CC_AVX512_SUPPORT
- if (dev->tx_pkt_burst == i40e_xmit_pkts_vec_avx512) {
- struct ci_tx_entry_vec *swr = (void *)txq->sw_ring;
-
- i = txq->tx_next_dd - txq->tx_rs_thresh + 1;
- if (txq->tx_tail < i) {
- for (; i < txq->nb_tx_desc; i++) {
- rte_pktmbuf_free_seg(swr[i].mbuf);
- swr[i].mbuf = NULL;
- }
- i = 0;
- }
- for (; i < txq->tx_tail; i++) {
- rte_pktmbuf_free_seg(swr[i].mbuf);
- swr[i].mbuf = NULL;
- }
- return;
- }
-#endif
- if (dev->tx_pkt_burst == i40e_xmit_pkts_vec_avx2 ||
- dev->tx_pkt_burst == i40e_xmit_pkts_vec) {
- i = txq->tx_next_dd - txq->tx_rs_thresh + 1;
- if (txq->tx_tail < i) {
- for (; i < txq->nb_tx_desc; i++) {
- rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
- txq->sw_ring[i].mbuf = NULL;
- }
- i = 0;
- }
- for (; i < txq->tx_tail; i++) {
- rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
- txq->sw_ring[i].mbuf = NULL;
- }
- } else {
- for (i = 0; i < txq->nb_tx_desc; i++) {
- if (txq->sw_ring[i].mbuf) {
- rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
- txq->sw_ring[i].mbuf = NULL;
- }
- }
- }
-}
-
static int
i40e_tx_done_cleanup_full(struct ci_tx_queue *txq,
uint32_t free_cnt)
@@ -3127,7 +3071,7 @@ i40e_dev_clear_queues(struct rte_eth_dev *dev)
for (i = 0; i < dev->data->nb_tx_queues; i++) {
if (!dev->data->tx_queues[i])
[PATCH v4 12/24] net/_common_intel: add Tx buffer free fn for AVX-512
AVX-512 code paths for ice and i40e drivers are common, and differ from
the regular post-Tx free function in that the SW ring from which the
buffers are freed does not contain anything other than the mbuf pointer.
Merge these into a common function in intel_common to reduce
duplication.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h | 92 +++
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 114 +--
drivers/net/ice/ice_rxtx_vec_avx512.c | 117 +---
3 files changed, 94 insertions(+), 229 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index a930309c05..84ff839672 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -178,4 +178,96 @@ ci_tx_free_bufs(struct ci_tx_queue *txq, ci_desc_done_fn
desc_done)
return txq->tx_rs_thresh;
}
+static __rte_always_inline int
+ci_tx_free_bufs_vec(struct ci_tx_queue *txq, ci_desc_done_fn desc_done)
+{
+ int nb_free = 0;
+ struct rte_mbuf *free[IETH_VPMD_TX_MAX_FREE_BUF];
+ struct rte_mbuf *m;
+
+ /* check DD bits on threshold descriptor */
+ if (!desc_done(txq, txq->tx_next_dd))
+ return 0;
+
+ const uint32_t n = txq->tx_rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+* tx_next_dd - (tx_rs_thresh - 1)
+*/
+ struct ci_tx_entry_vec *txep = txq->sw_ring_vec;
+ txep += txq->tx_next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0)
{
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ void **cache_objs;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
rte_lcore_id());
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+* 1. Add the objects to the cache
+* 2. Anything greater than the cache min value (if it
+* crosses the cache flush threshold) is flushed to the ring.
+*/
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ memcpy(&cache_objs[copied], &txep[copied], 32 *
sizeof(void *));
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
&cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (uint32_t i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
(void *)free, nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (uint32_t i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq->tx_rs_thresh);
+ txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq->tx_rs_thresh);
+ if (txq->tx_next_dd >= txq->nb_tx_desc)
+ txq->tx_next_dd = (uint16_t)(txq->tx_rs_thresh - 1);
+
+ return txq->tx_rs_thresh;
+}
+
#endif /* _COMMON_INTEL_TX_H_ */
diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx512.c
b/drivers/net/i40e/i40e_rxtx_vec_avx512.c
index a3f6d1667f..9bb2a44231 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_avx512.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_avx512.c
@@ -754,118 +754,6 @@ i40e_recv_scattered_pkts_vec_avx512(void *rx_queue,
rx_pkts + retval, nb_pkts);
}
-static __rte_always_inline int
-i40e_tx_free_bufs_avx512(struct c
[PATCH v4 10/24] net/_common_intel: pack Tx queue structure
Move some fields about to better pack the Tx queue structure and make
sure all data used by the vector codepaths is on the first cacheline of
the structure. Checking with "pahole" on 64-bit build, only one 6-byte
hole is left in the structure - on second cacheline - after this patch.
As part of the reordering, move the p/h/wthresh values to the
ixgbe-specific part of the union. That is the only driver which actually
uses those values. i40e and ice drivers just record the values for later
return, so we can drop them from the Tx queue structure for those
drivers and just report the defaults in all cases.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h | 12 +---
drivers/net/i40e/i40e_rxtx.c | 9 +++--
drivers/net/ice/ice_rxtx.c | 9 +++--
3 files changed, 11 insertions(+), 19 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index 51ae3b051d..c372d2838b 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -41,7 +41,6 @@ struct ci_tx_queue {
struct ci_tx_entry *sw_ring; /* virtual address of SW ring */
struct ci_tx_entry_vec *sw_ring_vec;
};
- rte_iova_t tx_ring_dma;/* TX ring DMA address */
uint16_t nb_tx_desc; /* number of TX descriptors */
uint16_t tx_tail; /* current value of tail register */
uint16_t nb_tx_used; /* number of TX desc used since RS bit set */
@@ -55,16 +54,14 @@ struct ci_tx_queue {
uint16_t tx_free_thresh;
/* Number of TX descriptors to use before RS bit is set. */
uint16_t tx_rs_thresh;
- uint8_t pthresh; /**< Prefetch threshold register. */
- uint8_t hthresh; /**< Host threshold register. */
- uint8_t wthresh; /**< Write-back threshold reg. */
uint16_t port_id; /* Device port identifier. */
uint16_t queue_id; /* TX queue index. */
uint16_t reg_idx;
- uint64_t offloads;
uint16_t tx_next_dd;
uint16_t tx_next_rs;
+ uint64_t offloads;
uint64_t mbuf_errors;
+ rte_iova_t tx_ring_dma;/* TX ring DMA address */
bool tx_deferred_start; /* don't start this queue in dev start */
bool q_set; /* indicate if tx queue has been configured */
union { /* the VSI this queue belongs to */
@@ -95,9 +92,10 @@ struct ci_tx_queue {
const struct ixgbe_txq_ops *ops;
struct ixgbe_advctx_info *ctx_cache;
uint32_t ctx_curr;
-#ifdef RTE_LIB_SECURITY
+ uint8_t pthresh; /**< Prefetch threshold register. */
+ uint8_t hthresh; /**< Host threshold register. */
+ uint8_t wthresh; /**< Write-back threshold reg. */
uint8_t using_ipsec; /**< indicates that IPsec TX
feature is in use */
-#endif
};
};
};
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 305bc53480..539b170266 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -2539,9 +2539,6 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev,
txq->nb_tx_desc = nb_desc;
txq->tx_rs_thresh = tx_rs_thresh;
txq->tx_free_thresh = tx_free_thresh;
- txq->pthresh = tx_conf->tx_thresh.pthresh;
- txq->hthresh = tx_conf->tx_thresh.hthresh;
- txq->wthresh = tx_conf->tx_thresh.wthresh;
txq->queue_id = queue_idx;
txq->reg_idx = reg_idx;
txq->port_id = dev->data->port_id;
@@ -3310,9 +3307,9 @@ i40e_txq_info_get(struct rte_eth_dev *dev, uint16_t
queue_id,
qinfo->nb_desc = txq->nb_tx_desc;
- qinfo->conf.tx_thresh.pthresh = txq->pthresh;
- qinfo->conf.tx_thresh.hthresh = txq->hthresh;
- qinfo->conf.tx_thresh.wthresh = txq->wthresh;
+ qinfo->conf.tx_thresh.pthresh = I40E_DEFAULT_TX_PTHRESH;
+ qinfo->conf.tx_thresh.hthresh = I40E_DEFAULT_TX_HTHRESH;
+ qinfo->conf.tx_thresh.wthresh = I40E_DEFAULT_TX_WTHRESH;
qinfo->conf.tx_free_thresh = txq->tx_free_thresh;
qinfo->conf.tx_rs_thresh = txq->tx_rs_thresh;
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index bcc7c7a016..e2e147ba3e 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -1492,9 +1492,6 @@ ice_tx_queue_setup(struct rte_eth_dev *dev,
txq->nb_tx_desc = nb_desc;
txq->tx_rs_thresh = tx_rs_thresh;
txq->tx_free_thresh = tx_free_thresh;
- txq->pthresh = tx_conf->tx_thresh.pthresh;
- txq->hthresh = tx_conf->tx_thresh.hthresh;
- txq->wthresh = tx_conf->tx_thresh.wthresh;
txq->queue_id = queue_idx;
txq->reg_idx = vsi->base_queue + queue_idx;
@@ -1583,9 +1580,9 @@ ice_txq_info_get(struct rte_eth_dev *dev, uint16_t
queue_id,
qinfo->nb_desc = txq->nb_tx_desc;
- qinfo->conf.tx_th
[PATCH v4 13/24] net/iavf: use common Tx free fn for AVX-512
Switch the iavf driver to use the common Tx free function. This requires
one additional parameter to that function, since iavf sometimes uses
context descriptors which means that we have double the descriptors per
SW ring slot.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h | 6 +-
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 2 +-
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 119 +---
drivers/net/ice/ice_rxtx_vec_avx512.c | 2 +-
4 files changed, 7 insertions(+), 122 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index 84ff839672..26aef528fa 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -179,7 +179,7 @@ ci_tx_free_bufs(struct ci_tx_queue *txq, ci_desc_done_fn
desc_done)
}
static __rte_always_inline int
-ci_tx_free_bufs_vec(struct ci_tx_queue *txq, ci_desc_done_fn desc_done)
+ci_tx_free_bufs_vec(struct ci_tx_queue *txq, ci_desc_done_fn desc_done, bool
ctx_descs)
{
int nb_free = 0;
struct rte_mbuf *free[IETH_VPMD_TX_MAX_FREE_BUF];
@@ -189,13 +189,13 @@ ci_tx_free_bufs_vec(struct ci_tx_queue *txq,
ci_desc_done_fn desc_done)
if (!desc_done(txq, txq->tx_next_dd))
return 0;
- const uint32_t n = txq->tx_rs_thresh;
+ const uint32_t n = txq->tx_rs_thresh >> ctx_descs;
/* first buffer to free from S/W ring is at index
* tx_next_dd - (tx_rs_thresh - 1)
*/
struct ci_tx_entry_vec *txep = txq->sw_ring_vec;
- txep += txq->tx_next_dd - (n - 1);
+ txep += (txq->tx_next_dd >> ctx_descs) - (n - 1);
if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0)
{
struct rte_mempool *mp = txep[0].mbuf->pool;
diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx512.c
b/drivers/net/i40e/i40e_rxtx_vec_avx512.c
index 9bb2a44231..c555c3491d 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_avx512.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_avx512.c
@@ -829,7 +829,7 @@ i40e_xmit_fixed_burst_vec_avx512(void *tx_queue, struct
rte_mbuf **tx_pkts,
uint64_t rs = I40E_TX_DESC_CMD_RS | I40E_TD_CMD;
if (txq->nb_tx_free < txq->tx_free_thresh)
- ci_tx_free_bufs_vec(txq, i40e_tx_desc_done);
+ ci_tx_free_bufs_vec(txq, i40e_tx_desc_done, false);
nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
if (unlikely(nb_pkts == 0))
diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx512.c
b/drivers/net/iavf/iavf_rxtx_vec_avx512.c
index 9cf7171524..8543490c70 100644
--- a/drivers/net/iavf/iavf_rxtx_vec_avx512.c
+++ b/drivers/net/iavf/iavf_rxtx_vec_avx512.c
@@ -1844,121 +1844,6 @@
iavf_recv_scattered_pkts_vec_avx512_flex_rxd_offload(void *rx_queue,
true);
}
-static __rte_always_inline int
-iavf_tx_free_bufs_avx512(struct ci_tx_queue *txq)
-{
- struct ci_tx_entry_vec *txep;
- uint32_t n;
- uint32_t i;
- int nb_free = 0;
- struct rte_mbuf *m, *free[IAVF_VPMD_TX_MAX_FREE_BUF];
-
- /* check DD bits on threshold descriptor */
- if ((txq->iavf_tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
-rte_cpu_to_le_64(IAVF_TXD_QW1_DTYPE_MASK)) !=
- rte_cpu_to_le_64(IAVF_TX_DESC_DTYPE_DESC_DONE))
- return 0;
-
- n = txq->tx_rs_thresh >> txq->use_ctx;
-
-/* first buffer to free from S/W ring is at index
- * tx_next_dd - (tx_rs_thresh-1)
- */
- txep = (void *)txq->sw_ring;
- txep += (txq->tx_next_dd >> txq->use_ctx) - (n - 1);
-
- if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0)
{
- struct rte_mempool *mp = txep[0].mbuf->pool;
- struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
- rte_lcore_id());
- void **cache_objs;
-
- if (!cache || cache->len == 0)
- goto normal;
-
- cache_objs = &cache->objs[cache->len];
-
- if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
- rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
- goto done;
- }
-
- /* The cache follows the following algorithm
-* 1. Add the objects to the cache
-* 2. Anything greater than the cache min value (if it
crosses the
-* cache flush threshold) is flushed to the ring.
-*/
- /* Add elements back into the cache */
- uint32_t copied = 0;
- /* n is multiple of 32 */
- while (copied < n) {
-#ifdef RTE_ARCH_64
- const __m512i a = _mm512_loadu_si512(&txep[copied]);
- const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
- const __m512i c = _mm
[PATCH v4 18/24] net/ice: use vector SW ring for all vector paths
The AVX-512 code path used a smaller SW ring structure only containing
the mbuf pointer, but no other fields. The other fields are only used in
the scalar code path, so update all vector driver code paths to use the
smaller, faster structure.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h| 7 +++
drivers/net/ice/ice_rxtx.c| 2 +-
drivers/net/ice/ice_rxtx_vec_avx2.c | 12 ++--
drivers/net/ice/ice_rxtx_vec_avx512.c | 14 ++
drivers/net/ice/ice_rxtx_vec_common.h | 6 --
drivers/net/ice/ice_rxtx_vec_sse.c| 12 ++--
6 files changed, 22 insertions(+), 31 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index 310b51adcf..aa42b9b49f 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -109,6 +109,13 @@ ci_tx_backlog_entry(struct ci_tx_entry *txep, struct
rte_mbuf **tx_pkts, uint16_
txep[i].mbuf = tx_pkts[i];
}
+static __rte_always_inline void
+ci_tx_backlog_entry_vec(struct ci_tx_entry_vec *txep, struct rte_mbuf
**tx_pkts, uint16_t nb_pkts)
+{
+ for (uint16_t i = 0; i < nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
#define IETH_VPMD_TX_MAX_FREE_BUF 64
typedef int (*ci_desc_done_fn)(struct ci_tx_queue *txq, uint16_t idx);
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index ad0ddf6a88..77cb6688a7 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -825,7 +825,7 @@ ice_tx_queue_start(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
/* record what kind of descriptor cleanup we need on teardown */
txq->vector_tx = ad->tx_vec_allowed;
- txq->vector_sw_ring = ad->tx_use_avx512;
+ txq->vector_sw_ring = txq->vector_tx;
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c
b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 12ffa0fa9a..98bab322b4 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -858,7 +858,7 @@ ice_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
{
struct ci_tx_queue *txq = (struct ci_tx_queue *)tx_queue;
volatile struct ice_tx_desc *txdp;
- struct ci_tx_entry *txep;
+ struct ci_tx_entry_vec *txep;
uint16_t n, nb_commit, tx_id;
uint64_t flags = ICE_TD_CMD;
uint64_t rs = ICE_TX_DESC_CMD_RS | ICE_TD_CMD;
@@ -867,7 +867,7 @@ ice_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
nb_pkts = RTE_MIN(nb_pkts, txq->tx_rs_thresh);
if (txq->nb_tx_free < txq->tx_free_thresh)
- ice_tx_free_bufs_vec(txq);
+ ci_tx_free_bufs_vec(txq, ice_tx_desc_done, false);
nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
if (unlikely(nb_pkts == 0))
@@ -875,13 +875,13 @@ ice_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
tx_id = txq->tx_tail;
txdp = &txq->ice_tx_ring[tx_id];
- txep = &txq->sw_ring[tx_id];
+ txep = &txq->sw_ring_vec[tx_id];
txq->nb_tx_free = (uint16_t)(txq->nb_tx_free - nb_pkts);
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- ci_tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, n);
ice_vtx(txdp, tx_pkts, n - 1, flags, offload);
tx_pkts += (n - 1);
@@ -896,10 +896,10 @@ ice_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
/* avoid reach the end of ring */
txdp = &txq->ice_tx_ring[tx_id];
- txep = &txq->sw_ring[tx_id];
+ txep = &txq->sw_ring_vec[tx_id];
}
- ci_tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, nb_commit);
ice_vtx(txdp, tx_pkts, nb_commit, flags, offload);
diff --git a/drivers/net/ice/ice_rxtx_vec_avx512.c
b/drivers/net/ice/ice_rxtx_vec_avx512.c
index f6ec593f96..481f784e34 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx512.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx512.c
@@ -924,16 +924,6 @@ ice_vtx(volatile struct ice_tx_desc *txdp, struct rte_mbuf
**pkt,
}
}
-static __rte_always_inline void
-ice_tx_backlog_entry_avx512(struct ci_tx_entry_vec *txep,
- struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
-{
- int i;
-
- for (i = 0; i < (int)nb_pkts; ++i)
- txep[i].mbuf = tx_pkts[i];
-}
-
static __rte_always_inline uint16_t
ice_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts, bool do_offload)
@@ -964,7 +954,7 @@ ice_xmit_fixed_burst_vec_avx512(void *tx_queue, struct
rte_mbuf **tx_pkts,
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- ice_tx
[PATCH v4 19/24] net/i40e: use vector SW ring for all vector paths
The AVX-512 code path used a smaller SW ring structure only containing
the mbuf pointer, but no other fields. The other fields are only used in
the scalar code path, so update all vector driver code paths (AVX2, SSE,
Neon, Altivec) to use the smaller, faster structure.
Signed-off-by: Bruce Richardson
---
drivers/net/i40e/i40e_rxtx.c | 8 +---
drivers/net/i40e/i40e_rxtx_vec_altivec.c | 12 ++--
drivers/net/i40e/i40e_rxtx_vec_avx2.c| 12 ++--
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 14 ++
drivers/net/i40e/i40e_rxtx_vec_common.h | 6 --
drivers/net/i40e/i40e_rxtx_vec_neon.c| 12 ++--
drivers/net/i40e/i40e_rxtx_vec_sse.c | 12 ++--
7 files changed, 31 insertions(+), 45 deletions(-)
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 081d743e62..745c467912 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -1891,7 +1891,7 @@ i40e_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
tx_queue_id);
txq->vector_tx = ad->tx_vec_allowed;
- txq->vector_sw_ring = ad->tx_use_avx512;
+ txq->vector_sw_ring = txq->vector_tx;
/*
* tx_queue_id is queue id application refers to, while
@@ -3550,9 +3550,11 @@ i40e_set_tx_function(struct rte_eth_dev *dev)
}
}
+ if (rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_128)
+ ad->tx_vec_allowed = false;
+
if (ad->tx_simple_allowed) {
- if (ad->tx_vec_allowed &&
- rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ if (ad->tx_vec_allowed) {
#ifdef RTE_ARCH_X86
if (ad->tx_use_avx512) {
#ifdef CC_AVX512_SUPPORT
diff --git a/drivers/net/i40e/i40e_rxtx_vec_altivec.c
b/drivers/net/i40e/i40e_rxtx_vec_altivec.c
index 500bba2cef..b6900a3e15 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_altivec.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_altivec.c
@@ -553,14 +553,14 @@ i40e_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
{
struct ci_tx_queue *txq = (struct ci_tx_queue *)tx_queue;
volatile struct i40e_tx_desc *txdp;
- struct ci_tx_entry *txep;
+ struct ci_tx_entry_vec *txep;
uint16_t n, nb_commit, tx_id;
uint64_t flags = I40E_TD_CMD;
uint64_t rs = I40E_TX_DESC_CMD_RS | I40E_TD_CMD;
int i;
if (txq->nb_tx_free < txq->tx_free_thresh)
- i40e_tx_free_bufs(txq);
+ ci_tx_free_bufs_vec(txq, i40e_tx_desc_done, false);
nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
nb_commit = nb_pkts;
@@ -569,13 +569,13 @@ i40e_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
tx_id = txq->tx_tail;
txdp = &txq->i40e_tx_ring[tx_id];
- txep = &txq->sw_ring[tx_id];
+ txep = &txq->sw_ring_vec[tx_id];
txq->nb_tx_free = (uint16_t)(txq->nb_tx_free - nb_pkts);
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- ci_tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, n);
for (i = 0; i < n - 1; ++i, ++tx_pkts, ++txdp)
vtx1(txdp, *tx_pkts, flags);
@@ -589,10 +589,10 @@ i40e_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
/* avoid reach the end of ring */
txdp = &txq->i40e_tx_ring[tx_id];
- txep = &txq->sw_ring[tx_id];
+ txep = &txq->sw_ring_vec[tx_id];
}
- ci_tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, nb_commit);
vtx(txdp, tx_pkts, nb_commit, flags);
diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx2.c
b/drivers/net/i40e/i40e_rxtx_vec_avx2.c
index 29bef64287..2477573c01 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_avx2.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_avx2.c
@@ -745,13 +745,13 @@ i40e_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
{
struct ci_tx_queue *txq = (struct ci_tx_queue *)tx_queue;
volatile struct i40e_tx_desc *txdp;
- struct ci_tx_entry *txep;
+ struct ci_tx_entry_vec *txep;
uint16_t n, nb_commit, tx_id;
uint64_t flags = I40E_TD_CMD;
uint64_t rs = I40E_TX_DESC_CMD_RS | I40E_TD_CMD;
if (txq->nb_tx_free < txq->tx_free_thresh)
- i40e_tx_free_bufs(txq);
+ ci_tx_free_bufs_vec(txq, i40e_tx_desc_done, false);
nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
if (unlikely(nb_pkts == 0))
@@ -759,13 +759,13 @@ i40e_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
tx_id = txq->tx_tail;
txdp = &txq->i40e_tx_ring[tx_id];
- txep = &txq->sw_ring[tx_id];
+ txep = &txq->sw_ring_vec[tx_id];
txq->nb_tx_free = (uint16_t
[PATCH v4 16/24] net/ixgbe: use common Tx queue mbuf cleanup fn
Update driver to use the common cleanup function.
Signed-off-by: Bruce Richardson
---
drivers/net/ixgbe/ixgbe_rxtx.c| 22 +++---
drivers/net/ixgbe/ixgbe_rxtx.h| 1 -
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 28 ++-
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 7 --
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c| 7 --
5 files changed, 5 insertions(+), 60 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
index 344ef85685..bf9d461b06 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx.c
@@ -2334,21 +2334,6 @@ ixgbe_recv_pkts_lro_bulk_alloc(void *rx_queue, struct
rte_mbuf **rx_pkts,
*
**/
-static void __rte_cold
-ixgbe_tx_queue_release_mbufs(struct ci_tx_queue *txq)
-{
- unsigned i;
-
- if (txq->sw_ring != NULL) {
- for (i = 0; i < txq->nb_tx_desc; i++) {
- if (txq->sw_ring[i].mbuf != NULL) {
- rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
- txq->sw_ring[i].mbuf = NULL;
- }
- }
- }
-}
-
static int
ixgbe_tx_done_cleanup_full(struct ci_tx_queue *txq, uint32_t free_cnt)
{
@@ -2472,7 +2457,7 @@ static void __rte_cold
ixgbe_tx_queue_release(struct ci_tx_queue *txq)
{
if (txq != NULL && txq->ops != NULL) {
- txq->ops->release_mbufs(txq);
+ ci_txq_release_all_mbufs(txq);
txq->ops->free_swring(txq);
rte_memzone_free(txq->mz);
rte_free(txq);
@@ -2526,7 +2511,6 @@ ixgbe_reset_tx_queue(struct ci_tx_queue *txq)
}
static const struct ixgbe_txq_ops def_txq_ops = {
- .release_mbufs = ixgbe_tx_queue_release_mbufs,
.free_swring = ixgbe_tx_free_swring,
.reset = ixgbe_reset_tx_queue,
};
@@ -3380,7 +3364,7 @@ ixgbe_dev_clear_queues(struct rte_eth_dev *dev)
struct ci_tx_queue *txq = dev->data->tx_queues[i];
if (txq != NULL) {
- txq->ops->release_mbufs(txq);
+ ci_txq_release_all_mbufs(txq);
txq->ops->reset(txq);
dev->data->tx_queue_state[i] =
RTE_ETH_QUEUE_STATE_STOPPED;
}
@@ -5655,7 +5639,7 @@ ixgbe_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
}
if (txq->ops != NULL) {
- txq->ops->release_mbufs(txq);
+ ci_txq_release_all_mbufs(txq);
txq->ops->reset(txq);
}
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.h b/drivers/net/ixgbe/ixgbe_rxtx.h
index 4333e5bf2f..11689eb432 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.h
+++ b/drivers/net/ixgbe/ixgbe_rxtx.h
@@ -181,7 +181,6 @@ struct ixgbe_advctx_info {
};
struct ixgbe_txq_ops {
- void (*release_mbufs)(struct ci_tx_queue *txq);
void (*free_swring)(struct ci_tx_queue *txq);
void (*reset)(struct ci_tx_queue *txq);
};
diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
index 06e760867c..2b12bdcc9c 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
+++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
@@ -78,32 +78,6 @@ tx_backlog_entry(struct ci_tx_entry_vec *txep,
txep[i].mbuf = tx_pkts[i];
}
-static inline void
-_ixgbe_tx_queue_release_mbufs_vec(struct ci_tx_queue *txq)
-{
- unsigned int i;
- struct ci_tx_entry_vec *txe;
- const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
-
- if (txq->sw_ring == NULL || txq->nb_tx_free == max_desc)
- return;
-
- /* release the used mbufs in sw_ring */
- for (i = txq->tx_next_dd - (txq->tx_rs_thresh - 1);
-i != txq->tx_tail;
-i = (i + 1) % txq->nb_tx_desc) {
- txe = &txq->sw_ring_vec[i];
- rte_pktmbuf_free_seg(txe->mbuf);
- }
- txq->nb_tx_free = max_desc;
-
- /* reset tx_entry */
- for (i = 0; i < txq->nb_tx_desc; i++) {
- txe = &txq->sw_ring_vec[i];
- txe->mbuf = NULL;
- }
-}
-
static inline void
_ixgbe_rx_queue_release_mbufs_vec(struct ixgbe_rx_queue *rxq)
{
@@ -207,6 +181,8 @@ ixgbe_txq_vec_setup_default(struct ci_tx_queue *txq,
/* leave the first one for overflow */
txq->sw_ring_vec = txq->sw_ring_vec + 1;
txq->ops = txq_ops;
+ txq->vector_tx = 1;
+ txq->vector_sw_ring = 1;
return 0;
}
diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
index cb749a3760..2ccb399b64 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
@@ -633,12 +633,6 @@ ixgbe_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
[PATCH v4 17/24] net/iavf: use common Tx queue mbuf cleanup fn
Adjust iavf driver to also use the common mbuf freeing functions on Tx
queue release/cleanup. The implementation is complicated a little by the
need to integrate the additional "has_ctx" parameter for the iavf code,
but changes in other drivers are minimal - just a constant "false"
parameter.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h | 27 +-
drivers/net/i40e/i40e_rxtx.c| 6 ++--
drivers/net/iavf/iavf_rxtx.c| 37 ++---
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 24 ++--
drivers/net/iavf/iavf_rxtx_vec_common.h | 18
drivers/net/iavf/iavf_rxtx_vec_sse.c| 9 ++
drivers/net/ice/ice_dcf_ethdev.c| 4 +--
drivers/net/ice/ice_rxtx.c | 6 ++--
drivers/net/ixgbe/ixgbe_rxtx.c | 6 ++--
9 files changed, 31 insertions(+), 106 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index 1bf2a61b2f..310b51adcf 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -271,23 +271,23 @@ ci_tx_free_bufs_vec(struct ci_tx_queue *txq,
ci_desc_done_fn desc_done, bool ctx
return txq->tx_rs_thresh;
}
-#define IETH_FREE_BUFS_LOOP(txq, swr, start) do { \
+#define IETH_FREE_BUFS_LOOP(swr, nb_desc, start, end) do { \
uint16_t i = start; \
- if (txq->tx_tail < i) { \
- for (; i < txq->nb_tx_desc; i++) { \
+ if (end < i) { \
+ for (; i < nb_desc; i++) { \
rte_pktmbuf_free_seg(swr[i].mbuf); \
swr[i].mbuf = NULL; \
} \
i = 0; \
} \
- for (; i < txq->tx_tail; i++) { \
+ for (; i < end; i++) { \
rte_pktmbuf_free_seg(swr[i].mbuf); \
swr[i].mbuf = NULL; \
} \
} while (0)
static inline void
-ci_txq_release_all_mbufs(struct ci_tx_queue *txq)
+ci_txq_release_all_mbufs(struct ci_tx_queue *txq, bool use_ctx)
{
if (unlikely(!txq || !txq->sw_ring))
return;
@@ -306,15 +306,14 @@ ci_txq_release_all_mbufs(struct ci_tx_queue *txq)
* vPMD tx will not set sw_ring's mbuf to NULL after free,
* so need to free remains more carefully.
*/
- const uint16_t start = txq->tx_next_dd - txq->tx_rs_thresh + 1;
-
- if (txq->vector_sw_ring) {
- struct ci_tx_entry_vec *swr = txq->sw_ring_vec;
- IETH_FREE_BUFS_LOOP(txq, swr, start);
- } else {
- struct ci_tx_entry *swr = txq->sw_ring;
- IETH_FREE_BUFS_LOOP(txq, swr, start);
- }
+ const uint16_t start = (txq->tx_next_dd - txq->tx_rs_thresh + 1) >>
use_ctx;
+ const uint16_t nb_desc = txq->nb_tx_desc >> use_ctx;
+ const uint16_t end = txq->tx_tail >> use_ctx;
+
+ if (txq->vector_sw_ring)
+ IETH_FREE_BUFS_LOOP(txq->sw_ring_vec, nb_desc, start, end);
+ else
+ IETH_FREE_BUFS_LOOP(txq->sw_ring, nb_desc, start, end);
}
#endif /* _COMMON_INTEL_TX_H_ */
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index b70919c5dc..081d743e62 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -1933,7 +1933,7 @@ i40e_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t
tx_queue_id)
return err;
}
- ci_txq_release_all_mbufs(txq);
+ ci_txq_release_all_mbufs(txq, false);
i40e_reset_tx_queue(txq);
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
@@ -2608,7 +2608,7 @@ i40e_tx_queue_release(void *txq)
return;
}
- ci_txq_release_all_mbufs(q);
+ ci_txq_release_all_mbufs(q, false);
rte_free(q->sw_ring);
rte_memzone_free(q->mz);
rte_free(q);
@@ -3071,7 +3071,7 @@ i40e_dev_clear_queues(struct rte_eth_dev *dev)
for (i = 0; i < dev->data->nb_tx_queues; i++) {
if (!dev->data->tx_queues[i])
continue;
- ci_txq_release_all_mbufs(dev->data->tx_queues[i]);
+ ci_txq_release_all_mbufs(dev->data->tx_queues[i], false);
i40e_reset_tx_queue(dev->data->tx_queues[i]);
}
diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c
index 7e381b2a17..f0ab881ac5 100644
--- a/drivers/net/iavf/iavf_rxtx.c
+++ b/drivers/net/iavf/iavf_rxtx.c
@@ -387,24 +387,6 @@ release_rxq_mbufs(struct iavf_rx_queue *rxq)
rxq->rx_nb_avail = 0;
}
-static inline void
-release_txq_mbufs(struct ci_tx_queue *txq)
-{
- uint16_t i;
-
- if (!txq || !txq->sw_ring) {
- PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
- return;
- }
-
- for (i = 0; i < txq->nb_tx_desc; i++) {
- if (txq->sw
[PATCH v4 20/24] net/iavf: use vector SW ring for all vector paths
The AVX-512 code path used a smaller SW ring structure only containing
the mbuf pointer, but no other fields. The other fields are only used in
the scalar code path, so update all vector driver code paths (AVX2, SSE)
to use the smaller, faster structure.
Signed-off-by: Bruce Richardson
---
drivers/net/iavf/iavf_rxtx.c| 7 ---
drivers/net/iavf/iavf_rxtx_vec_avx2.c | 12 ++--
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 8
drivers/net/iavf/iavf_rxtx_vec_common.h | 6 --
drivers/net/iavf/iavf_rxtx_vec_sse.c| 14 +++---
5 files changed, 13 insertions(+), 34 deletions(-)
diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c
index f0ab881ac5..6692f6992b 100644
--- a/drivers/net/iavf/iavf_rxtx.c
+++ b/drivers/net/iavf/iavf_rxtx.c
@@ -4193,14 +4193,7 @@ iavf_set_tx_function(struct rte_eth_dev *dev)
txq = dev->data->tx_queues[i];
if (!txq)
continue;
-#ifdef CC_AVX512_SUPPORT
- if (use_avx512)
- iavf_txq_vec_setup_avx512(txq);
- else
- iavf_txq_vec_setup(txq);
-#else
iavf_txq_vec_setup(txq);
-#endif
}
if (no_poll_on_link_down) {
diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx2.c
b/drivers/net/iavf/iavf_rxtx_vec_avx2.c
index fdb98b417a..b847886081 100644
--- a/drivers/net/iavf/iavf_rxtx_vec_avx2.c
+++ b/drivers/net/iavf/iavf_rxtx_vec_avx2.c
@@ -1736,14 +1736,14 @@ iavf_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
{
struct ci_tx_queue *txq = (struct ci_tx_queue *)tx_queue;
volatile struct iavf_tx_desc *txdp;
- struct ci_tx_entry *txep;
+ struct ci_tx_entry_vec *txep;
uint16_t n, nb_commit, tx_id;
/* bit2 is reserved and must be set to 1 according to Spec */
uint64_t flags = IAVF_TX_DESC_CMD_EOP | IAVF_TX_DESC_CMD_ICRC;
uint64_t rs = IAVF_TX_DESC_CMD_RS | flags;
if (txq->nb_tx_free < txq->tx_free_thresh)
- iavf_tx_free_bufs(txq);
+ ci_tx_free_bufs_vec(txq, iavf_tx_desc_done, false);
nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
if (unlikely(nb_pkts == 0))
@@ -1752,13 +1752,13 @@ iavf_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
tx_id = txq->tx_tail;
txdp = &txq->iavf_tx_ring[tx_id];
- txep = &txq->sw_ring[tx_id];
+ txep = &txq->sw_ring_vec[tx_id];
txq->nb_tx_free = (uint16_t)(txq->nb_tx_free - nb_pkts);
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- ci_tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, n);
iavf_vtx(txdp, tx_pkts, n - 1, flags, offload);
tx_pkts += (n - 1);
@@ -1773,10 +1773,10 @@ iavf_xmit_fixed_burst_vec_avx2(void *tx_queue, struct
rte_mbuf **tx_pkts,
/* avoid reach the end of ring */
txdp = &txq->iavf_tx_ring[tx_id];
- txep = &txq->sw_ring[tx_id];
+ txep = &txq->sw_ring_vec[tx_id];
}
- ci_tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, nb_commit);
iavf_vtx(txdp, tx_pkts, nb_commit, flags, offload);
diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx512.c
b/drivers/net/iavf/iavf_rxtx_vec_avx512.c
index 007759e451..641f3311eb 100644
--- a/drivers/net/iavf/iavf_rxtx_vec_avx512.c
+++ b/drivers/net/iavf/iavf_rxtx_vec_avx512.c
@@ -2357,14 +2357,6 @@ iavf_xmit_pkts_vec_avx512(void *tx_queue, struct
rte_mbuf **tx_pkts,
return iavf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts, false);
}
-int __rte_cold
-iavf_txq_vec_setup_avx512(struct ci_tx_queue *txq)
-{
- txq->vector_tx = true;
- txq->vector_sw_ring = true;
- return 0;
-}
-
uint16_t
iavf_xmit_pkts_vec_avx512_offload(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
diff --git a/drivers/net/iavf/iavf_rxtx_vec_common.h
b/drivers/net/iavf/iavf_rxtx_vec_common.h
index 6f94587eee..c69399a173 100644
--- a/drivers/net/iavf/iavf_rxtx_vec_common.h
+++ b/drivers/net/iavf/iavf_rxtx_vec_common.h
@@ -24,12 +24,6 @@ iavf_tx_desc_done(struct ci_tx_queue *txq, uint16_t idx)
rte_cpu_to_le_64(IAVF_TX_DESC_DTYPE_DESC_DONE);
}
-static __rte_always_inline int
-iavf_tx_free_bufs(struct ci_tx_queue *txq)
-{
- return ci_tx_free_bufs(txq, iavf_tx_desc_done);
-}
-
static inline void
_iavf_rx_queue_release_mbufs_vec(struct iavf_rx_queue *rxq)
{
diff --git a/drivers/net/iavf/iavf_rxtx_vec_sse.c
b/drivers/net/iavf/iavf_rxtx_vec_sse.c
index 3adf2a59e4..9f7db80bfd 100644
--- a/drivers/net/iavf/iavf_rxtx_vec_sse.c
+++ b/drivers/net/iavf/iavf_rxtx_vec_sse.c
@@ -1368,14 +1368,14 @@ iavf_xm
[PATCH v4 23/24] net/_common_intel: create common mbuf initializer fn
Across a number of drivers, the same code is used for initializing the
"mbuf_initializer" value inside the rx queue structure for use with the
vector drivers. Since the rx queue structures are (currently) different
across the drivers, we cannot just move a single copy of the function to
a common location. Instead, we create a dedicated function which just
creates the mbuf initializer for a particular port.
In creating this function, we can shorten it vs the original versions by
initializing the mbuf fields as they are defined, rather than
afterwards. We can also remove the use of the barrier and temporary
uintptr_t variable, because the mbuf has been reworked so that
rearm_data is a proper single-element array in a union.
Across ixgbe, i40e, iavf and i40e, we can call this function to
initialize the rxq data, replacing the "*_rxq_vec_setup_default"
functions. Only the i40e was slightly different, having an extra
assignment in it, to set the "sse" flag (even in case of neon and
altivec paths). This assignment was just duplicated to the calling sites
for simplicity and to keep existing behaviour.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/rx.h| 13 +
drivers/net/i40e/i40e_rxtx_vec_altivec.c | 4 +++-
drivers/net/i40e/i40e_rxtx_vec_common.h | 19 ---
drivers/net/i40e/i40e_rxtx_vec_neon.c | 4 +++-
drivers/net/i40e/i40e_rxtx_vec_sse.c | 4 +++-
drivers/net/iavf/iavf_rxtx_vec_common.h | 18 --
drivers/net/iavf/iavf_rxtx_vec_neon.c | 3 ++-
drivers/net/iavf/iavf_rxtx_vec_sse.c | 3 ++-
drivers/net/ice/ice_rxtx_vec_common.h | 18 --
drivers/net/ice/ice_rxtx_vec_sse.c| 3 ++-
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 18 --
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 3 ++-
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c| 3 ++-
13 files changed, 32 insertions(+), 81 deletions(-)
diff --git a/drivers/net/_common_intel/rx.h b/drivers/net/_common_intel/rx.h
index 5bd2fea7e3..ca0485875c 100644
--- a/drivers/net/_common_intel/rx.h
+++ b/drivers/net/_common_intel/rx.h
@@ -76,4 +76,17 @@ ci_rx_reassemble_packets(struct rte_mbuf **rx_bufs, uint16_t
nb_bufs, uint8_t *s
return pkt_idx;
}
+static inline uint64_t
+ci_rxq_mbuf_initializer(uint16_t port_id)
+{
+ struct rte_mbuf mb_def = {
+ .nb_segs = 1,
+ .data_off = RTE_PKTMBUF_HEADROOM,
+ .port = port_id,
+ };
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ return mb_def.rearm_data[0];
+}
+
#endif /* _COMMON_INTEL_RX_H_ */
diff --git a/drivers/net/i40e/i40e_rxtx_vec_altivec.c
b/drivers/net/i40e/i40e_rxtx_vec_altivec.c
index b6900a3e15..e8046b5ce5 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_altivec.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_altivec.c
@@ -621,7 +621,9 @@ i40e_rx_queue_release_mbufs_vec(struct i40e_rx_queue *rxq)
int __rte_cold
i40e_rxq_vec_setup(struct i40e_rx_queue *rxq)
{
- return i40e_rxq_vec_setup_default(rxq);
+ rxq->rx_using_sse = 1;
+ rxq->mbuf_initializer = ci_rxq_mbuf_initializer(rxq->port_id);
+ return 0;
}
int __rte_cold
diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h
b/drivers/net/i40e/i40e_rxtx_vec_common.h
index 733dc797cd..1ccdbd3fdb 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_common.h
+++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
@@ -54,25 +54,6 @@ _i40e_rx_queue_release_mbufs_vec(struct i40e_rx_queue *rxq)
memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
}
-static inline int
-i40e_rxq_vec_setup_default(struct i40e_rx_queue *rxq)
-{
- uintptr_t p;
- struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
-
- mb_def.nb_segs = 1;
- mb_def.data_off = RTE_PKTMBUF_HEADROOM;
- mb_def.port = rxq->port_id;
- rte_mbuf_refcnt_set(&mb_def, 1);
-
- /* prevent compiler reordering: rearm_data covers previous fields */
- rte_compiler_barrier();
- p = (uintptr_t)&mb_def.rearm_data;
- rxq->mbuf_initializer = *(uint64_t *)p;
- rxq->rx_using_sse = 1;
- return 0;
-}
-
static inline int
i40e_rx_vec_dev_conf_condition_check_default(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/i40e/i40e_rxtx_vec_neon.c
b/drivers/net/i40e/i40e_rxtx_vec_neon.c
index b398d66154..1c7e9bf1fa 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_neon.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_neon.c
@@ -749,7 +749,9 @@ i40e_rx_queue_release_mbufs_vec(struct i40e_rx_queue *rxq)
int __rte_cold
i40e_rxq_vec_setup(struct i40e_rx_queue *rxq)
{
- return i40e_rxq_vec_setup_default(rxq);
+ rxq->rx_using_sse = 1;
+ rxq->mbuf_initializer = ci_rxq_mbuf_initializer(rxq->port_id);
+ return 0;
}
int __rte_cold
diff --git a/drivers/net/i40e/i40e_rxtx_vec_sse.c
b/drivers/net/i40e/i40e_rxtx_vec_sse.c
index 90c57e59d0..42255a20af 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_sse.c
+++ b/drivers/net/i40e/i40e_rxtx_vec_sse.
[PATCH v4 21/24] net/_common_intel: remove unneeded code
With all drivers using the common Tx structure updated so that their
vector paths all use the simplified Tx mbuf ring format, it's no longer
necessary to have a separate flag for the ring format and for use of a
vector driver.
Remove the former flag and base all decisions off the vector flag. With
that done, we go from having only two paths to consider for releasing
all mbufs in the ring, not three. That allows further simplification of
the "ci_txq_release_all_mbufs" function.
The separate function to free buffers from the vector driver not using
the simplified ring format can similarly be removed as no longer
necessary.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/tx.h| 97 +++
drivers/net/i40e/i40e_rxtx.c | 1 -
drivers/net/iavf/iavf_rxtx_vec_sse.c | 1 -
drivers/net/ice/ice_rxtx.c| 1 -
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 1 -
5 files changed, 10 insertions(+), 91 deletions(-)
diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h
index aa42b9b49f..d9cf4474fc 100644
--- a/drivers/net/_common_intel/tx.h
+++ b/drivers/net/_common_intel/tx.h
@@ -66,7 +66,6 @@ struct ci_tx_queue {
bool tx_deferred_start; /* don't start this queue in dev start */
bool q_set; /* indicate if tx queue has been configured */
bool vector_tx; /* port is using vector TX */
- bool vector_sw_ring;/* port is using vectorized SW ring
(ieth_tx_entry_vec) */
union { /* the VSI this queue belongs to */
struct i40e_vsi *i40e_vsi;
struct iavf_vsi *iavf_vsi;
@@ -120,72 +119,6 @@ ci_tx_backlog_entry_vec(struct ci_tx_entry_vec *txep,
struct rte_mbuf **tx_pkts,
typedef int (*ci_desc_done_fn)(struct ci_tx_queue *txq, uint16_t idx);
-static __rte_always_inline int
-ci_tx_free_bufs(struct ci_tx_queue *txq, ci_desc_done_fn desc_done)
-{
- struct ci_tx_entry *txep;
- uint32_t n;
- uint32_t i;
- int nb_free = 0;
- struct rte_mbuf *m, *free[IETH_VPMD_TX_MAX_FREE_BUF];
-
- /* check DD bits on threshold descriptor */
- if (!desc_done(txq, txq->tx_next_dd))
- return 0;
-
- n = txq->tx_rs_thresh;
-
-/* first buffer to free from S/W ring is at index
- * tx_next_dd - (tx_rs_thresh-1)
- */
- txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)];
-
- if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {
- for (i = 0; i < n; i++) {
- free[i] = txep[i].mbuf;
- /* no need to reset txep[i].mbuf in vector path */
- }
- rte_mempool_put_bulk(free[0]->pool, (void **)free, n);
- goto done;
- }
-
- m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
- if (likely(m != NULL)) {
- free[0] = m;
- nb_free = 1;
- for (i = 1; i < n; i++) {
- m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
- if (likely(m != NULL)) {
- if (likely(m->pool == free[0]->pool)) {
- free[nb_free++] = m;
- } else {
- rte_mempool_put_bulk(free[0]->pool,
-(void *)free,
-nb_free);
- free[0] = m;
- nb_free = 1;
- }
- }
- }
- rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
- } else {
- for (i = 1; i < n; i++) {
- m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
- if (m != NULL)
- rte_mempool_put(m->pool, m);
- }
- }
-
-done:
- /* buffers were freed, update counters */
- txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq->tx_rs_thresh);
- txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq->tx_rs_thresh);
- if (txq->tx_next_dd >= txq->nb_tx_desc)
- txq->tx_next_dd = (uint16_t)(txq->tx_rs_thresh - 1);
-
- return txq->tx_rs_thresh;
-}
-
static __rte_always_inline int
ci_tx_free_bufs_vec(struct ci_tx_queue *txq, ci_desc_done_fn desc_done, bool
ctx_descs)
{
@@ -278,21 +211,6 @@ ci_tx_free_bufs_vec(struct ci_tx_queue *txq,
ci_desc_done_fn desc_done, bool ctx
return txq->tx_rs_thresh;
}
-#define IETH_FREE_BUFS_LOOP(swr, nb_desc, start, end) do { \
- uint16_t i = start; \
- if (end < i) { \
- for (; i < nb_desc; i++) { \
- rte_pktmbuf_free_seg(swr[i].mbuf); \
- swr[i].mbuf = NULL; \
- } \
[PATCH v4 22/24] net/ixgbe: use common Tx backlog entry fn
Remove the custom vector Tx backlog entry function and use the standard
intel_common one, now that all vector drivers are using the same,
smaller ring structure.
Signed-off-by: Bruce Richardson
---
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 10 --
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 4 ++--
drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c| 4 ++--
3 files changed, 4 insertions(+), 14 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
index 53d1fed6f8..9c3752a12a 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
+++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
@@ -68,16 +68,6 @@ ixgbe_tx_free_bufs(struct ci_tx_queue *txq)
return txq->tx_rs_thresh;
}
-static __rte_always_inline void
-tx_backlog_entry(struct ci_tx_entry_vec *txep,
-struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
-{
- int i;
-
- for (i = 0; i < (int)nb_pkts; ++i)
- txep[i].mbuf = tx_pkts[i];
-}
-
static inline void
_ixgbe_rx_queue_release_mbufs_vec(struct ixgbe_rx_queue *rxq)
{
diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
index 2ccb399b64..f879f6fa9a 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
@@ -597,7 +597,7 @@ ixgbe_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, n);
for (i = 0; i < n - 1; ++i, ++tx_pkts, ++txdp)
vtx1(txdp, *tx_pkts, flags);
@@ -614,7 +614,7 @@ ixgbe_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
txep = &txq->sw_ring_vec[tx_id];
}
- tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, nb_commit);
vtx(txdp, tx_pkts, nb_commit, flags);
diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c
b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c
index fa26365f06..915358e16b 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c
@@ -720,7 +720,7 @@ ixgbe_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
n = (uint16_t)(txq->nb_tx_desc - tx_id);
if (nb_commit >= n) {
- tx_backlog_entry(txep, tx_pkts, n);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, n);
for (i = 0; i < n - 1; ++i, ++tx_pkts, ++txdp)
vtx1(txdp, *tx_pkts, flags);
@@ -737,7 +737,7 @@ ixgbe_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf
**tx_pkts,
txep = &txq->sw_ring_vec[tx_id];
}
- tx_backlog_entry(txep, tx_pkts, nb_commit);
+ ci_tx_backlog_entry_vec(txep, tx_pkts, nb_commit);
vtx(txdp, tx_pkts, nb_commit, flags);
--
2.43.0
[PATCH v4 24/24] net/_common_intel: extract common Rx vector criteria
While some drivers have specific criteria for when a vector driver can
be enabled on the Rx path, there are a number of basic criteria which
apply across all drivers. Centralize those in the _common_intel folder,
and then update drivers to use the common conditional checks. This adds
some additional restrictions to some drivers like ixgbe, where those
conditions were necessary but never checked.
Signed-off-by: Bruce Richardson
---
drivers/net/_common_intel/rx.h| 20 +
drivers/net/i40e/i40e_rxtx_vec_common.h | 35 +--
drivers/net/iavf/iavf_rxtx.c | 15 +-
drivers/net/iavf/iavf_rxtx.h | 1 +
drivers/net/ice/ice_rxtx_vec_common.h | 14 +
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 7 +
6 files changed, 37 insertions(+), 55 deletions(-)
diff --git a/drivers/net/_common_intel/rx.h b/drivers/net/_common_intel/rx.h
index ca0485875c..abb01ba5e7 100644
--- a/drivers/net/_common_intel/rx.h
+++ b/drivers/net/_common_intel/rx.h
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#define CI_RX_BURST 32
@@ -89,4 +90,23 @@ ci_rxq_mbuf_initializer(uint16_t port_id)
return mb_def.rearm_data[0];
}
+/* basic checks for a vector-driver capable queue.
+ * Individual drivers may have other further tests beyond this.
+ */
+static inline bool
+ci_rxq_vec_capable(uint16_t nb_desc, uint16_t rx_free_thresh, uint64_t
offloads)
+{
+ if (!rte_is_power_of_2(nb_desc) ||
+ rx_free_thresh < CI_RX_BURST ||
+ (nb_desc % rx_free_thresh) != 0)
+ return false;
+
+ /* no driver supports timestamping or buffer split on vector path */
+ if ((offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) ||
+ (offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
+ return false;
+
+ return true;
+}
+
#endif /* _COMMON_INTEL_RX_H_ */
diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h
b/drivers/net/i40e/i40e_rxtx_vec_common.h
index 1ccdbd3fdb..5d0b777e0d 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_common.h
+++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
@@ -61,9 +61,6 @@ i40e_rx_vec_dev_conf_condition_check_default(struct
rte_eth_dev *dev)
struct i40e_adapter *ad =
I40E_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_rxmode *rxmode = &dev->data->dev_conf.rxmode;
- struct i40e_rx_queue *rxq;
- uint16_t desc, i;
- bool first_queue;
/* no QinQ support */
if (rxmode->offloads & RTE_ETH_RX_OFFLOAD_VLAN_EXTEND)
@@ -73,31 +70,13 @@ i40e_rx_vec_dev_conf_condition_check_default(struct
rte_eth_dev *dev)
* Vector mode is allowed only when number of Rx queue
* descriptor is power of 2.
*/
- if (!dev->data->dev_started) {
- first_queue = true;
- for (i = 0; i < dev->data->nb_rx_queues; i++) {
- rxq = dev->data->rx_queues[i];
- if (!rxq)
- continue;
- desc = rxq->nb_rx_desc;
- if (first_queue)
- ad->rx_vec_allowed =
- rte_is_power_of_2(desc);
- else
- ad->rx_vec_allowed =
- ad->rx_vec_allowed ?
- rte_is_power_of_2(desc) :
- ad->rx_vec_allowed;
- first_queue = false;
- }
- } else {
- /* Only check the first queue's descriptor number */
- for (i = 0; i < dev->data->nb_rx_queues; i++) {
- rxq = dev->data->rx_queues[i];
- if (!rxq)
- continue;
- desc = rxq->nb_rx_desc;
- ad->rx_vec_allowed = rte_is_power_of_2(desc);
+ ad->rx_vec_allowed = true;
+ for (uint16_t i = 0; i < dev->data->nb_rx_queues; i++) {
+ struct i40e_rx_queue *rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+ if (!ci_rxq_vec_capable(rxq->nb_rx_desc, rxq->rx_free_thresh,
rxq->offloads)) {
+ ad->rx_vec_allowed = false;
break;
}
}
diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c
index 6692f6992b..e4c4b9682c 100644
--- a/drivers/net/iavf/iavf_rxtx.c
+++ b/drivers/net/iavf/iavf_rxtx.c
@@ -199,19 +199,6 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
-static inline bool
-check_rx_vec_allow(struct iavf_rx_queue *rxq)
-{
- if (rxq->rx_free_thresh >= IAVF_VPMD_RX_MAX_BURST &&
- rxq->nb_rx_desc % rxq->rx_free_thresh == 0) {
- PMD_INIT_LOG(DEBUG, "Vector Rx can be enabled on this rxq.");
-
[PATCH] net/vhost: improve devargs documentation
This patch adds missing 'client' devarg documentation and improve other devargs. Bugzilla ID: 1603 Signed-off-by: Maxime Coquelin --- doc/guides/nics/vhost.rst | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/vhost.rst b/doc/guides/nics/vhost.rst index 39c6fcff58..0013dc526f 100644 --- a/doc/guides/nics/vhost.rst +++ b/doc/guides/nics/vhost.rst @@ -32,13 +32,18 @@ The user can specify below arguments in `--vdev` option. #. ``iface``: -It is used to specify a path to connect to a QEMU virtio-net device. +It is used to specify a path to connect to a Vhost-user frontend or VDUSE device. #. ``queues``: -It is used to specify the number of queues virtio-net device has. +It is used to specify the number of queue pairs virtio-net device has. (Default: 1) +#. ``client``: + + It is used to specify whether the Vhost PMD acts as client or server. + (Default: 0 (server)) + #. ``iommu-support``: It is used to enable iommu support in vhost library. -- 2.47.0
