[dpdk-dev] [dpdk-announce] DPDK 16.11 released
Thomas, it's great to hear that DPDK16.11 released just in time. As prospect for 17.02, our intel validation team have some concern about the release date. The official day off for Chinese Sprint Festival holiday will be from 27th Jan to 3th Feb. Most of our members may ask for more days leave either before or after the official day off. >From our previous experience, it will take 3~4 weeks to do the full function >and performance test. If the first candidate release in the middle of Jan, we can do first round of validation and raise issues to developers. And after the holiday, we can keep on the validation process and finish in two weeks. If release date is after Feb, it will be hard for us to cover all cases in release window. Thanks, DPDK STV team > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Thomas Monjalon > Sent: Monday, November 14, 2016 6:23 AM > To: announce at dpdk.org > Subject: [dpdk-dev] [dpdk-announce] DPDK 16.11 released > > The new features for the 17.02 cycle must be submitted before December 4. > There is a long list of expected works: > http://dpdk.org/dev/roadmap > It means we will have a huge workload to properly review all the new stuff > before the end of the year. Do not forget to help reviewing patches from > others if we want to have a chance to integrate everything in time. > > > Thanks everyone
[dpdk-dev] acl: delete/modify rule support
Hi, I had a couple of questions about ACL support in DPDK. 1) Is it possible to build the ACL context over a period of time, one rule at a time by calling build post each add operation? Something like this : rte_acl_add_rules(ctx, rule1, 1); rte_acl_build(ctx, build_cfg); rte_acl_add_rules(ctx, rule2, 1); rte_acl_build(ctx, build_cfg); rte_acl_add_rules(ctx, rule3, 1); rte_acl_build(ctx, build_cfg); I tried this, and it seems to be working - but still wanted to confirm. 2) I did not find any delete-rule API. I understand from one of the email archives (http://dpdk.org/ml/archives/dev/2015-June/018868.html) that it is not supported. Any specific reason why deletion of rule is not supported? Any plan to support it? Any alternative way other than destroying the whole context, adding the new set of rules and building the rules again? Regards, Nikhil
[dpdk-dev] acl: delete/modify rule support
On Tue, Nov 15, 2016 at 10:40 AM, Nikhil Jagtap wrote: > Is it possible to build the ACL context over a period of time, one rule > at a time by calling build post each add operation? > Something like this : > rte_acl_add_rules(ctx, rule1, 1); > rte_acl_build(ctx, build_cfg); > rte_acl_add_rules(ctx, rule2, 1); > rte_acl_build(ctx, build_cfg); > rte_acl_add_rules(ctx, rule3, 1); > rte_acl_build(ctx, build_cfg); > > I tried this, and it seems to be working - but still wanted to confirm. > ?yes this is fine. but building the querying data-structure is not thread safe. so more often than not, you are better of having a rule-list, and then building it all in one fell swoop... ? > > 2) I did not find any delete-rule API. I understand from one of the email > archives (http://dpdk.org/ml/archives/dev/2015-June/018868.html) that it > is > not supported. Any specific reason why deletion of rule is not supported? > Any plan to support it? > Any alternative way other than destroying the whole context, adding the new > set of rules and building the rules again? > current implementation uses multibit-trie?s. to conserve memory, rules are split into multiple non-overlapping/non-intersecting rules. i would wager that implementing deletion would be non-trivial :) better option might be to add/modify/delete your rule-list, and rebuild, and then re-use... ?-- kind regards anupam? In the beginning was the lambda, and the lambda was with Emacs, and Emacs was the lambda.
[dpdk-dev] [PATCH] lib/librte_mempool: a redundant word in comment
Hi, john
> -Original Message-
> From: Mcnamara, John
> Sent: Monday, November 14, 2016 6:30 PM
> To: Zhao1, Wei ; dev at dpdk.org
> Cc: olivier.matz at 6wind.com; Zhao1, Wei
> Subject: RE: [dpdk-dev] [PATCH] lib/librte_mempool: a redundant word in
> comment
>
>
>
> > -Original Message-
> > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Wei Zhao
> > Sent: Monday, November 14, 2016 2:47 AM
> > To: dev at dpdk.org
> > Cc: olivier.matz at 6wind.com; Zhao1, Wei
> > Subject: [dpdk-dev] [PATCH] lib/librte_mempool: a redundant word in
> > comment
> >
> > From: zhao wei
>
> I think you need to add your name to gitconfig file on the sending machine to
> avoid this "From:"
>
> >
> > There is a redundant repetition word "for" in commnet line the file
> > rte_mempool.h after the definition of RTE_MEMPOOL_OPS_NAMESIZE.
> > The word "for"appear twice in line 359 and 360.One of them is
> > redundant, so delete it.
> >
> > Fixes: 449c49b93a6b ("lib/librte_mempool: mempool: support handler
> > operations")
> >
> > Signed-off-by: zhao wei
>
> /commnet/comment/
>
> And same comment as before about the title. Apart from that:
>
> Acked-by: John McNamara
>
>
Thank you for your suggestion, I will change as your comment in following
patch!
[dpdk-dev] [PATCH v4 0/3] Expanded statistics reporting
This patchset extends statistics reporting to include peak and average data-rate metrics. It comes in two parts: a statistics reporting library, and a bitrate calculation library that uses it. This structure is intended to seperate statistic reporting from ethdev and allow more flexible metric registration. -- v4 changes: * References to 16.11 changed to 17.02 * Fetching of non-port values was broken * Added sanity checks to value fetching * rte_stat_value renamed to rte_metric_value * Corrected doxygen descriptions * Added MAINTAINERS entries * Added #ifdef directives to bitrate code in test-pmd v3 changes: * Marked rte_stats_bitrate_s as internal * Minor integer roundoff correction * Coding style corrections * Removed spurious object allocation * Changes to rte_metrics.[ch] moved from Patch 2/3 to 1/3. * Reintroduced non-port values (RTE_METRICS_NONPORT) * Added spinlocks to metric library * Removed spurious test registration/update * Added release notes entries v2 changes: * Uses a new metrics library rather than being part of ethdev Remy Horton (3): lib: add information metrics library lib: add bitrate statistics library app/test-pmd: add support for bitrate statistics MAINTAINERS| 9 + app/test-pmd/testpmd.c | 36 +++ config/common_base | 10 + doc/api/doxy-api-index.md | 2 + doc/api/doxy-api.conf | 2 + doc/guides/rel_notes/release_17_02.rst | 11 + lib/Makefile | 2 + lib/librte_bitratestats/Makefile | 53 lib/librte_bitratestats/rte_bitrate.c | 128 + lib/librte_bitratestats/rte_bitrate.h | 80 ++ .../rte_bitratestats_version.map | 9 + lib/librte_metrics/Makefile| 51 lib/librte_metrics/rte_metrics.c | 308 + lib/librte_metrics/rte_metrics.h | 190 + lib/librte_metrics/rte_metrics_version.map | 13 + mk/rte.app.mk | 3 + 16 files changed, 907 insertions(+) create mode 100644 lib/librte_bitratestats/Makefile create mode 100644 lib/librte_bitratestats/rte_bitrate.c create mode 100644 lib/librte_bitratestats/rte_bitrate.h create mode 100644 lib/librte_bitratestats/rte_bitratestats_version.map create mode 100644 lib/librte_metrics/Makefile create mode 100644 lib/librte_metrics/rte_metrics.c create mode 100644 lib/librte_metrics/rte_metrics.h create mode 100644 lib/librte_metrics/rte_metrics_version.map -- 2.5.5
[dpdk-dev] [PATCH v4 1/3] lib: add information metrics library
This patch adds a new information metric library that allows other modules to register named metrics and update their values. It is intended to be independent of ethdev, rather than mixing ethdev and non-ethdev information in xstats. Signed-off-by: Remy Horton --- MAINTAINERS| 5 + config/common_base | 5 + doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf | 1 + doc/guides/rel_notes/release_17_02.rst | 6 + lib/Makefile | 1 + lib/librte_metrics/Makefile| 51 + lib/librte_metrics/rte_metrics.c | 308 + lib/librte_metrics/rte_metrics.h | 190 ++ lib/librte_metrics/rte_metrics_version.map | 13 ++ mk/rte.app.mk | 2 + 11 files changed, 583 insertions(+) create mode 100644 lib/librte_metrics/Makefile create mode 100644 lib/librte_metrics/rte_metrics.c create mode 100644 lib/librte_metrics/rte_metrics.h create mode 100644 lib/librte_metrics/rte_metrics_version.map diff --git a/MAINTAINERS b/MAINTAINERS index d6bb8f8..52bd8a9 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -595,6 +595,11 @@ F: lib/librte_jobstats/ F: examples/l2fwd-jobstats/ F: doc/guides/sample_app_ug/l2_forward_job_stats.rst +Metrics +M: Remy Horton +F: lib/librte_metrics/ +F: doc/guides/sample_app_ug/keep_alive.rst + Test Applications - diff --git a/config/common_base b/config/common_base index 4bff83a..dedc4c3 100644 --- a/config/common_base +++ b/config/common_base @@ -589,3 +589,8 @@ CONFIG_RTE_APP_TEST_RESOURCE_TAR=n CONFIG_RTE_TEST_PMD=y CONFIG_RTE_TEST_PMD_RECORD_CORE_CYCLES=n CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n + +# +# Compile the device metrics library +# +CONFIG_RTE_LIBRTE_METRICS=y diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 6675f96..ca50fa6 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -147,4 +147,5 @@ There are many libraries, so their headers may be grouped by topics: [common] (@ref rte_common.h), [ABI compat] (@ref rte_compat.h), [keepalive] (@ref rte_keepalive.h), + [Device Metrics] (@ref rte_metrics.h), [version](@ref rte_version.h) diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf index 9dc7ae5..fe830eb 100644 --- a/doc/api/doxy-api.conf +++ b/doc/api/doxy-api.conf @@ -57,6 +57,7 @@ INPUT = doc/api/doxy-api-index.md \ lib/librte_reorder \ lib/librte_ring \ lib/librte_sched \ + lib/librte_metrics \ lib/librte_table \ lib/librte_timer \ lib/librte_vhost diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst index 3b65038..e1b8894 100644 --- a/doc/guides/rel_notes/release_17_02.rst +++ b/doc/guides/rel_notes/release_17_02.rst @@ -34,6 +34,12 @@ New Features Refer to the previous release notes for examples. + * **Added information metric library.** + + A library that allows information metrics to be added and update. It is + intended to provide a reporting mechanism that is independent of the + ethdev library. + This section is a comment. do not overwrite or remove it. Also, make sure to start the actual text at the margin. = diff --git a/lib/Makefile b/lib/Makefile index 990f23a..5d85dcf 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -58,6 +58,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_TABLE) += librte_table DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += librte_pipeline DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump +DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile new file mode 100644 index 000..8d6e23a --- /dev/null +++ b/lib/librte_metrics/Makefile @@ -0,0 +1,51 @@ +# BSD LICENSE +# +# Copyright(c) 2016 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +#
[dpdk-dev] [PATCH v4 2/3] lib: add bitrate statistics library
This patch adds a library that calculates peak and average data-rate statistics. For ethernet devices. These statistics are reported using the metrics library. Signed-off-by: Remy Horton --- MAINTAINERS| 4 + config/common_base | 5 + doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf | 1 + doc/guides/rel_notes/release_17_02.rst | 5 + lib/Makefile | 1 + lib/librte_bitratestats/Makefile | 53 + lib/librte_bitratestats/rte_bitrate.c | 128 + lib/librte_bitratestats/rte_bitrate.h | 80 + .../rte_bitratestats_version.map | 9 ++ mk/rte.app.mk | 1 + 11 files changed, 288 insertions(+) create mode 100644 lib/librte_bitratestats/Makefile create mode 100644 lib/librte_bitratestats/rte_bitrate.c create mode 100644 lib/librte_bitratestats/rte_bitrate.h create mode 100644 lib/librte_bitratestats/rte_bitratestats_version.map diff --git a/MAINTAINERS b/MAINTAINERS index 52bd8a9..d6bbdd5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -600,6 +600,10 @@ M: Remy Horton F: lib/librte_metrics/ F: doc/guides/sample_app_ug/keep_alive.rst +Bit-rate statistica +M: Remy Horton +F: lib/librte_bitratestats/ + Test Applications - diff --git a/config/common_base b/config/common_base index dedc4c3..beca7ec 100644 --- a/config/common_base +++ b/config/common_base @@ -594,3 +594,8 @@ CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n # Compile the device metrics library # CONFIG_RTE_LIBRTE_METRICS=y + +# +# Compile the bitrate statistics library +# +CONFIG_RTE_LIBRTE_BITRATE=y diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index ca50fa6..91e8ea6 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -148,4 +148,5 @@ There are many libraries, so their headers may be grouped by topics: [ABI compat] (@ref rte_compat.h), [keepalive] (@ref rte_keepalive.h), [Device Metrics] (@ref rte_metrics.h), + [Bitrate Statistics] (@ref rte_bitrate.h), [version](@ref rte_version.h) diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf index fe830eb..8765ddd 100644 --- a/doc/api/doxy-api.conf +++ b/doc/api/doxy-api.conf @@ -58,6 +58,7 @@ INPUT = doc/api/doxy-api-index.md \ lib/librte_ring \ lib/librte_sched \ lib/librte_metrics \ + lib/librte_bitratestats \ lib/librte_table \ lib/librte_timer \ lib/librte_vhost diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst index e1b8894..f949e88 100644 --- a/doc/guides/rel_notes/release_17_02.rst +++ b/doc/guides/rel_notes/release_17_02.rst @@ -40,6 +40,11 @@ New Features intended to provide a reporting mechanism that is independent of the ethdev library. + * **Added bit-rate calculation library.** + + A library that can be used to calculate device bit-rates. Calculated + bitrates are reported using the metrics library. + This section is a comment. do not overwrite or remove it. Also, make sure to start the actual text at the margin. = diff --git a/lib/Makefile b/lib/Makefile index 5d85dcf..e211bc0 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -59,6 +59,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += librte_pipeline DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics +DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni diff --git a/lib/librte_bitratestats/Makefile b/lib/librte_bitratestats/Makefile new file mode 100644 index 000..b725d4e --- /dev/null +++ b/lib/librte_bitratestats/Makefile @@ -0,0 +1,53 @@ +# BSD LICENSE +# +# Copyright(c) 2016 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors m
[dpdk-dev] [PATCH v4 3/3] app/test-pmd: add support for bitrate statistics
Signed-off-by: Remy Horton
---
app/test-pmd/testpmd.c | 36
1 file changed, 36 insertions(+)
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index a0332c2..60c635f 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -78,6 +78,10 @@
#ifdef RTE_LIBRTE_PDUMP
#include
#endif
+#include
+#ifdef RTE_LIBRTE_BITRATE
+#include
+#endif
#include "testpmd.h"
@@ -322,6 +326,9 @@ uint16_t nb_rx_queue_stats_mappings = 0;
unsigned max_socket = 0;
+/* Bitrate statistics */
+struct rte_stats_bitrates_s *bitrate_data;
+
/* Forward function declarations */
static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port
*port);
static void check_all_ports_link_status(uint32_t port_mask);
@@ -921,12 +928,32 @@ run_pkt_fwd_on_lcore(struct fwd_lcore *fc, packet_fwd_t
pkt_fwd)
struct fwd_stream **fsm;
streamid_t nb_fs;
streamid_t sm_id;
+#ifdef RTE_LIBRTE_BITRATE
+ uint64_t tics_per_1sec;
+ uint64_t tics_datum;
+ uint64_t tics_current;
+ uint8_t idx_port, cnt_ports;
+#endif
+#ifdef RTE_LIBRTE_BITRATE
+ cnt_ports = rte_eth_dev_count();
+ tics_datum = rte_rdtsc();
+ tics_per_1sec = rte_get_timer_hz();
+#endif
fsm = &fwd_streams[fc->stream_idx];
nb_fs = fc->stream_nb;
do {
for (sm_id = 0; sm_id < nb_fs; sm_id++)
(*pkt_fwd)(fsm[sm_id]);
+#ifdef RTE_LIBRTE_BITRATE
+ tics_current = rte_rdtsc();
+ if (tics_current - tics_datum >= tics_per_1sec) {
+ /* Periodic bitrate calculation */
+ for (idx_port = 0; idx_port < cnt_ports; idx_port++)
+ rte_stats_bitrate_calc(bitrate_data, idx_port);
+ tics_datum = tics_current;
+ }
+#endif
} while (! fc->stopped);
}
@@ -2133,6 +2160,15 @@ main(int argc, char** argv)
FOREACH_PORT(port_id, ports)
rte_eth_promiscuous_enable(port_id);
+ /* Setup bitrate stats */
+#ifdef RTE_LIBRTE_BITRATE
+ bitrate_data = rte_stats_bitrate_create();
+ if (bitrate_data == NULL)
+ rte_exit(EXIT_FAILURE, "Could not allocate bitrate data.\n");
+ rte_stats_bitrate_reg(bitrate_data);
+#endif
+
+
#ifdef RTE_LIBRTE_CMDLINE
if (interactive == 1) {
if (auto_start) {
--
2.5.5
[dpdk-dev] [dpdk-announce] DPDK 16.11 released
Hi and thanks for sharing your time constraints, 2016-11-15 01:46, Liu, Yong: > As prospect for 17.02, our intel validation team have some concern about the > release date. > The official day off for Chinese Sprint Festival holiday will be from 27th > Jan to 3th Feb. > Most of our members may ask for more days leave either before or after the > official day off. > From our previous experience, it will take 3~4 weeks to do the full function > and performance test. > If the first candidate release in the middle of Jan, > we can do first round of validation and raise issues to developers. The integration deadline is January 5. So we can target/expect a RC1 on January 11. > And after the holiday, we can keep on the validation process and finish in > two weeks. > If release date is after Feb, it will be hard for us to cover all cases in > release window. Yes, we must remind that mid-January is a hard deadline for RC1. Then the release will be in mid-February to make sure you have some time after the holidays. What about Valentine's day? :)
[dpdk-dev] pmdinfogen issues: cross compilation for ARM fails with older host compiler
> On Mon, Nov 14, 2016 at 02:29:24AM +0530, Jerin Jacob wrote:
> > On Fri, Nov 11, 2016 at 10:34:39AM +, Hemant Agrawal wrote:
> > > Hi Neil,
> > >Pmdinfogen compiles with host compiler. It usages
> > > rte_byteorder.h
> of the target platform.
> > > However, if the host compiler is older than 4.8, it will be an issue
> > > during cross
> compilation for some platforms.
> > > e.g. if we are compiling on x86 host for ARM, x86 host compiler will not
> understand the arm asm instructions.
> > >
> > > /* fix missing __builtin_bswap16 for gcc older then 4.8 */ #if
> > > !(__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)) static
> > > inline uint16_t rte_arch_bswap16(uint16_t _x) {
> > >register uint16_t x = _x;
> > >asm volatile ("rev16 %0,%1"
> > > : "=r" (x)
> > > : "r" (x)
> > > );
> > >return x;
> > > }
> > > #endif
> > >
> > > One easy solution is that we add compiler platform check in this
> > > code section of rte_byteorder.h e.g #if !(defined __arm__ || defined
> > > __aarch64__) static inline uint16_t rte_arch_bswap16(uint16_t _x) {
> > >return (_x >> 8) | ((_x << 8) & 0xff00); } #else ?.
> > >
> > > Is there a better way to fix it?
> >
> > IMO, It is a HOST build infrastructure issue. If a host app is using
> > the dpdk service then it should compile and link against HOST
> > target(in this specific case, build/x86_64-native-linuxapp-gcc). I
> > think, introducing the HOSTTARGET kind of scheme is a clean solution.
> >
> > /Jerin
> >
> >
> That would be accurate. That is to say, pmdinfogen is a tool that should
> only be
> run on the host doing the build, by the host doing the build, and so should be
> compiled to run on the host, not on the target being built for.
>
> Yeah, so what we need is a way to get to the host version of rte_byteorder.h
> when building in a cross environment
>
+1
> Neil
[dpdk-dev] [PATCH v1 0/2] XStats fixes
On 11/14/2016 6:14 AM, Remy Horton wrote:
> The offsets used in rte_i40evf_stats_strings for transmission
> statistics were wrong, returning the total byte count rather than
> the respective (unicast, multicast, broadcast, drop, & error)
> packet counts.
>
> This patchset also fixes some spelling errors.
>
> Fixes: da61cd084976 ("i40evf: add extended stats")
> Fixes: 0eedec25ea36 ("i40e: clean log messages")
>
> Remy Horton (2):
> net/i40e: fix incorrect xstats value mapping
> net/i40e: fix spelling errors
>
> drivers/net/i40e/i40e_ethdev.c| 2 +-
> drivers/net/i40e/i40e_ethdev_vf.c | 16
> 2 files changed, 9 insertions(+), 9 deletions(-)
>
Series applied to dpdk-next-net/master, thanks.
[dpdk-dev] [PATCH] net/i40e: add additional prefetch instructions for bulk rx
On 10/13/2016 11:30 AM, Ananyev, Konstantin wrote: <...> Actually I can see some valid use cases where it is beneficial to have this prefetch in driver. In our sw distributor case it is trivial to just prefetch next packet on each iteration because packets are processed one by one. However when we move this functionality to hw by means of RSS/vfunction/FlowDirector(our long term goal) worker threads will >> receive packets directly from rx queues of NIC. First operation of worker thread is to perform bulk lookup in hash table by destination MAC. This will cause cache miss on accessing >> each eth header and can't be easily mitigated in application code. I assume it is ubiquitous use case for DPDK. >>> >>> Yes it is a quite common use-case. >>> Though I many cases it is possible to reorder user code to hide (or >>> minimize) that data-access latency. >>> From other side there are scenarios where this prefetch is excessive and >>> can cause some drop in performance. >>> Again, as I know, none of PMDs for Intel devices prefetches packet's data >>> in simple (single segment) RX mode. >>> Another thing that some people may argue then - why only one cache line is >>> prefetched, >>> in some use-cases might need to look at 2-nd one. >>> >> There is a build-time config setting for this behaviour for exactly the >> reasons >> called out here - in some apps you get a benefit, in others you see a perf >> hit. The default is "on", which makes sense for most cases, I think. >> From common_base: >> >> CONFIG_RTE_PMD_PACKET_PREFETCH=y$ > > Yes, but right now i40e and ixgbe non-scattered RX (both vector and scalar) > just ignore that flag. > Though yes, might be a good thing to make them to obey that flag properly. Hi Vladyslav, According Konstantin's comment, what do you think updating patch to do prefetch within CONFIG_RTE_PMD_PACKET_PREFETCH ifdef? But since config option is enabled by default, performance concern is still valid and needs to be investigated. Thanks, ferruh
[dpdk-dev] [PATCH] net/i40e: add additional prefetch instructions for bulk rx
> -Original Message- > From: Ferruh Yigit [mailto:ferruh.yigit at intel.com] > Sent: Tuesday, November 15, 2016 2:19 PM > To: Ananyev, Konstantin; Richardson, Bruce > Cc: Vladyslav Buslov; Wu, Jingjing; Zhang, Helin; dev at dpdk.org > Subject: Re: [dpdk-dev] [PATCH] net/i40e: add additional prefetch > instructions for bulk rx > > On 10/13/2016 11:30 AM, Ananyev, Konstantin wrote: > > <...> > > > Actually I can see some valid use cases where it is beneficial to have > this > prefetch in driver. > In our sw distributor case it is trivial to just prefetch next packet on > each iteration because packets are processed one by one. > However when we move this functionality to hw by means of > RSS/vfunction/FlowDirector(our long term goal) worker threads will > >> receive > packets directly from rx queues of NIC. > First operation of worker thread is to perform bulk lookup in hash > table by destination MAC. This will cause cache miss on accessing > >> each > eth header and can't be easily mitigated in application code. > I assume it is ubiquitous use case for DPDK. > >>> > >>> Yes it is a quite common use-case. > >>> Though I many cases it is possible to reorder user code to hide (or > minimize) that data-access latency. > >>> From other side there are scenarios where this prefetch is excessive and > can cause some drop in performance. > >>> Again, as I know, none of PMDs for Intel devices prefetches packet's > data in simple (single segment) RX mode. > >>> Another thing that some people may argue then - why only one cache > >>> line is prefetched, in some use-cases might need to look at 2-nd one. > >>> > >> There is a build-time config setting for this behaviour for exactly > >> the reasons called out here - in some apps you get a benefit, in > >> others you see a perf hit. The default is "on", which makes sense for most > cases, I think. > >> From common_base: > >> > >> CONFIG_RTE_PMD_PACKET_PREFETCH=y$ > > > > Yes, but right now i40e and ixgbe non-scattered RX (both vector and scalar) > just ignore that flag. > > Though yes, might be a good thing to make them to obey that flag > properly. > > Hi Vladyslav, > > According Konstantin's comment, what do you think updating patch to do > prefetch within CONFIG_RTE_PMD_PACKET_PREFETCH ifdef? > > But since config option is enabled by default, performance concern is still > valid and needs to be investigated. > > Thanks, > ferruh Hi Ferruh, I'll update my patch according to code review suggestions. Regards, Vlad
[dpdk-dev] [PATCH v5] latencystats: added new library for latency stats
Add a library designed to calculate latency statistics and report them
to the application when queried. The library measures minimum, average and
maximum latencies, and jitter in nano seconds. The current implementation
supports global latency stats, i.e. per application stats.
Added new field to mbuf struct to mark the packet arrival time on Rx.
Modify testpmd code to initialize/uninitialize latency statistics
calulation.
Modify the dpdk-procinfo process to display the newly added metrics.
Added new command line option "--metrics" to display metrics.
This pacth is dependent on http://dpdk.org/dev/patchwork/patch/16927/
APIs:
* Added APIs to initialize and un initialize latency stats
calculation.
* Added API to retrieve latency stats names and values.
Functionality:
* The library will register ethdev Rx/Tx callbacks for each active port,
queue combinations.
* The library will register latency stats names with new metrics library.
* Rx packets will be marked with time stamp on each sampling interval.
* On Tx side, packets with time stamp will be considered for calculating
the minimum, maximum, average latencies and also jitter.
* Average latency is calculated using exponential weighted moving average
method.
* Minimum and maximum latencies will be low and high latency values
observed so far.
* Jitter calculation is done based on inter packet delay variation.
* Measured stats are reported to the metrics library in a separate
pthread.
* Measured stats can be retrieved via get API of the libray (or)
by calling generic get API of the new metrics library.
Signed-off-by: Reshma Pattan
---
v5:
* References to 16.11 changed to 17.02
* Updated comments and doxygen
* rte_stat_value changed to rte_metric_value in library and proc_info
* Updated doc for doxygen
* Updated release notes
---
MAINTAINERS| 4 +
app/proc_info/main.c | 70
app/test-pmd/testpmd.c | 10 +
config/common_base | 5 +
doc/api/doxy-api-index.md | 1 +
doc/api/doxy-api.conf | 1 +
doc/guides/rel_notes/release_17_02.rst | 5 +
lib/Makefile | 1 +
lib/librte_latencystats/Makefile | 57 +++
lib/librte_latencystats/rte_latencystats.c | 389 +
lib/librte_latencystats/rte_latencystats.h | 146
.../rte_latencystats_version.map | 10 +
lib/librte_mbuf/rte_mbuf.h | 3 +
mk/rte.app.mk | 2 +
14 files changed, 704 insertions(+)
create mode 100644 lib/librte_latencystats/Makefile
create mode 100644 lib/librte_latencystats/rte_latencystats.c
create mode 100644 lib/librte_latencystats/rte_latencystats.h
create mode 100644 lib/librte_latencystats/rte_latencystats_version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index d6bb8f8..6e5e26b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -704,3 +704,7 @@ F: examples/tep_termination/
F: examples/vmdq/
F: examples/vmdq_dcb/
F: doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst
+
+Latency Stats
+M: Reshma Pattan
+F: lib/librte_latencystats/
diff --git a/app/proc_info/main.c b/app/proc_info/main.c
index 2c56d10..33a4b39 100644
--- a/app/proc_info/main.c
+++ b/app/proc_info/main.c
@@ -57,6 +57,7 @@
#include
#include
#include
+#include
/* Maximum long option length for option parsing. */
#define MAX_LONG_OPT_SZ 64
@@ -68,6 +69,8 @@ static uint32_t enabled_port_mask;
static uint32_t enable_stats;
/**< Enable xstats. */
static uint32_t enable_xstats;
+/**< Enable metrics. */
+static uint32_t enable_metrics;
/**< Enable stats reset. */
static uint32_t reset_stats;
/**< Enable xstats reset. */
@@ -85,6 +88,8 @@ proc_info_usage(const char *prgname)
" --stats: to display port statistics, enabled by default\n"
" --xstats: to display extended port statistics, disabled by "
"default\n"
+ " --metrics: to display derived metrics of the ports, disabled
by "
+ "default\n"
" --stats-reset: to reset port statistics\n"
" --xstats-reset: to reset port extended statistics\n",
prgname);
@@ -127,6 +132,7 @@ proc_info_parse_args(int argc, char **argv)
{"stats", 0, NULL, 0},
{"stats-reset", 0, NULL, 0},
{"xstats", 0, NULL, 0},
+ {"metrics", 0, NULL, 0},
{"xstats-reset", 0, NULL, 0},
{NULL, 0, 0, 0}
};
@@ -159,6 +165,10 @@ proc_info_parse_args(int argc, char **argv)
else if (!strncmp(long_option[option_index].name,
"xstats",
MAX_LONG_OPT_SZ))
enable_xstats = 1;
+
[dpdk-dev] pmdinfogen issues: cross compilation for ARM fails with older host compiler
On Tue, Nov 15, 2016 at 09:34:16AM +, Hemant Agrawal wrote:
> > On Mon, Nov 14, 2016 at 02:29:24AM +0530, Jerin Jacob wrote:
> > > On Fri, Nov 11, 2016 at 10:34:39AM +, Hemant Agrawal wrote:
> > > > Hi Neil,
> > > >Pmdinfogen compiles with host compiler. It usages
> > > > rte_byteorder.h
> > of the target platform.
> > > > However, if the host compiler is older than 4.8, it will be an issue
> > > > during cross
> > compilation for some platforms.
> > > > e.g. if we are compiling on x86 host for ARM, x86 host compiler will not
> > understand the arm asm instructions.
> > > >
> > > > /* fix missing __builtin_bswap16 for gcc older then 4.8 */ #if
> > > > !(__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)) static
> > > > inline uint16_t rte_arch_bswap16(uint16_t _x) {
> > > >register uint16_t x = _x;
> > > >asm volatile ("rev16 %0,%1"
> > > > : "=r" (x)
> > > > : "r" (x)
> > > > );
> > > >return x;
> > > > }
> > > > #endif
> > > >
> > > > One easy solution is that we add compiler platform check in this
> > > > code section of rte_byteorder.h e.g #if !(defined __arm__ || defined
> > > > __aarch64__) static inline uint16_t rte_arch_bswap16(uint16_t _x) {
> > > >return (_x >> 8) | ((_x << 8) & 0xff00); } #else ?.
> > > >
> > > > Is there a better way to fix it?
> > >
> > > IMO, It is a HOST build infrastructure issue. If a host app is using
> > > the dpdk service then it should compile and link against HOST
> > > target(in this specific case, build/x86_64-native-linuxapp-gcc). I
> > > think, introducing the HOSTTARGET kind of scheme is a clean solution.
> > >
> > > /Jerin
> > >
> > >
> > That would be accurate. That is to say, pmdinfogen is a tool that should
> > only be
> > run on the host doing the build, by the host doing the build, and so should
> > be
> > compiled to run on the host, not on the target being built for.
> >
> > Yeah, so what we need is a way to get to the host version of rte_byteorder.h
> > when building in a cross environment
> >
> +1
>
> > Neil
>
Actually, looking at this, I think we're 90% there anyway. The buildtools
already uses the hostapp.mk file (as it should), to target the host build
system, rather than any cross target, so we're good there. The only question
is, should we be using rte_byteorder.h at all, and I think the answer is no, we
shouldn't. I assert that because the byteorder file is configured for the
target, not the host, and so we shouldn't be touching it at all. Instead we
should just be using the posix htobe*/htole*/letoh*/betoh* variants to do the
endian conversion, as those are always configured to work on the local build
host.
I'll propose a patch shortly for you to test.
Neil
[dpdk-dev] [PATCH] doc: add pdump library to API doxygen
Add pdump library to API doxygen. Signed-off-by: Reshma Pattan --- doc/api/doxy-api-index.md | 3 ++- doc/api/doxy-api.conf | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 6675f96..492a0af 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -140,7 +140,8 @@ There are many libraries, so their headers may be grouped by topics: [debug] (@ref rte_debug.h), [log](@ref rte_log.h), [warnings] (@ref rte_warnings.h), - [errno] (@ref rte_errno.h) + [errno] (@ref rte_errno.h), + [pdump] (@ref rte_pdump.h) - **misc**: [EAL config] (@ref rte_eal.h), diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf index 9dc7ae5..b340fcf 100644 --- a/doc/api/doxy-api.conf +++ b/doc/api/doxy-api.conf @@ -51,6 +51,7 @@ INPUT = doc/api/doxy-api-index.md \ lib/librte_mempool \ lib/librte_meter \ lib/librte_net \ + lib/librte_pdump \ lib/librte_pipeline \ lib/librte_port \ lib/librte_power \ -- 2.7.4
[dpdk-dev] pmdinfogen issues: cross compilation for ARM fails with older host compiler
On Tue, Nov 15, 2016 at 09:34:16AM +, Hemant Agrawal wrote:
> > On Mon, Nov 14, 2016 at 02:29:24AM +0530, Jerin Jacob wrote:
> > > On Fri, Nov 11, 2016 at 10:34:39AM +, Hemant Agrawal wrote:
> > > > Hi Neil,
> > > >Pmdinfogen compiles with host compiler. It usages
> > > > rte_byteorder.h
> > of the target platform.
> > > > However, if the host compiler is older than 4.8, it will be an issue
> > > > during cross
> > compilation for some platforms.
> > > > e.g. if we are compiling on x86 host for ARM, x86 host compiler will not
> > understand the arm asm instructions.
> > > >
> > > > /* fix missing __builtin_bswap16 for gcc older then 4.8 */ #if
> > > > !(__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)) static
> > > > inline uint16_t rte_arch_bswap16(uint16_t _x) {
> > > >register uint16_t x = _x;
> > > >asm volatile ("rev16 %0,%1"
> > > > : "=r" (x)
> > > > : "r" (x)
> > > > );
> > > >return x;
> > > > }
> > > > #endif
> > > >
> > > > One easy solution is that we add compiler platform check in this
> > > > code section of rte_byteorder.h e.g #if !(defined __arm__ || defined
> > > > __aarch64__) static inline uint16_t rte_arch_bswap16(uint16_t _x) {
> > > >return (_x >> 8) | ((_x << 8) & 0xff00); } #else ?.
> > > >
> > > > Is there a better way to fix it?
> > >
> > > IMO, It is a HOST build infrastructure issue. If a host app is using
> > > the dpdk service then it should compile and link against HOST
> > > target(in this specific case, build/x86_64-native-linuxapp-gcc). I
> > > think, introducing the HOSTTARGET kind of scheme is a clean solution.
> > >
> > > /Jerin
> > >
> > >
> > That would be accurate. That is to say, pmdinfogen is a tool that should
> > only be
> > run on the host doing the build, by the host doing the build, and so should
> > be
> > compiled to run on the host, not on the target being built for.
> >
> > Yeah, so what we need is a way to get to the host version of rte_byteorder.h
> > when building in a cross environment
> >
> +1
>
> > Neil
>
Give this a try, I've tested it on linux, but not BSD. From what I read the
functions are not posix compliant, though they should exist on all BSD and Linux
systems in recent history. There may be some fiddling needed for Net and
OpenBSD variants, but I think this is the right general direction.
diff --git a/buildtools/pmdinfogen/pmdinfogen.h
b/buildtools/pmdinfogen/pmdinfogen.h
index 1da2966..c5ef89d 100644
--- a/buildtools/pmdinfogen/pmdinfogen.h
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -21,7 +21,6 @@
#include
#include
#include
-#include
/* On BSD-alike OSes elf.h defines these according to host's word size */
#undef ELF_ST_BIND
@@ -75,9 +74,9 @@
#define CONVERT_NATIVE(fend, width, x) ({ \
typeof(x) ___x; \
if ((fend) == ELFDATA2LSB) \
- ___x = rte_le_to_cpu_##width(x); \
+ ___x = le##width##toh(x); \
else \
- ___x = rte_be_to_cpu_##width(x); \
+ ___x = be##width##toh(x); \
___x; \
})
[dpdk-dev] [PATCH v4 2/3] lib: add bitrate statistics library
Hi, > Signed-off-by: Remy Horton > --- > a/doc/guides/rel_notes/release_17_02.rst > b/doc/guides/rel_notes/release_17_02.rst > index e1b8894..f949e88 100644 > --- a/doc/guides/rel_notes/release_17_02.rst > +++ b/doc/guides/rel_notes/release_17_02.rst > @@ -40,6 +40,11 @@ New Features > intended to provide a reporting mechanism that is independent of the > ethdev library. > > + * **Added bit-rate calculation library.** > + > + A library that can be used to calculate device bit-rates. Calculated > + bitrates are reported using the metrics library. > + Shared Library Versions section also need to be updated for bitrate and metrics libraries. Thanks, Reshma
[dpdk-dev] pmdinfogen issues: cross compilation for ARM fails with older host compiler
2016-11-15 09:27, Neil Horman: > On Tue, Nov 15, 2016 at 09:34:16AM +, Hemant Agrawal wrote: > > > > On Fri, Nov 11, 2016 at 10:34:39AM +, Hemant Agrawal wrote: > > > > > Hi Neil, > > > > >Pmdinfogen compiles with host compiler. It usages rte_byteorder.h > > > > >of the target platform. [...] > > > Yeah, so what we need is a way to get to the host version of > > > rte_byteorder.h > > > when building in a cross environment > > > > > +1 > > > Actually, looking at this, I think we're 90% there anyway. The buildtools > already uses the hostapp.mk file (as it should), to target the host build > system, rather than any cross target, so we're good there. The only question > is, should we be using rte_byteorder.h at all, and I think the answer is no, > we > shouldn't. I assert that because the byteorder file is configured for the > target, not the host, and so we shouldn't be touching it at all. Instead we > should just be using the posix htobe*/htole*/letoh*/betoh* variants to do the > endian conversion, as those are always configured to work on the local build > host. Yes there are 2 possible fixes: - get a host version of EAL - do not use EAL for host applications As Neil, I think there is no point in using EAL for a host application.
[dpdk-dev] [PATCH] doc: add pdump library to API doxygen
2016-11-15 14:41, Reshma Pattan: > --- a/doc/api/doxy-api-index.md > +++ b/doc/api/doxy-api-index.md > @@ -140,7 +140,8 @@ There are many libraries, so their headers may be grouped > by topics: >[debug] (@ref rte_debug.h), >[log](@ref rte_log.h), >[warnings] (@ref rte_warnings.h), > - [errno] (@ref rte_errno.h) > + [errno] (@ref rte_errno.h), > + [pdump] (@ref rte_pdump.h) The end of this list are trivial headers, what pdump is not. I think it should be at the top of the list, between jobstats and hexdump.
[dpdk-dev] [PATCH] net/bnxt: fix a segfault encountered during PMD exit
This patch fixes segfault encountered during dev_uninit/close routine.
KNI sample app can be used to reproduce the issue.
backported from upstream commit 316e412
Cc:
Signed-off-by: Ajit Khaparde
---
drivers/net/bnxt/bnxt.h| 1 +
drivers/net/bnxt/bnxt_ethdev.c | 28 ++--
2 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index df1f771..0e21ace 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -171,6 +171,7 @@ struct bnxt {
struct bnxt_pf_info pf;
struct bnxt_vf_info vf;
+ uint8_t dev_stopped;
};
#endif
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 3795fac..deeb54c 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -384,6 +384,7 @@ static int bnxt_dev_start_op(struct rte_eth_dev *eth_dev)
struct bnxt *bp = (struct bnxt *)eth_dev->data->dev_private;
int rc;
+ bp->dev_stopped = 0;
rc = bnxt_hwrm_func_reset(bp);
if (rc) {
RTE_LOG(ERR, PMD, "hwrm chip reset failure rc: %x\n", rc);
@@ -427,16 +428,6 @@ static int bnxt_dev_set_link_down_op(struct rte_eth_dev
*eth_dev)
return 0;
}
-static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
-{
- struct bnxt *bp = (struct bnxt *)eth_dev->data->dev_private;
-
- bnxt_free_tx_mbufs(bp);
- bnxt_free_rx_mbufs(bp);
- bnxt_free_mem(bp);
- rte_free(eth_dev->data->mac_addrs);
-}
-
/* Unload the driver, release resources */
static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
{
@@ -449,6 +440,19 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
bnxt_shutdown_nic(bp);
}
+static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
+{
+ struct bnxt *bp = (struct bnxt *)eth_dev->data->dev_private;
+
+ if (bp->dev_stopped == 0)
+ bnxt_dev_stop_op(eth_dev);
+
+ bnxt_free_tx_mbufs(bp);
+ bnxt_free_rx_mbufs(bp);
+ bnxt_free_mem(bp);
+ rte_free(eth_dev->data->mac_addrs);
+}
+
static void bnxt_mac_addr_remove_op(struct rte_eth_dev *eth_dev,
uint32_t index)
{
@@ -1021,6 +1025,8 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->pci_dev->mem_resource[0].phys_addr,
eth_dev->pci_dev->mem_resource[0].addr);
+ bp->dev_stopped = 0;
+
return 0;
error_free:
@@ -1040,6 +1046,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) {
rte_free(bp->grp_info);
rc = bnxt_hwrm_func_driver_unregister(bp, 0);
bnxt_free_hwrm_resources(bp);
+ if (bp->dev_stopped == 0)
+ bnxt_dev_close_op(eth_dev);
return rc;
}
--
2.8.4 (Apple Git-73)
