Re: [RFC] graph/node: feedback and future improvements
On Wed, Apr 24, 2024 at 6:54 PM Robin Jarry wrote: > > Robin Jarry, Apr 14, 2024 at 12:35: > > Jerin Jacob, Apr 06, 2024 at 01:11: > > > Great. > > > > > > You may consider improving and/or adding inbuilt nodes for generic > > > protocol processing. Furthermore, consider contributing on app/graph. > > > I think, most likely, you should be able to leverage app/graph. > > > > Thanks! I am definitely planning to upstream nodes into DPDK as much as > > possible. I still need to determine what is the correct level of data > > path node public API so that the nodes can be agnostic of the control > > plane implementation. > > Hey Jerin, > > while working on ARP support [1], I noticed that we need to have > configurability of nodes (or set of nodes to be more precise) from the > outside. But the nodes in the set also need to specify metadata that > they expect from other nodes to store in mbufs. > > For now, I have used a single dynamic field which I repurpose for every > node set depending on the use case [2]. However this raises a question: > how can we make it generic (and agnostic to the application) before > inclusion in lib/node? > > I would be glad to have your opinion on the subject. Looking at the brouter[1] project, Based on my understanding it has following 1)Data plane code: IMO, Since it is generic and any consumers can reuse this code. IMO, We can make as generic and upstream to lib/node. 2)Control plane code: IMO, if you are willing, I will be glad to see it is hosted at https://www.dpdk.org/hosted-projects/ like pktgen. This may attract more developers for control plane code for brouter and show good reference of using DPDK graph library. I think, TB needs to approve for this. If you are OK, We can add this to one of TB meeting agenda. [1] https://github.com/rjarry/brouter/ > Cheers! > > [1] https://github.com/rjarry/brouter/commit/e05246f51b736821b6689d40 > [2] > https://github.com/rjarry/brouter/blob/main/modules/infra/datapath/br_mbuf.h >
Re: [PATCH] devtools: add .clang-format file
Hello,
I am facing a problem while creating the configuration. I am trying to make
a clang-format that formats codes without changing any codebases. Here is
the question:
clang-format related config:
...
BraceWrapping:
AfterFunction: true
...
If the configuration supports braces after function, then MACRO's formats
fail.
An example of the formatted MACRO is given below:
...
#define RTE_RX_OFFLOAD_BIT2STR(_name) \
{ \
...
If the AfterFunction is false, the functions' formats fail; here is a
formatted example.
...
if (devargs_str == NULL) {
...
I will send the current configuration as v2. You can test it.
On Mon, Apr 29, 2024 at 6:43 PM Stephen Hemminger <
step...@networkplumber.org> wrote:
> On Mon, 29 Apr 2024 14:32:43 +0100
> Ferruh Yigit wrote:
>
> > On 4/29/2024 2:04 PM, Abdullah Ömer Yamaç wrote:
> > > clang-format is a tool to format C/C++/Objective-C code. It can be used
> > > to reformat code to match a given coding style, or to ensure that code
> > > adheres to a specific coding style. It helps to maintain a consistent
> > > coding style across the DPDK codebase.
> > >
> > > .clang-format file overrides the default style options provided by
> > > clang-format and large set of IDEs and text editors support it.
> > >
> > > Signed-off-by: Abdullah Ömer Yamaç
> > >
> >
> > Overall +1 to have a format file, specially to help non-frequent
> > contributors.
> >
> > 1. Some options are failing for me [1], I don't know if it requires a
> > specific version of clang-format
> >
> > 2. Current options are not fully aligned with coding convention, it is
> > easier to see what is not compatible by running the tool on an existing
> > file [2].
> > we need to fix them.
> >
> >
> >
> > [1]
> > `clang-format -i ./lib/ethdev/rte_ethdev.c`
> > a.
> > /home/amd/development/dpdk-next-net/./.clang-format:28:1: error: unknown
> > key 'Whitespace'
> > Whitespace:
> > ^~
> > Error reading /home/amd/development/dpdk-next-net/./.clang-format:
> > Invalid argument
> >
> > b.
> > /home/amd/development/dpdk-next-net/./.clang-format:12:1: error: unknown
> > key 'EndOfLine'
> > EndOfLine: lf
> > ^
> > Error reading /home/amd/development/dpdk-next-net/./.clang-format:
> > Invalid argument
> >
> > c.
> > /home/amd/development/dpdk-next-net/./.clang-format:15:1: error: unknown
> > key 'File'
> > File:
> > ^~~~
> > Error reading /home/amd/development/dpdk-next-net/./.clang-format:
> > Invalid argument
> >
> >
> >
> > [2]
> > `clang-format -i ./lib/ethdev/rte_ethdev.c` (with failing part commented
> > out)
> > `git diff`
> >
> > A few diff samples to fix at first glance:
> > ```
> > static const struct rte_eth_xstats_name_off eth_dev_txq_stats_strings[]
> = {
> > - {"packets", offsetof(struct rte_eth_stats, q_opackets)},
> > - {"bytes", offsetof(struct rte_eth_stats, q_obytes)},
> > +{"packets", offsetof(struct rte_eth_stats, q_opackets)},
> > +{"bytes", offsetof(struct rte_eth_stats, q_obytes)},
> > };
> > ```
> >
> > ```
> > -enum {
> > - STAT_QMAP_TX = 0,
> > - STAT_QMAP_RX
> > -};
> > +enum { STAT_QMAP_TX = 0, STAT_QMAP_RX };
> > ```
> >
> > ```
> > -int
> > -rte_eth_iterator_init(struct rte_dev_iterator *iter, const char
> > *devargs_str)
> > +int rte_eth_iterator_init(struct rte_dev_iterator *iter, const char
> > *devargs_str)
> > ```
> >
> > ```
> > RTE_ETH_FOREACH_DEV(p)
>
> Add this
>
> # Take from
> #
> # git grep -h '^#define [^[:space:]]*FOREACH[^[:space:]]*(' lib/ \
> # | sed "s,^#define \([^[:space:]]*FOREACH[^[:space:]]*\)(.*$, - '\1'," \
> # | LC_ALL=C sort -u
> ForeachMacros:
> - 'CIRBUF_FOREACH'
> - 'RTE_BBDEV_FOREACH'
> - 'RTE_DEV_FOREACH'
> - 'RTE_DMA_FOREACH_DEV'
> - 'RTE_EAL_DEVARGS_FOREACH'
> - 'RTE_ETH_FOREACH_DEV'
> - 'RTE_ETH_FOREACH_DEV_OF'
> - 'RTE_ETH_FOREACH_DEV_OWNED_BY'
> - 'RTE_ETH_FOREACH_DEV_SIBLING'
> - 'RTE_ETH_FOREACH_MATCHING_DEV'
> - 'RTE_ETH_FOREACH_VALID_DEV'
> - 'RTE_GPU_FOREACH'
> - 'RTE_GPU_FOREACH_CHILD'
> - 'RTE_GPU_FOREACH_PARENT'
> - 'RTE_LCORE_FOREACH'
> - 'RTE_LCORE_FOREACH_WORKER'
> - 'RTE_TAILQ_FOREACH'
> - 'RTE_TAILQ_FOREACH_SAFE'
>
>
>
[PATCH v2] devtools: add .clang-format file
clang-format is a tool to format C/C++/Objective-C code. It can be used to reformat code to match a given coding style, or to ensure that code adheres to a specific coding style. It helps to maintain a consistent coding style across the DPDK codebase. .clang-format file overrides the default style options provided by clang-format and large set of IDEs and text editors support it. Signed-off-by: Abdullah Ömer Yamaç --- .clang-format | 42 ++ 1 file changed, 42 insertions(+) create mode 100644 .clang-format diff --git a/.clang-format b/.clang-format new file mode 100644 index 00..480beaca20 --- /dev/null +++ b/.clang-format @@ -0,0 +1,42 @@ +--- +BasedOnStyle: LLVM + +# Place opening and closing parentheses on the same line for control statements +BreakBeforeBraces: Custom +BraceWrapping: +AfterFunction: false +AfterControlStatement: false +AfterEnum: false + +# Set maximum line length to 100 characters +ColumnLimit: 100 + +# Use LF (line feed) as the end-of-line character +LineEnding: LF + +# Insert a newline at the end of the file +InsertNewlineAtEOF: true + +# Set indentation width to 8 spaces +IndentWidth: 8 + +# Set continuation indentation width to 8 spaces +ContinuationIndentWidth: 8 + +# Set tab width to 8 spaces +TabWidth: 8 + +# Use tabs for indentation +UseTab: Always + +# Preserve include blocks as they are +IncludeBlocks: Preserve + +# Never sort includes +SortIncludes: Never + +# Always break after return type for top-level definitions +AlwaysBreakAfterReturnType: TopLevelDefinitions + +# Always break before multiline string literals +AlignEscapedNewlines: Left -- 2.34.1
Re: [RFC v6 5/6] eal: add atomic bit operations
On 2024-05-04 01:30, Tyler Retzlaff wrote:
On Fri, May 03, 2024 at 08:41:09AM +0200, Mattias Rönnblom wrote:
On 2024-05-02 07:57, Mattias Rönnblom wrote:
Add atomic bit test/set/clear/assign/flip and
test-and-set/clear/assign/flip functions.
All atomic bit functions allow (and indeed, require) the caller to
specify a memory order.
RFC v6:
* Have rte_bit_atomic_test() accept const-marked bitsets.
RFC v4:
* Add atomic bit flip.
* Mark macro-generated private functions experimental.
RFC v3:
* Work around lack of C++ support for _Generic (Tyler Retzlaff).
RFC v2:
o Add rte_bit_atomic_test_and_assign() (for consistency).
o Fix bugs in rte_bit_atomic_test_and_[set|clear]().
o Use to support MSVC.
Signed-off-by: Mattias Rönnblom
Acked-by: Morten Brørup
Acked-by: Tyler Retzlaff
---
lib/eal/include/rte_bitops.h | 428 +++
1 file changed, 428 insertions(+)
diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h
index caec4f36bb..9cde982113 100644
--- a/lib/eal/include/rte_bitops.h
+++ b/lib/eal/include/rte_bitops.h
@@ -21,6 +21,7 @@
#include
#include
+#include
#ifdef __cplusplus
extern "C" {
@@ -399,6 +400,202 @@ extern "C" {
uint32_t *: __rte_bit_once_flip32, \
uint64_t *: __rte_bit_once_flip64)(addr, nr)
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Test if a particular bit in a word is set with a particular memory
+ * order.
+ *
+ * Test a bit with the resulting memory load ordered as per the
+ * specified memory order.
+ *
+ * @param addr
+ * A pointer to the word to query.
+ * @param nr
+ * The index of the bit.
+ * @param memory_order
+ * The memory order to use. See for details.
+ * @return
+ * Returns true if the bit is set, and false otherwise.
+ */
+#define rte_bit_atomic_test(addr, nr, memory_order)\
+ _Generic((addr),\
+uint32_t *: __rte_bit_atomic_test32, \
+const uint32_t *: __rte_bit_atomic_test32, \
+uint64_t *: __rte_bit_atomic_test64, \
+const uint64_t *: __rte_bit_atomic_test64)(addr, nr, \
+ memory_order)
Should __rte_bit_atomic_test32()'s addr parameter be marked
volatile, and two volatile-marked branches added to the above list?
off-topic comment relating to the generic type selection list above, i was
reading C17 DR481 recently and i think we may want to avoid providing
qualified and unauqlified types in the list.
DR481: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2396.htm#dr_481
"the controlling expression of a generic selection shall have type
compatibile with at most one of the types named in its generic
association list."
Const and unqualified pointers are not compatible. Without the "const
uint32_t *" element in the association list, passing const-qualified
pointers to rte_bit_test() will cause a compiler error.
So, if you want to support both passing const-qualified and unqualified
pointers (which you do, obviously), then you have no other option than
to treat them separately.
GCC, clang and ICC all seem to agree on this. The standard also is
pretty clear on this, from what I can tell. "No two generic associations
in the same generic selection shall specify compatible types." (6.5.1.1,
note *compatible*). "For two pointer types to be compatible, both shall
be identically qualified and both shall be pointers to compatible
types." (6.7.6.1)
"the type of the controlling expression is the type of the expression as
if it had undergone an lvalue conversion"
"lvalue conversion drops type qualifiers"
so the unqualified type of the controlling expression is only matched
selection list which i guess that means the qualified entries in the
list are never selected.
i suppose the implication here is we couldn't then provide 2 inline
functions one for volatile qualified and for not volatile qualified.
as for a single function where the parameter is or isn't volatile
qualified. if we're always forwarding to an intrinsic i've always
assumed (perhaps incorrectly) that the intrinsic itself did what was
semantically correct even without qualification.
as you note i believe there is a convenience element in providing the
volatile qualified version since it means the function like macro /
inline function will accept both volatile qualified and unqualified
whereas if we did not qualify the parameter it would require the
caller/user to strip the volatile qualification if present with casts.
i imagine in most cases we are just forwarding, in which case it seems
not horrible to provide the qualified version.
Both the C11-style GCC built-ins and the C11-proper atomic functions
have addresses marked volatile. The Linux kernel and the old __sync
GCC
Re: [PATCH v2] devtools: add .clang-format file
On Sat, 4 May 2024 13:41:35 + Abdullah Ömer Yamaç wrote: > clang-format is a tool to format C/C++/Objective-C code. It can be used > to reformat code to match a given coding style, or to ensure that code > adheres to a specific coding style. It helps to maintain a consistent > coding style across the DPDK codebase. > > .clang-format file overrides the default style options provided by > clang-format and large set of IDEs and text editors support it. > > Signed-off-by: Abdullah Ömer Yamaç You need to add the ForEachMacros: I sent in earlier email
Re: [PATCH v2] devtools: add .clang-format file
I was confused about the kernel functions. I didn't understand what you meant at first, but now I understand. I will send a new patch with all DPDK-related foreach macros. On Sat, May 4, 2024 at 7:27 PM Stephen Hemminger wrote: > On Sat, 4 May 2024 13:41:35 + > Abdullah Ömer Yamaç wrote: > > > clang-format is a tool to format C/C++/Objective-C code. It can be used > > to reformat code to match a given coding style, or to ensure that code > > adheres to a specific coding style. It helps to maintain a consistent > > coding style across the DPDK codebase. > > > > .clang-format file overrides the default style options provided by > > clang-format and large set of IDEs and text editors support it. > > > > Signed-off-by: Abdullah Ömer Yamaç > > You need to add the ForEachMacros: I sent in earlier email > >
[PATCH v3] devtools: add .clang-format file
clang-format is a tool to format C/C++/Objective-C code. It can be used to reformat code to match a given coding style, or to ensure that code adheres to a specific coding style. It helps to maintain a consistent coding style across the DPDK codebase. .clang-format file overrides the default style options provided by clang-format and large set of IDEs and text editors support it. Signed-off-by: Abdullah Ömer Yamaç --- .clang-format | 138 ++ 1 file changed, 138 insertions(+) create mode 100644 .clang-format diff --git a/.clang-format b/.clang-format new file mode 100644 index 00..16164ef1de --- /dev/null +++ b/.clang-format @@ -0,0 +1,138 @@ +--- +BasedOnStyle: LLVM + +# Place opening and closing parentheses on the same line for control statements +BreakBeforeBraces: Custom +BraceWrapping: +AfterFunction: false +AfterControlStatement: false +AfterEnum: false + +# Set maximum line length to 100 characters +ColumnLimit: 100 + +# Use LF (line feed) as the end-of-line character +LineEnding: LF + +# Insert a newline at the end of the file +InsertNewlineAtEOF: true + +# Set indentation width to 8 spaces +IndentWidth: 8 + +# Set continuation indentation width to 8 spaces +ContinuationIndentWidth: 8 + +# Set tab width to 8 spaces +TabWidth: 8 + +# Use tabs for indentation +UseTab: Always + +# Preserve include blocks as they are +IncludeBlocks: Preserve + +# Never sort includes +SortIncludes: Never + +# Always break after return type for top-level definitions +AlwaysBreakAfterReturnType: TopLevelDefinitions + +# Always break before multiline string literals +AlignEscapedNewlines: Left + +# Foreach macros +ForEachMacros: [ +"CIRBUF_FOREACH", +"DLB2_LIST_FOR_EACH", +"DLB2_LIST_FOR_EACH_SAFE", +"ECORE_LIST_FOR_EACH_ENTRY", +"ECORE_LIST_FOR_EACH_ENTRY_SAFE", +"FOR_EACH", +"FOR_EACH_BUCKET", +"FOR_EACH_CNIC_QUEUE", +"FOR_EACH_COS_IN_TX_QUEUE", +"FOR_EACH_ETH_QUEUE", +"FOR_EACH_MEMBER", +"FOR_EACH_NONDEFAULT_ETH_QUEUE", +"FOR_EACH_NONDEFAULT_QUEUE", +"FOR_EACH_PORT", +"FOR_EACH_PORT_IF", +"FOR_EACH_QUEUE", +"FOR_EACH_SUITE_TESTCASE", +"FOR_EACH_SUITE_TESTSUITE", +"FOREACH_ABS_FUNC_IN_PORT", +"FOREACH_DEVICE_ON_AUXILIARY_BUS", +"FOREACH_DEVICE_ON_CDXBUS", +"FOREACH_DEVICE_ON_PCIBUS", +"FOREACH_DEVICE_ON_PLATFORM_BUS", +"FOREACH_DEVICE_ON_UACCEBUS", +"FOREACH_DEVICE_ON_VMBUS", +"FOREACH_DRIVER_ON_AUXILIARY_BUS", +"FOREACH_DRIVER_ON_CDXBUS", +"FOREACH_DRIVER_ON_PCIBUS", +"FOREACH_DRIVER_ON_PLATFORM_BUS", +"FOREACH_DRIVER_ON_UACCEBUS", +"FOREACH_DRIVER_ON_VMBUS", +"FOREACH_SUBDEV", +"FOREACH_SUBDEV_STATE", +"HLIST_FOR_EACH_ENTRY", +"ILIST_FOREACH", +"LIST_FOR_EACH_ENTRY", +"LIST_FOR_EACH_ENTRY_SAFE", +"LIST_FOREACH", +"LIST_FOREACH_FROM", +"LIST_FOREACH_FROM_SAFE", +"LIST_FOREACH_SAFE", +"ML_AVG_FOREACH_QP", +"ML_AVG_FOREACH_QP_MVTVM", +"ML_AVG_RESET_FOREACH_QP", +"ML_MAX_FOREACH_QP", +"ML_MAX_FOREACH_QP_MVTVM", +"ML_MAX_RESET_FOREACH_QP", +"ML_MIN_FOREACH_QP", +"ML_MIN_FOREACH_QP_MVTVM", +"ML_MIN_RESET_FOREACH_QP", +"MLX5_ETH_FOREACH_DEV", +"MLX5_IPOOL_FOREACH", +"MLX5_L3T_FOREACH", +"OSAL_LIST_FOR_EACH_ENTRY", +"OSAL_LIST_FOR_EACH_ENTRY_SAFE", +"PLT_TAILQ_FOREACH_SAFE", +"RTE_BBDEV_FOREACH", +"RTE_DEV_FOREACH", +"RTE_DMA_FOREACH_DEV", +"RTE_EAL_DEVARGS_FOREACH", +"RTE_ETH_FOREACH_DEV", +"RTE_ETH_FOREACH_DEV_OF", +"RTE_ETH_FOREACH_DEV_OWNED_BY", +"RTE_ETH_FOREACH_DEV_SIBLING", +"RTE_ETH_FOREACH_MATCHING_DEV", +"RTE_ETH_FOREACH_VALID_DEV", +"RTE_GPU_FOREACH", +"RTE_GPU_FOREACH_CHILD", +"RTE_GPU_FOREACH_PARENT", +"RTE_LCORE_FOREACH", +"RTE_LCORE_FOREACH_WORKER", +"RTE_TAILQ_FOREACH", +"RTE_TAILQ_FOREACH_SAFE", +"SILIST_FOREACH", +"SLIST_FOREACH", +"SLIST_FOREACH_FROM", +"SLIST_FOREACH_FROM_SAFE", +"SLIST_FOREACH_PREVPTR", +"SLIST_FOREACH_SAFE", +"STAILQ_FOREACH", +"STAILQ_FOREACH_FROM", +"STAILQ_FOREACH_FROM_SAFE", +"STAILQ_FOREACH_SAFE", +"TAILQ_FOREACH", +"TAILQ_FOREACH_ENTRY", +"TAILQ_FOREACH_ENTRY_SAFE", +"TAILQ_FOREACH_FROM", +"TAILQ_FOREACH_FROM_SAFE", +"TAILQ_FOREACH_REVERSE", +"TAILQ_FOREACH_REVERSE_FROM", +"TAILQ_FOREACH_REVERSE_FROM_SAFE", +"TAILQ_FOREACH_REVERSE_SAFE", +"TAILQ_FOREACH_SAFE", ] -- 2.34.1
RE: Need help with reducing VIRT memory
Hi Dmitry, I tried your settings tonight and the application VIRT memory is now 7.9G and is in the ball park, you are amazing. #define RTE_MAX_MEMSEG_LISTS 2 #define RTE_MAX_MEMSEG_PER_LIST 1024 #define RTE_MAX_MEM_MB_PER_LIST 2048 #define RTE_MAX_MEMSEG_PER_TYPE 1024 #define RTE_MAX_MEM_MB_PER_TYPE 2048 There is no possibility to change these settings at run-time? Maybe I asking too much? Maybe would be something DPDK developers can consider? Thanks, Ed -Original Message- From: Dmitry Kozlyuk Sent: Friday, May 3, 2024 4:57 PM To: Lombardo, Ed Cc: dev@dpdk.org Subject: Re: Need help with reducing VIRT memory External Email: This message originated outside of NETSCOUT. Do not click links or open attachments unless you recognize the sender and know the content is safe. 2024-05-03 23:54 (UTC+0300), Dmitry Kozlyuk: > #define RTE_MAX_MEM_MB_PER_LIST 2024 // see item 4 below Typo: 2048
