Add a chapter to the programmer's guide describing the new rte_flow_compile library: API summary, BNF grammar, field qualifier semantics (is/spec/last/mask/prefix), diagnostic format, and the table-driven extension model for adding items and actions.
Notes that the grammar is a strict subset of testpmd's, and documents the limitations of the initial implementation (item/action coverage, missing RSS and RAW handling). Signed-off-by: Stephen Hemminger <[email protected]> --- MAINTAINERS | 6 + doc/guides/prog_guide/flow_compile_lib.rst | 302 +++++++++++++++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/rel_notes/release_26_07.rst | 6 + 4 files changed, 315 insertions(+) create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index 0f5539f851..4923e126df 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -448,6 +448,12 @@ F: app/test-pmd/cmdline_flow.c F: doc/guides/prog_guide/ethdev/flow_offload.rst F: lib/ethdev/rte_flow* +Flow Compiler API +M: Stephen Hemminger <[email protected]> +T: git://dpdk.org/next/dpdk-next-net +F: lib/flow_compile/ +F: doc/guides/prog_guide/flow_compile_lib.rst + Traffic Management API M: Cristian Dumitrescu <[email protected]> T: git://dpdk.org/next/dpdk-next-net diff --git a/doc/guides/prog_guide/flow_compile_lib.rst b/doc/guides/prog_guide/flow_compile_lib.rst new file mode 100644 index 0000000000..2c38c3d2d6 --- /dev/null +++ b/doc/guides/prog_guide/flow_compile_lib.rst @@ -0,0 +1,302 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2026 Stephen Hemminger <[email protected]> + +Flow Rule Compiler +================== + +The flow rule compiler (``rte_flow_compile``) turns a textual +description of an ``rte_flow`` rule into the +``struct rte_flow_attr`` / ``struct rte_flow_item`` / +``struct rte_flow_action`` arrays accepted by ``rte_flow_create()``. + +It is modelled on ``pcap_compile()`` from libpcap: a single string in, +an opaque compiled object out, with human readable diagnostics +written to a caller supplied buffer. + +Runtime dependencies are limited to ``rte_flow`` (currently part of +``rte_ethdev``) and ``rte_net`` (for MAC address parsing). In +particular the compiler does not pull in ``rte_cmdline``, so it is +suitable for use from libraries, control planes and unit tests. + +Flex and bison are required at build time to regenerate the lexer +and parser sources; if either tool is missing the library is +silently skipped via meson's ``has_flex_bison`` check. + + +Example +------- + +.. code-block:: c + + char errbuf[RTE_FLOW_COMPILE_ERRBUF_SIZE]; + const char *src = + "ingress group 0 priority 1 " + "pattern eth / ipv4 src is 10.0.0.1 / udp dst is 4789 / end " + "actions queue index 3 / count / end"; + + struct rte_flow_compile *fc = rte_flow_compile(src, errbuf); + if (fc == NULL) { + fprintf(stderr, "%s\n", errbuf); + return -1; + } + + struct rte_flow_error err; + struct rte_flow *f = rte_flow_compile_create(port_id, fc, &err); + + /* fc may be reused on multiple ports or freed now. */ + rte_flow_compile_free(fc); + + +API summary +----------- + +.. code-block:: c + + struct rte_flow_compile * + rte_flow_compile(const char *str, + char errbuf[RTE_FLOW_COMPILE_ERRBUF_SIZE]); + + void + rte_flow_compile_free(struct rte_flow_compile *fc); + + const struct rte_flow_attr *rte_flow_compile_attr(...); + const struct rte_flow_item *rte_flow_compile_pattern(..., unsigned int *n); + const struct rte_flow_action *rte_flow_compile_actions(..., unsigned int *n); + + int rte_flow_compile_validate(uint16_t port_id, ..., struct rte_flow_error *); + struct rte_flow *rte_flow_compile_create (uint16_t port_id, ..., struct rte_flow_error *); + +The compiled object owns every buffer it returns: attributes, +patterns, actions and all underlying spec/mask/last/conf payloads. +Pointers are valid until ``rte_flow_compile_free()`` is called. +A single compiled rule may be installed on many ports and validated +or created concurrently from multiple threads; the parser itself +holds no static mutable state. + + +Grammar +------- + +The grammar is pure ASCII; ``#`` starts an end-of-line comment. +Whitespace is insignificant. + +.. code-block:: bnf + + rule ::= attribute* "pattern" item-list "actions" action-list + attribute ::= "ingress" | "egress" | "transfer" + | "group" UINT + | "priority" UINT + item-list ::= ( item "/" )+ "end" + item ::= IDENT field-spec* + field-spec ::= IDENT qualifier value + qualifier ::= "is" | "spec" | "last" | "mask" | "prefix" + action-list ::= ( action "/" )+ "end" + action ::= IDENT param* + param ::= IDENT value + value ::= UINT | IPV4 | IPV6 | MAC | HEXSTR + +Both the pattern list and the action list must contain at least one +entry before the trailing ``end``: ``pattern end`` and +``actions end`` are parse errors. A truly catch-all rule must list +at least one item (typically ``eth``) and at least one action +(typically ``passthru`` or a queue assignment). + +The ``STRING`` token is recognised by the lexer for use by future +custom setters but is not currently accepted by any production. + +Lexical tokens: + +.. code-block:: bnf + + IDENT ::= [A-Za-z_][A-Za-z0-9_]* + UINT ::= [0-9]+ | "0x" [0-9A-Fa-f]+ ; up to 16 hex digits + IPV4 ::= UINT "." UINT "." UINT "." UINT ; each 0..255 + IPV6 ::= RFC 4291 / 5952 textual form, including the + embedded-IPv4 form ``::ffff:a.b.c.d`` + MAC ::= XX ":" XX ":" XX ":" XX ":" XX ":" XX + | XX "-" XX "-" XX "-" XX "-" XX "-" XX + | XXXX "." XXXX "." XXXX + HEXSTR ::= "0x" [0-9A-Fa-f]+ ; > 16 hex digits + STRING ::= '"' character* '"' ; \\ escapes recognised + +The grammar follows ``testpmd`` closely so that flow rules already +familiar to users carry over. The lexer is generated by flex from +``flow_compile.l`` and the parser by bison from ``flow_compile.y``; +both are reentrant (``%option reentrant``, ``%define api.pure``) +and share state through a per-compile context, so multiple +compilations may run concurrently. Neither depends on testpmd, +``rte_cmdline`` or ``cmdline_parse_*``. + +.. note:: + + This library implements a strict subset of testpmd's flow rule + syntax. Some forms accepted by testpmd are rejected here -- for + example, empty pattern or action lists, and quoted-string values + in contexts that have no custom setter wired up. Rules written + for testpmd should parse, but rules written for this library may + not be exercise-equivalent in testpmd if they rely on a + construct testpmd treats more permissively. + + +Field qualifier semantics +------------------------- + +For each parsed ``field qualifier value`` triple the compiler writes +into one or more of the spec/mask/last buffers. Semantics match +``testpmd``: + +.. list-table:: + :header-rows: 1 + :widths: 10 30 30 20 + + * - Qualifier + - spec + - mask + - last + * - ``is`` + - value + - all-ones over the field + - -- + * - ``spec`` + - value + - -- + - -- + * - ``mask`` + - -- + - value + - -- + * - ``last`` + - -- + - -- + - value + * - ``prefix`` + - -- + - high N bits set (CIDR style); IPv4/IPv6 only + - -- + +Last write wins. ``ipv4 src spec 10.0.0.0 src prefix 16`` therefore +matches the entire ``10.0.0.0/16`` range with mask ``255.255.0.0``; +``src is 10.0.0.0`` would have set the mask to all-ones, which is +exact match. + + +Diagnostics +----------- + +Errors are reported as ``LINE:COL: message`` in the caller-supplied +``errbuf`` of at least ``RTE_FLOW_COMPILE_ERRBUF_SIZE`` (256) bytes. +The first error wins; subsequent errors are suppressed so that the +user sees the original cause rather than a cascade. + +On failure ``rte_errno`` is set to ``EINVAL`` for parse errors and +``ENOMEM`` for allocation failures. + + +Extending the compiler +---------------------- + +The grammar itself is generic: it knows about attributes, items, +fields, actions and parameters, but has no per-type productions. +All per-type knowledge lives in descriptor tables. Adding a new +flow item type therefore requires no changes to ``flow_compile.l`` +or ``flow_compile.y``: + +#. In ``flow_compile_tables.c``, define a static + ``struct field_desc`` array describing the parsable fields of the + item's spec struct. +#. Add an ``ITEM(...)`` entry to ``flow_items[]``. + +Each ``field_desc`` lists the field's offset, byte width and a +``field_kind`` (``FK_U32``, ``FK_BE16``, ``FK_MAC``, ``FK_IPV4``, +``FK_IPV6``, ``FK_BYTES``, ...). Default setters honor every kind +and produce the correct byte order automatically. + +Fields whose layout cannot be expressed as a plain byte range +(C bitfields, indirect arrays, RSS keys, ...) are not currently +supported; the descriptor mechanism handles fixed-shape fields +only. + +Adding a new action type follows the same pattern with +``flow_actions[]`` and ``ACTION(...)``. + + +Source layout +------------- + +The library sits in ``lib/flow_compile`` and is split for clarity: + +================================ ================================== +File Contents +================================ ================================== +``rte_flow_compile.h`` Public API. +``flow_compile_priv.h`` Internal types: descriptors, + parser context, error helpers. +``flow_compile.l`` Flex lexer. Reentrant, with + source-position tracking via + ``YY_USER_ACTION`` for diagnostics. +``flow_compile.y`` Bison grammar. Pure parser; all + semantic actions delegate to the + setter helpers below. +``flow_compile_setters.c`` Default field setters for every + ``field_kind`` and the begin/end + helpers called from the grammar. +``flow_compile_tables.c`` Per-item and per-action descriptor + tables. All extension work + happens here. +``rte_flow_compile_api.c`` Public entry points: compile, + free, accessors, validate, create. +================================ ================================== + + +Implementation notes +-------------------- + +Locale independence + The flex regular expressions use byte-literal character classes + (``[0-9A-Fa-f]``, ``[A-Za-z_]``) rather than locale-aware + classifications, and address parsing goes through + ``inet_pton()`` and ``rte_ether_unformat_addr()`` which are + themselves locale-independent. The active locale therefore + cannot affect tokenisation. + +Endianness + All multibyte writes go through ``rte_cpu_to_be_{16,32,64}`` or + raw byte copies from already network-order tokens + (``TK_IPV4``, ``TK_MAC``, ``TK_IPV6``). + +Alignment + Spec and mask buffers may contain unaligned multibyte fields + inside packed-ish header structs. All writes go through + ``memcpy`` to handle this portably. + +Memory + All allocations go through ``rte_zmalloc`` and ``rte_free``. Each + spec, mask, last and conf payload is its own allocation; the + pattern and action arrays are grown with ``rte_realloc``, + doubling from an initial capacity of 8. + ``rte_flow_compile_free()`` walks the pattern and action arrays + and frees every non-NULL slot before freeing the arrays + themselves, so a partially compiled rule on a parse-error path + is cleaned up uniformly. + +Reentrancy + The parser holds no static mutable state. Multiple threads may + compile rules in parallel and a single compiled rule may be + installed concurrently on multiple ports. + + +Limitations +----------- + +The initial implementation covers the most common items +(``eth``, ``vlan``, ``ipv4``, ``ipv6``, ``tcp``, ``udp``, ``vxlan``, +``port_id``, ``port_representor``, ``represented_port``) and actions +(``drop``, ``passthru``, ``queue``, ``mark``, ``jump``, ``count``, +``port_id``, ``port_representor``, ``represented_port``, +``of_pop_vlan``, ``vxlan_decap``). Adding more is purely a matter +of extending the descriptor tables. + +Items and actions whose conf has a variable-length payload +(``RSS``, ``RAW``, the various ``RAW_ENCAP``/``RAW_DECAP`` actions) +are not yet supported. Adding them will require extending the +descriptor mechanism beyond fixed-shape fields. diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index e6f24945b0..3476dfecfd 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -121,6 +121,7 @@ Utility Libraries argparse_lib cmdline + flow_compile_lib ptr_compress_lib timer_lib rcu_lib diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst index f012d47a4b..addb9ff94b 100644 --- a/doc/guides/rel_notes/release_26_07.rst +++ b/doc/guides/rel_notes/release_26_07.rst @@ -64,6 +64,12 @@ New Features * ``--auto-probing`` enables the initial bus probing, which is the current default behavior. +* **Added library to compile flow definitions.** + + New library that works like libpcap ``pcap_compile`` function to compile + a text string into flow rules. + + Removed Items ------------- -- 2.53.0
