st 6. 5. 2026 v 5:33 odesílatel Stephen Hemminger < [email protected]> napsal:
> Background > ---------- > > Multiple efforts over the past few cycles have tried to make > testpmd's flow rule grammar reusable from outside testpmd. > External applications that need rte_flow want a documented way > to turn human-written rules into the rte_flow_attr/item/action > arrays accepted by rte_flow_create(). > > The most recent attempt is Lukas Sismis's series, currently at > v12: > > http://patches.dpdk.org/project/dpdk/list/?series=37384 (or > most recent thread on [email protected]) > > That series factors testpmd's existing cmdline_flow.c into a > library and updates testpmd to consume it. It works, but > inherits two properties of cmdline_flow.c that I think are worth > avoiding in a reusable library: > > - Coupling to librte_cmdline. Even after the v12 split into > a "simple" part and a "cmdline" part, the parser is still > organized around testpmd's command interpreter, and v12 has > cmdline depending on ethdev to break a previous circular > dependency. A library used by daemons, control planes, or > unit tests should not need that. > > - Ad-hoc grammar. cmdline_flow.c implements parsing per-token > in long dispatch logic; the grammar emerges from the code > rather than being stated, and adding a new flow item > requires touching the parser. > > This RFC explores a different shape and is posted to ask the > list which one is preferred before more work goes into either. > > So I started a new green field library for parsing flow rules > (with the help of AI assistance). It is only a few hours old, > but passes tests and passes review. > > This series > ----------- > > lib/flow_compile -- a small new library providing the same > service via a pcap_compile()-style API: > > char errbuf[RTE_FLOW_COMPILE_ERRBUF_SIZE]; > struct rte_flow_compile *fc = rte_flow_compile(rule, errbuf); > if (fc == NULL) > fail(errbuf); /* "line:col: message" */ > > rte_flow_compile_create(port_id, fc, &flow_error); > rte_flow_compile_free(fc); > > Design properties: > > - Hand-rolled lexer + recursive descent parser. No flex/bison. > - Parser is driven entirely by descriptor tables of items and > actions. Adding a new flow item is a table edit, not a > parser change. A custom-setter hook on each field covers > the layouts that do not fit a plain byte range (bitfields, > indirect arrays). > - Dependencies: rte_ethdev and rte_net only. No librte_cmdline, > no flex/bison. Builds clean on Linux, FreeBSD, and Windows. > - Per-allocation rte_zmalloc for spec/mask/last/conf payloads; > rte_flow_compile_free() walks the pattern and action arrays. > ASan/LSan run clean on the autotest. > > The grammar follows testpmd's syntax closely so familiar rules > carry over: > > ingress pattern eth / ipv4 src is 10.0.0.1 / end > actions queue index 3 / count / end > > and is documented as a formal BNF in the programmer's guide > chapter (patch 2). > > Initial coverage: eth, vlan, ipv4, ipv6, tcp, udp, vxlan, > port_id, port_representor, represented_port items; drop, > passthru, queue, mark, jump, count, port_id and representor > variants, of_pop_vlan, vxlan_decap actions. Variable-conf > items and actions (RSS, RAW) require custom setters and are > deferred to a follow-up. > > What this RFC is *not* > ---------------------- > > Not a replacement for cmdline_flow.c in testpmd. If the shape > here is acceptable, the next step is a separate series adding a > "flow compile <port> <rule>" command in testpmd alongside the > existing parser, so users can adopt the library incrementally > without breaking scripts that depend on the current syntax. > > What I'd like feedback on > ------------------------- > > Hi Stephen, Thank you for looking into this. I genuinely appreciate it. I found this almost by accident, so here are some of my answers: > 1. API shape. pcap_compile-style (one string -> opaque object -> > arrays) versus the three-call attr/pattern/actions form > Sismis's v12 exposes. What does your application actually > want? > Generally, I am fine with pcap_compile style as per your suggestions. The original intent was to only use traffic patterns because everything else was set by the application itself. You can see an example of the use here: https://github.com/OISF/suricata/pull/15290/changes#diff-4d5f5239a7124ed4b4d45501788bddffe803911f19c157e3c4ac7676a84d68b1R301 That is why I divided the simple API into 3 calls -- to let the application decide which part of the flow expression to parse. But a single entry point to the library is fine too, as the application could just fill it into the desired rule. 2. Library placement. Stand-alone at lib/flow_compile/ versus > addition to lib/ethdev. This series treats it as a > control-path parser layered on top of ethdev rather than > part of ethdev itself; v12 places its parser inside ethdev. > My general take on this is that it can be so useful for developers, so it should be part of the lib/ethdev to make it more visible. As highlighted by Thomas, it heavily interacts with the rte_flow objects. But, as it probes for the flex/bison tools and skips the library if they are unavailable, wouldn't it be a problem if it is part of lib/ethdev directly? This can be a very strong argument for a separate library, then. > > 3. Table-driven extension model. Is "to add a new flow item, > add a row to the descriptor table" the right contract? > Should the tables live alongside each rte_flow_item_* > definition in rte_flow.h, or in their own file as here? > 4. Convergence. If this design is preferred, I'm happy to > coordinate with Lukas to fold in the testpmd-side changes > from his series. > (My) original idea was to reuse one parser for both the testpmd and the library to avoid multiple implementations of essentially the same thing. How would "flow compile" be useful if testpmd already has the "flow create" command? Perhaps as a library demo only? I would expect testpmd users to like the `flow create` endpoint more for its more comprehensive support. Similarly, at this moment, since testpmd supports interactivity, I cannot see how the `_compile` API would replace functions in testpmd. But perhaps it would be part of a greater effort to migrate to flex/bison solution. Since this is also a simple, single-rule, stateless parser, it is not fully compatible with all of TestPMD's commands (e.g., "set raw_encap"). (Ok, noticed, you highlighted this) Perhaps it is fine, just wanted to highlight this. >From the user-as-an-engineer's perspective, I would be generally happy for this parser to exist. The drop-filter feature in Suricata, in my view, requires just a simple network pattern specification so looking at the supported patterns already ticks off most of the items I've had in mind, except, e.g., MPLS. I can also see this as a viable way for Suricata user-specified decap options for better applicability of RSS. > 5. API code. Is it readable enough. Lots of code here > but doing lexer/parser is prime template territory > and AI has a good chance here. > > Stephen Hemminger (3): > flow_compile: introduce textual flow rule compiler > doc: add programmer's guide for flow rule compiler > test/flow_compile: add unit tests for flow rule compiler > > MAINTAINERS | 8 + > app/test/meson.build | 2 + > app/test/test_flow_compile.c | 230 +++ > doc/guides/prog_guide/flow_compile_lib.rst | 170 ++ > doc/guides/prog_guide/index.rst | 1 + > doc/guides/rel_notes/release_26_07.rst | 5 + > lib/flow_compile/flow_compile_lex.c | 488 +++++ > lib/flow_compile/flow_compile_parse.c | 634 ++++++ > lib/flow_compile/flow_compile_priv.h | 181 ++ > lib/flow_compile/flow_compile_tables.c | 245 +++ > lib/flow_compile/meson.build | 15 + > lib/flow_compile/rte_flow_compile.h | 158 ++ > lib/flow_compile/rte_flow_compile_api.c | 132 ++ > lib/flow_compile/version.map | 13 + > lib/meson.build | 1 + > 15 files changed, 2283 insertions(+) > create mode 100644 app/test/test_flow_compile.c > create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst > create mode 100644 lib/flow_compile/flow_compile_lex.c > create mode 100644 lib/flow_compile/flow_compile_parse.c > create mode 100644 lib/flow_compile/flow_compile_priv.h > create mode 100644 lib/flow_compile/flow_compile_tables.c > create mode 100644 lib/flow_compile/meson.build > create mode 100644 lib/flow_compile/rte_flow_compile.h > create mode 100644 lib/flow_compile/rte_flow_compile_api.c > create mode 100644 lib/flow_compile/version.map > > -- > 2.43.7 > > > Stephen Hemminger (3): > flow_compile: introduce textual flow rule compiler > doc: add programmer's guide for flow rule compiler > test/flow_compile: add unit tests for flow rule compiler > > MAINTAINERS | 7 + > app/test/meson.build | 1 + > app/test/test_flow_compile.c | 234 ++++++++ > doc/guides/prog_guide/flow_compile_lib.rst | 272 +++++++++ > doc/guides/prog_guide/index.rst | 1 + > doc/guides/rel_notes/release_26_07.rst | 6 + > lib/flow_compile/flow_compile_lex.c | 510 +++++++++++++++++ > lib/flow_compile/flow_compile_parse.c | 634 +++++++++++++++++++++ > lib/flow_compile/flow_compile_priv.h | 181 ++++++ > lib/flow_compile/flow_compile_tables.c | 245 ++++++++ > lib/flow_compile/meson.build | 15 + > lib/flow_compile/rte_flow_compile.h | 158 +++++ > lib/flow_compile/rte_flow_compile_api.c | 132 +++++ > lib/meson.build | 1 + > 14 files changed, 2397 insertions(+) > create mode 100644 app/test/test_flow_compile.c > create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst > create mode 100644 lib/flow_compile/flow_compile_lex.c > create mode 100644 lib/flow_compile/flow_compile_parse.c > create mode 100644 lib/flow_compile/flow_compile_priv.h > create mode 100644 lib/flow_compile/flow_compile_tables.c > create mode 100644 lib/flow_compile/meson.build > create mode 100644 lib/flow_compile/rte_flow_compile.h > create mode 100644 lib/flow_compile/rte_flow_compile_api.c > > -- > 2.53.0 > >
