Hi Christophe,
Please see inline comments
Thanks,
Nitin
On Thu, Oct 17, 2024 at 2:02 PM Christophe Fontaine <cfont...@redhat.com> wrote:
>
> Hi all,
>
> What about the following steps:
> - update the nodes so they work on the current layer (example: for all L3
> nodes, the current mbuf data offset *must* be pointing to the IP header)
Agreed. It would be better if nodes uses
rte_pktmbuf_[append()/shrink() etc..] APIs to manipulate layer data
offset
> - define a public data structure that would be shared across nodes through
> priv data, and not dynfields ?
Eventually public data structures should be defined to serve *a
purpose*. Do you refer to creating a generic public structure? If yes,
IMO, it may not be tuned for performance
IMO, we need to create public structures for each specific purpose.
Feature arc is also a public data structure which optimally saves
following variables in 8 byte compact structure
(rte_graph_feature_daa_t) for every interface
- rte_edge_t (uint16_t)
- next enabled feature (uint8_t) per index (from current node)
- Per interface feature specific user_data (uint32_t)
Due to its compact nature, 8 such objects per interface can be saved
in one 64B cache line. So IMO, it is better to create public
structures for a given purpose and optimally define them fields and
APIs.
Also feature arc specific 3B data is saved in mbuf dynfield. Hard to
say if priv data would provide a better solution.
> This structure would be the "internal api" (so, that has to be tracked across
> dpdk releases) between nodes.
> We’d need common data shared for all the nodes as well as specific data
> between 2 nodes.
> As we get to this point, this (hopefully) will help with the node reusability.
Feature arc also maintains data between 2 nodes per interface and also
for all nodes which are added as features.
>
> - Update the feature arcs to leverage this well known structure, and refine
> the api
> - Define which part of the stack needs to be defined as a feature arc, with
> the benefit of the generic API to enable/disable that feature, and which part
> needs to be dynamically pluggable.
> For instance, for a router, it may not make sense to define IPv4 support as a
> feature arc.
> So, we’d statically connect eth_input to ip_input.
Agreed
> Yet, lldp support is a good candidate for a feature arc: we need to configure
> it per interface, and this is independent of the main graph.
>
There would be more protocols which need to be enabled per interface
> WDYT?
> Christophe
>
> > On 17 Oct 2024, at 09:50, Robin Jarry <rja...@redhat.com> wrote:
> >
> > Hi Nitin, all,
> >
> > Nitin Saxena, Oct 17, 2024 at 09:03:
> >> Hi Robin/David and all,
> >>
> >> We realized the feature arc patch series is difficult to understand as a
> >> new concept. Our objectives are following with feature arc changes
> >>
> >> 1. Allow reusability of standard DPDK nodes (defined in lib/nodes/*)
> >> with out-of-tree applications (like grout). Currently out-of-tree graph
> >> applications are duplicating standard nodes but not reusing the
> >> standard ones which are available. In the long term, we would like to
> >> mature standard DPDK nodes with flexibility of hooking them to
> >> out-of-tree application nodes.
> >
> > It would be ideal if the in-built nodes could be reused. When we started
> > working on grout, I tried multiple approaches where I could reuse these
> > nodes, but all failed. The nodes public API seems tailored for app/graph
> > but does not fit well with other control plane implementations.
> >
> > One of the main issues I had is that the ethdev_rx and ethdev_tx nodes are
> > cloned per rxq / txq associated with a graph worker. The rte_node API
> > requires that every clone has a unique name. This in turn makes hot
> > plugging of DPDK ports very complex, if not impossible.
> >
> > For example, with the in-built nodes, it is not possible to change the
> > number of ports or their number of RX queues without destroying the whole
> > graph and creating a new one from scratch.
> >
> > Also, the current implementation of "ip{4,6}-rewrite" handles writing
> > ethernet header data. This would prevent it from using this node for an
> > IP-in-IP tunnel interface as we did in grout.
> >
> > Do you think we could change the in-built nodes to enforce OSI layer
> > separation of concerns? It would make them much more flexible. It may cause
> > a slight drop of performance because you'd be splitting processing in two
> > different nodes. But I think flexibility is more important. Otherwise, the
> > in-built nodes can only be used for very specific use-cases.
> >
> > Finally, I would like to improve the rte_node API to allow defining and
> > enforcing per-packet metadata that every node expects as input. The current
> > in-built nodes rely on mbuf dynamic fields for this but this means you only
> > have 9x32 bits available. And using all of these may break some drivers
> > (ixgbe) that rely on dynfields to work. Have you considered using mbuf
> > private data for this?
> >
> >>
> >> 2. Flexibility to enable/disable sub-graphs per interface based on the
> >> runtime configuration updates. Protocol sub-graphs can be selectively
> >> enabled for few (or all interfaces) at runtime
> >>
> >> 3. More than one sub-graphs/features can be enabled on an interface. So
> >> a packet has to follow a sequential ordering node path on worker cores.
> >> Packets may need to move from one sub-graph to another sub-graph per
> >> interface
> >>
> >> 4. Last but not least, an optimized implementation which does not (or
> >> minimally) stop worker cores for any control plane runtime updates. Any
> >> performance regression should also be avoided
> >>
> >> I am planning to create a draft presentation on feature arc which I can
> >> share, when ready, to discuss. If needed, I can also plan to present that
> >> in one of the DPDK community meetings. Their we can also discuss if there
> >> are any alternatives of achieving above objectives
> >
> > Looking forward to this.
> >
> > Thanks!
> >
>
