On 2019-04-26 13:42, Maxim Mikityanskiy wrote:
Currently, the drivers that implement AF_XDP zero-copy support (e.g.,
i40e) switch the channel into a different mode when an XSK is opened. It
causes some issues that have to be taken into account. For example, RSS
needs to be reconfigured to skip the XSK-enabled channels, or the XDP
program should filter out traffic not intended for that socket and
XDP_PASS it with an additional copy. As nothing validates or forces the
proper configuration, it's easy to have packets drops, when they get
into an XSK by mistake, and, in fact, it's the default configuration.
There has to be some tool to have RSS reconfigured on each socket open
and close event, but such a tool is problematic to implement, because no
one reports these events, and it's race-prone.
This commit extends XSK to support both kinds of traffic (XSK and
non-XSK) in the same channel. It implies having two RX queues in
XSK-enabled channels: one for the regular traffic, and the other for
XSK. It solves the problem with RSS: the default configuration just
works without the need to manually reconfigure RSS or to perform some
possibly complicated filtering in the XDP layer. It makes it easy to run
both AF_XDP and regular sockets on the same machine. In the XDP program,
the QID's most significant bit will serve as a flag to indicate whether
it's the XSK queue or not. The extension is compatible with the legacy
configuration, so if one wants to run the legacy mode, they can
reconfigure RSS and ignore the flag in the XDP program (implemented in
the reference XDP program in libbpf). mlx5e will support this extension.
A single XDP program can run both with drivers supporting or not
supporting this extension. The xdpsock sample and libbpf are updated
accordingly.
Signed-off-by: Maxim Mikityanskiy<maxi...@mellanox.com>
I acknowledge the problem you're addressing, but I'm not a fan of this
design.
Netdevs, queues and now channels. This is too complicated. The setup is
complex as it is. Adding more cognitive load is not good.
There's already an assumption that a user attaching to a certain netdev
queue has to take care of the flow steering.
What about adding a mode where the AF_XDP user just get a *new* queue,
by binding to (say) netdev;qid==-1? This new queue wouldn't be included
in the existing queue set. Create a bunch of AF_XDP sockets and steer
traffic to that set. And yes, we need a mechanism to steer to that
queue, in-app/socket or out-of-band. Please see this [1] discussion.
I don't like this "let's split a queue into channels".
Björn
[1] https://lore.kernel.org/netdev/20190415183258.36dcee9a@carbon/
