On 5/18/20 12:00 PM, Toke Høiland-Jørgensen wrote: > I meant 'less powerful' in the obvious sense: it only sees a subset of > the packets going out of the interface. And so I worry that it will (a) > make an already hard to use set of APIs even more confusing, and (b) > turn out to not be enough so we'll end up needing a "real" egress hook. > > As I said in my previous email, a post-REDIRECT hook may or may not be > useful in its own right. I'm kinda on the fence about that, but am > actually leaning towards it being useful; however, I am concerned that > it'll end up being redundant if we do get a full egress hook. >
I made the changes to mlx5 to run programs in the driver back in early March. I have looked at both i40e and mlx5 xmit functions all the way to h/w handoff to get 2 vendor perspectives. With xdp I can push any header I want - e.g., mpls - and as soon as I do the markers are wrong. Take a look at mlx5e_sq_xmit and how it gets the transport header offset. Or i40e_tso. Those markers are necessary for the offloads so there is no 'post skb' location to run a bpf program in the driver and have the result be sane for hardware handoff. [ as an aside, a co-worker just happened to hit something like this today (unrelated to xdp). He called dev_queue_xmit with a large, manually crafted packet and no skb markers. Both the boxes (connected back to back) had to be rebooted.] >From what I can see there are 3 ways to run an XDP program on skbs in the Tx path: 1. disable hardware offloads (which is nonsense - you don't disable H/W acceleration for S/W acceleration), 2. neuter XDP egress and not allow bpf_xdp_adjust_head (that is a key feature of XDP), or 3. walk the skb afterwards and reset the markers (performance killer). I have stared at this code for months; I would love for someone to prove me wrong.
