Hello Harry,
My concern is how to properly maintain Rust crate once DPDK starts to implement
it's own API.
I'm not really sure what is meant here. I don't understand what "own" word
refers to?
I see it like this:
- DPDK has the public C API exported (and that stays the same as today, with
ABI rules, version.map files, etc)
- The Rust crate consumes the public C API (via bindgen, as done in this patch.
More detail about bindgen below.)
Bindgen cannot provide access to all DPDK public API.
Ah - you're referring to C static inline functions, declared in header files,
which bindgen doesn't wrap.
Correct - thanks - I understand your point now.
A good example here is rte_eth_rx_burst().
That function is defined as inline and bindgen does not translate it.
Also, the function definition references rte_eth_fp_ops array that is not part
of the
public DPDK API. That means Rust cannot duplicate rte_eth_rx_burst() "as-is" and
the solution can require extensions to existing DPDK API.
I added a new public API that exports rte_eth_fp_ops for a given port Id.
Rust implementation of rte_eth_rx_burst() does not have to follow the original
approach.
Usage of rte_eth_fp_ops is good for C, but Rust has different methods.
Agreed there is a certain "mismatch" sometimes, if functions aren't in the
actually "C ABI" then they can't be called via bindgen.
Agree that elegant solutions (clean, maintainable, and high performance) will
have
to be found here. Many existing solutions just wrap the "static inline"
function into
a "non-static" function, and export it as a public symbol. That allows calling
into it
from Rust (via bindgen-generated header) however causes an actual function
call..
(LTO _might_ fix/inline it, but not everybody compiles with LTO.. link times!)
The core DPDK code is maintained as a self-contained pure C project.
What about extending that model and provide direct access to DPDK resources that
are needed for Rust API ?
That can be arranged as part of "native" DPDK API or as extension for
Rust-enabled DPDK only.
I'm currently experimenting with the latter in
https://github.com/getelson-at-mellanox/rdpdk/tree/main/dpdk-patches
As DPDK uses static-inline functions primarily for "packet-at-a-time"
performance reasons,
it is unfortunate to give-up (some small amounts of..?) performance by having a
C->Rust FFI call.
We have work to do to find/propose the best solution.
What exactly is a small amount of performance degradation ?
For some existing performance oriented projects loosing performance to
eliminate the `unsafe {}` construct is not an option.
For conclusion, Rust DPDK infrastructure cannot relay on bindgen only and needs
to provide native implementation for some public DPDK API.
It can be easier to maintain Rust files separately.
OK, now I understand your "kind-of-C-DPDK, kind of Rust-DPDK" code, or the
"own" code reference above.
I'm not sure right now where that would be best implemented/maintained, I'll
have to think about it and do a POC.
You can check out my Rust DPDK version here:
https://github.com/getelson-at-mellanox/rdpdk
Thanks for the link - I see you've been pushing code actively! Good to see
opportunities,
and compare approaches and concepts. Have you investigated wrapping the various
pointers
into structs, and providing safer APIs? For example, the [*mut rte_mbuf; 64]
array for RX causes
raw pointers to be handled for all packet-processing - resulting in "unsafe{ /* work
here */ }" blocks.
This construct can work:
struct PktsBuf<const SIZE: usize> {
buffer: [*mut rte_mbuf; SIZE]
}
The code in the above repo feels like "DPDK C code written in Rust".
Agree.
At this stage there is no native Rust DPDK API.
It is a great step towards better
understanding, and having something that works is very valuable; thanks for
sharing it.
A "top down" application view might help to brainstorm idiomatic/Safe Rust
APIs, and then we can
discuss/understand how to map these high level APIs onto the "DPDK C in Rust" or even
"DPDK C API/ABI" layers.
Does that seem like a good method to you, to achieve an ergonomic Safe Rust API
as the end result?
I think we need to conclude what the safe API means in terms of DPDK project.
Because it's possible to provide native Rust API for DPDK what will use FFI.
Specially, if Rust PMD is not in plans and performance is one of the main goals.
Next steps are to "allowlist" more DPDK public API functions, and start building
"Safe Rust" APIs over
them, in order to expose an ergonomic and misuse-resistant API. As you note,
this is where a network ports,
lcores, mempools, and ethdev configuration are all required. First goal something like
"Safe L2 macswap"?
check out for Rx, L2 addr swap, Tx sequence here:
https://github.com/getelson-at-mellanox/rdpdk/blob/253fde7c16a5514bf99d3823725f825262e244c8/app/runpmd/runpmd.rs#L156
Regards,
Gregory
