From: Owen Hilyard <ohily...@iol.unh.edu>
Currently, DTS uses Testpmd for most of its testing. This has been
successful in reducing the need to create more test apps, but it has a few
drawbacks. First, if some part of DPDK is not exposed via Testpmd or one of the
example applications, for the purposes of DTS it is not testable. This is a
situation I’d like to avoid. However, adding new functionality to Testpmd is
labor-intensive. Testpmd currently uses a hand-written LL(1) parser
(https://en.wikipedia.org/wiki/LL_parser) to parse command line options. This
makes adding new functionality difficult since the parser is stored as a series
of several thousand line long lookup tables. To look at it another way, 64% of
the 52238 lines in Testpmd are related to command line input in some way. The
command line interface of testpmd also presents several challenges for the
underlying implementation, since it requires that everything a user might want
to reference is identified via something that is reasonable to ask a user to
type. As of right now, this is handled via either strings or integers. This can
be handled by creating a global registry for objects, but it is still extra
work that I think can be avoided. In addition, this leads to more places where
things can go wrong.
This is what DTS running a single command in testpmd looks like right now:
https://drive.google.com/file/d/1hvTcjfVdh8-I3CUNoq6bx82EuNQSK6qW/view?usp=sharing
This approach has a number of disadvantages. First, it requires assembling
all commands as strings inside of the test suite and sending them through a
full round trip of SSH. This means that any non-trivial command, such as
creating an RTE flow, will involve a lot of string templating. This normally
wouldn’t be a big issue, except that some of the test suites are designed to
hundreds of commands over the course of a test, paying the cost of an SSH round
trip for each. Once Testpmd has the commands, it will then call the appropriate
functions inside of DPDK, and then print out all of the state to standard out.
All of this is sent back to DTS, where the author of the test case then needs
to handle all possible outputs of Trex, often by either declaring the presence
of a single word or short phrase in the output as meaning success or failure.
In my opinion, this is something that is perfectly fine for humans to interact
with, but it causes a lot of issues with automation due to its inherent
inflexibility and the less-than-ideal methods of information transfer. This is
why I am proposing the creation of an automation-oriented pmd, with a focus on
exposing as much.
https://drive.google.com/file/d/1wj4-RnFPVERCzM8b68VJswAOEI9cg-X8/view?usp=sharing
That diagram is a high-level overview of the design, which explicitly
excludes implementation details. However, it already has some benefits. First,
making DPDK do something is a normal method call, instead of needing to format
things into a string. This provides a much better interface for people working
in both DTS and DPDK. Second, the ability to return structured data means that
there won’t be parsers on both sides of communication anymore. Structured data
also allows much more verbosity, since it is no longer an interface designed
for humans. If a test case author needs to return the bytes of every received
packet back to DTS for comparison with the expected value, they can. If you
need to return a pointer for DTS to use later, that becomes reasonable. Simply
moving to shuffling structured data around and using RPC already provides a lot
of benefits.
The next obvious question would be what to use for the implementation.
The initial attempt was made using Python on both sides and the standard
library xmlrpc module. The RPC aspect of this approach worked very well, with
the ability to send arbitrary python objects back and forth between DTS and
app. However, having Python interacting with DPDK has a few issues. First, DPDK
is generally very multi-threaded and the most common implementation of Python,
CPython, does not have concurrency. It has something known as the global
interpretr lock, which is a global mutex. This makes it very difficult to
interact with blocking, multi-threaded code. The other issue is that I was not
able to find a binding generator that I feel would be sufficient for DPDK. Many
generators assumed sizeof(int) == 4 or had other portability issues such as
assuming GCC or Clang as a C compiler. Others focused on some subset of C,
meaning they would throw errors on alignment annotations.
Given this, I decided to look for cross-language RPC libraries. Although
libraries exist for performing xmlrpc in C, they generally appeared quite
difficult to use and required a lot of manual work. The next best option was
gRPC. gRPC allows using a simple language, protobuf, with a language extension
for rpc. It provides code generation to make it easy to use multiple languages
together, since it was developed to make polyglot microservice interaction
easier. The only drawback is that it considers C++ good enough for C support.
In this case, I was able to easily integrate DPDK with C++, so that isn’t much
of a concern. I used C++17 in the attached patches, but the minimum
requirements are C++11. If there is concern about modern C++ causing too much
mental overhead, a “C with classes” subset of C++ could easily be used. I also
added an on-by-default option to use a C++ compiler, allowing anyone who does
not have a C++ compiler available to them to turn off everything that uses C++.
This disables the application written for this RFC.
One of the major benefits of gRPC is the asynchronous API. This allows
streaming data on both sides of an RPC call. This allows streaming logs back to
DTS, streaming large amounts of data from low-memory systems back to DTS for
processing, and would allow DTS to easily feed DPDK data, ending the test
quickly on a failure. Currently, due to the overhead of sending data to
Testpmd, it is common to just send all of the commands over and run everything
since that will be much faster when the test passes, but it can cost a lot of
time in the event of a failure. There are also optional security features for
requiring authentication before allowing code execution. I think that a
discussion on whether using them for DTS is necessary is warranted, although I
personally think that it’s not worth the effort given the type of environment
this sample application is expected to run in.
For this RFC, I ported test-acl because it was mostly self-contained and
was something I could run on my laptop. It should be fairly easy to see how you
would expand this proof of concept to cover more of DPDK, and I think that most
of the functions currently used in testpmd could be ported over to this
approach, saving a lot of development time. However, I would like to see some
more interest before I take on a task like that. This will require a lot of
work on the DTS side to implement, but it will make it much easier to add new
features to DTS.
Owen Hilyard (4):
app/test-pmd-api: Add C++ Compiler
app/test-pmd-api: Add POC with gRPC deps
app/test-pmd-api: Add protobuf file
app/test-pmd-api: Implementation files for the API
app/meson.build | 17 +
app/test-pmd-api/api.proto | 12 +
app/test-pmd-api/api_impl.cc | 1160 ++++++++++++++++++++++++++++++++++
app/test-pmd-api/api_impl.h | 10 +
app/test-pmd-api/main.c | 11 +
app/test-pmd-api/meson.build | 96 +++
meson.build | 3 +
meson_options.txt | 2 +
8 files changed, 1311 insertions(+)
create mode 100644 app/test-pmd-api/api.proto
create mode 100644 app/test-pmd-api/api_impl.cc
create mode 100644 app/test-pmd-api/api_impl.h
create mode 100644 app/test-pmd-api/main.c
create mode 100644 app/test-pmd-api/meson.build
--
2.30.2
