On Thu, Feb 16, 2023 at 03:01:28PM +0000, Van Haaren, Harry wrote:
> > -----Original Message-----
> > From: Liang Ma <lian...@liangbit.com>
> > Sent: Thursday, February 16, 2023 2:40 PM
> > To: Sam Kirubakaran <samkirubakar...@gmail.com>
> > Cc: dev@dpdk.org
> > Subject: Re: TCP stack support on DPDK
> >
> > On Tue, Jan 24, 2023 at 01:24:38PM +0530, Sam Kirubakaran wrote:
> > > Hi Liang,
> > >
> > > Thank you so much for your response.
> > > I would like to convert our in-house tool into a high performance traffic
> > > generator and it is written in Python.
> > > Is it possible to use any of the user space TCP stack by compiling into
> > > shared libraries and access it via Python using ctypes?
> > > Could you please let me know any viable solution to make use of user space
> > > TCP stack compatible with Python?
>
> Liang replied:
> > sorry, I have no idea bout the python bind with DPDK, but there are some
> > Lua bind with DPDK. Like Moonwire.
>
> Moongen: https://github.com/emmericp/MoonGen and its "library form"
> Libmoon may be of interest yes! (https://github.com/libmoon/libmoon)
>
> Note that LUA has certain properties that are nice for packet processing
> (soft-realtime capable).
> Python unfortunately does not have the same properties by default (but likely
> can be configured/used/extended to mitigate impact).
> This is only relevant if you want to generate packets "on the fly" with very
> low jitter (potentially caused by GarbageCollectors from Python runtimes).
+1
> Looking at the bigger picture:
> 1) What are your requirements around performance for generating traffic?
> 2) And what are the reasons that existing projects are not sufficient?
> Trex is very configurable for many things:
> https://github.com/cisco-system-traffic-generator
>
> As you likely know, writing a good reliable traffic generation tool is a lot
> of work,
> and re-using existing work is likely a good idea. There is a list of DPDK
> users/libraries etc on the
> official DPDK site here; https://www.dpdk.org/ecosystem/ Some of these
> libraries might
> provide a good starting point for your needs?
>
> My experience when developing a simple/minimal traffic generator for DPDK was
> that although
> It might seem that generating of packets can be done in
> Python/other-interpreted-language, it
> causes performance issues in the end. Imagine generating 1000 flows, the
> checksum of each
> packet is going to be different. That leaves two options:
> 1) Generate all packets in advance, and "replay" them from memory (memory
> intensive, and no other customization potential as it corrupts checksums!)
> 2) Calculate checksums on the fly in C or other high performance language:
> the complexity of checksum calculations, offsets etc must be present in
> high-perf language, removes benefit of using Python/high-level language for
> "quick" packet generation.
>
> The results of the Proof-of-Concept that Ronan Randles & I done is here,
> called TGen. I have
> Presented my experience of "handmade" traffic generators as part of DPDK
> userspace:
> - Video: https://www.youtube.com/watch?v=Djcjq59H1uo
> - Patches:
> http://patches.dpdk.org/project/dpdk/cover/20220121103122.2926856-1-ronan.rand...@intel.com/
>
> Hope the above information is of use! Regards, -Harry
>
