Early in the process of deciding how to make vpp multi-thread capable, I placed node subgraphs onto specific threads and handed off frames between threads. As expected, the resulting pipelines ran at the speed of the slowest stage.
When the workload changed, it was necessary to repartition the graph. Not so hard for a vpp expert, but the moment we let folks start playing with the code, we discovered it was simply not appropriate for use by ops folks. IMO trying to solve the pipeline partitioning problem algorithmically is probably not compatible with product-quality software. Imagine what happens if the offered load changes so frequently that the optimizer never converges. The current scheme – ingress load-balancing [typically via hardware RSS hashing] across identical worker thread / graph instances – is really simple to use and works well. If hardware RSS hashing doesn’t generate sufficient statistics, it’s possible to load-balance in software. Bottom line: you can try pipelining [as I did], but you probably won’t like the results [any better than I did]. All of the infra is there to make it easy enough to try; see the “handoff” node. Bottom line, and with apologies to George Lucas: “these are not the droids we’re looking for. You can go about your business. Move along…” HTH… Dave From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of raju Sent: Tuesday, May 8, 2018 9:41 AM To: vpp-dev <vpp-dev@lists.fd.io> Subject: [vpp-dev] VPP with pipe line processing. All, I tried to find this on VPP wiki and on web, but no luck hence question here. I wanted to try pipe line processing in VPP (each core doing specific job and passing on the packets further to next core for subsequent processing and finally send out). In other words, use case is multi threaded with Master core(as is today), I/O core and worker cores. Each worker core does specific job. Worker 1 does Job A and passes the packet to Worker 2 which does Job B and then transmits the packet out. And definitely I should be able to scale this (whole pipe line processing into multiple pipelines) by increasing the number of cores on the system. Does VPP support this ? How can I dedicate a worker core to execute specific nodes in the graph? How each worker can pass on packets to the next core in the pipeline for further processing? memif can be used or direct dpdk ring buffers can be used ? Thanks
