As Damjan wrote, the vpp graph dispatcher has known how to switch input nodes between interrupt and polling mode since before the dpdk even existed. Unfortunately, not all dpdk drivers support interrupt-mode processing, again, as discussed.
As a clock-cycle / power-consumption mitigation maneuver, we'll probably introduce a nanosleep(...) in either the dpdk input node or in epoll-based "pre-input" node. It's easy to check the current vector size and decide to take a break [or not]. We know from prior work that considerable hysteresis should be applied to the problem. For example: vector size > 20 => switch from "snoozy mode" to polling mode. Vector size < 10 => switch from polling to "snoozy mode". What we don't want is to oscillate between modes on every other frame. That would play hell with end-to-end jitter. Thanks… Dave -----Original Message----- From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Damjan Marion (damarion) Sent: Wednesday, January 25, 2017 4:38 AM To: Stephen Hemminger <step...@networkplumber.org> Cc: vpp-dev <vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] igb_uio -> uio_pci_generic > On 25 Jan 2017, at 06:00, Stephen Hemminger <step...@networkplumber.org> > wrote: > > On Tue, 24 Jan 2017 23:05:55 -0500 > Burt Silverman <bur...@gmail.com> wrote: > >> Hi Damjan, >> >> My understanding is that CONFIG_VFIO_NOIOMMU will never be set in a stock >> kernel, and you will need to build a custom kernel for that. I understand >> that with this option, the kernel cannot guarantee that applications are >> prevented from creating bugs that normally the kernel can guarantee will >> not occur (outside of a kernel bug.) It therefore violates the fundamental >> Linux system design. That being said, you may wish to accept the risk for >> performance reasons and build a custom kernel. The other strange thing >> would be that MSI or MSI-X style interrupts are not needed for performance. >> The people who developed them have made a lot of noise about how they came >> about for performance reasons. I have no direct experience, but to learn >> that they are not important is a shock. >> >> It seems to me that the Ubuntu 14.04 issue is really a separate one from >> all of this, although I would imagine that the conclusion to stop >> supporting it does not change. >> >> Burt >> > > The reality is any userspace I/O without IOMMU is insecure and can introduce > bugs. It was only when changes to UIO were proposed that the UIO maintainer > realized the problem and would not accept changes. The VFIO maintainer > was more enlightened "if you want to hang yourself, and you sign the > disclaimer, > here is a prettier rope”. Another reality is that on ubuntu systems we cannot use VFIO on systems without IOMMU. So the real choice is really between igb_uio and uio_pci_generic. Maybe we can convince ubuntu folks to enable vfio-noiommu in the next LTS, and then we can reconsider this decision. Please note that this is pure packaging problem, we are perfectly fine to use vfio-pci in the VPP, folks just need to enable iommu and change one line in vpp config file. > > MSI-X allows DPDK applications to be built with a hybrid polling model which > is better for power and CPU consumption. Unfortunately this is supportable > only > on some drivers, and configurations; plus from my terse reading of the FD.IO > code > it is not possible now. Pure polling is a great only if you don't have to ever > pay for power or CPU cycles. It sucks in virtual environments. Power consumption is valid point, and we have support for interrupt mode for many years in the codebase. If you look at src/vnet/devices/pci/ixge.c you will see our old native niantic driver which supports interrupt mode and it is able to dynamically switch between polling and interrupt mode based on load. At the moment when DPDK was integrated in VPP, we had to disable that for DPDK interfaces as PMDs were lacking interrupt mode. Now we can re-enable that but question is what to do when we have mix of PMDs and only some of the support interrupt mode. _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
