Hi Bruce, > -----Original Message----- > From: Richardson, Bruce <bruce.richard...@intel.com> > Sent: Friday, June 10, 2022 11:36 PM > To: dev@dpdk.org > Cc: Maxime Coquelin <maxime.coque...@redhat.com>; Xia, Chenbo > <chenbo....@intel.com>; Richardson, Bruce <bruce.richard...@intel.com>; > sta...@dpdk.org > Subject: [PATCH v3 1/4] doc/howto: rework section on virtio-user as > exception path > > This patch extensively reworks the howto guide on using virtio-user for > exception packets. Changes include: > > * rename "exceptional path" to "exception path" > * remove references to uio and just reference vfio-pci > * simplify testpmd command-lines, giving a basic usage example first > before adding on detail about checksum or TSO parameters > * give a complete working example showing traffic flowing through the > whole system from a testpmd loopback using the created TAP netdev > * replace use of "ifconfig" with Linux standard "ip" command > * other general rewording. > > CC: sta...@dpdk.org
Not sure you want to backport or not as there's no fixes tag. For this patch: Acked-by: Chenbo Xia <chenbo....@intel.com> > > Signed-off-by: Bruce Richardson <bruce.richard...@intel.com> > > --- > V3: > * fix error reported by Chenbo on review. > * add stable on CC, since this rework could be applicable for older > releases too, if desired for backport. > --- > .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++------- > 1 file changed, 100 insertions(+), 59 deletions(-) > > diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst > b/doc/guides/howto/virtio_user_as_exceptional_path.rst > index ec021af399..3d3d1b1219 100644 > --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst > +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst > @@ -3,116 +3,157 @@ > > .. _virtio_user_as_exceptional_path: > > -Virtio_user as Exceptional Path > -=============================== > +Virtio_user as Exception Path > +============================= > > -The virtual device, virtio-user, was originally introduced with vhost- > user > -backend, as a high performance solution for IPC (Inter-Process > Communication) > +.. note:: > + > + This solution is only applicable to Linux systems. > + > +The virtual device, virtio-user, was originally introduced with the > vhost-user > +backend as a high performance solution for IPC (Inter-Process > Communication) > and user space container networking. > > -Virtio_user with vhost-kernel backend is a solution for exceptional path, > -such as KNI which exchanges packets with kernel networking stack. This > -solution is very promising in: > +Beyond this originally intended use, virtio-user can be used in > conjunction with the vhost-kernel > +backend as a solution for dealing with exception path packets which need > to be injected into the > +Linux kernel for processing there. > +In this regard, virtio-user and vhost in kernel space are an alternative > to DPDK KNI for > +transferring packets between a DPDK packet processing application and the > kernel stack. > + > +This solution has a number of advantages over alternatives such as KNI: > > * Maintenance > > All kernel modules needed by this solution, vhost and vhost-net > (kernel), > - are upstreamed and extensively used kernel module. > + are upstreamed and extensively used. > > * Features > > - vhost-net is born to be a networking solution, which has lots of > networking > - related features, like multi queue, tso, multi-seg mbuf, etc. > + vhost-net is designed to be a networking solution, and, as such, has > lots of networking > + related features, such as multi queue support, TSO, multi-segment > buffer support, etc. > > * Performance > > - similar to KNI, this solution would use one or more kthreads to > - send/receive packets to/from user space DPDK applications, which has > little > - impact on user space polling thread (except that it might enter into > kernel > - space to wake up those kthreads if necessary). > + similar to KNI, this solution would uses one or more kthreads to > + send/receive packets to/from user space DPDK applications, which > minimises the impact > + on the polling DPDK threads. > > -The overview of an application using virtio-user as exceptional path is > shown > +The overview of an application using virtio-user as exception path is > shown > in :numref:`figure_virtio_user_as_exceptional_path`. > > .. _figure_virtio_user_as_exceptional_path: > > .. figure:: img/virtio_user_as_exceptional_path.* > > - Overview of a DPDK app using virtio-user as exceptional path > + Overview of a DPDK app using virtio-user as exception path > + > > +Example Usage With Testpmd > +--------------------------- > > -Sample Usage > ------------- > +.. note:: > + > + These instruction assume that the vhost/vhost-net kernel modules are > available and have already > + been loaded into the running kernel. > + It also assumes that the DPDK virtio driver has not been disabled in > the DPDK build. > > -As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen > before > -compiling the kernel and those kernel modules should be inserted. > +To run a simple test of virtio-user as exception path using testpmd: > > -#. Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio- > pci. > +#. Compile DPDK and bind a NIC to vfio-pci as documented > in :ref:`linux_gsg_linux_drivers`. > > - This physical NIC is for communicating with outside. > + This physical NIC is for communicating with the outside world, > + and serves as a packet source in this example. > > -#. Run testpmd. > +#. Run testpmd to forward packets from NIC to kernel, > + passing in a suitable list of logical cores to run on (``-l`` > parameter), > + and optionally the PCI address of the physical NIC to use (``-a`` > parameter). > + The virtio-user device for interfacing to the kernel is specified via > a ``-vdev`` argument, > + taking the parameters described below. > > .. code-block:: console > > - $(testpmd) -l 2-3 -n 4 \ > - --vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \ > - -- -i --tx-offloads=0x0000002c --enable-lro \ > - --txd=1024 --rxd=1024 > + /path/to/dpdk-testpmd -l <cores> -a <pci BDF> \ > + --vdev=virtio_user0,path=/dev/vhost- > net,queues=1,queue_size=1024 > > - This command runs testpmd with two ports, one physical NIC to > communicate > - with outside, and one virtio-user to communicate with kernel. > + * ``path`` > > -* ``--enable-lro`` > + The path to the kernel vhost-net device. > > - This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and > - VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can > be > - transmitted to DPDK application and further TSOed by physical NIC. > + * ``queue_size`` > > -* ``queue_size`` > + 256 by default. To avoid shortage of descriptors, we can increase > it to 1024. > > - 256 by default. To avoid shortage of descriptors, we can increase it > to 1024. > + * ``queues`` > > -* ``queues`` > + Number of virt-queues. Each queue will be served by a kthread. > > - Number of multi-queues. Each queue will be served by a kthread. For > example: > +#. Once testpmd is running, a new network interface - called ``tap0`` by > default - > + will be present on the system. > + This should be configured with an IP address and then enabled for use: > > .. code-block:: console > > - $(testpmd) -l 2-3 -n 4 \ > - --vdev=virtio_user0,path=/dev/vhost- > net,queues=2,queue_size=1024 \ > - -- -i --tx-offloads=0x0000002c --enable-lro \ > - --txq=2 --rxq=2 --txd=1024 --rxd=1024 > + ip addr add 192.168.1.1/24 dev tap0 > + ip link set dev tap0 up > > -#. Enable Rx checksum offloads in testpmd: > +#. To observe packet forwarding through the kernel, > + a second testpmd instance can be run on the system, > + taking packets from the kernel using an ``af_packet`` socket on the > ``tap0`` interface. > > - .. code-block:: console > + .. code-block:: console > > - (testpmd) port stop 0 > - (testpmd) port config 0 rx_offload tcp_cksum on > - (testpmd) port config 0 rx_offload udp_cksum on > - (testpmd) port start 0 > + /path/to/dpdk-testpmd -l <cores> --vdev=net_af_packet0,iface=tap0 > --in-memory --no-pci > > -#. Start testpmd: > + When running this instance, > + we can use ``--in-memory`` flag to avoid hugepage naming conflicts > with the previous instance, > + and we also use ``--no-pci`` flag to only use the ``af_packet`` > interface for all traffic forwarding. > > - .. code-block:: console > +#. Running traffic into the system through the NIC should see that > traffic returned back again, > + having been forwarded through both testpmd instances. > + This can be confirmed by checking the testpmd statistics on testpmd > exit. > > - (testpmd) start > +For more advanced use of virtio-user with testpmd in this scenario, > +some other more advanced options may also be used. > +For example: > > -#. Configure IP address and start tap: > +* ``--tx-offloads=0x02c`` > + > + This testpmd option enables TX offloads for UDP and TCP checksum on > transmit, > + as well as TCP TSO support. > + The list of the offload flag values can be seen in header > `rte_ethdev.h > + <https://doc.dpdk.org/api/rte__ethdev_8h.html>`_. > + > +* ``--enable-lro`` > + > + This testpmd option is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and > + VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from the kernel > can be > + transmitted to the DPDK application and further TSOed by physical NIC. > + If unsupported by the physical NIC, errors may be reported by testpmd > with this option. > + > + > +* Enabling Rx checksum offloads for physical port: > + > + Within testpmd, you can enable and disable offloads on a per-port > basis, > + rather than enabling them for both ports. > + For the physical NIC, it may be desirable to enable checksum offload > on packet RX. > + This may be done as below, if testpmd is run with ``-i`` flag for > interactive mode. > > .. code-block:: console > > - ifconfig tap0 1.1.1.1/24 up > + testpmd> port stop 0 > + testpmd> port config 0 rx_offload tcp_cksum on > + testpmd> port config 0 rx_offload udp_cksum on > + testpmd> port start 0 > > -.. note:: > +* Multiple queue support > > - The tap device will be named tap0, tap1, etc, by kernel. > + Better performance may be achieved by using multiple queues, > + so that multiple kernel threads are handling the traffic on the > kernel side. > + For example, to use 2 queues on both NIC and virtio ports, > + while also enabling TX offloads and LRO support: > > -Then, all traffic from physical NIC can be forwarded into kernel stack, > and all > -traffic on the tap0 can be sent out from physical NIC. > + .. code-block:: console > > -Limitations > ------------ > + /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost- > net,queues=2,queue_size=1024 -- \ > + -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 -- > txd=1024 --rxd=1024 > > -This solution is only available on Linux systems. > -- > 2.34.1
