As before, comments inline. On Fri, Nov 03, 2023 at 12:01:49AM -0400, David Young wrote: > --- > .../getting_started_guide/system_setup.rst | 195 ++++++++++++++++++ > 1 file changed, 195 insertions(+) > create mode 100644 doc/guides/getting_started_guide/system_setup.rst > > diff --git a/doc/guides/getting_started_guide/system_setup.rst > b/doc/guides/getting_started_guide/system_setup.rst > new file mode 100644 > index 0000000000..fa9d249ec7 > --- /dev/null > +++ b/doc/guides/getting_started_guide/system_setup.rst > @@ -0,0 +1,195 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2010-2025 Intel Corporation. > + > +.. _memory_setup: > + > +.. |reg| unicode:: U+000AE
Better to get this character via ".. include:: <isonum.txt>". That is what other files do. > + > +Setting up a System to Run DPDK Applications > +============================================ > + > +This section provides step-by-step instructions for setting up your system > to run DPDK applications. It covers system configurations for Linux, FreeBSD, > and Windows. Each section details the necessary memory and device setups for > these operating systems. > + > +Navigate to the section that corresponds to your operating system to begin > the setup process. > + > +.. contents:: Table of Contents > + :local: > + > +System Setup for Linux > +---------------------- > + > +Memory Setup: Reserving Hugepages > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +For Linux, DPDK requires hugepages be reserved for its use on the system. To > check if hugepages are are on your system, you can run the following command:: "are are" probably should be "are reserved". > + > + grep HugePages_Total /proc/meminfo > + > +If hugepages are not reserved, you will need to reserve them by following > these steps: > + > +1. Determine the number of hugepages you want to allocate. For example, to > allocate 1024 hugepages of 2MB each, you can use the following command:: > + > + echo 1024 | sudo tee > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > + > +2. To make the hugepages configuration persistent across reboots, add the > following line to your `/etc/sysctl.conf` file, adjusting the number of > hugepages as needed:: > + > + vm.nr_hugepages = 1024 > + > +3. Most distributions make hugepages available via `/dev/hugepages`, so this > step may not be necessary. If you need to manually mount the hugepages > filesystem, add the following line to your `/etc/fstab` file:: Starting a bullet point with a sentence saying "this step may not be necessary" seems a little strange - at least to me. Can we reword this perhaps? > + > + nodev /mnt/huge hugetlbfs defaults 0 0 > + > + Then, create the mount directory and mount the filesystem:: > + > + mkdir -p /mnt/huge > + mount -a > + We need a section somewhere in the docs about other page sizes - both 1G on x86, and the various sizes on Arm and other archs. Appendix would be a good place for it, with a reference here. For the large page sizes we need to talk about boot-time reservation. > +Device Setup: VFIO > +^^^^^^^^^^^^^^^^^^ > + > +VFIO is a robust and secure driver that relies on IOMMU protection. > +To make use of VFIO on Linux, the ``vfio-pci`` module must be loaded: > + > +.. code-block:: console > + > + sudo modprobe vfio-pci > + > +VFIO kernel is usually present by default in all distributions, > +however please consult your distribution's documentation to make sure that > is the case. > + > +To make use of full VFIO functionality, > +both kernel and BIOS must support and be configured > +to use IO virtualization (such as Intel\ |reg| VT-d). > + > +.. note:: > + > + In most cases, specifying "iommu=on" as kernel parameter should be enough > to > + configure the Linux kernel to use IOMMU. > + > +For proper operation of VFIO when running DPDK applications as a > non-privileged user, > +correct permissions should also be set up. > +For more information, refer to :ref:`running_dpdk_apps_without_root`. > + > +Refer to :ref:`vfio_no_iommu_mode` when there is no IOMMU available on the > system. > + > +Binding and Unbinding Network Ports to/from VFIO-PCI Module > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +To bind or unbind network ports to/from the `vfio-pci` module on Linux, > follow these steps: > + > +Replace ``<module_id>`` with the appropriate module identifier. On initial reading I thought module_id was referring to kernel module, i.e. vfio-pci. However, it instead is refering to the PCI device id or BDF address. Let's clarify this by using a better term. I'd suggest "<pci_id>". Also give an example: Replace <pci_id> with the appropriate device identifier, e.g. 01:00.0, a8:00.0. > + > +**Unbind from current module**: > + If a device is bound to a kernel driver, unbind it: > + > + .. code-block:: bash > + > + sudo dpdk-devbind.py -u <module_id> > + Actually, this step is not necessary. The devbind script is smart enough to know that to bind to a new driver, it needs to unbind the device first, so for binding to vfio-pci, it does unbind+bind with the one command. > +**Bind to vfio-pci module**: > + Bind the device to the `vfio-pci` module: > + > + .. code-block:: bash > + > + sudo dpdk-devbind.py -b vfio-pci <module_id> > + > +System Setup for FreeBSD > +------------------------ > + > +.. _loading_contigmem_module: > + > +Memory Setup: Loading the DPDK contigmem Module > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +To run a DPDK application on FreeBSD, physically contiguous memory is > required. In the absence of non-transparent superpages, the included sources > for the `contigmem` kernel module provides the ability to present contiguous > blocks of memory for the DPDK to use. > +The ``contigmem`` module must be loaded into the running kernel before any > DPDK is run. "any DPDK" -> "any DPDK application" > +Once DPDK is installed on the system, the module can be found in the > ``/boot/modules`` > +directory. > + > +The amount of physically contiguous memory along with the number of > physically > +contiguous blocks to be reserved by the module can be set at runtime prior > to module > +loading using:: > + > + kenv hw.contigmem.num_buffers=n > + kenv hw.contigmem.buffer_size=m > + > +The kernel environment variables can also be specified during boot by > placing the > +following in ``/boot/loader.conf``:: > + > + hw.contigmem.num_buffers=n > + hw.contigmem.buffer_size=m > + > +The variables can be inspected using the following command:: > + > + sysctl -a hw.contigmem > + > +The module can then be loaded using ``kldload``:: > + > + cd /boot/modules > + kldload contigmem > + > +Device Setup: Loading the DPDK nic_uio Module > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +After :ref:`loading_contigmem_module` on FreeBSD, the ``nic_uio`` module > must also be loaded into the running kernel prior to running any DPDK > application, e.g. using:: > + > + cd /boot/modules > + kldload nic_uio > + > +By default, the ``nic_uio`` module will take ownership of network ports if > they are > +recognized DPDK devices and are not owned by another module. > +However, since the FreeBSD kernel includes support, either built-in, > +or via a separate driver module, for most network card devices, > +it is likely that the ports to be used are already bound to a driver other > than > +``nic_uio``. > + > +To re-bind the ports to the `nic_uio` module upon loading, use the following > command:: > + > + kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." > + > +Where a comma-separated list of selectors is set, the list must not contain > any > +whitespace. > + > +The variable can also be specified during boot by placing the following into > +``/boot/loader.conf``, before the previously-described ``nic_uio_load`` > line:: Not previously-described! > + > + hw.nic_uio.bdfs="2:0:0,2:0:1" > + nic_uio_load="YES" > + > +Binding and Unbinding Network Ports to/from nic_uio Module > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +If the original driver for a network port has been compiled into the kernel, > +it is necessary to reboot FreeBSD to restore the original device binding. > +Before doing so, update or remove the ``hw.nic_uio.bdfs`` in > ``/boot/loader.conf``. > + > +If rebinding to a driver that is a loadable module, the network port binding > can be > +reset without rebooting. To do so, unload both the target kernel module and > the > +``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel > environment > +(``kenv``) value, and reload the two drivers - first the original kernel > driver, > +and then the ``nic_uio`` driver. > + > +Example commands to perform these steps are shown below:: > + > + kldunload nic_uio > + kldunload <original_driver> > + kenv -u hw.nic_uio.bdfs > + kldload <original_driver> > + kldload nic_uio # optional > + > +System Setup for Windows > +------------------------ > + > +Memory Setup: Installing Windows Modules > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +Before running DPDK applications on Windows, certain kernel-mode drivers > must be installed. Note that these drivers are not signed, so you'll need to > disable signature enforcement. However, be cautious as this can weaken your > OS security and is generally not recommended in production environments. > + > +Device Setup: Install Drivers > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +To run DPDK applications on Windows, you'll need to install specific > kernel-mode drivers: > + > +- **virt2phys**: This driver is essential for providing access to physical > addresses and is mandatory for allocating physically-contiguous memory, which > is required by hardware PMDs. Once loaded successfully, this driver will > appear in the Device Manager as ``Virtual to physical address translator > device`` under the Kernel bypass category. If DPDK cannot communicate with > the driver, a warning will be printed during initialization. > + > +- **NetUIO**: This driver provides access to device hardware resources and > is mandatory for all hardware PMDs, except for the mlx5 PMD. Devices > supported by NetUIO are listed in ``netuio.inf``. You can extend this list to > try running DPDK with new devices. > \ No newline at end of file > -- > 2.41.0.windows.1 >
