Thanks for the nice addition to the documentation, this is really needed. Some comments below.
17/01/2022 09:07, Dmitry Kozlyuk: > --- a/doc/guides/prog_guide/env_abstraction_layer.rst > +++ b/doc/guides/prog_guide/env_abstraction_layer.rst > - Memory reservations done using the APIs provided by rte_malloc are also > backed by pages from the hugetlbfs filesystem. > + Memory reservations done using the APIs provided by rte_malloc are also > backed by hugepages. Should we mention except if --no-huge is used? > +Hugepage Mapping > +^^^^^^^^^^^^^^^^ > + > +Below is an overview of methods used for each OS to obtain hugepages, > +explaining why certain limitations and options exist in EAL. > +See the user guide for a specific OS for configuration details. > + > +FreeBSD uses ``contigmem`` kernel module > +to reserve a fixed number of hugepages at system start, > +which are mapped by EAL at initialization using a specific ``sysctl()``. > + > +Windows EAL allocates hugepages from the OS as needed using Win32 API, > +so available amount depends on the system load. > +It uses ``virt2phys`` kernel module to obtain physical addresses, > +unless running in IOVA-as-VA mode (e.g. forced with ``--iova-mode=va``). > + > +Linux implements a variety of methods: > + > +* mapping each hugepage from its own file in hugetlbfs; > +* mapping multiple hugepages from a shared file in hugetlbfs; > +* anonymous mapping. > + > +Mapping hugepages from files in hugetlbfs is essential for multi-process, > +because secondary processes need to map the same hugepages. > +EAL creates files like ``rtemap_0`` > +in directories specified with ``--huge-dir`` option > +(or in the mount point for a specific hugepage size). > +The ``rtemap_`` prefix can be changed using ``--file-prefix``. > +This may be needed for running multiple primary processes > +that share a hugetlbfs mount point. > +Each backing file by default corresponds to one hugepage, > +it is opened and locked for the entire time the hugepage is used. > +See :ref:`segment-file-descriptors` section > +on how the number of open backing file descriptors can be reduced. > + > +Backing files may persist after the corresponding hugepage is freed > +and even after the application terminates, > +reducing the number of hugepages available to other processes. > +EAL removes existing files at startup > +and can remove newly created files before mapping them with > ``--huge-unlink``. This sentence require more explanations, as it is not clear when and why. > +However, since it disables multi-process anyway, > +using anonymous mapping (``--in-memory``) is recommended instead. > + > +:ref:`EAL memory allocator <malloc>` relies on hugepages being zero-filled. > +Hugepages are cleared by the kernel when a file in hugetlbfs or its part > +is mapped for the first time system-wide > +to prevent data leaks from previous users of the same hugepage. > +EAL ensures this behavior by removing existing backing files at startup > +and by recreating them before opening for mapping (as a precaution). > + > +Anonymous mapping does not allow multi-process architecture, > +but it is free of filename conflicts and leftover files on hugetlbfs. It is also easier to run as non-root. > +If memfd_create(2) is supported both at build and run time, > +DPDK memory manager can provide file descriptors for memory segments, > +which are required for VirtIO with vhost-user backend. > +This means open file descriptor issues may also affect this mode, > +with the same solution. This is not clear. Which issues? Which mode? Which solution?
