On Tue, Aug 25, 2020 at 01:17:49PM +0100, Anatoly Burakov wrote: > Current information regarding hugepage usage is a little out of date. > Update it to include information on in-memory mode, as well as on > default mountpoints provided by systemd. > > Cc: sta...@dpdk.org > > Signed-off-by: Anatoly Burakov <anatoly.bura...@intel.com> > --- > > Notes: > v2: > - Reworked the description > - Put runtime reservation first, and boot time as an alternative > - Clarified wording and fixed typos > - Mentioned that some kernel versions not supporting reserving 1G pages > > doc/guides/linux_gsg/sys_reqs.rst | 71 ++++++++++++++++++++----------- > 1 file changed, 45 insertions(+), 26 deletions(-) > > diff --git a/doc/guides/linux_gsg/sys_reqs.rst > b/doc/guides/linux_gsg/sys_reqs.rst > index a124656bcb..8782d05579 100644 > --- a/doc/guides/linux_gsg/sys_reqs.rst > +++ b/doc/guides/linux_gsg/sys_reqs.rst > @@ -155,8 +155,35 @@ Without hugepages, high TLB miss rates would occur with > the standard 4k page siz > Reserving Hugepages for DPDK Use > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > -The allocation of hugepages should be done at boot time or as soon as > possible after system boot > -to prevent memory from being fragmented in physical memory. > +The reservation of hugepages can be performed at run time. This is done by > +echoing the number of hugepages required to a ``nr_hugepages`` file in the > +``/sys/kernel/`` directory corresponding to a specific page size (in > +Kilobytes). For a single-node system, the command to use is as follows > +(assuming that 1024 of 2MB pages are required):: > + > + echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > + > +On a NUMA machine, the above command will usually divide the number of > hugepages > +equally across all NUMA nodes (assuming there is enough memory on all NUMA > +nodes). However, pages can also be reserved explicitly on individual NUMA > +nodes using a ``nr_hugepages`` file in the ``/sys/devices/`` directory:: > + > + echo 1024 > > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages > + echo 1024 > > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages > + > +.. note:: > + > + Some kernel versions may not allow reserving 1 GB hugepages at run time, > so > + reserving them at boot time may be the only option. Please see below for > + instructions. > + > +**Alternative:** > + > +In the general case, reserving hugepages at run time is perfectly fine, but > in > +use cases where having lots of physically contiguous memory is required, it > is > +preferable to reserve hugepages at boot time, as that will help in preventing > +physical memory from becoming heavily fragmented. > + > To reserve hugepages at boot time, a parameter is passed to the Linux kernel > on the kernel command line. > > For 2 MB pages, just pass the hugepages option to the kernel. For example, > to reserve 1024 pages of 2 MB, use:: > @@ -185,35 +212,27 @@ the number of hugepages reserved at boot time is > generally divided equally betwe > > See the Documentation/admin-guide/kernel-parameters.txt file in your Linux > source tree for further details of these and other kernel options. > > -**Alternative:** > - > -For 2 MB pages, there is also the option of allocating hugepages after the > system has booted. > -This is done by echoing the number of hugepages required to a nr_hugepages > file in the ``/sys/devices/`` directory. > -For a single-node system, the command to use is as follows (assuming that > 1024 pages are required):: > - > - echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > - > -On a NUMA machine, pages should be allocated explicitly on separate nodes:: > - > - echo 1024 > > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages > - echo 1024 > > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages > - > -.. note:: > - > - For 1G pages, it is not possible to reserve the hugepage memory after > the system has booted. > - > Using Hugepages with the DPDK > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > -Once the hugepage memory is reserved, to make the memory available for DPDK > use, perform the following steps:: > +If secondary process support is not required, DPDK is able to use hugepages > +without any configuration by using "in-memory" mode. Please see > +:ref:`linux_eal_parameters` for more details. > + > +If secondary process support is required, mount points for hugepages need to > be > +created. On modern Linux distributions, a default mount point for hugepages > is provided > +by the system and is located at ``/dev/hugepages``. This mount point will > use the > +default hugepage size set by the kernel parameters as described above. > + > +However, in order to use hugepage sizes other than default, it is necessary > to > +manually create mount points for hugepage sizes that are not provided by the > +system (e.g. 1GB pages).
This reads a bit strangely, as it implies that the hugepage sizes are not provided by the system, but I believe the intention is to say that the mount points are not provided by the system, correct? Perhaps look to reword. > + > +To make the hugepages of size 1GB available for DPDK use, perform the > following steps:: > > mkdir /mnt/huge > - mount -t hugetlbfs nodev /mnt/huge > + mount -t hugetlbfs pagesize=1GB /mnt/huge > > The mount point can be made permanent across reboots, by adding the > following line to the ``/etc/fstab`` file:: > > - nodev /mnt/huge hugetlbfs defaults 0 0 > - > -For 1GB pages, the page size must be specified as a mount option:: > - > - nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0 > + nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 > -- > 2.17.1 Apart from the one note above, LGTM: Acked-by: Bruce Richardson <bruce.richard...@intel.com>
