On Fri, Sep 04, 2020 at 03:58:03PM +0100, Burakov, Anatoly wrote:
> On 03-Sep-20 11:48 PM, Stephen Hemminger wrote:
> > This is an improved version of the setup of huge pages
> > bases on earlier DPDK setup. Differences are:
> > * it autodetects NUMA vs non NUMA
> > * it allows setting different page sizes
> > recent kernels support multiple sizes.
> > * it accepts a parameter in bytes (not pages).
> >
> > If necessary the steps of clearing old settings and mounting/umounting
> > can be done individually.
> >
> >
> > Signed-off-by: Stephen Hemminger <step...@networkplumber.org>
> > ---
> > v2 -- rewrite in python
> > The script is python3 only because supporting older versions
> > no longer makes any sense.
> >
> > usertools/hugepage-setup.py | 317 ++++++++++++++++++++++++++++++++++++
> > 1 file changed, 317 insertions(+)
> > create mode 100644 usertools/hugepage-setup.py
> >
> > diff --git a/usertools/hugepage-setup.py b/usertools/hugepage-setup.py
> > new file mode 100644
> > index 000000000000..8e7642428d9e
> > --- /dev/null
> > +++ b/usertools/hugepage-setup.py
> > @@ -0,0 +1,317 @@
> > +# Copyright (c) 2020 Microsoft Corporation
> > +#
> > +# Script to query and setup huge pages for DPDK applications.
> > +
> > +import sys
> > +import os
> > +import re
> > +import getopt
> > +import glob
> > +from os.path import exists, basename
> > +
> > +# convention for where to mount huge pages
> > +hugedir = '/dev/hugepages'
>
> This isn't a "convention", this is a default systemd mountpoint.
>
> > +
> > +# command-line flags
> > +show_flag = None
> > +reserve_kb = None
> > +clear_flag = None
> > +hugepagesize_kb = None
> > +mount_flag = None
> > +unmount_flag = None
> > +
> > +
> > +def usage():
> > + '''Print usage information for the program'''
> > + global hugedir
> > + mnt = hugedir
> > + argv0 = basename(sys.argv[0])
> > + print("""
> > +Usage:
> > +------
> > + %(argv0)s [options]
> > +
> > +Options:
> > + --help, --usage:
> > + Display usage information and quit
> > +
> > + -s, --show:
> > + Print the current huge page configuration.
> > +
> > + --setup:
> > + Simplified version of clear, umount, reserve, mount operations
> > +
> > + -c, --clear:
> > + Remove all huge pages
> > +
> > + -r, --reserve:
> > + Reserve huge pages. The size specified is in bytes, with
> > + optional K, M or G suffix. The size must be a multiple
> > + of the page size.
> > +
> > + -p, --pagesize
> > + Choose page size to use. If not specified, the default
> > + system page size will be used.
> > +
> > + -m, --mount
> > + Mount the system huge page directory %(mnt)s
> > +
> > + -u, --umount
> > + Unmount the system huge page directory %(mnt)s
> > +
> > +
> > +Examples:
> > +---------
> > +
> > +To display current huge page settings:
> > + %(argv0)s -s
> > +
> > +To a complete setup of with 2 Gigabyte of 1G huge pages:
> > + %(argv0)s -p 1G --setup 2G
> > +
> > +Equivalent to:
> > + %(argv0)s -p 1G -c -u -r 2G -m
> > +
> > +To clear existing huge page settings and umount %(mnt)s
> > + %(argv0)s -c -u
> > +
> > + """ % locals())
> > +
> > +
> > +def fmt_memsize(sz):
> > + '''Format memory size in conventional format'''
> > + sz_kb = int(sz)
> > + if sz_kb >= 1024 * 1024:
> > + return '{}Gb'.format(sz_kb / (1024 * 1024))
> > + elif sz_kb >= 1024:
> > + return '{}Mb'.format(sz_kb / 1024)
> > + else:
> > + return '{}Kb'.format(sz_kb)
>
> I've lost count how many times i've had to reimplement this code, but there
> is an easier way :) Off the top of my head,
>
> idx = log2(sz)
> # every 10th power of 2
> return '{}{}b'.format(sz, ' kMG'[int(idx) / 10])
>
> or something close to that.
>
Another minor nit, since these are memory sizes, not bandwidth rates, it's
bytes not bits, so the "b" should be "B" in all the prints, whatever way
it's calculated.
