Hi, this is a bug in the scheduler. Currently, it ignores hugepages when evaluating NUMA pinning.
There is a bugzilla ticket[1] that was originally reported as a similar case, but then later the reporter changed it. Could you open a new bugzilla ticket and attach the details from this email? As a workaround, if you don't want to migrate the VM or you are sure that it can run on the target host, you can clone a cluster policy and remove the 'NUMA' filter. (In Administration -> Configure -> Scheduling Policies). [1] - https://bugzilla.redhat.com/show_bug.cgi?id=1720558 Best regards, Andrej On Wed, 21 Aug 2019 at 12:16, Ralf Schenk <[email protected]> wrote: > Hello List, > > i ran into problems using numa-pinning and reserved hugepages. > > - My EPYC 7281 based Servers (Dual Socket) have 8 Numa-Nodes each having > 32 GB of memory for a total of 256 GB System Memory > > - I'm using 192 x 1 GB hugepages reserved on the kernel cmdline > default_hugepagesz=1G hugepagesz=1G hugepages=192 This reserves 24 > hugepages on each numa-node. > > I wanted to pin a MariaDB VM using 32 GB (Custom Property > hugepages=1048576) to numa-nodes 0-3 of CPU-Socket 1. Pinning in GUI etc. > no problem. > > When trying to start the vm this can't be done since ovirt claims that the > host can't fullfill the memory requirements - which is simply not correct > since there were > 164 hugepages free. > > It should have taken 8 hugepages from each numa node 0-3 to fullfill the > 32 GB Memory requirement. > > I also freed the system completely from other VM's but that didn't work > either. > > Is it possible that the scheduler only takes into account the "free > memory" (as seen in numactl -H below) *not reserved* by hugepages for its > decisions ? Since the host has only < 8 GB of free mem per numa-node I can > understand that VM was not able to start under that condition. > > VM is runnig and using 32 hugepages without pinning but a warning states > "VM dbserver01b does not fit to a single NUMA node on host > myhost.mydomain.de. This may negatively impact its performance. Consider > using vNUMA and NUMA pinning for this VM." > > This is the numa Hardware Layout and hugepages usage now with other VM's > running: > > from cat /proc/meminfo > > HugePages_Total: 192 > HugePages_Free: 160 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > > I can confirm that also under the condition of running other VM's there > are at least 8 hugepages free for each numa-node 0-3: > > grep "" > /sys/devices/system/node/*/hugepages/hugepages-1048576kB/free_hugepages > > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/free_hugepages:8 > > /sys/devices/system/node/node1/hugepages/hugepages-1048576kB/free_hugepages:23 > > /sys/devices/system/node/node2/hugepages/hugepages-1048576kB/free_hugepages:20 > > /sys/devices/system/node/node3/hugepages/hugepages-1048576kB/free_hugepages:22 > > /sys/devices/system/node/node4/hugepages/hugepages-1048576kB/free_hugepages:16 > > /sys/devices/system/node/node5/hugepages/hugepages-1048576kB/free_hugepages:5 > > /sys/devices/system/node/node6/hugepages/hugepages-1048576kB/free_hugepages:19 > > /sys/devices/system/node/node7/hugepages/hugepages-1048576kB/free_hugepages:24 > > numactl -h: > > available: 8 nodes (0-7) > node 0 cpus: 0 1 2 3 32 33 34 35 > node 0 size: 32673 MB > node 0 free: 3779 MB > node 1 cpus: 4 5 6 7 36 37 38 39 > node 1 size: 32767 MB > node 1 free: 6162 MB > node 2 cpus: 8 9 10 11 40 41 42 43 > node 2 size: 32767 MB > node 2 free: 6698 MB > node 3 cpus: 12 13 14 15 44 45 46 47 > node 3 size: 32767 MB > node 3 free: 1589 MB > node 4 cpus: 16 17 18 19 48 49 50 51 > node 4 size: 32767 MB > node 4 free: 2630 MB > node 5 cpus: 20 21 22 23 52 53 54 55 > node 5 size: 32767 MB > node 5 free: 2487 MB > node 6 cpus: 24 25 26 27 56 57 58 59 > node 6 size: 32767 MB > node 6 free: 3279 MB > node 7 cpus: 28 29 30 31 60 61 62 63 > node 7 size: 32767 MB > node 7 free: 5513 MB > node distances: > node 0 1 2 3 4 5 6 7 > 0: 10 16 16 16 32 32 32 32 > 1: 16 10 16 16 32 32 32 32 > 2: 16 16 10 16 32 32 32 32 > 3: 16 16 16 10 32 32 32 32 > 4: 32 32 32 32 10 16 16 16 > 5: 32 32 32 32 16 10 16 16 > 6: 32 32 32 32 16 16 10 16 > 7: 32 32 32 32 16 16 16 10 > > -- > > > *Ralf Schenk* > fon +49 (0) 24 05 / 40 83 70 > fax +49 (0) 24 05 / 40 83 759 > mail *[email protected]* <[email protected]> > > *Databay AG* > Jens-Otto-Krag-Straße 11 > D-52146 Würselen > *www.databay.de* <http://www.databay.de> > > Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 > Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. > Philipp Hermanns > Aufsichtsratsvorsitzender: Wilhelm Dohmen > ------------------------------ > _______________________________________________ > Users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/GXIWD6B6E3UMWARXARUDW6TIN6WWYQFU/ >
_______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/PIKRSWZGUZ5QTPRWDKL7RRMLQV6GEQYW/
