Hi Dmitry,
I am not clear on the DPDK memory layout and how to tweak these #define values.
#define RTE_MAX_MEMSEG_PER_LIST 8192
#define RTE_MAX_MEM_MB_PER_LIST 32768
#define RTE_MAX_MEMSEG_PER_TYPE 32768
#define RTE_MAX_MEM_MB_PER_TYPE 65536
I want to limit how much DPDK grabs for memory, but grabs what it absolutely
needs for our application. I don't want DPDK to plan for any hot-plug or
dynamic adjustments of hugepages. Our configuration is static, we allocate
hugepages, discover ports and initialize couple ring buffers and 1 Rx Q and 4
Tx Qs per port. Max of 4 ports.
My goal is to reduce our application VIRT memory by 80%. Not clear which
config variables to adjust.
What do you suggest, is there a formula?
Regards,
Ed
-----Original Message-----
From: Lombardo, Ed
Sent: Thursday, May 2, 2024 4:59 PM
To: Dmitry Kozlyuk <dmitry.kozl...@gmail.com>
Cc: dev@dpdk.org
Subject: RE: Need help with reducing VIRT memory
Hi Dmitry,
Yes. I will look into
The VM gives the following:
[root@vSTREAMO9-234 bin]# prlimit -v
RESOURCE DESCRIPTION SOFT HARD UNITS
AS address space limit unlimited unlimited bytes
I saw these optional configurations in my google searching, but was hesitant
because of unknown impact. I will try this path next.
It would be valuable to have meson be able to configure DPDK for low memory
devices.
Thank you for replying.
Regards,
Ed
-----Original Message-----
From: Dmitry Kozlyuk <dmitry.kozl...@gmail.com>
Sent: Thursday, May 2, 2024 4:04 PM
To: Lombardo, Ed <ed.lomba...@netscout.com>
Cc: dev@dpdk.org
Subject: Re: Need help with reducing VIRT memory
External Email: This message originated outside of NETSCOUT. Do not click links
or open attachments unless you recognize the sender and know the content is
safe.
Hi Ed,
I presume it's a revival of this thread:
https://urldefense.com/v3/__http://inbox.dpdk.org/users/ch3pr01mb8470c9675763e14954d6e3b88f...@ch3pr01mb8470.prod.exchangelabs.com/__;!!Nzg7nt7_!EBENSvDUv9LcvbAw6vLkHgh11JdjksApffakP2JU30OJaPLCkZPZVzk4U8rRTJsxJVCeRubpFQy7TB2-R6jDbrk4QpU$
2024-05-02 19:05 (UTC+0000), Lombardo, Ed:
[...]
> My situation is as follows:
> We were on DPDK 17.11.6 and upgraded to DPDK22.11.2 to support the Intel
> E810. Also upgraded from CentOS7 to Oracle 91.
> In DPDK 22.11.2 the VIRT memory has skyrocketed and causes major issues with
> our application that runs on 16 GB of memory (virtual appliances and low end
> remote appliances).
>
> I have also tried DPDK 23.11 and 24.03 and I also see across the DPDK
> versions that the VIRT memory has increase by a factor of 5 over DPDK
> 17.11
>
> Our application process shows VIRT memory is 7.6 GB with DPDK 17.11.6,
> and with DPDK 22.11.2, 23.11.2 and 24.03 the VIRT memory is ~36.6GB
>
> Our application minimum configuration needs to run with 16 GB memory when
> DPDK is enabled.
As explained in the linked thread,
high reserved VIRT does not mean high memory demand.
Running DPDK on a 16 GB machine is perfectly viable.
Something in your app or setup conflicts with the high VIRT reservation.
> Our application uses memory locking and rlimit settings to get optimal
> performance.
Does it use address space limit (prlimit -v)?
If so, this limit may be hit.
With default build options, DPDK reserves at least 32 GB of VIRT (one list) per
memory type (a combination of NUMA node + hugepage size).
This is controlled by defines in <rte_config.h>:
#define RTE_MAX_MEMSEG_LISTS 128
#define RTE_MAX_MEMSEG_PER_LIST 8192
#define RTE_MAX_MEM_MB_PER_LIST 32768
#define RTE_MAX_MEMSEG_PER_TYPE 32768
#define RTE_MAX_MEM_MB_PER_TYPE 65536
Try reducing those and rebuilding DPDK if the above is your case.
Reducing these values will limit the amount of memory available to DPDK
allocator in your app.
