On 10-Jun-20 4:28 PM, Stephen Hemminger wrote:
On Wed, 10 Jun 2020 10:24:44 +0100
"Burakov, Anatoly" <anatoly.bura...@intel.com> wrote:
On 09-Jun-20 4:35 PM, Stephen Hemminger wrote:
On Tue, 9 Jun 2020 14:39:54 +0100
"Burakov, Anatoly" <anatoly.bura...@intel.com> wrote:
On 09-Jun-20 2:13 PM, Ferruh Yigit wrote:
On 6/9/2020 1:46 PM, Burakov, Anatoly wrote:
On 08-Jun-20 12:03 PM, Francesco wrote:
Hi all,
I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
20.05 and I noticed that if I look at "top" I see that the VIRT memory
taken by my application is now 256.1GB while before it was <1GB.
I've seen this same behavior with also "testpmd" example... is this a known
issue with latest DPDK versions?
Can I tweak some setting to have VIRT memory usage more or less similar to
RSS ?
I forgot to add I'm working on Linux, Centos7
Thanks,
Francesco Montorsi
There was a discussion on this not too long ago, but i can't seem to
find it for some reason.
Can it be "Big spike in DPDK VSZ" ?
http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=eji2a...@mail.gmail.com/#t
Yes, that's the one, thanks Ferruh :)
Anyway, long story short, that's not a bug,
that's by design.
Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in
DPDK that allows growing and shrinking DPDK memory usage at runtime.
That means, you can start with zero hugepages preallocated, and then
allocate as you go, letting the memory subsystem decide how much memory
you need.
The catch is that all of this hugepage memory is allocated into
somewhere, some virtual address space. And *that* address space is
preallocated at startup, to allow for secondary processes to duplicate
primary process's address space exactly, and allow dynamic allocation of
*shared* memory at runtime.
This memory will show up in top et al. but the truth is, it's zero cost,
because it's anonymous memory. It isn't actually taking up any RAM. It
will show up in dumps (20.05 has already fixed that issue, and the fixes
will probably be backported to stable, including 18.11), so unless you
have a very specific problem, i don't think that's anything you should
be concerned about.
The one concern is for cases like cgroup memory accounting thinking
the process is huge and OOM killing it.
Is there any way to know the *actual* memory usage of the process (i.e.
not including anonymous memory)?
Huge pages do not count against the normal memory in cgroup.
There is a separate hugeTLB controller that limits that.
But hugepages only get mapped when they're required - the rest of the
memory is mapped anonymously with PROT_NONE. Would that count against
the limits?
--
Thanks,
Anatoly
