Hi everyone,
after upgrading my hardware I started to notice what seemed like memory leaks.
After booting my desktop machine usually around 450MiB(with i3) or
650MiB(with MATE) of RAM are being used and after extended use, when closing all
gui-programs except a terminal, based on prior experience I would expect the
RAM usage to not be more than +120MiB compared to after booting. I am *not*
counting file cache/buffers in this (or I severely misunderstand how to read
free's and top's output).
But now I would often end up with something between +600MiB and +2.5GiB.
When looking at top or htop no process using nearly that much memory is listed.
I tried to figure this out and will give some more details in a minute, but I'm
left confused as to why this is happening how to continue debugging this and
where I could file a bug report.
Any advice on how to proceed or where to file a bug would be appreciated.
Or maybe I'm just stupid and misreading top's and free's output.
Unfortunately the only reliable way to reproduce this, that I found, involves
manual interaction with some Firefox profiles. I know this means there's
probably an issue with Firefox, but afaik when a user-space process with a
memleak is closed, the leaked memory should be cleaned up as well. My failed
attempt to recreate the leaks with a simple C program seems to support this.
Also this seems to be hardware specific.
Affected are:
Debian 10 with kernels:
4.19 (from main)
5.4.0~bpo.3 5.4.0~bpo.4 (from backports)
5.4.23 with Manjaro patches and config
5.4.26 (vanilla; config based on Debian's config)
5.6-rc7 (vanilla; config based on Debian's config)
Manjaro with kernel:
5.4.23-MANJARO
Ubuntu-18.04 with kernels:
5.3
5.4.xy (vanilla)
In Manjaro and Ubuntu proprietary firmware was used.
In Debian I was first only using the proprietary AMD-GPU firmware but later also
added firmware for the Ethernet-controller and microcode for the CPU without any
change.
dmesg does not seem to show anything relevant*.
(*or atleast not anymore since cpu-bootcode is installed)
(The following was only tested with kernel 5.6-rc7):
Now yesterday on accident I noticed, that after running a backup, which "filled"
RAM with file cache, the reported memory fell again to a normal level.
With i3:
--After Boot:
$ free -h
total used free shared buff/cache available
Mem: 31Gi 397Mi 30Gi 11Mi 314Mi 30Gi
Swap: 6,0Gi 0B 6,0Gi
--After using and closing Firefox
--(no processes using that much RAM are listed in htop):
$ free -h
total used free shared buff/cache available
Mem: 31Gi 1,1Gi 2,4Gi 13Mi 27Gi 29Gi
Swap: 6,0Gi 0B 6,0Gi
--After Backing up which "filled" RAM with file caches:
$ free -h
total used free shared buff/cache available
Mem: 31Gi 471Mi 328Mi 13Mi 30Gi 30Gi
Swap: 6,0Gi 0B 6,0Gi
The reported memory usage of htop,top and free was all in accordance with each
other here.
Now as I couldn't find a way to reliably "fill" up my RAM with file caches, I
tried to recreate this (with MATE) with a C program that allocates memory,
writes pseudo-random data to it and exits (without freeing). I allocated 30GiB,
so more than MAX-used, but less than MAX-(expected usage).
After that (h)top and free are disagreeing on how much memory is being used
(see SWAP).
--After Firefox (htop,top and free report more or less the same):
$ free -h
total used free shared buff/cache available
Mem: 31Gi 1,5Gi 27Gi 92Mi 2,7Gi 29Gi
Swap: 6,0Gi 0B 6,0Gi
-- After allocating 30GiB and then exiting:
$ top -b -n 1 | grep "^MiB"
MiB Mem : 32092,0 total, 27747,1 free, 744,3 used, 3600,6 buff/cache
MiB Swap: 6144,0 total, 6143,5 free, 0,5 used. 30458,0 avail Mem
$ free -h
total used free shared buff/cache available
Mem: 31Gi 743Mi 27Gi 93Mi 3,5Gi 29Gi
Swap: 6,0Gi 0,0Ki 6,0Gi
htop was reporting around 500MiB swap usage (before and also after running the
above two commands).
So maybe this is not a memory leak but memory usage is being misreported ?
Where should I file a bug for this; or am I just misreading (h)top's and free's
output and everything is as expected ?
free's manpage states:
> used Used memory (calculated as total - free - buffers - cache)
which I take as this being the memory actually in use, that I could not just
allocated in a process without needing to fallback to swap.
Sorry for this wall of text.
Kind Regards
Nils
