Couldn't be SystemD, could it? That wasn't in use, in Debian 7. Just a guess...
Kenneth Parker On Thu, Apr 26, 2018, 9:09 AM Simon Beirnaert < simon.beirna...@lightspeedhq.com> wrote: > Hi Recently I've started moving a fleet of Debian 7, 32-bit machines > over to Debian 9, 64-bit. This migration is done by creating a fresh > Debian 9 image with the necessary services, moving over user data (some > wars and the content of /home) and rebooting into the new OS. > > Relevant services (ones we manage and use) are: > > - Jetty- Puppet - SSH - AutoSSH - NewRelic Infrastructure Through > Puppet, we enforce system configuration is pretty much identical, save > for stuff like host names and SSH keys. Now, we notice that on some > systems, the RAM usage is way higher than expected, to the point where > system memory is exhausted and processes (are) terminate(d). > Investigation into what is causing this, leaves me at a dead end. I > can't figure out where the memory is being consumed. Even after quitting > all services we manage (leaving a "clean" installation), RAM usage on > the system hovers just over 600MB, half a gig over what the same exact > image consumes just after boot. The only fix I found so far is to reboot > the system. The systems have ~1.8GB of system memory available. We don't > use swap. Enabling swap gives the system some breathing room. On one > system I enabled a swap volume of 512MB, which the kernel fills up and > leaves filled up indefinitely. This points me to unused memory being > allocated by mistake. A memory leak in the kernel, maybe? Below I've > posted the output of some of the things I checked (with all services > online). I also searched the internet and reached a topic about slab > allocation (1). However, that didn't seem to solve anything. Can anyone > here point me to some more stuff I can check out or try to debug this? > Thanks! Simon root@mysystem:~# uname -a Linux mysystem 4.9.0-6-amd64 #1 > SMP Debian 4.9.82-1+deb9u2 (2018-02-21) x86_64 GNU/Linux > root@mysystem:~# free -m total used free shared buff/cache available > Mem: 1831 1091 238 16 501 520 Swap: 511 511 0 root@mysystem:~# vmstat 1 > procs -----------memory---------- ---swap-- -----io---- -system-- > ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st > 2 0 524268 244480 46044 467248 1 2 2137 36 97 35 9 3 85 3 0 0 0 524268 > 244092 46052 467244 0 0 0 16 1125 2406 3 3 94 1 0 0 0 524268 244092 > 46060 467284 0 0 4 208 1230 3906 4 3 93 0 0 0 0 524268 244084 46068 > 467256 0 0 0 32 1003 1990 1 2 97 0 0 0 0 524268 244180 46068 467256 0 0 > 0 0 1099 2121 4 1 95 0 0 0 0 524268 244204 46076 467272 0 0 0 20 1000 > 1978 1 2 97 1 0 0 0 524268 244080 46076 467272 0 0 0 0 1135 2315 2 2 96 > 0 0 0 0 524268 244080 46084 467264 0 0 0 16 1079 2103 1 3 96 1 0 0 0 > 524268 244080 46092 467272 0 0 0 56 1002 1973 2 2 96 0 0 1 0 524268 > 244080 46100 467264 0 0 0 16 997 1979 1 2 97 1 0 0 0 524268 244144 46100 > 467268 0 0 0 0 988 1957 1 2 97 0 0 0 1 524268 244228 46108 467260 0 0 0 > 980 1292 2700 4 5 81 9 0 root@mysystem:~# smem PID User Command Swap USS > PSS RSS 528 root /sbin/agetty -f /etc/issue. 148 4 4 8 554 myuser > /usr/lib/autossh/autossh -o 80 24 40 184 220 root > /lib/systemd/systemd-udevd 552 108 140 688 10665 root > /usr/lib/autossh/autossh -o 0 104 143 648 432 root /usr/sbin/cron -f 164 > 120 147 500 434 root /usr/sbin/irqbalance --fore 252 156 172 508 12042 > mail /usr/sbin/nullmailer-send - 120 144 219 1228 382 systemd-timesync > /lib/systemd/systemd-timesy 456 152 301 1060 439 messagebus > /usr/bin/dbus-daemon --syst 272 280 348 1116 430 root > /lib/systemd/systemd-logind 380 192 448 1224 557 myuser /usr/bin/ssh -o > StrictHostK 564 360 515 1472 216 root /sbin/lvmetad -f 188 484 517 1028 > 10668 root /usr/bin/ssh -o StrictHostK 0 736 825 1112 14225 ntp > /usr/sbin/ntpd -p /var/run/ 0 808 849 1404 435 root /usr/sbin/rsyslogd > -n 716 896 980 2036 11612 root /usr/sbin/sshd -D 0 856 991 1560 500 root > /sbin/dhclient -4 -v -pf /r 840 928 1010 2068 1330 root sudo -i 0 920 > 1375 3548 1234 myuser -bash 0 600 1407 3684 1331 root -bash 0 640 1447 > 3712 1233 myuser sshd: myuser@pts/0 0 300 1697 4568 1 root /sbin/init > 524 1568 1935 3524 1227 root sshd: posios [priv] 0 1372 2984 6380 193 > root /lib/systemd/systemd-journa 312 4304 4521 5828 8885 root > /usr/bin/python /usr/bin/sm 0 9100 9452 11292 14574 root > /usr/bin/newrelic-infra 1440 17348 17378 17828 520 root > /opt/puppetlabs/puppet/bin/ 20636 40140 40168 40592 566 jetty > /usr/lib/jvm/java-8-openjdk 493896 958124 958381 959804 [1] > > https://unix.stackexchange.com/questions/244735/why-are-slab-objects-not-reclaimed-automatically > > >
