Once a week, save the output of df -h -F tmpfs
and top -o size -b -c 10 Look for filesystem growth in the former, and a process that's getting larger in the latter. If it's the kernel, you'd have to use mdb -K from the console or boot with kmdb, and then use ::findleaks The following covers some of that, if differently; but their use of mdb is for a process, not for the kernel. https://www.joyent.com/blog/bruning-questions-where-s-my-memory-going For the kernel, google for mdb findleaks and look around a little. > On Feb 21, 2018, at 06:27, Markus Wernig <liste...@wernig.net> wrote: > > Hi all > > I have 2 OI installations in kvm/qemu virtual machines. Both are showing > the same symptom: The available swap/stack space is continuously > decreasing, up to the point (roughly every 6-7 weeks) where none is left > and no new processes can be forked. Meaning a forced reboot, as also no > login shells can be spawned any more. > I have already moved /tmp to a separate dataset in the global zone (and > to plain directories within the two other zones). > > What I see is a slow, continuous decrease in the sisze of all file > systems marked as swap (~6.5 GB after reboot): > > df -h | grep swap over the last 24 hours: > > swap 6.45G 1020K 6.45G 1% /etc/svc/volatile > swap 6.45G 60K 6.45G 1% /var/run > .. > swap 5.94G 1020K 5.94G 1% /etc/svc/volatile > swap 5.94G 60K 5.94G 1% /var/run > .. > swap 5.89G 204K 5.89G 1% /etc/svc/volatile > swap 5.89G 12K 5.89G 1% /var/run > .. > swap 5.62G 204K 5.62G 1% /etc/svc/volatile > swap 5.62G 12K 5.62G 1% /var/run > .. > swap 5.04G 1020K 5.04G 1% /etc/svc/volatile > swap 5.04G 48K 5.04G 1% /var/run > > The swap device on the other hand remains stable: > > # swap -lh > swapfile dev swaplo blocks free > /dev/zvol/dsk/rpool/swap 270,2 4K 512M 512M > > As nothing seems to get stored in that space, I assume it must be used > by something else (processes not freeing it). > > Does anybody have an idea on how to start debugging this? All I get in > the log files are these messages, starting some hours before the boxes > stand still: > > Feb 14 03:51:09 xfer-srv05 tmpfs: [ID 518458 kern.warning] WARNING: > /etc/svc/volatile: File system full, swap space limit exceeded > Feb 14 03:51:10 xfer-srv05 last message repeated 1 time > Feb 14 04:03:20 xfer-srv05 genunix: [ID 470503 kern.warning] WARNING: > Sorry, no swap space to grow stack for pid 8106 (httpd) > Feb 14 04:28:36 xfer-srv05 genunix: [ID 470503 kern.warning] WARNING: > Sorry, no swap space to grow stack for pid 8920 (httpd) > Feb 14 05:58:20 xfer-srv05 genunix: [ID 470503 kern.warning] WARNING: > Sorry, no swap space to grow stack for pid 11050 (httpd) > Feb 14 05:58:24 xfer-srv05 last message repeated 1 time > > Thanks for any ideas. > > /markus > > > > _______________________________________________ > openindiana-discuss mailing list > openindiana-discuss@openindiana.org > https://openindiana.org/mailman/listinfo/openindiana-discuss >
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
