Hi all Thanks for the pointers, will try to follow up on them.
The fact that Solaris is interleaving disk and memory-backed storage into the tmpfs hadn't struck me until now, as there never seems to be an increase of the used space in these file systems, but just a decrease of the available overall (presumably memory) space. Just one other question: Shouldn't an increase in memory consumption by any process be visible in the output of tools like eg. top? (Which it isn't: no matter how low the available "swap" space reported by df becomes, top keeps reporting ~6GB free memory: # top Memory: 8192M phys mem, 5900M free mem, 512M total swap, 512M free swap ARC: 816M Total, 607M MRU, 159M MFU, 1579K Anon, 4144K Header, 45M Other # df -h -F tmpfs Filesystem Size Used Available Capacity Mounted on swap 4.98G 1020K 4.98G 1% /etc/svc/volatile swap 4.98G 48K 4.98G 1% /var/run) best /markus On 21.02.2018 12:27, Markus Wernig wrote: > Hi all > > I have 2 OI installations in kvm/qemu virtual machines. Both are showing > the same symptom: The available swap/stack space is continuously > decreasing, up to the point (roughly every 6-7 weeks) where none is left > and no new processes can be forked. Meaning a forced reboot, as also no > login shells can be spawned any more. > I have already moved /tmp to a separate dataset in the global zone (and > to plain directories within the two other zones). > > What I see is a slow, continuous decrease in the sisze of all file > systems marked as swap (~6.5 GB after reboot): > > df -h | grep swap over the last 24 hours: > > swap 6.45G 1020K 6.45G 1% /etc/svc/volatile > swap 6.45G 60K 6.45G 1% /var/run > .. > swap 5.94G 1020K 5.94G 1% /etc/svc/volatile > swap 5.94G 60K 5.94G 1% /var/run > .. > swap 5.89G 204K 5.89G 1% /etc/svc/volatile > swap 5.89G 12K 5.89G 1% /var/run > .. > swap 5.62G 204K 5.62G 1% /etc/svc/volatile > swap 5.62G 12K 5.62G 1% /var/run > .. > swap 5.04G 1020K 5.04G 1% /etc/svc/volatile > swap 5.04G 48K 5.04G 1% /var/run > > The swap device on the other hand remains stable: > > # swap -lh > swapfile dev swaplo blocks free > /dev/zvol/dsk/rpool/swap 270,2 4K 512M 512M > > As nothing seems to get stored in that space, I assume it must be used > by something else (processes not freeing it). > > Does anybody have an idea on how to start debugging this? All I get in > the log files are these messages, starting some hours before the boxes > stand still: > > Feb 14 03:51:09 xfer-srv05 tmpfs: [ID 518458 kern.warning] WARNING: > /etc/svc/volatile: File system full, swap space limit exceeded > Feb 14 03:51:10 xfer-srv05 last message repeated 1 time > Feb 14 04:03:20 xfer-srv05 genunix: [ID 470503 kern.warning] WARNING: > Sorry, no swap space to grow stack for pid 8106 (httpd) > Feb 14 04:28:36 xfer-srv05 genunix: [ID 470503 kern.warning] WARNING: > Sorry, no swap space to grow stack for pid 8920 (httpd) > Feb 14 05:58:20 xfer-srv05 genunix: [ID 470503 kern.warning] WARNING: > Sorry, no swap space to grow stack for pid 11050 (httpd) > Feb 14 05:58:24 xfer-srv05 last message repeated 1 time _______________________________________________ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
