On Tue, Jul 2, 2019 at 10:06 AM Raimo Niskanen < raimo+open...@erix.ericsson.se> wrote:
> In /var/log/messages I find suspicious entries "/bsd: proc: table is full" > possibly before the machines become inresponsive, but these entries appear > many more times before that point. And after this "table is full" message > there are many syslog entries; on one machine smartd constatly complains > about > an unreadable (pending) sector and atascsi_passthru_done timeout, and on > the other the kernel complains about a probed monitor but no|invalid EDID. > In addition to Stuart's suggestion to leave top(1) running, and periodically save "ps ax" output, it might also be a good idea to start up a bunch of nested shells and just leave them running. This will reserve a bunch of process table slots, which you will be able to use via "exec", the idea being that if you can't fork new processes, you can at least use exec to replace an existing ksh process with something else. This will hopefully give you some limited ability to run a few post-mortem diagnostic commands before you run out of reserved process table slots. -ken