On Fri, Aug 16, 2024 at 10:08 AM Brian Filipek <bfili...@betterworldbooks.com.invalid> wrote:
> Hi Nick, > > This problem continues in FreeBSD with guacd randomly signal 6 aborting > approximately once a week.When this happens I need to log in and restart > the guacd service. We had a crash last night and here are the last few > lines of /var/log/messages: > > Aug 15 11:53:14 guacamole guacd[17699]: Client did not terminate in a > timely manner. Forcibly terminating client and any child processes. > Aug 15 15:35:14 guacamole guacd[17752]: Client did not terminate in a > timely manner. Forcibly terminating client and any child processes. > Aug 15 15:59:19 guacamole guacd[32874]: User is not responding. > Aug 16 00:09:22 guacamole guacd[65139]: User is not responding. > Aug 16 03:05:28 guacamole guacd[79034]: User is not responding. > Aug 16 03:25:22 guacamole kernel: pid 75166 (guacd), jid 0, uid 899: > exited on signal 6 (no core dump - other error) > Aug 16 03:25:22 guacamole kernel: pid 1758 (guacd), jid 0, uid 899: exited > on signal 6 (no core dump - other error) > Aug 16 03:25:22 guacamole guacd[67047]: Unable to shutdown internal socket > for connection $2bbbb77e-d090-441a-b2d5-c337bf55b13c. Corresponding process > may remain running but inactive. > Aug 16 08:20:40 guacamole login[1783]: ROOT LOGIN (root) ON ttyv0 > > I did a quick Google search on "signal 6" and found that the general consensus seems to be that this is a "voluntary" termination by the process (guacd), usually triggered by a call to the "abort()" function. A quick search through the guacd source code shows that the only times we every call this function are in the libguac memory allocation functions ( https://github.com/apache/guacamole-server/blob/main/src/libguac/mem.c), during memory arithmetic processes (add, subtract, multiple), and during the realloc process. So, my _guess_ here is that it is failing to allocate required memory for some operation and is aborting. It's possible there's a memory leak somewhere in guacd that is manifesting itself in your particular case on a semi-regular basis, but hard to say. The interesting things to try to track this down further would be: * Any information you have on resources utilization - particularly memory - that would show any sort of a trend of memory utilization growing up to the point where the process crashes, resetting, then growing again, etc. * Any information on resource constraints for the process - that is, are you running it in such a way that you've limited the available memory to that specific process (a FreeBSD jail, or whatever the FreeBSD equivalent might be of the Linux "limits" - any limits on heap space, threads, etc. * If you can collect a stack trace during the abort, it would be really useful to drill down and figure where in the code this is being triggered - if it's in one of the four places in the mem.c file that I mentioned, or if something else is causing it. -Nick >
