Hi Nick, We have doubled the memory on this virtual machine from 2GB to 4GB. It has been nearly a month and so far the random signal 6 aborting of the guacd processes have stopped. However, now we're getting a new error message every couple days (see attached). When this happens I need to log in to the server and kill -9 the guacd processes and then restart the process. We had never received this message before doubling the memory. The network in our virtual environment has never been an issue. Any ideas? Thanks!
On Fri, Aug 16, 2024 at 3:05 PM Brian Filipek <bfili...@betterworldbooks.com> wrote: > Hi Nick, > > This is very helpful by narrowing this down to a memory allocation > problem. I have the "top" application open so I'll keep an eye on it. We > gave 2GB of memory to this particular virtual machine. We never have more > than 5 users connecting at a time and all they use it for is RDP. Looks > like there's a guacd process of about 150MB resident memory for each > connection but also an extra guacd process. I assume that's a parent > process. I'm also going to keep an eye on Tomcat though. It's currently > referencing 2GB of memory but with 236MB resident. If Tomcat has a memory > leak maybe that ends up preventing new guacd connections. > > Thank you for your help! I'll see how these numbers look on Monday. > > Brian > > On Fri, Aug 16, 2024 at 10:49 AM Nick Couchman <vn...@apache.org> wrote: > >> On Fri, Aug 16, 2024 at 10:08 AM Brian Filipek >> <bfili...@betterworldbooks.com.invalid> wrote: >> >>> Hi Nick, >>> >>> This problem continues in FreeBSD with guacd randomly signal 6 aborting >>> approximately once a week.When this happens I need to log in and restart >>> the guacd service. We had a crash last night and here are the last few >>> lines of /var/log/messages: >>> >>> Aug 15 11:53:14 guacamole guacd[17699]: Client did not terminate in a >>> timely manner. Forcibly terminating client and any child processes. >>> Aug 15 15:35:14 guacamole guacd[17752]: Client did not terminate in a >>> timely manner. Forcibly terminating client and any child processes. >>> Aug 15 15:59:19 guacamole guacd[32874]: User is not responding. >>> Aug 16 00:09:22 guacamole guacd[65139]: User is not responding. >>> Aug 16 03:05:28 guacamole guacd[79034]: User is not responding. >>> Aug 16 03:25:22 guacamole kernel: pid 75166 (guacd), jid 0, uid 899: >>> exited on signal 6 (no core dump - other error) >>> Aug 16 03:25:22 guacamole kernel: pid 1758 (guacd), jid 0, uid 899: >>> exited on signal 6 (no core dump - other error) >>> Aug 16 03:25:22 guacamole guacd[67047]: Unable to shutdown internal >>> socket for connection $2bbbb77e-d090-441a-b2d5-c337bf55b13c. Corresponding >>> process may remain running but inactive. >>> Aug 16 08:20:40 guacamole login[1783]: ROOT LOGIN (root) ON ttyv0 >>> >>> >> I did a quick Google search on "signal 6" and found that the general >> consensus seems to be that this is a "voluntary" termination by the process >> (guacd), usually triggered by a call to the "abort()" function. A quick >> search through the guacd source code shows that the only times we every >> call this function are in the libguac memory allocation functions ( >> https://github.com/apache/guacamole-server/blob/main/src/libguac/mem.c), >> during memory arithmetic processes (add, subtract, multiple), and during >> the realloc process. So, my _guess_ here is that it is failing to allocate >> required memory for some operation and is aborting. >> >> It's possible there's a memory leak somewhere in guacd that is >> manifesting itself in your particular case on a semi-regular basis, but >> hard to say. The interesting things to try to track this down further would >> be: >> * Any information you have on resources utilization - particularly memory >> - that would show any sort of a trend of memory utilization growing up to >> the point where the process crashes, resetting, then growing again, etc. >> * Any information on resource constraints for the process - that is, are >> you running it in such a way that you've limited the available memory to >> that specific process (a FreeBSD jail, or whatever the FreeBSD equivalent >> might be of the Linux "limits" - any limits on heap space, threads, etc. >> * If you can collect a stack trace during the abort, it would be really >> useful to drill down and figure where in the code this is being triggered - >> if it's in one of the four places in the mem.c file that I mentioned, or if >> something else is causing it. >> >> -Nick >> >>> > > -- > > Regards, > > Brian Filipek > > Technical Support Specialist > > 55740 Currant Road > > Mishawaka, IN 46545 > > (574) 855-5745 > > bfili...@betterworldbooks.com > > betterworldbooks.com > > Better World Books is a for-profit social enterprise that collects and > sells books online with each sale generating funds for literacy initiatives > around the world. > > -- Regards, Brian Filipek Technical Support Specialist 55740 Currant Road Mishawaka, IN 46545 (574) 855-5745 bfili...@betterworldbooks.com betterworldbooks.com Better World Books is a for-profit social enterprise that collects and sells books online with each sale generating funds for literacy initiatives around the world.
--------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@guacamole.apache.org For additional commands, e-mail: user-h...@guacamole.apache.org
