Any insights into this issue please, Mr. Shultz? Anyone?
​Thanks and regards, Quoc Nguyen Sr. Software Engineer | Octo an IBM Company M: 407.404.4912 quoc.ngu...@octo.us ________________________________ From: Quoc Nguyen <quoc.ngu...@octo.us> Sent: Friday, August 30, 2024 1:03 PM To: Tomcat Users List <users@tomcat.apache.org> Subject: Re: Tomcat takes over 1 minute to stop Thank you Mr. Shultz !!! We've had warnings like the below forever. I can confirm that these have not affected stopping in anyway whatsoever. WARNING [Catalina-utility-11] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [web app name] appears to have started a thread named [thread name] but has failed to stop it. This is very likely to create a memory leak...... We did try to fix this in the form of a context listener servlet to clean up these threads during destroying the context process but, again, this has had no effects whatsoever. =============== Tomcat9w.exe --> Shutdown tab --> Timeout From the logs that I sent, I believe that: Apache Commons Daemon procrun (1.3.4.0 64-bit) --> took this Timeout into account Apache Commons Daemon procrun (1.4.0.0 64-bit) --> did NOT take this Timeout into account ________________________________________ From: Christopher Schultz <ch...@christopherschultz.net> Sent: Thursday, August 29, 2024 5:59 PM To: users@tomcat.apache.org Subject: Re: Tomcat takes over 1 minute to stop Quoc, On 8/29/24 15:23, Quoc Nguyen wrote: > Thank you Mr. Thomas !!! > > Yes sir !!! I noticed that a clean Tomcat 9.0.93 install (as with other > 9.0.9x versions) stops around 1s. I believe that it is so because it has no > managed web apps/resources. Just Tomcat itself. I could be wrong. > > Yes, I noticed that there are warnings of non-daemon threads that weren't > stopped in catalina.log. I read somewhere that they're just warnings; thus > don't affect this process. > > There are no requests running at all while stopping Tomcat. Essentially, > install/deploy different versions (9.0.89, 9.0.9x) of Tomcat with the same > set of non-changing web apps and stop Tomcat via Windows service and record > the stop times. > > Yes, I took the thread dumps while stopping version 9.0.90. There is a thread > "DestroyJavaVM" that, after a few seconds after Tomcat receives the shutdown > signal (maybe after it's done stopping its main stuff), was running for 60s. > > That said, there're no explanations for what happened: > > a) between version 9.0.13 and 9.0.14 with the introduction of "..scheduled > executor to the Server..", which has default wait time of 60s and forces a > Timeout in Tomcat9w.exe to get Tomcat stop under 60s. > b) between version 9.0.89 and 9.0.90 after the upgrade of Apache Commons > Daemon procrun to version 1.4.0.0, which has a default pause of 60s. > > More work/data to confirm: I'm working with Tomcat version 9.0.89 and 9.0.90 > (where I noticed the change) in two different boxes. > > For version 9.0.90 box: started and stopped. Daemon logs below. > > Note: > > * Apache Commons Daemon procrun (1.4.0.0 64-bit) > * exactly 60s wait until finished regardless of the Timeout set in > Tomcat9w.exe > > [2024-08-29 13:41:58] [info] [13472] Apache Commons Daemon procrun (1.4.0.0 > 64-bit) started. > [2024-08-29 13:41:58] [info] [13472] Running Service 'Tomcat9'... > [2024-08-29 13:41:58] [info] [ 9148] Starting service... > [2024-08-29 13:41:58] [error] [12380] Could not create instance of > java/io/FileOutputStream > [2024-08-29 13:41:59] [info] [ 9148] Service started in 1636 milliseconds. > [2024-08-29 13:42:40] [info] [13472] Service SERVICE_CONTROL_STOP signalled. > [2024-08-29 13:42:40] [info] [11996] Stopping service... > [2024-08-29 13:43:06] [info] [11996] Service stop thread completed. > [2024-08-29 13:44:06] [info] [13472] Run service finished. > [2024-08-29 13:44:06] [info] [13472] Apache Commons Daemon procrun finished. > > > For version 9.0.90 box: switched version 9.0.89 of Tomcat9.exe into this box. > Started and stopped. Daemon logs below. > > Note: > > * Apache Commons Daemon procrun (1.3.4.0 64-bit) > * stop time is definitely less than 60s if the Timeout set in Tomcat9w.exe is > less than 60 and ~63s (or 1 min 3 secs as reported) when set to 0 (out of > the box) > * the last two log lines of "out of the box" don't appear in the log for > Timeout being set to 5. Speculation: the process is short-circuited taking > into account the set Timeout. > > The Timeout was set for 5: > > [2024-08-29 14:08:04] [info] [11012] Apache Commons Daemon procrun (1.3.4.0 > 64-bit) started. > [2024-08-29 14:08:04] [info] [11012] Running Service 'Tomcat9'... > [2024-08-29 14:08:04] [info] [14356] Starting service... > [2024-08-29 14:08:05] [error] [ 6740] Could not create instance of > java/io/FileOutputStream > [2024-08-29 14:08:06] [info] [14356] Service started in 1648 milliseconds. > [2024-08-29 14:08:47] [info] [11012] Service SERVICE_CONTROL_STOP signalled. > [2024-08-29 14:08:47] [info] [14432] Stopping service... > [2024-08-29 14:08:58] [info] [14432] Service stop thread completed. > > > The Timeout was set for 0 (out of the box): > > [2024-08-29 14:43:51] [info] [ 8848] Apache Commons Daemon procrun (1.3.4.0 > 64-bit) started. > [2024-08-29 14:43:51] [info] [ 8848] Running Service 'Tomcat9'... > [2024-08-29 14:43:51] [info] [ 8796] Starting service... > [2024-08-29 14:43:52] [error] [ 1688] Could not create instance of > java/io/FileOutputStream > [2024-08-29 14:43:53] [info] [ 8796] Service started in 1641 milliseconds. > [2024-08-29 14:44:47] [info] [ 8848] Service SERVICE_CONTROL_STOP signalled. > [2024-08-29 14:44:47] [info] [15996] Stopping service... > [2024-08-29 14:45:00] [info] [15996] Service stop thread completed. > [2024-08-29 14:46:00] [info] [ 8848] Run service finished. > [2024-08-29 14:46:00] [info] [ 8848] Apache Commons Daemon procrun finished. > > > All that said, I believe the procrun version is the difference. > > Greatly appreciate your help !!! The non-daemon threads are highly likely to be involved in this. I'm not sure why you weren't having any issues in previous versions of Tomcat. You should definitely try to fix those whether or not they are related to this 1-minute stop. What "timeout" are you setting in Tomcat9w.exe? -chris --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
