Thank you Mr. Thomas !!! Yes sir !!! I noticed that a clean Tomcat 9.0.93 install (as with other 9.0.9x versions) stops around 1s. I believe that it is so because it has no managed web apps/resources. Just Tomcat itself. I could be wrong.
Yes, I noticed that there are warnings of non-daemon threads that weren't stopped in catalina.log. I read somewhere that they're just warnings; thus don't affect this process. There are no requests running at all while stopping Tomcat. Essentially, install/deploy different versions (9.0.89, 9.0.9x) of Tomcat with the same set of non-changing web apps and stop Tomcat via Windows service and record the stop times. Yes, I took the thread dumps while stopping version 9.0.90. There is a thread "DestroyJavaVM" that, after a few seconds after Tomcat receives the shutdown signal (maybe after it's done stopping its main stuff), was running for 60s. That said, there're no explanations for what happened: a) between version 9.0.13 and 9.0.14 with the introduction of "..scheduled executor to the Server..", which has default wait time of 60s and forces a Timeout in Tomcat9w.exe to get Tomcat stop under 60s. b) between version 9.0.89 and 9.0.90 after the upgrade of Apache Commons Daemon procrun to version 1.4.0.0, which has a default pause of 60s. More work/data to confirm: I'm working with Tomcat version 9.0.89 and 9.0.90 (where I noticed the change) in two different boxes. For version 9.0.90 box: started and stopped. Daemon logs below. Note: * Apache Commons Daemon procrun (1.4.0.0 64-bit) * exactly 60s wait until finished regardless of the Timeout set in Tomcat9w.exe [2024-08-29 13:41:58] [info] [13472] Apache Commons Daemon procrun (1.4.0.0 64-bit) started. [2024-08-29 13:41:58] [info] [13472] Running Service 'Tomcat9'... [2024-08-29 13:41:58] [info] [ 9148] Starting service... [2024-08-29 13:41:58] [error] [12380] Could not create instance of java/io/FileOutputStream [2024-08-29 13:41:59] [info] [ 9148] Service started in 1636 milliseconds. [2024-08-29 13:42:40] [info] [13472] Service SERVICE_CONTROL_STOP signalled. [2024-08-29 13:42:40] [info] [11996] Stopping service... [2024-08-29 13:43:06] [info] [11996] Service stop thread completed. [2024-08-29 13:44:06] [info] [13472] Run service finished. [2024-08-29 13:44:06] [info] [13472] Apache Commons Daemon procrun finished. For version 9.0.90 box: switched version 9.0.89 of Tomcat9.exe into this box. Started and stopped. Daemon logs below. Note: * Apache Commons Daemon procrun (1.3.4.0 64-bit) * stop time is definitely less than 60s if the Timeout set in Tomcat9w.exe is less than 60 and ~63s (or 1 min 3 secs as reported) when set to 0 (out of the box) * the last two log lines of "out of the box" don't appear in the log for Timeout being set to 5. Speculation: the process is short-circuited taking into account the set Timeout. The Timeout was set for 5: [2024-08-29 14:08:04] [info] [11012] Apache Commons Daemon procrun (1.3.4.0 64-bit) started. [2024-08-29 14:08:04] [info] [11012] Running Service 'Tomcat9'... [2024-08-29 14:08:04] [info] [14356] Starting service... [2024-08-29 14:08:05] [error] [ 6740] Could not create instance of java/io/FileOutputStream [2024-08-29 14:08:06] [info] [14356] Service started in 1648 milliseconds. [2024-08-29 14:08:47] [info] [11012] Service SERVICE_CONTROL_STOP signalled. [2024-08-29 14:08:47] [info] [14432] Stopping service... [2024-08-29 14:08:58] [info] [14432] Service stop thread completed. The Timeout was set for 0 (out of the box): [2024-08-29 14:43:51] [info] [ 8848] Apache Commons Daemon procrun (1.3.4.0 64-bit) started. [2024-08-29 14:43:51] [info] [ 8848] Running Service 'Tomcat9'... [2024-08-29 14:43:51] [info] [ 8796] Starting service... [2024-08-29 14:43:52] [error] [ 1688] Could not create instance of java/io/FileOutputStream [2024-08-29 14:43:53] [info] [ 8796] Service started in 1641 milliseconds. [2024-08-29 14:44:47] [info] [ 8848] Service SERVICE_CONTROL_STOP signalled. [2024-08-29 14:44:47] [info] [15996] Stopping service... [2024-08-29 14:45:00] [info] [15996] Service stop thread completed. [2024-08-29 14:46:00] [info] [ 8848] Run service finished. [2024-08-29 14:46:00] [info] [ 8848] Apache Commons Daemon procrun finished. All that said, I believe the procrun version is the difference. Greatly appreciate your help !!! ________________________________________ From: Mark Thomas <ma...@apache.org> Sent: Thursday, August 29, 2024 9:22 AM To: users@tomcat.apache.org Subject: Re: Tomcat takes over 1 minute to stop On 27/08/2024 21:41, Christopher Schultz wrote: > Quoc, > > On 8/27/24 14:58, Quoc Nguyen wrote: >> Apache Tomcat version: 9.0.90 and above >> OS: Windows Server 2019 >> >> Tomcat Version Used Time Taken to stop >> >> Apache Tomcat/8.5.66 ~ 9 seconds >> Apache Tomcat/9.0.1 ~ 9 seconds >> Apache Tomcat/9.0.10 ~ 9 seconds >> Apache Tomcat/9.0.13 ~ 9 seconds >> Apache Tomcat/9.0.14 ~ 1 min 3 secs >> Apache Tomcat/9.0.16 ~ 1 min 3 secs >> Apache Tomcat/9.0.20 ~ 1 min 3 secs >> Apache Tomcat/9.0.30 ~ 1 min 3 secs >> Apache Tomcat/9.0.40 ~ 1 min 3 secs >> Apache Tomcat/9.0.44 ~ 1 min 3 secs >> Apache Tomcat/9.0.46 ~ 1 min 3 secs >> Apache Tomcat/9.0.75 ~ 1 min 3 secs >> Apache Tomcat/9.0.89 ~ 1 min 3 secs >> Apache Tomcat/9.0.90 ~ 1 min 3 secs >> Apache Tomcat/9.0.93 ~ 1 min 3 secs >> >> From Tomcat changelog >> (https://tomcat.apache.org/tomcat-9.0-doc/changelog.html#Tomcat_9.0.14_(markt): >> Version 9.0.14 "Add a scheduled executor to the Server, which can be used >> to process periodic utility tasks. The utility threads are non daemon by >> default. (remm)", which has the default timeout of 60s for procrun. >> >> Workaround: to reduce the default timeout of 60s, I can supply it via >> Tomcat Monitor (Tomcat9w.exe) or while creating the Windows service. >> >> This workaround has worked up to version 9.0.89 and has stopped for >> version 9.0.90 and above. The difference is version 9.0.89 and lower >> uses Apache Commons Daemon procrun (1.3.x.0 64-bit) whereas version >> 9.0.90 and above uses Apache Commons Daemon procrun (1.4.0.0 64-bit). >> >> Questions: >> >> 1) Am I on the right rack with this procrun timeout? No. >> >> 2) If #1 is yes, does the upgrade from Apache Commons Daemon procrun >> (1.3.x.0 64-bit) to Apache Commons Daemon procrun (1.4.0.0 64-bit) >> cause the supplied timeout to be ignored so that the default of 60s is >> always in effect? >> >> 2) If #2 is yes, is there a workaround while waiting for a fix or must >> wait for a fix? > > Since this symptom lasts for more than a minute and (I assume) is > trivially reproducible, you should take a thread dump to find out what > Tomcat is doing during that time. It is unlikely to be Tomcat. A clean Tomcat 9.0.93 install stops in around 1 second. It will be something the application is doing. Probably with a non-daemon thread or maybe a long running request. +1 to the thread dump suggestion. Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
