On Thu, Jul 2, 2020 at 7:21 AM Coty Sutherland <csuth...@apache.org> wrote:
> The RHEL 7 Tomcat package uses systemd and the journal to capture stdout > instead of catalina.out. Did you check the journal to see if the thread > dumps are logged there from your invocations of kill -3? You can use > `journalctl -u tomcat` to check it. > > On Wed, Jul 1, 2020 at 6:58 PM Sean Neeley <sean.nee...@producepro.com> > wrote: > > > On Wed, Jul 1, 2020 at 5:24 PM calder <calder....@gmail.com> wrote: > > > > > On Wed, Jul 1, 2020, 15:32 Sean Neeley <sean.nee...@producepro.com> > > wrote: > > > > > > > I tried switching from Java 1.8 to Java 11 to see if that makes a > > > > difference. Now the VM Thread is using a lot less CPU: > > > > > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > > > COMMAND > > > > 2320 tomcat 20 0 4659072 47872 19904 R 99.9 0.6 22:15.16 > java > > > > 2326 tomcat 20 0 4659072 47872 19904 R 4.6 0.6 0:56.43 VM > > > > Thread > > > > > > > > I tried running jstack on the processes, but I get this: > > > > > > > > 2320: Unable to open socket file: target process not responding or > > > HotSpot > > > > VM not loaded > > > > > > > > > > Did you attempt to run the command as the "Tomcat user"? > > > > > > BTW, Oracle recommends the use of "jcmd" over "jstack". Personally, > I'd > > > give Mission Control/Flight Recorder a go. > > > > > > > I'm definitely running it as the tomcat user. I just tried jcmd with no > > arguments and the command completely hangs. The only way to terminate it > > is a kill -9. This seems almost like an OS level issue. We are opening > a > > ticket with Red Hat support to see what they say. > > > Thanks for the tip Coty. The logs show a normal start. The kill -3 output does not appear in the logs. I then did a kill -9 and that showed up. Logs: -- Logs begin at Wed 2020-07-01 12:37:09 PDT, end at Thu 2020-07-02 06:36:59 PDT. -- Jul 02 06:36:31 ecom-main.spokaneproduce.com systemd[1]: Started Apache Tomcat We b Application Container. -- Subject: Unit tomcat.service has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit tomcat.service has finished starting up. -- -- The start-up result is done. Jul 02 06:36:32 ecom-main.spokaneproduce.com server[26466]: Java virtual machine used: /usr/lib/jvm/jre/bin/java Jul 02 06:36:32 ecom-main.spokaneproduce.com server[26466]: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar Jul 02 06:36:32 ecom-main.spokaneproduce.com server[26466]: main class used: org.apache.catalina.startup.Bootstrap Jul 02 06:36:32 ecom-main.spokaneproduce.com server[26466]: flags used: Jul 02 06:36:32 ecom-main.spokaneproduce.com server[26466]: options used: -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager Jul 02 06:36:32 ecom-main.spokaneproduce.com server[26466]: arguments used: start Jul 02 06:36:59 ecom-main.spokaneproduce.com systemd[1]: tomcat.service: main process exited, code=killed, status=9/KILL Jul 02 06:36:59 ecom-main.spokaneproduce.com systemd[1]: Unit tomcat.service entered failed state. Jul 02 06:36:59 ecom-main.spokaneproduce.com systemd[1]: tomcat.service failed.
