Update.... I totally disabled Windows Real Time Virus protection.  Apparently it was 'one' of the problems.  However...

Original startup after reboot (with WIndows Real Time Virus scan) -- ~21 minutes

With Windows Real Time Virus scan totally disabled immediately after reboot -- ~3 minutes

Second TC startup -- ~35 seconds.

3 minutes beats 21 minutes, and I guess I can live with it.  But there's still something else slowing it down on the first startup after system reboot.   I'll see about getting some stack dumps.

Thanks for the help.

Jerry


On 5/21/2019 9:55 AM, Christopher Schultz wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Jerry,

On 5/20/19 14:05, Jerry Malcolm wrote:
Andre,

Your theory sounds like a real possibility.  It would explain why
it only blows up the first time after a reboot.  I'll test the
theory tonight during off-peak use hours for the web sites I host.
If André's theory is correct, you will probably find that 2-3 months
ago the version of your virus scanner was upgraded, and all of the
whitelisted applications (like Tomcat) were lost.

If that's the case, you can probably re-whitelist Tomcat and get your
performance back. At least until the next upgrade.

- -chris

On 5/20/2019 3:43 AM, André Warnier (tomcat) wrote:
On 20.05.2019 00:09, Jerry Malcolm wrote:
Just an FYI.... this server has been in production a little
over a year.  This TC load problem only started 2-3 months ago.
So 'something' changed to cause this problem to manifest
itself. It's been too long to try to correlate a specific
server change to the start of this problem. But I just wanted
to point out that this problem has NOT been around since day 1
on this server.
Just in case, since this is a Windows server .. 2-3 months ago,
an update of a virus scanner ? (of the intrusive kind which
pre-checks each file that wants to be opened) Checking may be
easy or not, depending on your access : disable the virus scanner
just the time to start tomcat and check.

Jerry

On 5/19/2019 5:03 PM, Jerry Malcolm wrote:
Rainer,

No change with the urandom parm.  I am attaching a portion of
the Catalina log.  The first half shows between 8 and 15
seconds to deploy each app for a single virtual host (there
are no war files, the app is already exploded in the appbase
dir).  I have quite a few virtual hosts with several apps
each.  That initial server start took 21+ minutes. I then
restarted the TC service and got the ~.5 sec start per webapp
shown in the 2nd half of the log below.

Can you refresh me on how to capture the stack dumps you
suggested? It's been a while....

Thx.

Jerry

First start of Tomcat after server reboot
-------------------------------------------------------------------
- --

[C:\domains\<myDomain>.com\webapps\JSPWiki.war] has finished in
[8,579] ms tory [C:\domains\<myDomain>.com\webapps\cis]
irectory [C:\domains\<myDomain>.com\webapps\cis] has finished
in [11,486] ms tory
[C:\domains\<myDomain>.com\webapps\gallery] irectory
[C:\domains\<myDomain>.com\webapps\gallery] has finished in
[9,204] ms tory [C:\domains\<myDomain>.com\webapps\gl]
irectory [C:\domains\<myDomain>.com\webapps\gl] has finished
in [8,469] ms tory
[C:\domains\<myDomain>.com\webapps\idmanager] irectory
[C:\domains\<myDomain>.com\webapps\idmanager] has finished in
[8,689] ms tory
[C:\domains\<myDomain>.com\webapps\itemtrack] irectory
[C:\domains\<myDomain>.com\webapps\itemtrack] has finished in
[6,907] ms tory
[C:\domains\<myDomain>.com\webapps\malcolment] irectory
[C:\domains\<myDomain>.com\webapps\malcolment] has finished
in [8,469] ms tory
[C:\domains\<myDomain>.com\webapps\notebook] irectory
[C:\domains\<myDomain>.com\webapps\notebook] has finished in
[10,189] ms tory [C:\domains\<myDomain>.com\webapps\order]
irectory [C:\domains\<myDomain>.com\webapps\order] has
finished in [8,501] ms tory
[C:\domains\<myDomain>.com\webapps\payment] irectory
[C:\domains\<myDomain>.com\webapps\payment] has finished in
[14,209] ms tory
[C:\domains\<myDomain>.com\webapps\projectmanager] irectory
[C:\domains\<myDomain>.com\webapps\projectmanager] has
finished in [9,018] ms [C:\Tomcat
9.0\conf\Catalina\myridetx.net\manager.xml] ptor [C:\Tomcat
9.0\conf\Catalina\myridetx.net\manager.xml] has finished in
[62] ms

Restart of Tomcat
-------------------------------------------------------------------
- --

tory [C:\domains\<myDomain>.com\webapps\cis]
irectory [C:\domains\<myDomain>.com\webapps\cis] has finished
in [594] ms tory [C:\domains\<myDomain>.com\webapps\gallery]
irectory [C:\domains\<myDomain>.com\webapps\gallery] has
finished in [547] ms tory
[C:\domains\<myDomain>.com\webapps\gl] irectory
[C:\domains\<myDomain>.com\webapps\gl] has finished in [562]
ms tory [C:\domains\<myDomain>.com\webapps\idmanager]
irectory [C:\domains\<myDomain>.com\webapps\idmanager] has
finished in [578] ms tory
[C:\domains\<myDomain>.com\webapps\itemtrack] irectory
[C:\domains\<myDomain>.com\webapps\itemtrack] has finished in
[547] ms tory [C:\domains\<myDomain>.com\webapps\malcolment]
irectory [C:\domains\<myDomain>.com\webapps\malcolment] has
finished in [579] ms tory
[C:\domains\<myDomain>.com\webapps\notebook] irectory
[C:\domains\<myDomain>.com\webapps\notebook] has finished in
[547] ms tory [C:\domains\<myDomain>.com\webapps\order]
irectory [C:\domains\<myDomain>.com\webapps\order] has
finished in [531] ms tory
[C:\domains\<myDomain>.com\webapps\payment] irectory
[C:\domains\<myDomain>.com\webapps\payment] has finished in
[562] ms tory
[C:\domains\<myDomain>.com\webapps\projectmanager] irectory
[C:\domains\<myDomain>.com\webapps\projectmanager] has
finished in [531] ms

On 5/18/2019 3:18 PM, Rainer Jung wrote:
Most likely it hangs waiting for enough entropy for random
number generator seeding.

Try whether the problem goes away if you add

-Djava.security.egd=file:/dev/urandom

to you process flags. If you are using older Java than Java
8 (not possible for Tomcat 9 but just in case you also have
older software stacks running), then it would be

-Djava.security.egd=file:/dev/./urandom

The cryptic /./ is not a typo, you would need it for Java
before Java 8.

If this doesn't help, then you should try capturing a few
stack dumps (thread dumps) during the long startup time.
People here can help interpret them.

Regards,

Rainer

Am 18.05.2019 um 21:55 schrieb Jerry Malcolm:
This is a weird one.  It started a few months ago.   I
have TC 9 running on Windows Server 16.  After I reboot
the entire server, Tomcat takes forever on startup. It
normally starts in about 30 seconds.  But after a server
reboot it takes up to 15 minutes... chugging along at a
snail's pace starting up all of the apps on all of the
virtual hosts.  It always finally gets there with
everything successfully running. Other servers on the
same box (Apache, JAMES, ISC BIND, MySQL) don't have any
problem starting up. CPU, Disk, Memory, etc. usages are
barely showing on the performance graphs. There's nothing
in the Catalina log or system.err other than showing a
couple of minute gap in the time stamp between each app
as it starts up. If I need to reboot again later, it
boots again in about 30 sec as expected.

First question... any ideas off the top of your head that
might be causing this?

Second question... is there any other logging I can turn
on that might explain what TC is blocking on?

Thanks.

Jerry
------------------------------------------------------------------
- ---

To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org

-------------------------------------------------------------------
- --

To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org

--------------------------------------------------------------------
- -

To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


---------------------------------------------------------------------


To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

---------------------------------------------------------------------


To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlzkEWoACgkQHPApP6U8
pFinLRAAttjrmJnG6lwB3pd7kT5IpBk5H/MwJdXvEX8/S/k6MLLgWqWmqQhgRQAZ
UeVqJFJFDCrgXymxXNI2f7k3saVA+2Lxafjuft9WQGKs+mTSQR0s82VG6Iko5JzL
dZxA5XD+SSU6PafTMQ1hmlVMMDeug+ONqMcUY2XFoZSV6TUk6KqjMiod74We7/b8
7PSqI4VWSS6fA4/ZoPJNRvGLXtYDu1obVX3YysMb4tSky/5OenJX6rz0mwMenY0z
F2p2G2ObNlLKUq7DuAhBcLHB4lbkEv+20Zv6hqMGPlFB8rjiOpUpL86/gfdJY29z
KGgWLAw1M4rw2n0GjVihcfYfjVBA931wWwkaXpa6cXCrcyjOcgCH+OlhIqjtoHkW
w6mM9m99LqrWRwqqfiHDDxqywdqukfGANwi1QvaPgVkd0+FwGkinh5G3P9au3cLs
JKdpGgEog1hJnjkkXWwROlozKiKjMD82V474E4mwjw7nqPI/YXXL5E8VTlhm11JB
OWDZ8yNgwA//Aw7JbGdAUPqRSsSqrKIyGYr4voDd6nhs+83R/7z27IyIxgw/kzFU
6xRFOorztK5dOR5TqPZYB67qUL39++fpdayTYDs7f5Y8guajOsWgVTrvWo3bxCkg
CxUE7V91tWJYJKlxgbFZyZp4vvSzabf4dq5fxKhPTkYH9pmHhCs=
=z1un
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Reply via email to