Thanks a lot. Comments below.
On Wed, Feb 3, 2016 at 5:02 PM, Christopher Schultz <
ch...@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Yuval,
>
> On 2/2/16 6:11 PM, Yuval Schwartz wrote:
> > Thanks. Answers below
> >
> > On Tuesday, 2 February 2016, Christopher Schultz <
> > ch...@christopherschultz.net
> > <javascript:_e(%7B%7D,'cvml','ch...@christopherschultz.net');>>
> > wrote:
> >
> > Yuval,
> >
> > On 2/2/16 9:28 AM, Yuval Schwartz wrote:
> >>>> On Tue, Feb 2, 2016 at 4:15 PM, Christopher Schultz <
> >>>> ch...@christopherschultz.net> wrote:
> >>>>
> >>>> Yuval,
> >>>>
> >>>> On 2/2/16 6:21 AM, Yuval Schwartz wrote:
> >>>>>>> Hello again Christoph,
> >>>>>>>
> >>>>>>> I think there is a problem with the way that I
> >>>>>>> installed the APR library. If you recall, I installed a
> >>>>>>> compiled version from the distros with the commands yum
> >>>>>>> install apr-devel openssl-devel tomcat-native
> >>>>>>>
> >>>>>>> I get the desired "Loaded APR based library" in my
> >>>>>>> logs. However, every time I shut down tomcat I get a
> >>>>>>> warning:
> >>>>>>>
> >>>>>>> WARNING [main]
> >>>>>>> org.apache.tomcat.util.net.AbstractEndpoint.shutdownExecutor
> >>>>>>>
> >>>>>>>
> The executor associated with thread pool [http-apr-8080] has
> >>>>>>> not fully shutdown. Some application threads may still
> >>>>>>> be running
> >>>>>>>
> >>>>>>> If I remove the packages that I installed (yum remove
> >>>>>>> apr-devel openssl-devel tomcat-native) then this
> >>>>>>> warning goes away.
> >>>>>>>
> >>>>>>> Is there something that I forgot to configure when
> >>>>>>> installing the APR library?
> >>>>
> >>>> What makes you think that something is wrong with the
> >>>> library? Perhaps you are getting that WARNING because you
> >>>> have still-running request-processing threads?
> >>>>
> >>>>
> >>>>> Thanks, any ideas on how I can look into this then? I've
> >>>>> tried looking at thread dumps and at the threads that are
> >>>>> created and destroyed in my ServletContextListener's
> >>>>> contextInitialize and contextDestroyed methods.
> >
> > So, just to confirm, when you attempt to shut-down Tomcat, you get
> > that WARNING and if you take a thread dump after that warning, you
> > have threads still running contextInitialized and
> > contextDestroyed?
> >
> >
> >> When I shut down I get that message (I am able to shut down
> >> successfully, I just get that message right before). I can't take
> >> a thread dump after I get the message because tomcat is no longer
> >> running (i.e. There are no Java processes taking place for me to
> >> perform a thread dump on).
>
> Try this. Before you are ready to shut down, take note of the pid of
> the JVM process. Then run this command:
>
> $ CATALINA_HOME/bin/shutdown.sh && sleep 2 && jstack [pid] > stack.txt
>
(Nice idea!)
I performed this and it worked.
What stood out was a line in 2 http-apr-8080 threads:
- locked <[some number]> (a
org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper)
Does this tell us something?
>
> This should request a Tomcat shutdown and take a thread dump 2 seconds
> later. If that doesn't work, adjust the sleep time until you get a
> good snapshot. You should see a thread with a name like
> http-apr-8080-[#] that is doing something inside your application.
> then its up to you to figure out why it's still running.
>
As far as I can tell, I only get this WARNING when my elastic load balancer
is pinging pages on my web app.
>
> >> I mentioned contextInitialized and contextDestroyed simply
> >> because I used those methods to print information about threads
> >> that were running in order to try to get a better idea of what's
> >> going on (i.e. Using Thread.getAllStackTraces()).
>
> Well, if you are using contextInitialized and contextDestroyed to dump
> the stacks of all threads, then of course those methods will be listed
> - -- they are currently running (to generate the thread dumps).
>
> > The thread the WARNING is talking about will be one of the HTTP
> > request processor threads (it even tells you which thread pool is
> > still in use: http-apr-8080). If you have some long-running
> > requests, you may get that WARNING during shutdown, but shutdown
> > can still succeed after those threads complete.
> >
> >
> >> When I take a thread dump while the application is running I see
> >> threads called http-apr-8080-exec-1, http-apr-8080-exec-2, etc.
> >> do these threads belong to the pool that the warning is referring
> >> to?
>
> Yes, but while the server is running, you should be expecting to
> handle requests from clients, right? The fact that those threads are
> running is not out of the ordinary. Remember that even threads not
> actively servicing requests will have a non-null stack trace: they'll
> be waiting for new requests.
>
> >> Are you saying I don't need to pay attention to this warning? As
> >> a side question: why should it concern me that a thread might
> >> stay open when tomcat is shutting down? If tomcat is shutting
> >> down then won't all threads be destroyed anyway?
>
> If a request is in-progress when you ask Tomcat to shut down, Tomcat
> will wait for a certain amount of time for those requests to complete.
> If they don't complete within a certain amount of time (configurable:
> see the Configuration section of the user's guide), Tomcat will stop
> them anyway and shut down.
>
> If you get this error every time you shut down, you should find out
> which thread is running long and figure out why.
>
> >>>> If the library loads, I think you installed it properly ;)
> >>>>
> >>>>
> >>>>> Yes, I just reinstalled by compiling the source code and I
> >>>>> still get the same error...so it's not the library.
> >
> > If you are okay with the library version available from your Linux
> > distro, you should stick with that.
> >
> >> I mentioned this point because, interestingly, when I don't use
> >> the apr library (i.e. I use the standard nio) then I don't see
> >> the warning that we're discussing.
>
> So if you use the BIO connector, you never get this warning?
>
> What is your connector configuration?
>
The connector element in $CATALINA_HOME/conf/server.xml looks like this:
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
(there is a commented connector after it with an attribute of
"executor=tomcatThreadPool"...but this is commented out).
>
> - -chris
>
Thank you for your help
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlayFpgACgkQ9CaO5/Lv0PAJ0ACghxKS1I+POEIlrCvIGOrp0nzM
> 0tcAn1Fxi1neSvL5z8bTS5cE+xEEw5VN
> =/4Yk
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
