On Thu, Aug 7, 2014 at 11:35 AM, Peter Rifel <pri...@mixpo.com> wrote:
> Dan, > > On 8/6/14, 4:52 AM, "Daniel Mikusa" <dmik...@pivotal.io> wrote: > > >On Tue, Aug 5, 2014 at 7:54 PM, Peter Rifel <pri...@mixpo.com> wrote: > > > >> Hello, > >> > >> I am in the process of upgrading from Tomcat 7.0.54 to Tomcat 8.0.9 and > >>am > >> seeing some performance degradation in my webapp. > >> > >> Configuration info: > >> > >> Ubuntu 14.04.1 LTS > >> java version "1.8.0_11" > >> Java(TM) SE Runtime Environment (build 1.8.0_11-b12) > >> Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode) > >> > >> Both versions of tomcat use the APR Connector with TC Native 1.1.31 > >> statically linked against APR 1.5.1 and OpenSSL 1.0.1h > >> > >> > >> I'm running identical requests to identical web apps between these > >>tomcats > >> and in Tomcat 8 I'm seeing an ~20% response time increase and as such a > >> similar reqs / sec decrease. After using VisualVM to profile the > >> application I've narrowed it down to the JRE's JAXP library taking > >> significantly longer to create its xml validation "SchemaFactory" which > >> doesn't make sense to me because I'm using the same JRE in both tests. > >>I > >> know that the Resources implementation was overhauled in Tomcat 8 so is > >> there any possibility that this could be a side effect? For a better > >>idea > >> of what I'm talking about, see the below VisualVM screenshots of a > >> profiling snapshot's call tree: > >> > >> > >> Tomcat 7 - http://i.imgur.com/pbX5xBG.jpg > >> Notice how the time spent in the javax.xml.validation.* methods is <6% > >> > >> Tomcat 8 - http://i.imgur.com/Txv20lg.jpg > >> Notice how the time spent in these methods is significantly higher. > >> > >> > >> Again, I'm confused as to how the performance of the JRE's library > >>changes > >> between tomcat versions, can anyone explain this to me or point me in > >>the > >> right direction on where to look? Is it as simple as a config change or > >> will it require a java code change? The XmlUtils.validate method you > >>see > >> in the screenshot is simply the following: > >> > >> javax.xml.validation.SchemaFactory factory = > >> SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema";); > >> > >> Here are the Tomcat configurations: > >> > >> > >> Tomcat 8 server.xml: http://pastebin.com/Tn3d6bXd > >> Tomcat 8 context.xml: http://pastebin.com/reuJCGNq > >> > >> Tomcat 7 server.xml: http://pastebin.com/Kq6SfFd0 > >> Tomcat 7 context.xml: http://pastebin.com/vdp0PGPh > >> > >> > >> Does anyone have any idea why this might be performing so differently > >> between versions of tomcat? Could it be a bug in the JRE? I can try > >>and > >> provide test code for repro if it helps. > >> > > > >Are you using the "endorsed" directory with Tomcat 7 (and not with 8) by > >any chance? > > > > > > > http://tomcat.apache.org/tomcat-7.0-doc/class-loader-howto.html#XML_Parser > >s_and_Java > > > >Dan > > I double checked, and no I'm not using that or any special class-loading > in either tomcat 7 or 8. For reference, here is my CATALINA_OPTS for > both: (The Xverify:none at the end was to fix a problem with the profiler) > > -Djava.library.path=$CATALINA_HOME/lib/apr/osx:$CATALINA_HOME/lib/apr/linux > -javaagent:$CATALINA_HOME/bin/classmexer.jar -XX:+PrintGCDetails > -XX:+PrintGCTimeStamps > -Xloggc:$CATALINA_HOME/logs/garbage-collection_$(date > +'%y_%m_%d__%H_%M').log > -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n > -Dcom.sun.management.jmxremote.port=64850 > -Dcom.sun.management.jmxremote.authenticate=false > -Dcom.sun.management.jmxremote.ssl=false -Djava.net.preferIPv4Stack=true > -Xms128m -Xmx5120m -XX:MaxPermSize=256m -Xverify:none > You could add the "-verbose" option just to double check. That would show you where the classes are being loaded from. It would catch an unintentional change to the class path. Not saying that's the issue, but different classes could certainly be a cause. Verifying that the same versions are being used in both setups would help to rule out that possibility. Also, you mentioned a test case. Do you have a small test case that can reproduce the problem that you can share? Dan
