Thanks again Pat. Tomcat's initial server.xml sets maxThreads to be 150 so its not that.
I'll get some monitoring enabled on the server and see what's happening there.
Its strange that we can see it happening from as little as 2 users though. That
says to me that it definitely looks like some sort of resource contention
issue. Time to keep on plugging.
I'll let anyone interested know if I find it.
Rob Cole
"Patrick Casey" <[EMAIL PROTECTED]>
01/08/2005 14:49
To
"'Tapestry users'" <[email protected]>
cc
Please respond to
"Tapestry users" <[email protected]>
Subject
RE: Tapestry fails performance testing. HELP!
Please see comments below.
--- Pat
PS You *did* turn tapestry's internal page cache back on, right? :)
> -----Original Message-----
> From: Robert Cole [mailto:[EMAIL PROTECTED]
> Sent: Monday, August 01, 2005 6:37 AM
> To: Tapestry users
> Subject: RE: Tapestry fails performance testing. HELP!
>
>
> Hi Pat,
>
> To answer your questions:
>
> 1) We shouldn't be as we can see the problem from as little as 5
> concurrent users. We're on a 2CPU server with plenty of memory so its
> unlikely, but not something to rule out.
As you said, sounds unlikely, but it's definatley worth whipping
out
permon on the server and graphic CPU and disk utilization vs number of
connections. If you're saturating out within a connection or two then you've
got a run of the mill gremlin hunt because your problem isn't tapestry or
zee web per-se, it's just that some of your code is slow :). Not the likely
case though, I agree.
> 2) Hmm. Do you mean the HTTPConnector? If so its 150, i.e. the default.
> Otherwise I'm not sure what you mean. Could you point me in the right
> direction, e.g. config file? We've not changed anything on that side since
> installing Tomat
No, I mean the number of handler threads defined in the web.xml.
<Connector port="9090" maxThreads="150" minSpareThreads="2"
maxSpareThreads="75" enableLookups="false" redirectPort="8443"
acceptCount="100" debug="0" connectionTimeout="20000"
disableUploadTimeout="true" />
Make sure maxThreads is a decent size number.
> 3) I've checked this. There was one section (in our entire code base) that
> was synchronized and this was a simple log.debug statement that is carried
> out at server start up so it shouldn't be an issue
Probably not a synchronization issue, although they can sometimes
crop up at strange times e.g. if you're using libraries or collections that
are *themselves* synchronized internally (like Hashtable for example).
> 4) On server start we cache our database into memory (its not that large,
> < 10 megs) so we don't rely on any database connections
I'd still take a quick look at your data access paths as a likely
first culprit. Generally when mutli-threaded apps have scaling problems, the
first place to look is at contention over shared resources. Once you rule
out the hardware, that leaves things like your static variables, anything
you pool internally, etc.
>
> Thanks,
>
> Rob Cole
>
>
>
>
>
> "Patrick Casey" <[EMAIL PROTECTED]>
>
> 01/08/2005 14:21
> To
>
> "'Tapestry users'" <[email protected]>
>
> cc
>
> Please respond to
>
> "Tapestry users" <[email protected]>
> Subject
>
> RE: Tapestry fails performance testing. HELP!
>
>
>
>
>
>
>
>
>
>
>
> Things to check:
>
> 1) Are you physically saturating your CPU or disk? If so,
> that's
> that and you're hardware limited.
>
> 2) How many threads is your tomcat set to? If it's set to one
> thread, it'll de-facto serialize everything. A lot of people set it to one
> thread for development and forget to set it back for unit test.
>
> 3) Are you synchronizing internally around a slow resource
> e.g. the
> database, the network, etc? Snippets of code like:
>
> Synchronized (session) {
> List l = query.execute();
> }
>
> Can be murder in production.
>
> 4) How many database connections do you have in your pool?
> Try using
> more.
>
> When all else fails, either get out a profiler and see where
> your
> time is going or instrument your sample transaction with lots of
> log.debug()
> calls to see where your time is going.
>
> --- Pat
>
> > -----Original Message-----
> > From: Robert Cole [mailto:[EMAIL PROTECTED]
> > Sent: Monday, August 01, 2005 5:31 AM
> > To: [email protected]
> > Subject: Tapestry fails performance testing. HELP!
> >
> > Hi all,
> >
> > First off: Tapestry 3.0.1, Tomcat 5.5.9, JDK 1.5.0_1
> >
> > We've built our application in Tapestry and we're now trying to do some
> > performance testing to ensure that we will be able to support our user
> > base. However, we've hit a performance/scalability issue that we need to
> > resolve before we go live.
> >
> > The test we're doing is for a user to go to 1 page, then another page 5
> > times. This ensures that the session is up and running so that our
> > DirectLink works for the 2nd page. We're using The Grinder to test the
> > server. So far I've tested 1, 2, 5, 10, 20, 40 and 80 concurrent users.
> >
> > The results show that we are scaling nice and linearly, but the problem
> is
> > it looks like Tapestry is almost dealing with our requests sequentially
> > rather than in parallel. For example take the results below:
> >
> > Users Time(ms)
> > 1 662
> > 2 1280
> > 5 2140
> > 10 4900
> > 20 9120
> > 40 20200
> >
> > As you can see, a response time of 20.2 seconds for 40 users is a bit
> > much.
> >
> > So far I've checked the Tapestry Wiki, where the performance page is
> still
> > being built despite being a common question, the "Tapestry In Action"
> book
> > (absolutely nothing on performance) and attempted to search the
> archives.
> > The archives redirect me to a page that groups threads into months and
> > doesn't allow you to search. The "searchable" archive link just takes
> you
> > back to the un-searchable one.
> >
> > So, can anyone offer a couple of pointers as to where to go? I take it
> > Tapestry does NOT use the single threaded servlet model? Are there any
> > pools that we should configure? Is there a common coding mistake that we
> > could have made that will impact performance? Is there anything in
> > particular that Tapestry requires in Tomcat? I've checked our code for
> any
> > threading/synchronized issues and there are none that are obvious.
> >
> > Any ideas would be gratefully received.
> >
> > Thanks,
> >
> > Rob Cole
> >
> >
> >
> > --
> >
> > This e-mail may contain confidential and/or privileged information. If
> you
> > are not the intended recipient (or have received this e-mail in error)
> > please notify the sender immediately and destroy this e-mail. Any
> > unauthorized copying, disclosure or distribution of the material in this
> > e-mail is strictly forbidden.
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
>
>
>
>
> --
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy this e-mail. Any
> unauthorized copying, disclosure or distribution of the material in this
> e-mail is strictly forbidden.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and destroy this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is strictly
forbidden.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
