Just to check ... which version of Tapestry? On Tue, Jun 29, 2010 at 8:48 AM, Blower, Andy <andy.blo...@proquest.co.uk> wrote: > I doubt anyone remembers this thread except me, but we're still having > problems that I could do with help on. > > So, two months later, we have managed to improve things significantly, but > still have very large pages and we're having a lot of trouble with the page > pool size. What we see when we push the app too hard is that the heap fills > until the gc thread is permanently running, and the app is essentially dead > and unresponsive. I've been testing with 2g & 4g heaps up to now, with 100ms > soft-wait & 200 hard limit because we need a lot of instances of some pages, > and were running out with a 100 limit. I know I need to understand and do > more with these values, but it's worrying because our tests are only > requesting English pages at present and we'll have 17 languages for launch! > So I'm rather concerned to say the least... > > I have a theory about what is happening that I'd like to run by the list and > see what others think. I'm having some trouble figuring out what's happening > at load. > > What I think is that when the heap gets full-ish at high load, the gc kicks > in more and more often, slowing request processing down a little. (there's a > fair amount of soft referenced stuff it can boot) This means that pages are > returned to the page pool a bit later causing more page instances to be > created which fills the heap and enters an vicious cycle with the gc trying > to free memory and the T5 page pool trying to create more page instances. The > reason I suspect this is because heap dumps show about 40 PageImpl's that > have not completed loading, and many more that have but are much smaller in > size than others for the same page. (e.g. a Results PageImpl retaining 28k > when a normal one retains 7.5Mb - very big I know, but I've cut it down from > 14Mb) > > Our site has 223 pages which consume 213Mb if 1 instance of each is > instantiated. Multiply this by 200 (hard limit) and then by 17 languages and > worst case we'll need a heap size of 725Gb which is a little ridiculous! I > will do more work on reducing the size of the pages but I've already done the > easy stuff. > > Here's a list of the heap dominators using Eclipse MAT, > http://lh6.ggpht.com/_YwJn8TJTqJU/TCoNT11ouZI/AAAAAAAAACI/G8eTGsth4zM/HeapDominators.png > > Anyone think I'm on the right track, or barking up the wrong tree completely? > > Thanks, > > Andy > >> -----Original Message----- >> From: Blower, Andy [mailto:andy.blo...@proquest.co.uk] >> Sent: 20 April 2010 17:42 >> To: 'Tapestry users' >> Subject: RE: Tapestry using 1.3Gb of heap space after capacity testing >> >> Thanks for the link. As I said before, I don't think that we've fallen >> prey to that exactly. Doing some local testing and looking at one of >> our largest pages which displays search results, it has 76 component >> objects, each of those will have components nested within (pretty >> heavily in some cases) but nothing which has a bunch of components >> where only one is used and the others are redundant. (what I think is >> this uber-component thing) >> >> It looks like explicit repeated use of a component, rather than using a >> single component in a loop, creates a lot more component objects in the >> heap. I've never seen anything warning against this, and we do usually >> use loops but it's not always the best solution. >> >> Again, looking at our results page, a single instance seems to get >> 11,283 ComponentPageElementImpl instances created for it. I'm finding >> it hard to find a good view of the tree to figure out what component(s) >> are the main culprits for this huge number of instances. >> >> Thanks for any help or guidance you can give me. >> >> Andy >> >> > -----Original Message----- >> > From: Thiago H. de Paula Figueiredo [mailto:thiag...@gmail.com] >> > Sent: 20 April 2010 15:56 >> > To: Tapestry users >> > Subject: Re: Tapestry using 1.3Gb of heap space after capacity >> testing >> > >> > On Tue, 20 Apr 2010 11:41:25 -0300, Blower, Andy >> > <andy.blo...@proquest.co.uk> wrote: >> > >> > > Uber-component anti-pattern, not to my knowledge, but we have a lot >> > of >> > > pages and components now. And a lot of logic in them. The project's >> > > large enough I can't be sure. Do you have a handy link to Howard's >> > > description so I can get my team to check? >> > >> > It's here: >> > http://old.nabble.com/-T5.0.18--Out-of-Memory-Error---Potential-Leak- >> > (doesn't-reduce-after-forced-GC)-to25403474s302.html#a25497441 >> > That thread is interesting, by the way. The problem is more about how >> > components and pages are written than the number of them. >> > What Tapestry version are you using? >> > >> > > Page pool config is default, I don't know enough to fiddle yet. >> Also, >> > > the test is on a single stand alone server with no clustering, we >> are >> > > using Tomcat clustering for production. >> > >> > You should take a look a them, as they can affect the memory >> > consumption >> > directly. >> > >> > > I did fire the GC manually before getting concerned BTW. A lot of >> > heap >> > > seems to be taken up with 100-300k descriptions on the >> > > InternalClassTransformationImpl class. An example of one is >> attached >> > > below - seems wrong to me and more like the debug that can be >> > requested >> > > through log4j rather than a description. >> > >> > They look really bigger than they should. 300k is a lot. I have one >> > application with a reasonable number of pages and components and it >> > runs >> > inside a 512 MB slide at SliceHost without using the whole memory. >> > >> > -- >> > Thiago H. de Paula Figueiredo >> > Independent Java, Apache Tapestry 5 and Hibernate consultant, >> > developer, >> > and instructor >> > Owner, software architect and developer, Ars Machina Tecnologia da >> > Informação Ltda. >> > http://www.arsmachina.com.br >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org >> > For additional commands, e-mail: users-h...@tapestry.apache.org >> > > >
-- Howard M. Lewis Ship Creator of Apache Tapestry The source for Tapestry training, mentoring and support. Contact me to learn how I can get you up and productive in Tapestry fast! (971) 678-5210 http://howardlewisship.com --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
