Hi Jesse -- Thanks for your work on Tapestry. I have noticed an issue around memory usage with our automated tests which extensively use hivemind. I was wondering if anyone has investigated hivemind memory leaks. I am going to do some further investigating as time permits after Tuesday. That being said, we have a publicly-accessible machine with the latest yourkit 7m2 release on it that could be used to get long-running memory information.Contact me off-list about getting access to the instance.This is our production machine - but since we are still in beta we don't have many users (i.e. no one will notice a little memory profiling).
I would suggest the Tapestry community pitch in a little to help gather data to track down this issue - or determine it is a non-issue. One of open source's advantages is the power of the many to solve issues like this. I for one don't want to rely just on jesse to solve what sounds to be very serious barrier to my own personal success. -Pat Moore Amplafi On 8/28/07, Jesse Kuhnert <[EMAIL PROTECTED]> wrote: > > Umm yeah. I have looked in to it as I just said. > > The only thing left is for me to look at this youkit snapshot profile > and play from there but I'm not capable of debugging issues that > aren't reported properly. > > On 8/28/07, Peter Stavrinides <[EMAIL PROTECTED]> wrote: > > Hi Jessie > > > > I'm gonna try update my version of ognl, if that won't work I will > > downgrade. I have not filed any bugs yet, but I sent a couple of emails > > to the list. I don't get any exceptions during development, but that is > > because its not running for 3 or 4 days. I strongly suggest you consider > > looking into it, because I use the standard libraries provided by > > Tapestry through Maven. > > > > my configuration is as follows: > > > > JDK 6.01 32bit JVM (I have also tested on a 64 bit with no luck) > > Tomcat 5.5.20 > > Debian Linux (2.6.15.28 kernel) > > Tapestry 4.1.2-SNAPSHOT with ognl 2.7 > > > > regards, > > Peter > > > > Jesse Kuhnert wrote: > > > I'd downgrade then if I were you. Extensive profiling with yourkit > > > hasn't shed any new light on whatever problems others are having. > > > The OGNL classes indeed aren't being kept in the javassist pool from > > > what I can tell. > > > > > > I have another yourkit snapshot binary to look at so we'll see what > that says. > > > > > > I don't remember - have you filed any bug reports or reported any > > > problems? Are you getting ognl exceptions during development as well > > > as production? Do these errors sound like ummmmm....potential errors > > > that should be looked at further? > > > > > > On 8/28/07, Peter Stavrinides <[EMAIL PROTECTED]> wrote: > > > > > >> Hi Jessie > > >> > > >> Any progress on this? .... sorry to bug you, but I have to take a > > >> decision soon, I have two production machines that will need to > upgrade > > >> or downgrade Tapestry. > > >> > > >> Best wishes, > > >> Peter > > >> > > >> Jon Oakes wrote: > > >> > > >>> Hi Bryan, > > >>> > > >>> I am a relative newbie but I was wondering if you are running with > > >>> caching enabled or disabled? It might make sense to keep things > > >>> around when caching is disabled whereas I think it would clearly be > a > > >>> bug to keep things around with it disabled. > > >>> > > >>> Jon Oakes > > >>> > > >>> > > >>> Jesse Kuhnert wrote: > > >>> > > >>>> Hmmm...well, I don't think I like the sound of any of that. I'm > just > > >>>> going to pretend this problem doesn't exist. (just kidding) > > >>>> > > >>>> I had thought I was doing something special with the ognl > compilations > > >>>> that would cause its generated classes to not hang around > afterwards > > >>>> in any pools. > > >>>> > > >>>> I'll take a look at things this weekend and am sure a fix will > appear > > >>>> in between now and Monday - if there is a reasonable fix to be > found. > > >>>> (in 4.1.3 snapshot form) > > >>>> > > >>>> On 8/24/07, Bryan Dotzour <[EMAIL PROTECTED]> <mailto: > [EMAIL PROTECTED]> wrote: > > >>>> > > >>>> > > >>>>> I and another colleague of mine have been investigating what seems > to be > > >>>>> a "memory leak" in our Tapestry application for about a month > since we > > >>>>> upgraded to T4.1.2. I won't bore you with the saga of the last > month, > > >>>>> but I would like to present the data I've gathered and look to the > list > > >>>>> for a proposed solution. I was reading a recent thread in which > Jesse > > >>>>> said (08/09/2007): > > >>>>> > > >>>>> > > >>>>> > > >>>>> "There is a map that grows as large as the system using it > internally to > > >>>>> javassist of various cached reflection info - but it doesn't leak > in any > > >>>>> way." > > >>>>> > > >>>>> This is precisely what I've found in profiling our application and > it > > >>>>> *appears* to be this map that is causing our applications to > eventually > > >>>>> run out of memory. > > >>>>> > > >>>>> The YourKit profiler shows me that, as time goes on, there is an > > >>>>> instance of HiveMindClassPool that grows and grows as class > instances > > >>>>> are created. This class extends from javassist.ClassPool and is > the map > > >>>>> that Jesse is talking about in his quote above. And he's right, I > > >>>>> wouldn't say that the class pool "leaks" either because it looks > like > > >>>>> it's designed to retain that memory until the class pool itself is > no > > >>>>> longer needed. > > >>>>> > > >>>>> > > >>>>> > > >>>>> Take this quote from the javassist.ClassPool javadocs: > > >>>>> > > >>>>> "Memory consumption memo: > > >>>>> ClassPool objects hold all the CtClasses that have been created so > that > > >>>>> the consistency among modified classes can be guaranteed. Thus if > a > > >>>>> large number of CtClasses are processed, the ClassPool will > consume a > > >>>>> huge amount of memory. To avoid this, a ClassPool object should be > > >>>>> recreated, for example, every hundred classes processed. Note that > > >>>>> getDefault() is a singleton factory. Otherwise, detach() in > CtClass > > >>>>> should be used to avoid huge memory consumption. " > > >>>>> > > >>>>> This huge memory consumption by the ClassPool is what I was > seeing. In > > >>>>> particular it is the ClassPool that is held onto by OgnlRuntime. > > >>>>> Inspecting this object in the profiler showed that it has a map > > >>>>> containing about 45,000 classes. All of the keys into this map > were > > >>>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and > the > > >>>>> values are instances of javassist.CtNewClass. Each entry in this > map > > >>>>> looks like it retains about 1,900 bytes, for a grand total of > about 90 > > >>>>> MB of memory used. > > >>>>> > > >>>>> These numbers came from my staging deployment where I had the > profiler > > >>>>> attached, using some reflection tricks I was able to look at a > > >>>>> production site and found that it had about 240,000 items in that > class > > >>>>> pool.. approximately 450 MB of memory. > > >>>>> > > >>>>> So I guess the questions in my mind are: Why are there so many > classes > > >>>>> in the pool? Why does the number only ever go up? Do those > classes > > >>>>> really need to stay in the pool forever? > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>> > --------------------------------------------------------------------- > > >>> To unsubscribe, e-mail: [EMAIL PROTECTED] For > > >>> additional commands, e-mail: [EMAIL PROTECTED] > > >>> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >> For additional commands, e-mail: [EMAIL PROTECTED] > > >> > > >> > > >> > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > -- > Jesse Kuhnert > Tapestry/Dojo team member/developer > > Open source based consulting work centered around > dojo/tapestry/tacos/hivemind. http://blog.opencomponentry.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
