Actually Robert, I'd love it if you could patch/override T5 core just enough to disable SoftReferences and re-run your test. The results may surprise you. I could almost guarantee you'd see the same performance pattern for any modern jpa 2.x application. At 1.2GB, it doesn't look like your test setup is just a synthetic, lightweight t5 app with no back end, is it?
Kalle On Apr 1, 2015 3:44 PM, "Kalle Korhonen" <kalle.o.korho...@gmail.com> wrote: > A configurable cache might be ok but what Robert is showing is a highly > typical performance degradation pattern for any sufficiently large Java > application. Tapestry's page cache is hardly the only place where soft > references are used. When your memory budget is too small, most system > engineers would argue that it's far better to slow down the application > than OoM, but obviously that depends on the type of application and the > traffic patterns you are facing. For the consumer facing application, it's > not uncommon to see peak traffic 30-100 times over the averages at least > with the applications I've been involved with and I would hate to to budget > all resources based on peak consumption only. On the other hand, if the > number of pages on the site is small and the site is evenly in use, then > sure, it'd make sense to never purge. > > Kalle > > On Wed, Apr 1, 2015 at 3:01 PM, Howard Lewis Ship <hls...@gmail.com> > wrote: > >> I'm feeling that Robert is making a very good case here. I could imagine a >> page-level annotation to either enable or disable evication of a page >> instance after a period of time ... but that can come later. I do think >> that hard-caching of pages will leading to more predictable response >> performance. >> >> On Wed, Apr 1, 2015 at 7:31 AM, Robert Schmelzer <rob...@schmelzer.cc> >> wrote: >> >> > Hi, >> > >> > I now found time to sum up a short report about that topic. >> > >> > I summarized my results in following pdf file: >> > http://www.schmelzer.cc/Downloads/Files/Tapestry-Memory-Performance.pdf >> > >> > The main issue is, that you are able to bring a Tapestry based system >> into >> > a situation where it gets slower and slower and finally event not >> > responding any more, just be decreasing memory on the JVM and you DO NOT >> > get any error message or OutOfMemory warning or GC overhead warning. >> And >> > that only because the PageImpl instances are held in SoftReferences. My >> > opinion is still, that this does not work as it is supposed to do and I >> > keep with my example about other infrastructure. You would not expect >> e.g. >> > Spring beans or a hibernate configuration to get thrown away under >> memory >> > preasure - you would expect them to fail with OutOfMemory if they are >> not >> > able to hold their necassary static information in memory. >> > >> > Regards >> > Robert >> > >> > >> > >> > >> > Am 19.03.2015 um 17:55 schrieb Kalle Korhonen: >> > >> >> On Thu, Mar 19, 2015 at 12:24 AM, Robert Schmelzer <rob...@schmelzer.cc >> > >> >> wrote: >> >> >> >> Sorry, I was unprecise - my example should have referenced to the >> >>> EntityManagerFactory (SessionFactoryImpl in Hibernate). You would not >> >>> expect them, to throw away its cached configuration on memory >> preasure. I >> >>> do not either expect that from Tapestry. >> >>> I cannot make our results public because of regulatory issues. I will >> try >> >>> to setup a show case for that and will offer a patch. This will take >> me a >> >>> few days. >> >>> >> >>> I don't think we are going to simply do away with the SoftReferences >> >> without any replacements so I wouldn't even attempt at offering such a >> >> patch. I just don't agree that a memory cache should be permanent >> >> construct. If your object is not in a cache, you'll simply incur a >> cache >> >> miss and re-create the object on the fly. It is not typical that a >> cache >> >> will grow indefinitely. If you are adamant on this approach, you could >> >> probably convince us to add a symbol to control the cache behavior >> (i.e. >> >> to >> >> never purge objects from it). Guava has excellent, easily configurable >> >> cache implementations. >> >> >> >> Kalle >> >> >> >> >> >> Robert >> >>> >> >>> Am 18.03.2015 um 18:19 schrieb Kalle Korhonen: >> >>> >> >>> On Wed, Mar 18, 2015 at 12:44 AM, Robert Schmelzer >> <rob...@schmelzer.cc >> >>> > >> >>> >> >>>> wrote: >> >>>> >> >>>> I do not agree with you on that point. Tapestry is designed to >> cache >> >>>> the >> >>>> >> >>>>> page. When you do not have enough memory to hold your pages cached >> >>>>> basically the system does not work as designed so you should fail >> >>>>> early. >> >>>>> Otherwise you possible defer the problem to production use. Fail >> early >> >>>>> means you should try to see the problem in the early stages on dev, >> >>>>> where >> >>>>> you try out all your pages. As I mentioned in my other post - you >> would >> >>>>> also not expect the EntityManager to work soft-refereences or spring >> >>>>> application context to work soft referenced. >> >>>>> >> >>>>> >> >>>>> That's the definition of a memory cache - it trades memory for >> better >> >>>>> >> >>>> performance. The primary use case for soft refences is for caching so >> >>>> seems >> >>>> to me it works exactly as designed. Your comparison to the >> EntityManager >> >>>> is >> >>>> flawed since it's created per request. An EntityManager is designed >> to >> >>>> be >> >>>> inexpensive to create. There are many areas that need improvements in >> >>>> Tapestry but this is not one in my opinion. However, you seem to >> >>>> strongly >> >>>> think otherwise, so you probably have some data to back this up. Do >> you >> >>>> have a memory dump and trending cpu/memory charts of a sufficiently >> >>>> large >> >>>> system you can share with us to demonstrate the problem? Jvisualvm >> >>>> snapshots should work fine. And furthermore - how would you like this >> >>>> changed? If it's just adding a Page as a threadlocal, perhaps you can >> >>>> just >> >>>> write a patch for it. >> >>>> >> >>>> Kalle >> >>>> >> >>>> >> >>>> Am 18.03.2015 um 04:23 schrieb Kalle Korhonen: >> >>>> >> >>>>> In my opinion, soft referencing page objects is highly >> appropriate >> >>>>> usage >> >>>>> >> >>>>> here. If there's pressure on the available memory, it makes sense >> to >> >>>>>> trade >> >>>>>> performance for memory instead of exiting with OoM. This is simple >> >>>>>> condition to detect and should be visible with any reasonable >> >>>>>> monitoring >> >>>>>> tool. If you are hitting memory limits, you'll need to allocate >> more >> >>>>>> memory >> >>>>>> for the application for optimal performance. Soft references are >> >>>>>> especially >> >>>>>> useful here because you can optimize its behavior with the >> >>>>>> -client/-server >> >>>>>> setting depending on your preferences. >> >>>>>> >> >>>>>> Kalle >> >>>>>> >> >>>>>> On Tue, Mar 17, 2015 at 4:26 PM, Howard Lewis Ship < >> hls...@gmail.com> >> >>>>>> wrote: >> >>>>>> >> >>>>>> Possibly we need something more advanced; our own reference type >> >>>>>> that >> >>>>>> can >> >>>>>> >> >>>>>> react to memory pressure by discarding pages that haven't been >> used >> >>>>>>> in >> >>>>>>> configurable amount of time. >> >>>>>>> >> >>>>>>> Or perhaps we could just assume that any page that has been used >> once >> >>>>>>> need >> >>>>>>> to be used in the future and get rid of the SoftReference entirely >> >>>>>>> (or >> >>>>>>> otherwise janitorize it in some way). >> >>>>>>> >> >>>>>>> On Tue, Mar 17, 2015 at 1:24 AM, Robert Schmelzer >> >>>>>>> <rob...@schmelzer.cc >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>> Hello, >> >>>>>>> >> >>>>>>> I recently came accross the implementation of PageSourceImpl >> where >> >>>>>>>> PageImpl instances are softly refereneced into the pageCache: >> >>>>>>>> >> >>>>>>>> private final Map<CachedPageKey, SoftReference<Page>> pageCache = >> >>>>>>>> CollectionFactory.newConcurrentMap(); >> >>>>>>>> >> >>>>>>>> This implementation caused troubles, when you bring your system >> into >> >>>>>>>> memory preassure. The JVM will start to throw away the PageImpl >> to >> >>>>>>>> free >> >>>>>>>> >> >>>>>>>> up >> >>>>>>>> >> >>>>>>> memory - but during request processing he needs the PageImpl >> again >> >>>>>>> and >> >>>>>>> >> >>>>>>>> starts creating it again. So basically you end up loosing your >> >>>>>>>> pageCache >> >>>>>>>> >> >>>>>>>> at >> >>>>>>>> >> >>>>>>> all and start creating the PageImpl instances on every request, >> >>>>>>> which >> >>>>>>> >> >>>>>>>> take >> >>>>>>>> >> >>>>>>> way to much time and takes load onto the CPU. So basically you >> are >> >>>>>>> >> >>>>>>>> hiding a >> >>>>>>>> >> >>>>>>> memory problem by making the system slow and raise CPU load. >> >>>>>>> >> >>>>>>>> I would suggest to use "normal" references for the PageCache or >> at >> >>>>>>>> least >> >>>>>>>> only do SoftReferences only when not in production mode. >> Otherwise >> >>>>>>>> we >> >>>>>>>> are >> >>>>>>>> going to cover memory problems for too long. >> >>>>>>>> >> >>>>>>>> What do you think about that? >> >>>>>>>> >> >>>>>>>> Robert >> >>>>>>>> >> >>>>>>>> ------------------------------------------------------------ >> >>>>>>>> --------- >> >>>>>>>> To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org >> >>>>>>>> For additional commands, e-mail: users-h...@tapestry.apache.org >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> -- >> >>>>>>>> >> >>>>>>> Howard M. Lewis Ship >> >>>>>>> >> >>>>>>> Creator of Apache Tapestry >> >>>>>>> >> >>>>>>> The source for Tapestry training, mentoring and support. Contact >> me >> >>>>>>> to >> >>>>>>> learn how I can get you up and productive in Tapestry fast! >> >>>>>>> >> >>>>>>> (971) 678-5210 >> >>>>>>> http://howardlewisship.com >> >>>>>>> @hlship >> >>>>>>> >> >>>>>>> >> >>>>>>> ------------------------------------------------------------ >> >>>>>>> --------- >> >>>>>>> >> >>>>>> To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org >> >>>>> For additional commands, e-mail: users-h...@tapestry.apache.org >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> --------------------------------------------------------------------- >> >>> To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org >> >>> For additional commands, e-mail: users-h...@tapestry.apache.org >> >>> >> >>> >> >>> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org >> > For additional commands, e-mail: users-h...@tapestry.apache.org >> > >> > >> >> >> -- >> Howard M. Lewis Ship >> >> Creator of Apache Tapestry >> >> The source for Tapestry training, mentoring and support. Contact me to >> learn how I can get you up and productive in Tapestry fast! >> >> (971) 678-5210 >> http://howardlewisship.com >> @hlship >> > >
