yep but this is not enough, almost each "component" is doing n++. This is fast but since done under a lock if it can be removed it should. Romain Manni-Bucau Twitter: @rmannibucau Blog: http://rmannibucau.wordpress.com/ LinkedIn: http://fr.linkedin.com/in/rmannibucau Github: https://github.com/rmannibucau
2013/11/20 Phil Steitz <phil.ste...@gmail.com>: > On 11/20/13, 7:26 AM, Romain Manni-Bucau wrote: >> next version (rewrite/fork): >> https://svn.apache.org/repos/asf/incubator/sirona/trunk/core/src/main/java/org/apache/sirona/counters/OptimizedStatistics.java >> >> was easier to centralize everything in a single class > > That is exactly what I meant about the moment statistics. They can > share state. The problem is this approach eliminates the > pluggability and also cuts out the other stats. We can probably > have it both ways by allowing you to turn off stuff you don't want > and just having the moment stats share state. > > Phil >> Romain Manni-Bucau >> Twitter: @rmannibucau >> Blog: http://rmannibucau.wordpress.com/ >> LinkedIn: http://fr.linkedin.com/in/rmannibucau >> Github: https://github.com/rmannibucau >> >> >> >> 2013/11/20 Phil Steitz <phil.ste...@gmail.com>: >>> On 11/20/13, 12:43 AM, Romain Manni-Bucau wrote: >>>> Hi >>>> >>>> A quick mail to give some feedbacks of my tests. >>>> >>>> I started to hack a bit to get rid of not used stats by sirona, >>>> typically I do ATM: >>>> >>>> setSumsqImpl(NoopStat.INSTANCE); >>>> setSumLogImpl(NoopStat.INSTANCE); >>>> setGeoMeanImpl(NoopStat.INSTANCE); >>>> >>>> (NoopStat is a mock of StorelessUnivariateStatistic doijg nothing) >>>> >>>> Another point which could be improoved is the duplication of info >>>> accross sub StorelessUnivariateStatistic (typically n computed several >>>> times for instance). >>> Good point. Its kind of funny that simplest way to solve the >>> problem you mention at the end is to remove the flexibility that you >>> use in the beginning - i.e., to no longer use separate stats >>> instances to compute the bundled statistics. The setup is the way >>> it is precisely so that you can plug in alternative impls. I had >>> never thought of no-op-ing instances to suppress things, but it does >>> work. Having stats share state data is a little tricky but in >>> theory possible. The moment stats at least are set up to support >>> this. Patches are welcome. If you don't mind opening a JIRA to >>> suggest eliminating repeated computations that would be great. >>> >>> Phil >>>> Romain Manni-Bucau >>>> Twitter: @rmannibucau >>>> Blog: http://rmannibucau.wordpress.com/ >>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>>> Github: https://github.com/rmannibucau >>>> >>>> >>>> >>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>: >>>>> On 11/6/13 9:05 AM, Romain Manni-Bucau wrote: >>>>>> Great! >>>>>> >>>>>> Btw not sure for sirona we oculd use it. One constraint on sirona-core >>>>>> is to stay self contained. We already shade math3 so shading pool2 too >>>>>> would start to create a big jar for this need. I'll try to bench >>>>>> deeper next week too. >>>>> OK - and any ideas you have about how to implement something >>>>> lightweight inside [math] much appreciated. >>>>> >>>>> Phil >>>>>> Romain Manni-Bucau >>>>>> Twitter: @rmannibucau >>>>>> Blog: http://rmannibucau.wordpress.com/ >>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>>>>> Github: https://github.com/rmannibucau >>>>>> >>>>>> >>>>>> >>>>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>: >>>>>>> On 11/6/13 8:47 AM, Romain Manni-Bucau wrote: >>>>>>>> well pool are based on locks so I'm not sure (it would need deep >>>>>>>> benchs on a real app) it does worth it >>>>>>> Commons pool2 uses pretty lightweight locking and using a pool of >>>>>>> instances achieves the basic objective of reducing contention for >>>>>>> the single sync lock on one SummaryStatistics object. I bet it >>>>>>> would improve throughput over the single-instance approach if >>>>>>> maxActive, maxIdle were tuned. If I get some time to play with >>>>>>> this, I will report back with some benchmarks. >>>>>>> >>>>>>> Phil >>>>>>>> Romain Manni-Bucau >>>>>>>> Twitter: @rmannibucau >>>>>>>> Blog: http://rmannibucau.wordpress.com/ >>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>>>>>>> Github: https://github.com/rmannibucau >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>: >>>>>>>>> On 11/5/13 11:26 PM, Romain Manni-Bucau wrote: >>>>>>>>>> Hehe, right. >>>>>>>>>> >>>>>>>>>> I looked a bit more today and LongAdder is only a part of the >>>>>>>>>> solution. The stat computation still needs to lock to get acces to >>>>>>>>>> previous values (N -> N+1). Basically the gain wouldn't be as >>>>>>>>>> important as I thought :(. >>>>>>>>> Right, but I think your original idea of maintaining a pool of >>>>>>>>> instances (fewer that one per thread) to be periodically aggregated >>>>>>>>> is a good one. See below. >>>>>>>>>> As I said before we'll wait a bit to gather feedbacks, if it blocks >>>>>>>>>> I'll come back trying to find + propose a solution. >>>>>>>>>> >>>>>>>>>> Thanks in all cases for your answers! >>>>>>>>> A workaround that I have started playing with (partly for other >>>>>>>>> benchmarking reasons) might be to actually use a pool for the stats >>>>>>>>> objects that the monitoring threads use. Using a pool would allow >>>>>>>>> you to monitor and tune the parameters. We now have (well, once the >>>>>>>>> VOTE in progress completes :) a decently performing pool >>>>>>>>> implementation. The tricky bit is locking the instances during >>>>>>>>> aggregation. One way to handle this would be to have the factory's >>>>>>>>> passivate method and the aggregation thread contend for locks on the >>>>>>>>> pooled stats instances. The only contention would be when >>>>>>>>> aggregation is copying individual instances and contention would be >>>>>>>>> with at most one client thread (waiting to proceed in passivate). >>>>>>>>> >>>>>>>>> Phil >>>>>>>>>> Romain Manni-Bucau >>>>>>>>>> Twitter: @rmannibucau >>>>>>>>>> Blog: http://rmannibucau.wordpress.com/ >>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>>>>>>>>> Github: https://github.com/rmannibucau >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2013/11/5 Phil Steitz <phil.ste...@gmail.com>: >>>>>>>>>>> On 11/5/13 9:57 AM, Romain Manni-Bucau wrote: >>>>>>>>>>>> @Phil: hmm can be but the framework would create its own overhead >>>>>>>>>>>> which >>>>>>>>>>>> would be avoided with a dedicated solution, no? Well thought gain >>>>>>>>>>>> was great >>>>>>>>>>>> for small investment but ok to postpone it >>>>>>>>>>> As I said, patches welcome. Go for it. My point about the >>>>>>>>>>> framework was that when you actually get this implemented inside, >>>>>>>>>>> e.g. SummaryStatistics, you will have built a mini-framework. >>>>>>>>>>> Whatever overhead it has, it will have ;) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Phil >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Le 5 nov. 2013 18:54, "Romain Manni-Bucau" <rmannibu...@gmail.com> >>>>>>>>>>>> a écrit : >>>>>>>>>>>> >>>>>>>>>>>>> Well I didnt test sirona in prod but when using jamon (same kind >>>>>>>>>>>>> of >>>>>>>>>>>>> framework) locks were creating a serious overhead on some >>>>>>>>>>>>> benches. Not the >>>>>>>>>>>>> most important but enough to try to solve it. >>>>>>>>>>>>> >>>>>>>>>>>>> That said we are not yet in 1.0 so Im ok to wait for more serious >>>>>>>>>>>>> feedbacks if you think it is better >>>>>>>>>>>>> Le 5 nov. 2013 18:48, "Ted Dunning" <ted.dunn...@gmail.com> a >>>>>>>>>>>>> écrit : >>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Nov 4, 2013 at 10:09 PM, Romain Manni-Bucau >>>>>>>>>>>>>> <rmannibu...@gmail.com>wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Oh sorry, that's what I said early, in a real app no or not >>>>>>>>>>>>>>> enough to >>>>>>>>>>>>>> be an >>>>>>>>>>>>>>> issue buy on simple apps or very high thrououtput apps yes. >>>>>>>>>>>>>>> Le 5 nov. 2013 07:00, "Ted Dunning" <ted.dunn...@gmail.com> a >>>>>>>>>>>>>>> écrit : >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> That isn't what I meant. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Do you really think that more than one metric has to update >>>>>>>>>>>>>> (increment, >>>>>>>>>>>>>>>> say) at precisely the same time? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> I realize that is what you said. Do you have any serious >>>>>>>>>>>>>> examples where >>>>>>>>>>>>>> metrics have to be updated all or nothing? >>>>>>>>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>>> >>>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>> >>>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>> For additional commands, e-mail: dev-h...@commons.apache.org >>> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >> For additional commands, e-mail: dev-h...@commons.apache.org >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
