On 11/20/13, 7:26 AM, Romain Manni-Bucau wrote: > next version (rewrite/fork): > https://svn.apache.org/repos/asf/incubator/sirona/trunk/core/src/main/java/org/apache/sirona/counters/OptimizedStatistics.java > > was easier to centralize everything in a single class
That is exactly what I meant about the moment statistics. They can share state. The problem is this approach eliminates the pluggability and also cuts out the other stats. We can probably have it both ways by allowing you to turn off stuff you don't want and just having the moment stats share state. Phil > Romain Manni-Bucau > Twitter: @rmannibucau > Blog: http://rmannibucau.wordpress.com/ > LinkedIn: http://fr.linkedin.com/in/rmannibucau > Github: https://github.com/rmannibucau > > > > 2013/11/20 Phil Steitz <phil.ste...@gmail.com>: >> On 11/20/13, 12:43 AM, Romain Manni-Bucau wrote: >>> Hi >>> >>> A quick mail to give some feedbacks of my tests. >>> >>> I started to hack a bit to get rid of not used stats by sirona, >>> typically I do ATM: >>> >>> setSumsqImpl(NoopStat.INSTANCE); >>> setSumLogImpl(NoopStat.INSTANCE); >>> setGeoMeanImpl(NoopStat.INSTANCE); >>> >>> (NoopStat is a mock of StorelessUnivariateStatistic doijg nothing) >>> >>> Another point which could be improoved is the duplication of info >>> accross sub StorelessUnivariateStatistic (typically n computed several >>> times for instance). >> Good point. Its kind of funny that simplest way to solve the >> problem you mention at the end is to remove the flexibility that you >> use in the beginning - i.e., to no longer use separate stats >> instances to compute the bundled statistics. The setup is the way >> it is precisely so that you can plug in alternative impls. I had >> never thought of no-op-ing instances to suppress things, but it does >> work. Having stats share state data is a little tricky but in >> theory possible. The moment stats at least are set up to support >> this. Patches are welcome. If you don't mind opening a JIRA to >> suggest eliminating repeated computations that would be great. >> >> Phil >>> Romain Manni-Bucau >>> Twitter: @rmannibucau >>> Blog: http://rmannibucau.wordpress.com/ >>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>> Github: https://github.com/rmannibucau >>> >>> >>> >>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>: >>>> On 11/6/13 9:05 AM, Romain Manni-Bucau wrote: >>>>> Great! >>>>> >>>>> Btw not sure for sirona we oculd use it. One constraint on sirona-core >>>>> is to stay self contained. We already shade math3 so shading pool2 too >>>>> would start to create a big jar for this need. I'll try to bench >>>>> deeper next week too. >>>> OK - and any ideas you have about how to implement something >>>> lightweight inside [math] much appreciated. >>>> >>>> Phil >>>>> Romain Manni-Bucau >>>>> Twitter: @rmannibucau >>>>> Blog: http://rmannibucau.wordpress.com/ >>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>>>> Github: https://github.com/rmannibucau >>>>> >>>>> >>>>> >>>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>: >>>>>> On 11/6/13 8:47 AM, Romain Manni-Bucau wrote: >>>>>>> well pool are based on locks so I'm not sure (it would need deep >>>>>>> benchs on a real app) it does worth it >>>>>> Commons pool2 uses pretty lightweight locking and using a pool of >>>>>> instances achieves the basic objective of reducing contention for >>>>>> the single sync lock on one SummaryStatistics object. I bet it >>>>>> would improve throughput over the single-instance approach if >>>>>> maxActive, maxIdle were tuned. If I get some time to play with >>>>>> this, I will report back with some benchmarks. >>>>>> >>>>>> Phil >>>>>>> Romain Manni-Bucau >>>>>>> Twitter: @rmannibucau >>>>>>> Blog: http://rmannibucau.wordpress.com/ >>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>>>>>> Github: https://github.com/rmannibucau >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>: >>>>>>>> On 11/5/13 11:26 PM, Romain Manni-Bucau wrote: >>>>>>>>> Hehe, right. >>>>>>>>> >>>>>>>>> I looked a bit more today and LongAdder is only a part of the >>>>>>>>> solution. The stat computation still needs to lock to get acces to >>>>>>>>> previous values (N -> N+1). Basically the gain wouldn't be as >>>>>>>>> important as I thought :(. >>>>>>>> Right, but I think your original idea of maintaining a pool of >>>>>>>> instances (fewer that one per thread) to be periodically aggregated >>>>>>>> is a good one. See below. >>>>>>>>> As I said before we'll wait a bit to gather feedbacks, if it blocks >>>>>>>>> I'll come back trying to find + propose a solution. >>>>>>>>> >>>>>>>>> Thanks in all cases for your answers! >>>>>>>> A workaround that I have started playing with (partly for other >>>>>>>> benchmarking reasons) might be to actually use a pool for the stats >>>>>>>> objects that the monitoring threads use. Using a pool would allow >>>>>>>> you to monitor and tune the parameters. We now have (well, once the >>>>>>>> VOTE in progress completes :) a decently performing pool >>>>>>>> implementation. The tricky bit is locking the instances during >>>>>>>> aggregation. One way to handle this would be to have the factory's >>>>>>>> passivate method and the aggregation thread contend for locks on the >>>>>>>> pooled stats instances. The only contention would be when >>>>>>>> aggregation is copying individual instances and contention would be >>>>>>>> with at most one client thread (waiting to proceed in passivate). >>>>>>>> >>>>>>>> Phil >>>>>>>>> Romain Manni-Bucau >>>>>>>>> Twitter: @rmannibucau >>>>>>>>> Blog: http://rmannibucau.wordpress.com/ >>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau >>>>>>>>> Github: https://github.com/rmannibucau >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 2013/11/5 Phil Steitz <phil.ste...@gmail.com>: >>>>>>>>>> On 11/5/13 9:57 AM, Romain Manni-Bucau wrote: >>>>>>>>>>> @Phil: hmm can be but the framework would create its own overhead >>>>>>>>>>> which >>>>>>>>>>> would be avoided with a dedicated solution, no? Well thought gain >>>>>>>>>>> was great >>>>>>>>>>> for small investment but ok to postpone it >>>>>>>>>> As I said, patches welcome. Go for it. My point about the >>>>>>>>>> framework was that when you actually get this implemented inside, >>>>>>>>>> e.g. SummaryStatistics, you will have built a mini-framework. >>>>>>>>>> Whatever overhead it has, it will have ;) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Phil >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Le 5 nov. 2013 18:54, "Romain Manni-Bucau" <rmannibu...@gmail.com> >>>>>>>>>>> a écrit : >>>>>>>>>>> >>>>>>>>>>>> Well I didnt test sirona in prod but when using jamon (same kind of >>>>>>>>>>>> framework) locks were creating a serious overhead on some benches. >>>>>>>>>>>> Not the >>>>>>>>>>>> most important but enough to try to solve it. >>>>>>>>>>>> >>>>>>>>>>>> That said we are not yet in 1.0 so Im ok to wait for more serious >>>>>>>>>>>> feedbacks if you think it is better >>>>>>>>>>>> Le 5 nov. 2013 18:48, "Ted Dunning" <ted.dunn...@gmail.com> a >>>>>>>>>>>> écrit : >>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Nov 4, 2013 at 10:09 PM, Romain Manni-Bucau >>>>>>>>>>>>> <rmannibu...@gmail.com>wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Oh sorry, that's what I said early, in a real app no or not >>>>>>>>>>>>>> enough to >>>>>>>>>>>>> be an >>>>>>>>>>>>>> issue buy on simple apps or very high thrououtput apps yes. >>>>>>>>>>>>>> Le 5 nov. 2013 07:00, "Ted Dunning" <ted.dunn...@gmail.com> a >>>>>>>>>>>>>> écrit : >>>>>>>>>>>>>> >>>>>>>>>>>>>>> That isn't what I meant. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Do you really think that more than one metric has to update >>>>>>>>>>>>> (increment, >>>>>>>>>>>>>>> say) at precisely the same time? >>>>>>>>>>>>>>> >>>>>>>>>>>>> I realize that is what you said. Do you have any serious >>>>>>>>>>>>> examples where >>>>>>>>>>>>> metrics have to be updated all or nothing? >>>>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>>> >>>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>> >>>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>> For additional commands, e-mail: dev-h...@commons.apache.org >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >> For additional commands, e-mail: dev-h...@commons.apache.org >> > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
