Hi
A quick mail to give some feedbacks of my tests.
I started to hack a bit to get rid of not used stats by sirona,
typically I do ATM:
setSumsqImpl(NoopStat.INSTANCE);
setSumLogImpl(NoopStat.INSTANCE);
setGeoMeanImpl(NoopStat.INSTANCE);
(NoopStat is a mock of StorelessUnivariateStatistic doijg nothing)
Another point which could be improoved is the duplication of info
accross sub StorelessUnivariateStatistic (typically n computed several
times for instance).
Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau
2013/11/6 Phil Steitz <[email protected]>:
> On 11/6/13 9:05 AM, Romain Manni-Bucau wrote:
>> Great!
>>
>> Btw not sure for sirona we oculd use it. One constraint on sirona-core
>> is to stay self contained. We already shade math3 so shading pool2 too
>> would start to create a big jar for this need. I'll try to bench
>> deeper next week too.
>
> OK - and any ideas you have about how to implement something
> lightweight inside [math] much appreciated.
>
> Phil
>> Romain Manni-Bucau
>> Twitter: @rmannibucau
>> Blog: http://rmannibucau.wordpress.com/
>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>> Github: https://github.com/rmannibucau
>>
>>
>>
>> 2013/11/6 Phil Steitz <[email protected]>:
>>> On 11/6/13 8:47 AM, Romain Manni-Bucau wrote:
>>>> well pool are based on locks so I'm not sure (it would need deep
>>>> benchs on a real app) it does worth it
>>> Commons pool2 uses pretty lightweight locking and using a pool of
>>> instances achieves the basic objective of reducing contention for
>>> the single sync lock on one SummaryStatistics object. I bet it
>>> would improve throughput over the single-instance approach if
>>> maxActive, maxIdle were tuned. If I get some time to play with
>>> this, I will report back with some benchmarks.
>>>
>>> Phil
>>>> Romain Manni-Bucau
>>>> Twitter: @rmannibucau
>>>> Blog: http://rmannibucau.wordpress.com/
>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>> Github: https://github.com/rmannibucau
>>>>
>>>>
>>>>
>>>> 2013/11/6 Phil Steitz <[email protected]>:
>>>>> On 11/5/13 11:26 PM, Romain Manni-Bucau wrote:
>>>>>> Hehe, right.
>>>>>>
>>>>>> I looked a bit more today and LongAdder is only a part of the
>>>>>> solution. The stat computation still needs to lock to get acces to
>>>>>> previous values (N -> N+1). Basically the gain wouldn't be as
>>>>>> important as I thought :(.
>>>>> Right, but I think your original idea of maintaining a pool of
>>>>> instances (fewer that one per thread) to be periodically aggregated
>>>>> is a good one. See below.
>>>>>> As I said before we'll wait a bit to gather feedbacks, if it blocks
>>>>>> I'll come back trying to find + propose a solution.
>>>>>>
>>>>>> Thanks in all cases for your answers!
>>>>> A workaround that I have started playing with (partly for other
>>>>> benchmarking reasons) might be to actually use a pool for the stats
>>>>> objects that the monitoring threads use. Using a pool would allow
>>>>> you to monitor and tune the parameters. We now have (well, once the
>>>>> VOTE in progress completes :) a decently performing pool
>>>>> implementation. The tricky bit is locking the instances during
>>>>> aggregation. One way to handle this would be to have the factory's
>>>>> passivate method and the aggregation thread contend for locks on the
>>>>> pooled stats instances. The only contention would be when
>>>>> aggregation is copying individual instances and contention would be
>>>>> with at most one client thread (waiting to proceed in passivate).
>>>>>
>>>>> Phil
>>>>>> Romain Manni-Bucau
>>>>>> Twitter: @rmannibucau
>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>> Github: https://github.com/rmannibucau
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/11/5 Phil Steitz <[email protected]>:
>>>>>>> On 11/5/13 9:57 AM, Romain Manni-Bucau wrote:
>>>>>>>> @Phil: hmm can be but the framework would create its own overhead which
>>>>>>>> would be avoided with a dedicated solution, no? Well thought gain was
>>>>>>>> great
>>>>>>>> for small investment but ok to postpone it
>>>>>>> As I said, patches welcome. Go for it. My point about the
>>>>>>> framework was that when you actually get this implemented inside,
>>>>>>> e.g. SummaryStatistics, you will have built a mini-framework.
>>>>>>> Whatever overhead it has, it will have ;)
>>>>>>>
>>>>>>>
>>>>>>> Phil
>>>>>>>
>>>>>>>
>>>>>>>> Le 5 nov. 2013 18:54, "Romain Manni-Bucau" <[email protected]> a
>>>>>>>> écrit :
>>>>>>>>
>>>>>>>>> Well I didnt test sirona in prod but when using jamon (same kind of
>>>>>>>>> framework) locks were creating a serious overhead on some benches.
>>>>>>>>> Not the
>>>>>>>>> most important but enough to try to solve it.
>>>>>>>>>
>>>>>>>>> That said we are not yet in 1.0 so Im ok to wait for more serious
>>>>>>>>> feedbacks if you think it is better
>>>>>>>>> Le 5 nov. 2013 18:48, "Ted Dunning" <[email protected]> a écrit :
>>>>>>>>>
>>>>>>>>>> On Mon, Nov 4, 2013 at 10:09 PM, Romain Manni-Bucau
>>>>>>>>>> <[email protected]>wrote:
>>>>>>>>>>
>>>>>>>>>>> Oh sorry, that's what I said early, in a real app no or not enough
>>>>>>>>>>> to
>>>>>>>>>> be an
>>>>>>>>>>> issue buy on simple apps or very high thrououtput apps yes.
>>>>>>>>>>> Le 5 nov. 2013 07:00, "Ted Dunning" <[email protected]> a
>>>>>>>>>>> écrit :
>>>>>>>>>>>
>>>>>>>>>>>> That isn't what I meant.
>>>>>>>>>>>>
>>>>>>>>>>>> Do you really think that more than one metric has to update
>>>>>>>>>> (increment,
>>>>>>>>>>>> say) at precisely the same time?
>>>>>>>>>>>>
>>>>>>>>>> I realize that is what you said. Do you have any serious examples
>>>>>>>>>> where
>>>>>>>>>> metrics have to be updated all or nothing?
>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: [email protected]
>>>>>>> For additional commands, e-mail: [email protected]
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [email protected]
>>>>>> For additional commands, e-mail: [email protected]
>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [email protected]
>>>>> For additional commands, e-mail: [email protected]
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [email protected]
>>>> For additional commands, e-mail: [email protected]
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [email protected]
>>> For additional commands, e-mail: [email protected]
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
