On 11/21/2013 11:14 AM, Romain Manni-Bucau wrote:
> Hi
> 
> first maybe the thread should move over sirona list (I wouldn't bother
> commons ;)): http://sirona.incubator.apache.org/mail-lists.html
> 
> about hot topics I think we have:
> 1) Gui enhancement (a jira is about using another js library to draw
> graphes and get an analysis/correlation view, another topic is the GUI
> configuration: how to make dashboard etc...)
> 2) Storage backends (we support mainly cassandra and in memory ATM but
> a rotating JDBC impl could be great too)
> 3) Alerting is not yet done and needs to be in final versions
> 4) Agent/Plugins enhancement (we have a few today)
> 5) Surely a better configuration
> 6) Counter implementation with less locks (even if thanks to the stat
> impl we use we reduced the lock duration we still rely too much on
> lock IMO and I'm sure we can do better).
> 7) etc.. ;)
> 
> 
> It is a new project so all help is welcomed!

Hi Romain,

thanks for the pointers, I will take a closer look, some of them would
really be interesting to work on.

Thanks,

Thomas

> 
> PS: we have an IRC channel on freenode: #apache-sirona
> 
> 
> 
> 
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
> 
> 
> 
> 2013/11/21 Thomas Neidhart <thomas.neidh...@gmail.com>:
>> I did take a look at sirona, and it looks really nice.
>> Whats the best way to contribute to it? Any open issues you would like to
>> work somebody on?
>>
>> Thomas
>>
>>
>> On Wed, Nov 20, 2013 at 5:38 PM, Romain Manni-Bucau
>> <rmannibu...@gmail.com>wrote:
>>
>>> yep but this is not enough, almost each "component" is doing n++. This
>>> is fast but since done under a lock if it can be removed it should.
>>> Romain Manni-Bucau
>>> Twitter: @rmannibucau
>>> Blog: http://rmannibucau.wordpress.com/
>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>> Github: https://github.com/rmannibucau
>>>
>>>
>>>
>>> 2013/11/20 Phil Steitz <phil.ste...@gmail.com>:
>>>> On 11/20/13, 7:26 AM, Romain Manni-Bucau wrote:
>>>>> next version (rewrite/fork):
>>>>>
>>> https://svn.apache.org/repos/asf/incubator/sirona/trunk/core/src/main/java/org/apache/sirona/counters/OptimizedStatistics.java
>>>>>
>>>>> was easier to centralize everything in a single class
>>>>
>>>> That is exactly what I meant about the moment statistics.  They can
>>>> share state.  The problem is this approach eliminates the
>>>> pluggability and also cuts out the other stats.  We can probably
>>>> have it both ways by allowing you to turn off stuff you don't want
>>>> and just having the moment stats share state.
>>>>
>>>> Phil
>>>>> Romain Manni-Bucau
>>>>> Twitter: @rmannibucau
>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>> Github: https://github.com/rmannibucau
>>>>>
>>>>>
>>>>>
>>>>> 2013/11/20 Phil Steitz <phil.ste...@gmail.com>:
>>>>>> On 11/20/13, 12:43 AM, Romain Manni-Bucau wrote:
>>>>>>> Hi
>>>>>>>
>>>>>>> A quick mail to give some feedbacks of my tests.
>>>>>>>
>>>>>>> I started to hack a bit to get rid of not used stats by sirona,
>>>>>>> typically I do ATM:
>>>>>>>
>>>>>>>         setSumsqImpl(NoopStat.INSTANCE);
>>>>>>>         setSumLogImpl(NoopStat.INSTANCE);
>>>>>>>         setGeoMeanImpl(NoopStat.INSTANCE);
>>>>>>>
>>>>>>> (NoopStat is a mock of StorelessUnivariateStatistic doijg nothing)
>>>>>>>
>>>>>>> Another point which could be improoved is the duplication of info
>>>>>>> accross sub StorelessUnivariateStatistic (typically n computed several
>>>>>>> times for instance).
>>>>>> Good point.  Its kind of funny that simplest way to solve the
>>>>>> problem you mention at the end is to remove the flexibility that you
>>>>>> use in the beginning  - i.e., to no longer use separate stats
>>>>>> instances to compute the bundled statistics.  The setup is the way
>>>>>> it is precisely so that you can plug in alternative impls.  I had
>>>>>> never thought of no-op-ing instances to suppress things, but it does
>>>>>> work.  Having stats share state data is a little tricky but in
>>>>>> theory possible.  The moment stats at least are set up to support
>>>>>> this.   Patches are welcome.  If you don't mind opening a JIRA to
>>>>>> suggest eliminating repeated computations that would be great.
>>>>>>
>>>>>> Phil
>>>>>>> Romain Manni-Bucau
>>>>>>> Twitter: @rmannibucau
>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>:
>>>>>>>> On 11/6/13 9:05 AM, Romain Manni-Bucau wrote:
>>>>>>>>> Great!
>>>>>>>>>
>>>>>>>>> Btw not sure for sirona we oculd use it. One constraint on
>>> sirona-core
>>>>>>>>> is to stay self contained. We already shade math3 so shading pool2
>>> too
>>>>>>>>> would start to create a big jar for this need. I'll try to bench
>>>>>>>>> deeper next week too.
>>>>>>>> OK - and any ideas you have about how to implement something
>>>>>>>> lightweight inside [math] much appreciated.
>>>>>>>>
>>>>>>>> Phil
>>>>>>>>> Romain Manni-Bucau
>>>>>>>>> Twitter: @rmannibucau
>>>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>:
>>>>>>>>>> On 11/6/13 8:47 AM, Romain Manni-Bucau wrote:
>>>>>>>>>>> well pool are based on locks so I'm not sure (it would need deep
>>>>>>>>>>> benchs on a real app) it does worth it
>>>>>>>>>> Commons pool2 uses pretty lightweight locking and using a pool of
>>>>>>>>>> instances achieves the basic objective of reducing contention for
>>>>>>>>>> the single sync lock on one SummaryStatistics object.   I bet it
>>>>>>>>>> would improve throughput over the single-instance approach if
>>>>>>>>>> maxActive, maxIdle were tuned.  If I get some time to play with
>>>>>>>>>> this, I will report back with some benchmarks.
>>>>>>>>>>
>>>>>>>>>> Phil
>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>> Twitter: @rmannibucau
>>>>>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/11/6 Phil Steitz <phil.ste...@gmail.com>:
>>>>>>>>>>>> On 11/5/13 11:26 PM, Romain Manni-Bucau wrote:
>>>>>>>>>>>>> Hehe, right.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I looked a bit more today and LongAdder is only a part of the
>>>>>>>>>>>>> solution. The stat computation still needs to lock to get acces
>>> to
>>>>>>>>>>>>> previous values (N -> N+1). Basically the gain wouldn't be as
>>>>>>>>>>>>> important as I thought :(.
>>>>>>>>>>>> Right, but I think your original idea of maintaining a pool of
>>>>>>>>>>>> instances (fewer that one per thread) to be periodically
>>> aggregated
>>>>>>>>>>>> is a good one.  See below.
>>>>>>>>>>>>> As I said before we'll wait a bit to gather feedbacks, if it
>>> blocks
>>>>>>>>>>>>> I'll come back trying to find + propose a solution.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks in all cases for your answers!
>>>>>>>>>>>> A workaround that I have started playing with (partly for other
>>>>>>>>>>>> benchmarking reasons) might be to actually use a pool for the
>>> stats
>>>>>>>>>>>> objects that the monitoring threads use.  Using a pool would
>>> allow
>>>>>>>>>>>> you to monitor and tune the parameters.  We now have (well, once
>>> the
>>>>>>>>>>>> VOTE in progress completes :) a decently performing pool
>>>>>>>>>>>> implementation.  The tricky bit is locking the instances during
>>>>>>>>>>>> aggregation.  One way to handle this would be to have the
>>> factory's
>>>>>>>>>>>> passivate method and the aggregation thread contend for locks on
>>> the
>>>>>>>>>>>> pooled stats instances.  The only contention would be when
>>>>>>>>>>>> aggregation is copying individual instances and contention would
>>> be
>>>>>>>>>>>> with at most one client thread (waiting to proceed in passivate).
>>>>>>>>>>>>
>>>>>>>>>>>> Phil
>>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>>> Twitter: @rmannibucau
>>>>>>>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/11/5 Phil Steitz <phil.ste...@gmail.com>:
>>>>>>>>>>>>>> On 11/5/13 9:57 AM, Romain Manni-Bucau wrote:
>>>>>>>>>>>>>>> @Phil: hmm can be but the framework would create its own
>>> overhead which
>>>>>>>>>>>>>>> would be avoided with a dedicated solution, no? Well thought
>>> gain was great
>>>>>>>>>>>>>>> for small investment but ok to postpone it
>>>>>>>>>>>>>> As I said, patches welcome.  Go for it.  My point about the
>>>>>>>>>>>>>> framework was that when you actually get this implemented
>>> inside,
>>>>>>>>>>>>>> e.g. SummaryStatistics,  you will have built a mini-framework.
>>>>>>>>>>>>>> Whatever overhead it has, it will have ;)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Phil
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Le 5 nov. 2013 18:54, "Romain Manni-Bucau" <
>>> rmannibu...@gmail.com> a écrit :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Well I didnt test sirona in prod but when using jamon (same
>>> kind of
>>>>>>>>>>>>>>>> framework) locks were creating a serious overhead on some
>>> benches. Not the
>>>>>>>>>>>>>>>> most important but enough to try to solve it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> That said we are not yet in 1.0 so Im ok to wait for more
>>> serious
>>>>>>>>>>>>>>>> feedbacks if you think it is better
>>>>>>>>>>>>>>>> Le 5 nov. 2013 18:48, "Ted Dunning" <ted.dunn...@gmail.com>
>>> a écrit :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Nov 4, 2013 at 10:09 PM, Romain Manni-Bucau
>>>>>>>>>>>>>>>>> <rmannibu...@gmail.com>wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Oh sorry, that's what I said early, in a real app no or
>>> not enough to
>>>>>>>>>>>>>>>>> be an
>>>>>>>>>>>>>>>>>> issue buy on simple apps or very high thrououtput apps yes.
>>>>>>>>>>>>>>>>>>  Le 5 nov. 2013 07:00, "Ted Dunning" <
>>> ted.dunn...@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> That isn't what I meant.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Do you really think that more than one metric has to
>>> update
>>>>>>>>>>>>>>>>> (increment,
>>>>>>>>>>>>>>>>>>> say) at precisely the same time?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I realize that is what you said.  Do you have any serious
>>> examples where
>>>>>>>>>>>>>>>>> metrics have to be updated all or nothing?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>>>>
>>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>
>>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Reply via email to