Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-07 Thread Alexey Goncharuk
This is a tough topic, however, in my opinion, static bucket configuration for histograms will work worse than no configuration at all. The reason is simple - complexity. If we assume a need to change bucket configuration for some metric, we can assume that this metric can be related to a cache. No

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-06 Thread Павлухин Иван
Andrey, It seems that screenshot was rejected, I do not see it. вт, 6 авг. 2019 г. в 15:07, Andrey Gura : > > Good example of proper buckets configuration. Such configuration is > suitable for may cases. See attached screenshot (I hope it will not be > reject by mail system or forum). > > > On Tu

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-06 Thread Andrey Gura
Good example of proper buckets configuration. Such configuration is suitable for may cases. See attached screenshot (I hope it will not be reject by mail system or forum). On Tue, Aug 6, 2019 at 2:45 PM Andrey Gura wrote: > > > What do you mean by "exponential bounds"? > > Something like this if

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-06 Thread Andrey Gura
> What do you mean by "exponential bounds"? Something like this if we talk about latency in ms for example: 5, 10, 25, 50, 100, 200, 500, ... > Thanks, for the feedback, appreciate you ownesty. Nothing personal. It is just about functionality from user's stand point. > What is your proposal? >

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-05 Thread Ivan Rakov
Hi guys, DataStorageConfiguration#getMetricsSubIntervalCount was added by me as last resort to decrease number of intervals in HitRateMetrics in case of unexpected negative performance impact. As far as I know, no one ever used it - the precaution appeared to be premature. We can disregard its

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-05 Thread Nikolay Izhikov
Hello, Andrey. > Not necessary if we have exponential bounds' values for histograms. What do you mean by "exponential bounds"? > Anyway, in current solution it looks ugly and not usable. Thanks, for the feedback, appreciate you ownesty. > No. But we should admit that this is bad decision and d

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-05 Thread Andrey Gura
>> - metric configuration is node local (not cluster wide). > This issue is easy to solve on the user-side and in Ignite core. It's imaginary simplicity. The first, you need some additional automation on user-side in order to configure all nodes of the cluster. The second, new nodes can join to t

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-05 Thread Nikolay Izhikov
Hello, Andrey. > - metric configuration is node local (not cluster wide). This issue is easy to solve on the user-side and in Ignite core. > - metric configuration doesn't survive node restart. We decide to go with the simplest solution, for now. The easiest solution was implemented. Do we want

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-05 Thread Andrey Gura
Igniters, I've took a look to the PR and I want follow up this discussion again. Proposed solution has a couple of significant drawbacks: - metric configuration is node local (not cluster wide). - metric configuration doesn't survive node restart. This drawbacks make configuration complex, anno

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-09 Thread Nikolay Izhikov
Igniters, I made a PR for metrics configuration. Please, review https://github.com/apache/ignite/pull/6676/files В Вт, 09/07/2019 в 12:27 +0300, Nikolay Izhikov пишет: > Hello, Alex. > > OK, Let's go with the simplest solution. > I will provide API and JMX method for metrics configuration shor

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-09 Thread Nikolay Izhikov
Hello, Alex. OK, Let's go with the simplest solution. I will provide API and JMX method for metrics configuration shortly. В Пн, 08/07/2019 в 18:23 +0300, Alexey Goncharuk пишет: > Nikolay, > > To me a separate metrics configuration file seems to be not very > user-friendly. First of all, it doe

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-08 Thread Alexey Goncharuk
Nikolay, To me a separate metrics configuration file seems to be not very user-friendly. First of all, it does not allow to configure the system only from Java code. Second, having multiple configuration files seem to be quite confusing for end users (judging by the logging configuration questions

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-05 Thread Seliverstov Igor
Igniters, One more question on topic. Should we preserve metrics configuration on restart? (I think we should) If so, which configuration use after restart? Defined in config file or saved in config storage? (I guess, saved configuration should have a priority) So, how to tell users that any c

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-04 Thread Nikolay Izhikov
Hello, Andrey. > 3. I can't imagine that adequate values will be chosen on project > setup stage. Configuration file required in the case we adds new node or replace existing to the cluster. Use can have parameters similar to Ignite configuration, log configuration files. > My proposal is addi

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-04 Thread Andrey Gura
Igniters, I rethought the issue and I see some problems: 1. It seems that in most cases bucket boundaries configuration will be problem for user. Absolute values for latency boundaries it is very odd choice. 2. Also seems that latency for most caches (if we configure cache metrics fro example) wi

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-03 Thread Andrey Gura
Nikolai, Metric is disabled if it doesn't allocate any memory and doesn't update any variable because doesn't have any value. Ideally disabling metrics for some cache should be equal to cache stopping. On Fri, Jun 28, 2019 at 1:02 PM Nikolay Izhikov wrote: > > Hello, Alexey. > > Thanks for the f

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-07-03 Thread Andrey Gura
Ivan, Some notes about your comments: > 1. Specifying subset of metrics which are exported to an external system. I believe there are no any subset if metrics except of metrics related with one particular source. Also I think that there is no need to filter out metrics set on export stage. If so

Re: Fwd: [DISCUSSION][IEP-35] Metrics configuration

2019-07-03 Thread Павлухин Иван
Hi Nikolay, Thank you for clarifications. > What is "disabled" sensor? Why do we need it? (as long as sensort is just a > AtomicLong) I meant a possibility of disabling a particular metric value calculation (skipping AtomicLong.incrementAndGet call). My current understanding that we do NOT need

Re: Fwd: [DISCUSSION][IEP-35] Metrics configuration

2019-07-01 Thread Nikolay Izhikov
Hello, Ivan. Thanks for the feedback! > 1. Specifying subset of metrics which are exported to an external system. Covered by exporter filter [1] > 2. Subset of metrics which is collected (enable/disable sensor). What is "disabled" sensor? Why do we need it? (as long as sensort is just a Atomic

Fwd: [DISCUSSION][IEP-35] Metrics configuration

2019-07-01 Thread Павлухин Иван
Nikolay, Igniters, In my mind there are several configuration aspects: 1. Specifying subset of metrics which are exported to an external system. 2. Subset of metrics which is collected (enable/disable sensor). 3. A particular metric (sensor) parameters. Are we going to address all points in the s

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-06-28 Thread Nikolay Izhikov
Hello, Alexey. Thanks for the feedback! > My only concert is that we should have the metrics framework configuration > as the first-citizen of the framework itself Yes. I planned to add `void configure(String param)` method to the metric API. > but change the metrics parameters in > runtime fro

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-06-27 Thread Alexey Goncharuk
Nikolay, My only concert is that we should have the metrics framework configuration as the first-citizen of the framework itself. This way, we can configure the metrics not only from file, but change the metrics parameters in runtime from JMX or command-line, etc. Another concern is to have an abi

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-06-27 Thread Ivan Fedotov
Nikolay, I also agree with your approach thank, you for the explanation. One minor remark: maybe it will be better to explicitly indicate in config the type of the metric? In nutshell: HistrogramMetric: cache.my-cahe.GetLatency=50,100,250,500 HitRateMetric: cache.my-cache.RebalancingKeysRate=6000

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-06-27 Thread Nikolay Izhikov
Hello, Anton. Thanks for an answer! > Let's just make sure this syntax is extendable. Yes, I ~can~ will! > I'd like to count only latency greater than 37ms (0-37 - > ignored, 37-52, 52-infinity - recorded), it should be possible to skip some > values. Currently, histogram metric don't support

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-06-27 Thread Anton Vinogradov
Nikolay, The approach looks good to me. Let's just make sure this syntax is extendable. For example, I'd like to count only latency greater than 37ms (0-37 - ignored, 37-52, 52-infinity - recorded), it should be possible to skip some values. Another case is to specify 2+ windows for same metric, e

[DISCUSSION][IEP-35] Metrics configuration

2019-06-27 Thread Nikolay Izhikov
Hello, Igniters. As you may know, I've contributed Phase1 [1] for IEP-35 [2]. Now we have metrics subsystem and can create and export any metrics from Ignite. I think user(administrator of Ignite) should be able to configure some metrics params in a common way [3] I propose to use the same way