Re: Proper way to establish bucket counts

2017-08-04 Thread Fabian Hueske
Hi Robert, That's right. The count's are on a per operator-level. I think you can get down to the task-level but counts per bucket are not tracked. Maybe Chesnay (in CC) can help here. He knows the metrics system the best. @Chesnay, is there a way to expire metric counters? Alternatively, you cou

Re: Proper way to establish bucket counts

2017-08-02 Thread Fabian Hueske
Hi Robert, Flink collects many metrics by default, including the number of records / events that go into each operator (see [1], System Metrics, IO, "numRecordsIn"). So, you would only need to access that metric. Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/monito