Hi,
My issue has been resolved through discussion with AWS support.
It turns out that Kinesis Data Analytics reports to CloudWatch in a way I did
not expect. The way to view the accurate values for Flink counters is with
Average in CloudWatch metrics.
Below is the response from AWS support, fo
Hi Dawid,
My team's Flink application's primary purpose is not to count the number of SQS
messages received or the number of successful or failed S3 downloads. The
application's primary purpose is to process events and the corresponding data,
and for each event, create or update a new entry in
amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html
*From:* Zhanghao Chen
*Sent:* March 5, 2022 11:11 PM
*To:* Shane Bishop ; user@flink.apache.org
*Subject:* Re: Question about Flink counters
Hi Sha
.html
From: Zhanghao Chen
Sent: March 5, 2022 11:11 PM
To: Shane Bishop ; user@flink.apache.org
Subject: Re: Question about Flink counters
Hi Shane,
Could you share more information on what you would like to use the counter for?
The counter discussed here is primarily designed for exposing coun
e.org
Subject: Re: Question about Flink counters
If I used a thread-safe counter implementation, would that be enough to make
the count correct for a Flink cluster with multiple machines?
Best,
Shane
From: Zhanghao Chen
Sent: March 4, 2022 11:08 PM
To: Shane Bishop ;
: Question about Flink counters
Hi Shane,
Flink provides a generic counter interface with a few implementations. The
default implementation SimpleCounter, which is not thread-safe, is used when
you calling counter(String name) on a MetricGroup. Therefore, you'll need to
use your own thread
t: Question about Flink counters
Hi all,
For Flink counters [1], are increment operations guaranteed to be atomic across
all parallel tasks? I.e., is there a guarantee that the counter values will not
be higher than expected?
Thanks,
Shane
---
[1]
https://nightlies.apache.org/flink/flink-docs-r
Hi all,
For Flink counters [1], are increment operations guaranteed to be atomic across
all parallel tasks? I.e., is there a guarantee that the counter values will not
be higher than expected?
Thanks,
Shane
---
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/metrics/#c
