Re:Re: count the amount of data successfully processed by flink

2021-05-24 Thread zhanyuwei
Hi Guowei: Thanks for your help, it solves my problem. Best regards At 2021-05-24 15:15:57, "Guowei Ma" wrote: Hi I think you are right that the metrics are reset after the job restart. It is because the metrics are only stored in the memory. I think you could store the metrics

Re: count the amount of data successfully processed by flink

2021-05-24 Thread Guowei Ma
Hi I think you are right that the metrics are reset after the job restart. It is because the metrics are only stored in the memory. I think you could store the metrics to the Flink's state[1], which could be restored after the job restarted. [1] https://ci.apache.org/projects/flink/flink-docs-rele

count the amount of data successfully processed by flink

2021-05-23 Thread zzzyw
Dear all: I use flink for real-time data synchronization(mysql,oracle --> kafka --> mysql,oracle). I want to count how many pieces of data are synchronized every day(maybe need to count the last n days ). I am doing this now: flink metrics send to pushgateway, and then sum the metrics (flink_