Hello devs,
I would like to start a discussion about FLIP-XXX: Support
currentFetchEventTimeLag and processingLag metrics [1].
The main motivation for this change was that I had some difficulties in
implementing the currentFetchEventTimeLag metrics for KafkaSource [2]. So I
proposed to let the SourceReaderMetricGroup provide an interface to capture
the FetchTime so that all the FLIP-27 [3] sources can easily implement the
currentFetchEventTimeLag metrics.
In addition, I propose to support the processingLag metric for the FLIP-27
sources to measure the current processing latency of the source.
See the FLIP [1] and Jira [2] for more details.
Looking forward to your comments and opinions!
Thanks,
TanJiaLiang.
[1]
https://docs.google.com/document/d/1nPhh1A-v-a7zyQyl1A5-K5DeUqbfxNXdjr2TVBT-QMs/edit?usp=sharing
[2] https://issues.apache.org/jira/browse/FLINK-33173
[3] https://cwiki.apache.org/confluence/display/FLINK/FLIP-27
