Hi Arvid,
Thanks for the proposal. I like the idea of exposing concrete metric group
class so that users can access the predefined metrics.
A few questions are following:
1. When exposing the OperatorIOMetrics to the users, we are also exposing
the reuseInputMetricsForTask to the users. Should we hide these two methods
because users won't have enough information to decide whether the records
IO metrics should be reused by the task or not.
2. Similar to question 1, in the OperatorIOMetricGroup, we are adding
numBytesInCounter and numBytesOutCounter. Should these metrics be reusing
the task level metrics by default?
3. Regarding SourceMetricGroup#setLastFetchTimeGauge(), I am not sure how
it works with the FetchLag. Typically there are two cases when reporting
the fetch lag.
A. The EventTime is known at the point when the record is fetched in
the SplitFetcher, so the fetch lag can be derived and reported immediately.
B. The EventTime is known only after the fetched record was parsed in
the RecordEmitter. In this case, the RecordEmitter needs to get the fetch
time of that particular record.
I am not sure when users set the LastFetchTime in the above two cases. Can
you help elaborate on how users should use it?
Thanks,
Jiangjie (Becket) Qin
On Thu, Jul 8, 2021 at 10:25 PM Arvid Heise <ar...@apache.org> wrote:
> Dear devs,
>
> As a continuation and generalization of FLIP-33 (Standardize Connector
> Metrics) [1], we'd like to discuss how we actually expose the standardized
> operator metrics to users in terms of changes to the API.
>
> Please check out the FLIP [2] and provide feedback.
>
> Best,
>
> Arvid
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics
> [2]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
>
