Hi Shuyi, Thanks for bringing this issue up. Per partition lag is definitely something that should be exposed. I replied to the JIRA with some of my concerns. Do you mind keeping the discussion in the JIRA ticket so it is easier for future readers to follow the issue?
Thanks, Jiangjie (Becket) Qin On Mon, Mar 18, 2019 at 2:14 PM Shuyi Chen <suez1...@gmail.com> wrote: > Hi all, > > We found that Flink's Kafka connector does not expose the per-partition > Kafka lag. The metric is available in KafkaConsumer after Kafka 0.10.2. And > it's an important metric to diagnose which Kafka partition is lagging in > production. I've created a JIRA ( > https://issues.apache.org/jira/browse/FLINK-11912) and created a proposed > change in Flink to register and expose the metrics. Could someone help take > a look and give some suggestions? Thanks a lot. > > Shuyi >
