/cc @John Casey <johnjca...@google.com> - who might be able to help. On Wed, Jan 18, 2023 at 12:43 PM Lydian <lydia...@gmail.com> wrote:
> Hi, > I know that Beam KafkaIO doesn't use the native kafka offset, and > therefore I cannot use kafka metrics directly. > > Wondering what would be the right way to expose those metrics of my > KafkaIO pipeline? > Things I am interested includes: > > - bytes-consumed-rate > - fetch-latency-avg > - records-lag > - commit-rate > > consumer lagWondering how people get these metrics or instead of doing > this? or we should just enable `commit_offset_in_finalize` and then use the > Kafka metrics directly? > > also wondering if there's anything to notice when enabling the > commit_offset_in_finalize? Thanks! > > Sincerely, > Lydian Lee > >
