Hi Jorge, Thanks for the KIP. It looks like a very good addition. I skimmed through once and had a couple of questions =>
1) I am assuming the new metrics would be task level metric. Could you specify the way it's done for other sink/source connector? 2) I am slightly confused about the e2e latency metric. Let's consider the sink connector metric. If I look at the way it's supposed to be calculated, i.e the difference between the record timestamp and the wall clock time, it looks like a per record metric. However, the put-batch time measures the time to put a batch of records to external sink. So, I would assume the 2 can't be added as is to compute the e2e latency. Maybe I am missing something here. Could you plz clarify this. Thanks! Sagar. On Tue, Aug 30, 2022 at 8:43 PM Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Hi all, > > I'd like to start a discussion thread on KIP-864: Add End-To-End Latency > Metrics to Connectors. > This KIP aims to improve the metrics available on Source and Sink > Connectors to measure end-to-end latency, including source and sink record > conversion time, and sink record e2e latency (similar to KIP-613 for > Streams). > > The KIP is here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-864%3A+Add+End-To-End+Latency+Metrics+to+Connectors > > Please take a look and let me know what you think. > > Cheers, > Jorge. >
