Hi Srikanth, Looked at the source code once again and discussing with other committer I now remembered why we designed it that way: when you set the HandlerResponse to FAIL, it means that once a "poison record" is received, stop the world by throwing this exception all the way up. And hence at that time the application would be stopped anyways so we would not need to record this metric.
So in other words, I think it is rather a documentation improvement that we should do. Guozhang On Fri, Jan 26, 2018 at 8:56 AM, Guozhang Wang <wangg...@gmail.com> wrote: > Helo Srikanth, > > Thanks for reporting this, as I checked the code skippedDueToDeserializati > onError is effectively only recorded when the DeserializationHandlerResp > onse is not set to FAIL. I agree it is not exactly matching the > documentation's guidance, and will try to file a JIRA and fix it. > > As for skippedDueToDeserializationError and skipped-records-rate, there > is an open JIRA discussing about this: https://issues.apache. > org/jira/browse/KAFKA-6376, just FYI. > > > Could you share on which version of Kafka did you observe this issue? > > Guozhang > > On Fri, Jan 26, 2018 at 6:30 AM, Srikanth <srikanth...@gmail.com> wrote: > >> Hello, >> >> As per doc when LogAndContinueExceptionHandler is used it will set >> skippedDueToDeserializationError-rate metric to indicate deserialization >> error. >> I notice that it is never set. Instead skipped-records-rate is set. My >> understanding was that skipped-records-rate is set due to timestamp >> extraction errors. >> >> Ex, I sent a few invalid records to a topic and was able to see exception >> during deserialization. >> >> org.apache.kafka.common.errors.SerializationException: Error >> deserializing >> Avro message for id -1 >> Caused by: org.apache.kafka.common.errors.SerializationException: Unknown >> magic byte! >> 18/01/24 06:50:09 WARN StreamThread: Exception caught during >> Deserialization, taskId: 0_0, topic: docker.event.1, partition: 0, offset: >> 3764 >> >> These incremented skipped-records-[rate|total]. >> >> Thanks, >> Srikanth >> > > > > -- > -- Guozhang > -- -- Guozhang
