I have read through the logs and I am of the opinion that while we should not let the store and the change Log diverge. However, it is not obvious how we would be able to do that and allow the custom serializer to effect that topic. In order to get around this we can create a shell around the handler that will not trigger the custom handler if it is a changeLog topic. This is also a problem that we can fix in the general handler, so I think we can add that fix to this kip.
As for the repartition topics I can not see any reason to not apply the custom logic. By default it fails in any case and does not have to be changed from that. For the DSL I think that that can come later as this change would not affect the status quo. EOS and this handler can be made mutually exclusive or we could give users the option to use both with a warning. I would be interested in hearing other people's suggestions about that.Walker On Tue, Oct 15, 2019 at 11:50 PM Matthias J. Sax <matth...@confluent.io> wrote: > Walker, > > thanks for picking up this KIP. Did you read the previous discussion? > It's still unclear if we want to apply the handler to repartition topics > or not, and how errors for stores and changelog topic should be handled. > For the Processor API, users could catch `SerializationException` but > for the DSL this is not possible. Even if there is no serialization > problem, it's questionable if we should allow to swallow error when > writing into the changelog topic, because the store content and the > topic content could diverge, what is especially critical for the EOS case. > > > -Matthias > > On 10/15/19 10:25 AM, Walker Carlson wrote: > > Hello all, > > > > I would like to restart the discussion of this KIP 399 > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-399%3A+Extend+ProductionExceptionHandler+to+cover+serialization+exceptions > >. > > I think it is some low hanging fruit that could be quite beneficial. > > > > Thanks, > > Walker > > > >
