Hey Chris,
That's a cool idea! That can certainly be applied for failures other
than poll(), and could be useful when combined with the Offsets
modification API.
Perhaps failures inside of poll() can be handled by an extra
mechanism, similar to the ErrantRecordReporter, which allows reporting
aff
Hi Greg,
This was my understanding as well--if we can't turn a record into a byte
array on the source side, it's difficult to know exactly what to write to a
DLQ topic.
One idea I've toyed with recently is that we could write the source
partition and offset for the failed record (assuming, hopefu
Hi Yeikel,
Thanks for your question. It certainly isn't clear from the original
KIP-298, the attached discussion, or the follow-up KIP-610 as to why
the situation is asymmetric.
The reason as I understand it is: Source connectors are responsible
for importing data to Kafka. If an error occurs dur
Hi all,
Sink connectors support Dear Letter Queues[1], but Source connectors don't seem
to
What is the reason that we decided to do that?
In my data pipeline, I'd like to apply some transformations to the messages
before they are sink, but that leaves me vulnerable to failures as I need to
ei
