+1 we have a similar use case for message schema validation. Eleanore
On Wed, Jul 22, 2020 at 4:12 AM Tom Fennelly <tfenne...@cloudbees.com> wrote: > Hi. > > I've been searching blogs etc trying to see if there are > established patterns/mechanisms for reprocessing of failed messages via > something like a DLQ. I've read about using checkpointing and restarting > tasks (not what we want because we want to keep processing forward) and > then also how some use side outputs to filter "bad" data to a DLQ style > topic. Kafka has dead letter topic configs too but it seems that can't > really be used from inside Flink (from what I can see). > > We're running Flink+Kafka. I'm new to Flink+Kafka so not sure if there > just isn't a defined pattern for it, or if I'm just not asking the right > questions in my searches. I searched the archives here and don't see > anything either, which obviously makes me think that I'm not thinking about > this in the "Flink way" :-| > > Regards, > > Tom. >
