One way I thought to achieve this is - 1. For failures, add a special record to collection in RichAsyncFunction 2. Filter out those special records from the DataStream and push to another Kafka Let me know if it makes sense.
On Fri, Jun 11, 2021 at 10:40 AM Satish Saley <satishsa...@apache.org> wrote: > Hi, > - I have a kafka consumer to read events. > - Then, I have RichAsyncFunction to call a remote service to get > more information about that event. > > If the remote call fails after X number of retries, I don't want flink to > fail the job and start processing from the beginning. Instead I would like > to push info about failed call to another Kafka topic. Is there a way to > achieve this? >
