Hi Satish,
usually you would side-outputs [1] for that but afaik asyncIO doesn't
support that (yet).
So your option works well to use some union type. You can then chain a map
function that uses side-outputs.
[1]
https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/datastream/side_outp
One way I thought to achieve this is -
1. For failures, add a special record to collection in RichAsyncFunction
2. Filter out those special records from the DataStream and push to another
Kafka
Let me know if it makes sense.
On Fri, Jun 11, 2021 at 10:40 AM Satish Saley
wrote:
> Hi,
> - I have
Hi,
- I have a kafka consumer to read events.
- Then, I have RichAsyncFunction to call a remote service to get
more information about that event.
If the remote call fails after X number of retries, I don't want flink to
fail the job and start processing from the beginning. Instead I would like
to
