We have the same setup and it works quite well. One thing to take into
account is that your HTTP call may happen multiple times if you’re using
checkpointing/ fault tolerance mechanism, so it’s important that those
calls are idempotent and won’t duplicate data.
Also we’ve found that it’s important
Guys, thanks for the great advice. It works!
I used HttpAsyncClient from Apache Commons.
At first I tried to implement the async http client by implementing
AsyncFunction. I implemented the asyncInvoke method and used
try-with-resouce to instantiate the client (because it's
CloseableHttpAsyncClient
If I understand correctly he wants the HTTP result in the DB. So I do not
think side output works here. The DB would have to be the sink. Also sinks
in Flink are the final destination.
So it would have to be RabbitMQ -> Some Cool Business Logic Operators
Here > Async I/O HTTP Operator
So you have rabitmq source and http sink?
If so you can use side output in order to dump your data to db.
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/side_output.html
On Sat, Feb 29, 2020, 23:01 Gadi Katsovich wrote:
> Hi,
> I'm new to flink and am evaluating it to replace
