Hi Alexandru,
please share your code of the AsyncFunction. Your observed behaviour is
completely not in line how things should behave.
As long as you are not blocking AsyncFunction#asyncInvoke, checkpointing
will work.
On Fri, Feb 28, 2020 at 9:16 AM Alexandru Vasiu <
alexandru.va...@complyadvan
Hi,
That's how we used the executor. I think the problem is that the web
requests took too long time to complete (3-4 seconds) because the requests
are using a proxy server. I also transformed the asyncDataStream using a
flatMap and same issue (no successfull checkpoint). If I used a simple web
pa
Hi Alexandru,
I cannot follow what you posted, so let me point you to one example [1].
The executor needs to be used inside the async function.
[1]
https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/async/AsyncIOEx
Hi,
We tried to
use ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1)) as
executor and we still have this problem. We even tried even other
executors, but still the same. Only once we succeded to have a checkpoint
when we tried without the web requests.
Do you have any other ideas?
T
Hi Alexandru,
the most likely reason is that you are using *AsyncDataStream* incorrectly.
You have to ensure that all work is done in a separate thread.
AsyncIO will only guarantee that async results are merged back into the
sync stream. The reason is that many libraries have their own thread poo
Hi,
We have a pipeline which has somewhere a step of *AsyncDataStream.unorderedWait
*where some web requests are executed. The pipeline works, but when it
tries to make checkpoints it fails always with a timeout error (and it
stops at the component containing this async data stream). We are using
