Hi Alexandru, the most likely reason is that you are using *AsyncDataStream* incorrectly. You have to ensure that all work is done in a separate thread.
AsyncIO will only guarantee that async results are merged back into the sync stream. The reason is that many libraries have their own thread pool to send async requests and we didn't want to duplicate that. In the easiest way, you spawn an executor with Executors.newFixedThreadPool(10) and submit the jobs and then feed the results back. On Wed, Feb 26, 2020 at 2:27 PM Alexandru Vasiu < alexandru.va...@complyadvantage.com> wrote: > Hi, > > We have a pipeline which has somewhere a step of > *AsyncDataStream.unorderedWait > *where some web requests are executed. The pipeline works, but when it > tries to make checkpoints it fails always with a timeout error (and it > stops at the component containing this async data stream). We are using > Flink 1.10.0 in Scala 2.12.10 and this config for checkpoints: > > "checkpoints_interval": 180000, > "min_pause_between_checkpoints": 10000, > "checkpoints_timeout": 600000, > "tolerable_checkpoints_failure_number": 20, > "max_concurrent_checkpoints": 1, > "checkpoint_mode": CheckpointingMode.EXACTLY_ONCE > > Do you know why checkpointing doesn't work in this case? > > Thank you, > Alex Vasiu > > ComplyAdvantage is a trading name of IVXS TECHNOLOGY ROMANIA. This > message, including any attachments, is intended only for the use of the > individual(s) to whom it is addressed and may contain information that is > strictly privileged/confidential. Any other distribution, copying or > disclosure is strictly prohibited. If you are not the intended recipient or > have received this message in error, please notify the sender immediately > by reply email and permanently delete this message including any > attachments, without reading it or making a copy. Contact us > <https://complyadvantage.com/contact/>. Website > <https://complyadvantage.com/>.
