Thank you very much! I have two new questions:
1) the async operator must emit some value to the async collector (even it acts as a sink), right? 2) How could I use CheckpointListener with async operator? Could you give a simple example or doc page? 2017-12-08 18:25 GMT+08:00 Stefan Richter <s.rich...@data-artisans.com>: > Hi, > > Flink currently does not offer async sinks out of the box, but there is no > fundamental problem against having them and we will probably offer something > is this direction in the future. In the meantime, you can build something > like this by replacing the sink with an async io operator that acts as sink > (i.e. does the writes to the db) followed by a discarding sink for compliance > with the API. > > You need to be a bit careful if your sink needs exactly-once semantics. In > this case things should either be idempotent or the db must support rolling > back changes between checkpoints, e.g. via transactions. Commits should be > triggered for confirmed checkpoints („notifyCheckpointComplete“). > > Your assumptions about the blocking behavior of the non-async sinks is > correct. > > Best, > Stefan > >> Am 08.12.2017 um 08:11 schrieb Jinhua Luo <luajit...@gmail.com>: >> >> Hi, all. >> >> The invoke method of sink seems no way to make async io? e.g. returns Future? >> >> For example, the redis connector uses jedis lib to execute redis >> command synchronously: >> >> https://github.com/apache/bahir-flink/blob/master/flink-connector-redis/src/main/java/org/apache/flink/streaming/connectors/redis/RedisSink.java >> >> Then it will block the task thread of flink waiting the network >> response from redis server per command?! Is it possible for other >> operators running in the same thread with sink? If so, then it would >> block them too? >> >> I know flink has asyncio api, but it seems not for used by sink impl? >> >> https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/asyncio.html >> >> Thanks. >
