The intent is that users shouldn't have to use async I/O since the idea is that the runner should increase the number of workers/threads being processed automatically so that you never need to special case this. Unfortunately Dataflow is the only one who does this today so you are forced to use something like GroupIntoBatches[1] to gather input elements that you convert into requests you want to send and manage your own threads / completion.
1: https://beam.apache.org/documentation/transforms/java/aggregation/groupintobatches/ On Sun, Jun 14, 2020 at 7:21 PM Eleanore Jin <eleanore....@gmail.com> wrote: > Hi Community, > > I am trying to convert an existing Flink job into Beam pipeline. In the > current Flink job, we have async I/O operator ( > https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/asyncio.html) > which extends RichAsyncFunction > <https://ci.apache.org/projects/flink/flink-docs-master/api/java/index.html?org/apache/flink/streaming/api/functions/async/RichAsyncFunction.html> > . > > I did not find any document online for Beam to support this, if it is > documented somewhere, can you please point to me? > > In case Beam does not support it, is there any suggested 'work around' for > it? > > Thanks a lot! > Eleanore >
