Hi Eleanore, we understand side-input as something more general than simple broadcast input, see FLIP-17 for details [1].
If a broadcast fits your use case, you can use that of course. We are aiming for something, where a side input can also be co-partitioned. We are currently laying the foundations for that feature. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-17+Side+Inputs+for+DataStream+API On Thu, Feb 27, 2020 at 6:46 AM Jin Yi <eleanore....@gmail.com> wrote: > Hi All, > > there is a recent published article in the flink official website for > running beam on top of flink > https://flink.apache.org/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html > > In the article: > > - You get additional features like side inputs and cross-language > pipelines that are not supported natively in Flink but only supported when > using Beam with Flink > > Ultimately, Beam pipeline will be translated into Flink job. So does > beam's side input translates into Flink Broadcast stream? > > If I look at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators, > it looks like converting the the side input into broadcast stream, then why > it says Flink does not support it natively? > > Thanks a lot! > Eleanore > >
