Hi Arvid, Thanks a lot for the response and yes I am aware of FLIP-17.
Eleanore On Fri, Feb 28, 2020 at 2:16 AM Arvid Heise <ar...@ververica.com> wrote: > Hi Eleanore, > > we understand side-input as something more general than simple broadcast > input, see FLIP-17 for details [1]. > > If a broadcast fits your use case, you can use that of course. We are > aiming for something, where a side input can also be co-partitioned. We are > currently laying the foundations for that feature. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-17+Side+Inputs+for+DataStream+API > > On Thu, Feb 27, 2020 at 6:46 AM Jin Yi <eleanore....@gmail.com> wrote: > >> Hi All, >> >> there is a recent published article in the flink official website for >> running beam on top of flink >> https://flink.apache.org/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html >> >> In the article: >> >> - You get additional features like side inputs and cross-language >> pipelines that are not supported natively in Flink but only supported when >> using Beam with Flink >> >> Ultimately, Beam pipeline will be translated into Flink job. So does >> beam's side input translates into Flink Broadcast stream? >> >> If I look at >> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators, >> it looks like converting the the side input into broadcast stream, then why >> it says Flink does not support it natively? >> >> Thanks a lot! >> Eleanore >> >>
