Re: How to broadcast messages to all task manager instances in cluster?

2018-05-13 Thread Di Tang
> DataStream<> data = input. > .do() > .something() > .fancy(); > > controlConfigInput.broadcast() > .connect(data) > .flatMap(new MyFancyOperatorThatDependsOnConfigStream()) > > Or slide 36 from here: > https://www.slideshare.net/dataArtisans/apache-flink-datas

How to broadcast messages to all task manager instances in cluster?

2018-05-11 Thread Di Tang
Hi guys: I have a Flink job which contains multiple pipelines. Each pipeline depends on some configuration. I want to make the configuration dynamic and effective after change so I created a data source which periodically poll the database storing the configuration. However, how can I broadcast th