Hi Aljoscha, Thanks for quick reply. I didn't quite understand your suggestion. Say I have three Kafka Stream sources that my Flink program consumes. How can I modify those three sources to be Kafka source as well as consumer of this single element?
Thanks, +satish On Thu, Nov 3, 2016 at 6:37 PM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi, > I'm afraid this is not possible right now if you don't want to go with > completely custom sources/operators. > > If you want to go the custom source route you would have only one true > source in your job that does the global initialisation and then emits one > element. Your other sources would be operators that have this one source as > input and only start producing data once that one element arrived. In this > way, you would block all other sources until the first source has done the > initialisation. > > Cheers, > Aljoscha > > On Thu, 3 Nov 2016 at 09:26 Satish Chandra Gupta <scgupt...@gmail.com> > wrote: > >> Hi, >> >> I need to do set/initialize some config of a framework/util that is used >> in my Flink stream processing app. Basically, a piece of code that needs to >> be executed exactly once before anything else. Clearly doing it in the main >> flink processor function will not suffice, as apart from the client, the >> same needs to be done on other node before topology is executed. >> >> I have gone through Flink best practices >> <https://ci.apache.org/projects/flink/flink-docs-master/monitoring/best_practices.html>, >> and one way I can think about it to check whether init has been done in the >> open() all Rich Functions, and if not then call the initialization code. >> But that seems to be "right", basically to add any operator, I must do this >> initillazation call boilerplate code. >> >> Is there anyway to define a global initializations in Flink, or to define >> an operator that is guaranteed to be called the first thing on all nodes? >> >> Thanks, >> +satish >> >
