Hi Danny, Thanks for the response.
In short without restarting we cannot add new sinks or sources. For better understanding I will explain my problem more clearly. My scenario is I have two topics, one is configuration topic and second one is event activities. * In the configuration topic I will get details of the kafka cluster details and elasticsearch cluster details. * In the event activities i will get events and each event will have a tenantId * Suppose now we are getting a new tenantId data I need to send data to the respective elastic search cluster which I will come to know in the runtime from the configuration topic. * Is there a way to add a new elastic search sink in the same job without restarting. Before starting a job I can create two elastic search sinks and with a condition I can route the data to the respective elastic search cluster. Is there a way to do it in runtime? Thanks and Regards, Doinesh. On Wed, Jul 1, 2020 at 5:24 PM Danny Chan <yuzhao....@gmail.com> wrote: > Sorry, a job graph is solid while we compile it before submitting to the > cluster, not dynamic as what you want. > > You did can write some wrapper operators which response to your own PRCs > to run the appended operators you want, > But the you should keep the consistency semantics by yourself. > > Best, > Danny Chan > 在 2020年6月28日 +0800 PM3:30，C DINESH <dinesh.kitt...@gmail.com>，写道： > > Hi All, > > In a flink job I have a pipeline. It is consuming data from one kafka > topic and storing data to Elastic search cluster. > > without restarting the job can we add another kafka cluster and another > elastic search sink to the job. Which means i will supply the new kafka > cluster and elastic search details in the topic. After consuming the data > can our flink job add the new source and sink to the same job. > > > Thanks & Regards, > Dinesh. > >
