I see...
Because of the start-cluster script, I was under the impression that the 
jobmanager had to connect to each node upon start-up, which would make scaling 
an issue without restarting the job manager, but it makes sense now. Thanks for 
the clarification.

Side question:what happens if the job manager fails? Taskmanagers keep running 
the jobs? Should a typical production setup include multiple jobmanagers for 
replication, and if so how is that configured?

Emmanuel



-------- Original message --------
From: Stephan Ewen <[email protected]>
Date:03/16/2015  1:09 AM  (GMT-08:00)
To: [email protected]
Subject: Re: Scaling a Flink cluster

Hi Emmanuel!

The slaves file is not needed on every node. It is only used by the
"start-cluster.sh" Script, which makes an ssh call to every host in that
file to start a taskmanager.

You can add a taskmanager to an existing flink cluster by simply calling
"taskmanager.sh start" on that machine (which should have a flink-conf.yaml
file). The flink-conf.yaml may actually be different for every taskmanager
as well, but that is a more uncommon setup...

Greetings,
Stephan
Am 16.03.2015 08:27 schrieb "Emmanuel" <[email protected]>:

> Hello,
>
> In my understanding, the flink-conf.yaml is the one config file to
> configure a cluster.
> The slave file lists the slave nodes.
> they must both be on every node.
>
> I'm trying to understand what is the best strategy to scale a Flink
> cluster since:
> - adding a node means adding an entry to the slave list and replicating on
> every node.
>
> Does the cluster need to be restarted to take the new nodes into account?
> It seems like it.
> Having to replicate the file on all nodes is not super convenient.
> Restarting is even more trouble.
> Is there a way to scale a live cluster? If so how?
> Any link to the info would be helpful.
>
> Thanks
>

Reply via email to