The master node does not have to be similar to the worker nodes. It can be a smaller machine.
In case of C*, again you don't need to have C* on the master node. You need C* and Spark workers co-located. Master can be on one of the C* node or a non-C* node. Mohammed -----Original Message----- From: algermissen1971 [mailto:[email protected]] Sent: Sunday, July 12, 2015 12:35 PM To: Spark User Subject: Master vs. Slave Nodes Clarification Hi, I have a question that I really have problems with figuring out for myself: Does the master node in a spark cluster need to be a node similar to the slave nodes or should I rather view it as a coordinating node, that does not need much computing or storage power? For example, when using Spark Streaming and Checkpointing, would the master node need access to the shared file system (e.g. HDFS)? Or do I only need to mount that on the slaves? (likewise, if I use the Cassandra-Connector, does that (and C*) need to be installed on the master node, too?) Or, in other words: is the master just one node of similar cluster nodes, or is it merely a 'small control node', for which sort of any small VM would do? Jan --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
