If you want the machine that hosts the driver to also do work, you can designate it as a worker too, if I'm not mistaken. I don't think the driver should do work, logically, but, that's not to say that the machine it's on shouldn't do work. -- Sean Owen | Director, Data Science | London
On Tue, Apr 8, 2014 at 8:24 PM, Nicholas Chammas <nicholas.cham...@gmail.com> wrote: > So I have a cluster in EC2 doing some work, and when I take a look here > > http://driver-node:4040/executors/ > > I see that my driver node is snoozing on the job: No tasks, no memory used, > and no RDD blocks cached. > > I'm assuming that it was a conscious design choice not to have the driver > node partake in the cluster's workload. > > Why is that? It seems like a wasted resource. > > What's more, the slaves may rise up one day and overthrow the driver out of > resentment. > > Nick > > > ________________________________ > View this message in context: Why doesn't the driver node do any work? > Sent from the Apache Spark User List mailing list archive at Nabble.com.
