Re: Spark-on-YARN architecture

2015-03-10 Thread Sean Owen
I suppose you just provision enough resource to run both on that node... but it really shouldn't matter. The RM and your AM aren't communicating heavily. On Tue, Mar 10, 2015 at 10:23 AM, Harika Matha wrote: > Thanks for the quick reply. > > I am running the application in YARN client mode. > And

Re: Spark-on-YARN architecture

2015-03-10 Thread Harika Matha
Thanks for the quick reply. I am running the application in YARN client mode. And I want to run the AM on the same node as RM inorder use the node which otherwise would run AM. How can I get AM run on the same node as RM? On Tue, Mar 10, 2015 at 3:49 PM, Sean Owen wrote: > In YARN cluster mod

Re: Spark-on-YARN architecture

2015-03-10 Thread Sean Owen
In YARN cluster mode, there is no Spark master, since YARN is your resource manager. Yes you could force your AM somehow to run on the same node as the RM, but why -- what do think is faster about that? On Tue, Mar 10, 2015 at 10:06 AM, Harika wrote: > Hi all, > > I have Spark cluster setup on YA