Re: Multiple spark shell sessions

2014-10-01 Thread Sanjay Subramanian
: Error was: Failure(java.net.BindException: Address already in use)14/10/01 17:34:38 INFO SparkUI: Started SparkUI at http://hadoop02:4041 sanjay From: Matei Zaharia To: Sanjay Subramanian Cc: "user@spark.apache.org" Sent: Wednesday, October 1, 2014 5:19 PM Subject: Re: Mult

Re: Multiple spark shell sessions

2014-10-01 Thread Matei Zaharia
You need to set --total-executor-cores to limit how many total cores it grabs on the cluster. --executor-cores is just for each individual executor, but it will try to launch many of them. Matei On Oct 1, 2014, at 4:29 PM, Sanjay Subramanian wrote: > hey guys > > I am using spark 1.0.0+cdh

Re: Multiple spark shell sessions

2014-09-05 Thread Andrew Ash
Hi Dhimant, We also cleaned up these needless warnings on port failover in Spark 1.1 -- see https://issues.apache.org/jira/browse/SPARK-1902 Andrew On Thu, Sep 4, 2014 at 7:38 AM, Dhimant wrote: > Thanks Yana, > I am able to execute application and command via another session, i also > receiv

Re: Multiple spark shell sessions

2014-09-04 Thread Dhimant
Thanks Yana, I am able to execute application and command via another session, i also received another port for UI application. Thanks, Dhimant -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Multiple-spark-shell-sessions-tp13441p13459.html Sent from the Ap

Re: Multiple spark shell sessions

2014-09-04 Thread Yana Kadiyska
These are just warnings from the web server. Normally your application will have a UI page on port 4040. In your case, a little after the warning it should bind just fine to another port (mine picked 4041). Im running on 0.9.1. Do you actually see the application failing? The main thing when runnin