Roberto, I don't think shark is an issue -- I have shark server running on a node that also acts as a worker. What you can do is turn off shark server, just run start-all to start your spark cluster. then you can try bin/spark-shell --master <yourmasterip> and see if you can successfully run some "hello world" stuff. This will verify you have a working Spark cluster. Shark is just an application on top of it, so I can't imagine that's what's causing interference. But stopping it is the simplest way to check.
On Wed, Oct 29, 2014 at 10:54 PM, Pagliari, Roberto <rpagli...@appcomsci.com > wrote: > hi Yana, > in my case I did not start any spark worker. However, shark was definitely > running. Do you think that might be a problem? > > I will take a look > > Thank you, > > ------------------------------ > *From:* Yana Kadiyska [yana.kadiy...@gmail.com] > *Sent:* Wednesday, October 29, 2014 9:45 AM > *To:* Pagliari, Roberto > *Cc:* user@spark.apache.org > *Subject:* Re: problem with start-slaves.sh > > I see this when I start a worker and then try to start it again > forgetting it's already running (I don't use start-slaves, I start the > slaves individually with start-slave.sh). All this is telling you is that > there is already a running process on that machine. You can see it if you > do a ps -aef|grep worker > > you can look on the spark UI and see if your master shows this machine > as connected to it already. If it doesn't, you might want to kill the > worker process and restart it. > > On Tue, Oct 28, 2014 at 4:32 PM, Pagliari, Roberto < > rpagli...@appcomsci.com> wrote: > >> I ran sbin/start-master.sh followed by sbin/start-slaves.sh (I build >> with PHive option to be able to interface with hive) >> >> >> >> I’m getting this >> >> >> >> ip_address: org.apache.spark.deploy.worker.Worker running as process >> xxxx. Stop it first. >> >> >> >> Am I doing something wrong? In my specific case, shark+hive is running on >> the nodes. Does that interfere with spark? >> >> >> >> Thank you, >> > >
