Re: launch spark on mesos within a docker container

2016-12-30 Thread Timothy Chen
It seems like it's getting offer decline calls, which seems like it's getting the offer calls and was able to reply. Can you turn on TRACE logging in Spark with the Mesos coarse grain scheduler and see if it says if it is processing the offers? Tim On Fri, Dec 30, 2016 at 2:35 PM, Ji Yan wrote:

Re: launch spark on mesos within a docker container

2016-12-30 Thread Ji Yan
Thanks Timothy, Setting these four environment variables as you suggested has got the Spark running LIBPROCESS_ADVERTISE_IP=LIBPROCESS_ADVERTISE_PORT=40286 LIBPROCESS_IP=0.0.0.0 LIBPROCESS_PORT=40286 After that, it seems that Spark cannot accept any offer from mesos. If I run the same script out

Re: launch spark on mesos within a docker container

2016-12-30 Thread Timothy Chen
Hi Ji, One way to make it fixed is to set LIBPROCESS_PORT environment variable on the executor when it is launched. Tim > On Dec 30, 2016, at 1:23 PM, Ji Yan wrote: > > Dear Spark Users, > > We are trying to launch Spark on Mesos from within a docker container. We > have found that since t