You'd need the jar file (holding class definitions etc.) to do the
deserialisation on the executor.
On 18 June 2015 at 03:48, Shiyao Ma wrote:
> Hi,
>
> Looking from my executor logs, the submitted application jar is
> transmitted to each executors?
>
> Why does spark do the above? To my understa
Really excited to try out the new Docker executor support on 1.4.1, I'm
making progress but feel like I'm missing something.
(versions:
spark-1.4.1-hadoop2.6 - not using hadoop yet
mac os x yosemite java 8 spark-shell
mesos 0.22.1 : 2 slaves, 1 master + zk , all on centos 6.x
docker 1.8.x
)
I w
t we use: --mesos
> mesos://zk://prodMesosMaster01:2181,prodMesosMaster02:2181,prodMesosMaster03:2181/mesos
>
> And we followed the instructions here:
> https://spark.apache.org/docs/1.2.0/running-on-mesos.html
>
> On 23 September 2015 at 08:22, Dick Davies wrote:
>>
>> I'
Just wondered if anyone had managed to start spark
jobs on mesos wrapped in a docker container?
At present (i.e. very early testing) I'm able to submit executors
to mesos via spark-submit easily enough, but they fall over
as we don't have a JVM on our slaves out of the box.
I can push one out via
;> On Wednesday, December 3, 2014, Matei Zaharia
>> wrote:
>>>
>>> I'd suggest asking about this on the Mesos list (CCed). As far as I know,
>>> there was actually some ongoing work for this.
>>>
>>> Matei
>>>
>>>
