Spark on Mesos broken on 2.4 ?

Hello Everyone, I’m just trying out the spark-shell on mesos and I don’t get any executors. To debug it I started the vagrant box from aurora and try it out there and I can the same issue as I’m getting on my cluster. On Mesos the only active framework is the spark-shel, it is running 1.6.1 an

Re: Spark on Mesos - Weird behavior

Hi Susan, This is exactly what we have used. Thank you for your interest! - Thodoris > On 23 Jul 2018, at 20:55, Susan X. Huynh wrote: > > Hi Thodoris, > > Maybe setting "spark.scheduler.minRegisteredResourcesRatio" to > 0 would > help? Default value is 0 with Mesos. > > "The minimum ratio

Re: Spark on Mesos: Spark issuing hundreds of SUBSCRIBE requests / second and crashing Mesos

That does sound like it could be it - I checked our libmesos version and it is 1.4.1. I'll try upgrading libmesos. Thanks. On Mon, Jul 23, 2018 at 12:13 PM Susan X. Huynh wrote: > Hi Nimi, > > This sounds similar to a bug I have come across before. See: > https://jira.apache.org/jira/browse/SPA

Re: Spark on Mesos: Spark issuing hundreds of SUBSCRIBE requests / second and crashing Mesos

Hi Nimi, This sounds similar to a bug I have come across before. See: https://jira.apache.org/jira/browse/SPARK-22342?focusedCommentId=16429950&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16429950 It turned out to be a bug in libmesos (the client library used t

Re: Spark on Mesos - Weird behavior

Hi Thodoris, Maybe setting "spark.scheduler.minRegisteredResourcesRatio" to > 0 would help? Default value is 0 with Mesos. "The minimum ratio of registered resources (registered resources / total expected resources) (resources are executors in yarn mode and Kubernetes mode, CPU cores in standalon

Spark on Mesos: Spark issuing hundreds of SUBSCRIBE requests / second and crashing Mesos

I've come across an issue with Mesos 1.4.1 and Spark 2.2.1. We launch Spark tasks using the MesosClusterDispatcher in cluster mode. On a couple of occasions, we have noticed that when the Spark Driver crashes (to various causes - human error, network error), sometimes, when the Driver is restarted,

Re: Spark on Mesos - Weird behavior

Oh, sorry, i missed that you use spark without dynamic allocation. Anyway, i don't know does this parameters works without dynamic allocation. On Wed, Jul 11, 2018 at 5:11 PM Thodoris Zois wrote: > Hello, > > Yeah you are right, but I think that works only if you use Spark dynamic > allocation.

Re: Spark on Mesos - Weird behavior

Hello, Yeah you are right, but I think that works only if you use Spark dynamic allocation. Am I wrong? -Thodoris > On 11 Jul 2018, at 17:09, Pavel Plotnikov > wrote: > > Hi, Thodoris > You can configure resources per executor and manipulate with number of > executers instead using spark.ma

Re: Spark on Mesos - Weird behavior

Hi, Thodoris You can configure resources per executor and manipulate with number of executers instead using spark.max.cores. I think spark.dynamicAllocation.minExecutors and spark.dynamicAllocation.maxExecutors configuration values can help you. On Tue, Jul 10, 2018 at 5:07 PM Thodoris Zois wrote

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

Thanks for your suggestion. I have been checking Spark-jobserver. Just a off-topic question about this project: Does Apache Spark project have any support/connection to this Spark-jobserver project? I noticed that they do not have release for the newest version of Spark (e.g., 2.3.1). As you men

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

t;> latency? > >> > >> Best > >> > >> > >> > >> -- > >> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > >> > >> - > >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >> > >> > > > Quoted from: > > http://apache-spark-user-list.1001560.n3.nabble.com/SPARK-on-MESOS-Avoid-re-fetching-Spark-binary-tp32849p32865.html > > > _ > Sent from http://apache-spark-user-list.1001560.n3.nabble.com > >

Re: Spark on Mesos - Weird behavior

Actually after some experiments we figured out that spark.max.cores / spark.executor.cores is the upper bound for the executors. Spark apps will run even only if one executor can be launched. Is there any way to specify also the lower bound? It is a bit annoying that seems that we can’t contro

Re: Spark on Mesos - Weird behavior

Hello Thodoris! Have you checked this: - does mesos cluster have available resources? - if spark have waiting tasks in queue more than spark.dynamicAllocation.schedulerBacklogTimeout configuration value? - And then, have you checked that mesos send offers to spark app mesos framework at least w

Spark on Mesos - Weird behavior

Hello list, We are running Apache Spark on a Mesos cluster and we face a weird behavior of executors. When we submit an app with e.g 10 cores and 2GB of memory and max cores 30, we expect to see 3 executors running on the cluster. However, sometimes there are only 2... Spark applications are no

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

Essentially correct. The latency to start a Spark Job is nowhere close to 2-4 seconds under typical conditions. Creating a new Spark Application every time instead of running multiple Jobs in one Application is not going to lead to acceptable interactive or real-time performance, nor is that an exe

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

The latency to start a Spark Job is nowhere close to 2-4 seconds under typical conditions. You appear to be creating a new Spark Application everytime instead of running multiple Jobs in one Application. On Fri, Jul 6, 2018 at 3:12 AM Tien Dat wrote: > Dear Timothy, > > It works like a charm now

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

I know there are some community efforts shown in Spark summits before, mostly around reusing the same Spark context with multiple “jobs”. I don’t think reducing Spark job startup time is a community priority afaik. Tim On Fri, Jul 6, 2018 at 7:12 PM Tien Dat wrote: > Dear Timothy, > > It works

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

Dear Timothy, It works like a charm now. BTW (don't judge me if I am to greedy :-)), the latency to start a Spark job is around 2-4 seconds, unless I am not aware of some awesome optimization on Spark. Do you know if Spark community is working on reducing this latency? Best -- Sent from: http

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

Got it, then you can have an extracted Spark directory on each host on the same location, and don’t specify SPARK_EXECUTOR_URI. Instead, set spark.mesos.executor.home to that directory. This should effectively do what you want, which avoids extracting and fetching and just executed the command. T

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

Thank you for your answer. The think it I actually pointed to a local binary file. And Mesos locally copied the binary file to a specific folder in /var/lib/mesos/... and extract it to every time it launched an Spark executor. With the fetch cache, the copy time is reduced, but the reduction is no

Re: [SPARK on MESOS] Avoid re-fetching Spark binary

If its available locally on each host, then don’t specify a remote url but a local file uri instead. We have a fetcher cache in Mesos a while ago, I believe there is integration in the Spark framework if you look at the documentation as well. With the fetcher cache enabled Mesos agent will cache t

[SPARK on MESOS] Avoid re-fetching Spark binary

Dear all, We are running Spark with Mesos as the master for resource management. In our cluster, there are jobs that require very short response time (near real time applications), which usually around 3-5 seconds. In order to Spark to execute with Mesos, one has to specify the SPARK_EXECUTOR_URI

Spark on Mesos failure, when launching a simple job

I have been trying to learn spark on mesos, but the spark-shell just keeps on ignoring the offers. Here is my setup: All the components are in the same subnet - 1 mesos master on EC2 instance (t2.micro) command: `mesos-master --work_dir=/tmp/abc --hostname=` - 2 mesos agents (each with 4

Re: Spark on Mesos with Docker in bridge networking mode

t; at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks( > SingleThreadEventExecutor.java:357) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) > at > io.netty.util.concurrent.SingleThreadEventExecutor$2. > run(SingleThreadEventExecutor.java:

Spark on Mesos with Docker in bridge networking mode

ntExecutor.java:111) at java.lang.Thread.run(Thread.java:745) I was trying to follow instructions here: https://github.com/apache/spark/pull/15120 So in my Marathon json I'm defining the ports to use for the spark driver, spark ui and block manager. Can anyone help me get this running in

Re: Dynamic resource allocation to Spark on Mesos

re you using marathon to run the shuffle service? > > On Tue, Feb 7, 2017 at 7:36 PM, Sun Rui wrote: > >> Yi Jan, >> >> We have been using Spark on Mesos with dynamic allocation enabled, which >> works and improves the overall cluster utilization. >> >>

Re: Dynamic resource allocation to Spark on Mesos

e service? > > On Tue, Feb 7, 2017 at 7:36 PM, Sun Rui <mailto:sunrise_...@163.com>> wrote: > Yi Jan, > > We have been using Spark on Mesos with dynamic allocation enabled, which > works and improves the overall cluster utilization. > > In terms of job, do you me

Re: Dynamic resource allocation to Spark on Mesos

Sun, are you using marathon to run the shuffle service? On Tue, Feb 7, 2017 at 7:36 PM, Sun Rui wrote: > Yi Jan, > > We have been using Spark on Mesos with dynamic allocation enabled, which > works and improves the overall cluster utilization. > > In terms of job, do you

Re: Dynamic resource allocation to Spark on Mesos

Yi Jan, We have been using Spark on Mesos with dynamic allocation enabled, which works and improves the overall cluster utilization. In terms of job, do you mean jobs inside a Spark application or jobs among different applications? Maybe you can read http://spark.apache.org/docs/latest/job

Re: Dynamic resource allocation to Spark on Mesos

t;>>>>> have 180Gb of memory and 64 cores, when I run spark-submit ( on mesos ) >>>>>>>> with --cpu_cores set to 1000, the job starts up with 64 cores. but >>>>>>>> when I >>>>>>>> set --memory to 200Gb, the job fails to start with

Re: Dynamic resource allocation to Spark on Mesos

>>>>>>> not accepted any resources; check your cluster UI to ensure that workers >>>>>>> are registered and have sufficient resources" >>>>>>> >>>>>>> Also it is confusing to me that --cpu_cores specifies the number of >>>&g

Re: Dynamic resource allocation to Spark on Mesos

cpu_cores specifies the number of >>>>>> cpu cores across all executors, but --memory specifies per executor >>>>>> memory >>>>>> requirement. >>>>>> >>>>>> On Mon, Jan 30, 2017 at 11:34 AM, Michael Gumme

Re: Dynamic resource allocation to Spark on Mesos

s confusing to me that --cpu_cores specifies the number of >>>>> cpu cores across all executors, but --memory specifies per executor memory >>>>> requirement. >>>>> >>>>> On Mon, Jan 30, 2017 at 11:34 AM, Michael Gummelt < >>>>> mgumm...

Re: Dynamic resource allocation to Spark on Mesos

30, 2017 at 11:34 AM, Michael Gummelt < >>>> mgumm...@mesosphere.io> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Jan 30, 2017 at 9:47 AM, Ji Yan wrote: >>>>> >>>>>> Tasks begin scheduling as soon

Re: Dynamic resource allocation to Spark on Mesos

30, 2017 at 11:34 AM, Michael Gummelt < >>> mgumm...@mesosphere.io> wrote: >>> >>>> >>>> >>>> On Mon, Jan 30, 2017 at 9:47 AM, Ji Yan wrote: >>>> >>>>> Tasks begin scheduling as soon as the first executor come

Re: Dynamic resource allocation to Spark on Mesos

; > wrote: >> >>> >>> >>> On Mon, Jan 30, 2017 at 9:47 AM, Ji Yan wrote: >>> >>>> Tasks begin scheduling as soon as the first executor comes up >>>> >>>> >>>> Thanks all for the clarification. Is this the d

Re: Dynamic resource allocation to Spark on Mesos

> > On Mon, Jan 30, 2017 at 11:34 AM, Michael Gummelt > wrote: > >> >> >> On Mon, Jan 30, 2017 at 9:47 AM, Ji Yan wrote: >> >>> Tasks begin scheduling as soon as the first executor comes up >>> >>> >>> Thanks all for the clari

Re: Dynamic resource allocation to Spark on Mesos

reduce the "resource fragmentation"? >>>> >>>> Anyway, that is not supported at this moment. In all the supported >>>> cluster managers of spark (mesos, yarn, standalone, and the up-to-coming >>>> spark on kubernetes), you have to specify the co

Re: Dynamic resource allocation to Spark on Mesos

On Mon, Jan 30, 2017 at 9:47 AM, Ji Yan wrote: > Tasks begin scheduling as soon as the first executor comes up > > > Thanks all for the clarification. Is this the default behavior of Spark on > Mesos today? I think this is what we are looking for because sometimes a > job c

Re: Dynamic resource allocation to Spark on Mesos

> > Tasks begin scheduling as soon as the first executor comes up Thanks all for the clarification. Is this the default behavior of Spark on Mesos today? I think this is what we are looking for because sometimes a job can take up lots of resources and later jobs could not get all the res

Re: Dynamic resource allocation to Spark on Mesos

ling model. > > > On Sat, Jan 28, 2017 at 1:35 AM, Ji Yan wrote: > >> Dear Spark Users, >> >> Currently is there a way to dynamically allocate resources to Spark on >> Mesos? Within Spark we can specify the CPU cores, memory before running >> job. The way

Re: Dynamic resource allocation to Spark on Mesos

Ji Yan wrote: > Dear Spark Users, > > Currently is there a way to dynamically allocate resources to Spark on > Mesos? Within Spark we can specify the CPU cores, memory before running > job. The way I understand is that the Spark job will not run if the CPU/Mem > requirement is not m

Re: Dynamic resource allocation to Spark on Mesos

in Spark on Mesos, but we here at Mesosphere haven't been testing it much, and I'm not sure what the community adoption is.  So I can't yet speak to its robustness, but we will be investing in it soon.  Many users want it.   On Fri, Jan 27, 2017 at 9:35 AM, Ji Yan <ji...@drive.ai>

Re: Dynamic resource allocation to Spark on Mesos

nly allocating as many executors as a job needs, rather than a single static amount set up front. Dynamic Allocation is supported in Spark on Mesos, but we here at Mesosphere haven't been testing it much, and I'm not sure what the community adoption is. So I can't yet speak to its

Dynamic resource allocation to Spark on Mesos

Dear Spark Users, Currently is there a way to dynamically allocate resources to Spark on Mesos? Within Spark we can specify the CPU cores, memory before running job. The way I understand is that the Spark job will not run if the CPU/Mem requirement is not met. This may lead to decrease in overall

Re: launch spark on mesos within a docker container

hrome/55.0.2883.95 > Safari/537.36' > > I1230 14:30:12.473937 9572 master.cpp:5709] Sending 1 offers to framework > 993198d1-7393-4656-9f75-4f22702609d0-0251 (eval.py) at > scheduler-9300fd07-7cf5-4341-84c9-4f1930e8c145@172.16.1.101:40286 > > > > On Fri, Dec 30, 2016 a

Re: launch spark on mesos within a docker container

ment variable on > the executor when it is launched. > > Tim > > > On Dec 30, 2016, at 1:23 PM, Ji Yan wrote: > > Dear Spark Users, > > We are trying to launch Spark on Mesos from within a docker container. We > have found that since the Spark executors need to talk

Re: launch spark on mesos within a docker container

Hi Ji, One way to make it fixed is to set LIBPROCESS_PORT environment variable on the executor when it is launched. Tim > On Dec 30, 2016, at 1:23 PM, Ji Yan wrote: > > Dear Spark Users, > > We are trying to launch Spark on Mesos from within a docker container. We > hav

launch spark on mesos within a docker container

Dear Spark Users, We are trying to launch Spark on Mesos from within a docker container. We have found that since the Spark executors need to talk back at the Spark driver, there is need to do a lot of port mapping to make that happen. We seemed to have mapped the ports on what we could find from

Re: two spark-shells spark on mesos not working

What are the full driver logs? If you enable DEBUG logging, it should give you more information about the rejected offers. This can also happen if offers are being accepted, but tasks immediately die for some reason. You should check the Mesos UI for failed tasks. If they exist, please include

two spark-shells spark on mesos not working

Hi Everyone, There is probably an obvious answer to this, but not sure what it is. :) I am attempting to launch 2..n spark shells using Mesos as the master (this is to support 1..n researchers running pyspark stuff on our data). I can launch two or more spark shells without any problem. But, when

Re: Two questions about running spark on mesos

Hi Guys, > > > Two questions about running spark on mesos. > > 1, Does spark configuration of conf/slaves still work when running spark > on mesos? > > According to my observations, it seemed that conf/slaves still took > effect when running spark-shell. > >

Two questions about running spark on mesos

Hi Guys, Two questions about running spark on mesos. 1, Does spark configuration of conf/slaves still work when running spark on mesos? According to my observations, it seemed that conf/slaves still took effect when running spark-shell. However, it doesn't take effect when depl

Re: spark on mesos memory sizing with offheap

It doesn't look like we are. Can you file a JIRA? A workaround is to set spark.mesos.executor.overhead to be at least spark.memory.offheap.size. This is how the container is sized: https://github.com/apache/spark/blob/master/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSched

spark on mesos memory sizing with offheap

Hi, I am trying to understand how mesos allocate memory when offheap is enabled but it seems that the framework is only taking the heap + 400 MB overhead into consideration for resources allocation. Example: spark.executor.memory=3g spark.memory.offheap.size=1g ==> mesos report 3.4g allocated for t

Re: Spark on mesos in docker not getting parameters

ds) in the drive configuration. > > However, they are missing in subsequent child processes and the final java > process started doesn't contain them either. > > I "fixed" the classpath problem by putting my jar in /opt/spark/jars > (/opt/spark is the location I

Spark on mesos in docker not getting parameters

. I "fixed" the classpath problem by putting my jar in /opt/spark/jars (/opt/spark is the location I have spark installed in the docker container). Can someone tell me what I'm missing? Thanks Jim -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble

spark on mesos cluster - metrics with graphite sink

Hi, I'm launching spark application on mesos cluster. The namespace of the metric includes the framework id for driver metrics, and both framework id and executor id for executor metrics. These ids are obviously assigned by mesos, and they are not permanent - re-registering the application would re

Questions about Spark On Mesos

there is one import thing before making the decision: data locality. If we run spark on mesos, can it achieve good data locality when processing HDFS data? I think spark on yarn can achieve that out of the box, but not sure whether spark on mesos could do that. I've searched through the archive o

Re: Spark on Mesos with Centos 6.6 NFS

Executor.java:617) > at java.lang.Thread.run(Thread.java:745) > > The remoteMachineHost that throws the error has write access to the > specific > folder. > Any thoughts? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3

Spark on Mesos with Centos 6.6 NFS

this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-on-Mesos-with-Centos-6-6-NFS-tp25489.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user

--jars option not working for spark on Mesos in cluster mode

Hi, I am trying to run a spark job on mesos in cluster mode using the following command ./bin/spark-submit --deploy-mode cluster --master mesos://172.17.0.1:7077 —-jars http://172.17.0.2:18630/mesos/extraJars.jar --class MyClass http://172.17.0.2:18630/mesos/foo.jar The application jar “foo.ja

Re: Spark on Mesos / Executor Memory

ortional to the amount of CPU it >>>>>> has, because more CPU means more tasks and more tasks means more memory. >>>>>> Even if we're using coarse mode, the amount of executor memory should be >>>>>> proportionate to the amo

Re: Spark on Mesos / Executor Memory

le ago, and submitted a couple >>>>>> PRs to fix it: >>>>>> >>>>>> https://github.com/apache/spark/pull/2401 >>>>>> https://github.com/apache/spark/pull/3024 >>>>>> >>>>>> Do these look relevant? What version of Spark are you running? >>>>>> >>>>>> On Sat, Apr 11, 2015 at 9:33 AM, Tom Arnfeld wrote: >>>>>> >>>>>>> Hey, >>>>>>> >>>>>>> Not sure whether it's best to ask this on the spark mailing list or >>>>>>> the mesos one, so I'll try here first :-) >>>>>>> >>>>>>> I'm having a bit of trouble with out of memory errors in my spark >>>>>>> jobs... it seems fairly odd to me that memory resources can only be set >>>>>>> at >>>>>>> the executor level, and not also at the task level. For example, as far >>>>>>> as >>>>>>> I can tell there's only a *spark.executor.memory* config option. >>>>>>> >>>>>>> Surely the memory requirements of a single executor are quite >>>>>>> dramatically influenced by the number of concurrent tasks running? >>>>>>> Given a >>>>>>> shared cluster, I have no idea what % of an individual slave my >>>>>>> executor is >>>>>>> going to get, so I basically have to set the executor memory to a value >>>>>>> that's correct when the whole machine is in use... >>>>>>> >>>>>>> Has anyone else running Spark on Mesos come across this, or maybe >>>>>>> someone could correct my understanding of the config options? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> Tom. >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > >

Re: Spark on Mesos / Executor Memory

t 9:33 AM, Tom Arnfeld wrote: >>>>> >>>>>> Hey, >>>>>> >>>>>> Not sure whether it's best to ask this on the spark mailing list or >>>>>> the mesos one, so I'll try here first :-) >>>>>> >>>>>> I'm having a bit of trouble with out of memory errors in my spark >>>>>> jobs... it seems fairly odd to me that memory resources can only be set >>>>>> at >>>>>> the executor level, and not also at the task level. For example, as far >>>>>> as >>>>>> I can tell there's only a *spark.executor.memory* config option. >>>>>> >>>>>> Surely the memory requirements of a single executor are quite >>>>>> dramatically influenced by the number of concurrent tasks running? Given >>>>>> a >>>>>> shared cluster, I have no idea what % of an individual slave my executor >>>>>> is >>>>>> going to get, so I basically have to set the executor memory to a value >>>>>> that's correct when the whole machine is in use... >>>>>> >>>>>> Has anyone else running Spark on Mesos come across this, or maybe >>>>>> someone could correct my understanding of the config options? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Tom. >>>>>> >>>>> >>>>> >>>> >>> >> >

Re: Spark on Mesos / Executor Memory

11, 2015 at 9:33 AM, Tom Arnfeld wrote: >>>> >>>>> Hey, >>>>> >>>>> Not sure whether it's best to ask this on the spark mailing list or >>>>> the mesos one, so I'll try here first :-) >>>>> >>>>

Re: Spark on Mesos / Executor Memory

mple, as far as >>>> I can tell there's only a *spark.executor.memory* config option. >>>> >>>> Surely the memory requirements of a single executor are quite >>>> dramatically influenced by the number of concurrent tasks running? Given a >>>> shared cluster, I have no idea what % of an individual slave my executor is >>>> going to get, so I basically have to set the executor memory to a value >>>> that's correct when the whole machine is in use... >>>> >>>> Has anyone else running Spark on Mesos come across this, or maybe >>>> someone could correct my understanding of the config options? >>>> >>>> Thanks! >>>> >>>> Tom. >>>> >>> >>> >> >

Re: spark on mesos gets killed by cgroups for too much memory

tp://prodmesosfileserver01/spark-dist/1.2.2/spark-dist-1.2.2.tgz >> > >> > We increased the cgroup limit to 6GB and the memory resources from 3000 >> > to >> > 6000 for the startup of mesos and now cgroups doesn't kill the job >> > anymore. >>

spark on mesos gets killed by cgroups for too much memory

t isn't trying to take 3GB, even if when running it's only using 512MB? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-on-mesos-gets-killed-by-cgroups-for-too-much-memory-tp24769.html Sent from the Apache Spark

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

aults.conf from somewhere, >>>>> or >>>>> at least allow the user to inform the dispatcher through spark-submit that >>>>> those properties will be available once the job starts. >>>>> >>>>> Finally, I don't think the d

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

n the wrong >>>> direction. Also let me know if I should open some tickets for these >>>> issues. >>>> >>>> Thanks, >>>> - Alan >>>> >>>> On Fri, Sep 11, 2015 at 1:05 PM, Tim Chen wrote: >>>> >>>&

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

n't think the dispatcher should crash in this event. It >>>>> seems not exceptional that a job is misconfigured when submitted. >>>>> >>>>> Please direct me on the right path if I'm headed in the wrong direction. >>>>> Also l

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

. >>> >>> Thanks, >>> - Alan >>> >>> On Fri, Sep 11, 2015 at 1:05 PM, Tim Chen wrote: >>> >>>> Yes you can create an issue, or actually contribute a patch to update >>>> it :) >>>> >>>> Sorry the

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

t; On Fri, Sep 11, 2015 at 11:11 AM, Tom Waterhouse (tomwater) < >>> tomwa...@cisco.com> wrote: >>> >>>> Tim, >>>> >>>> Thank you for the explanation. You are correct, my Mesos experience is >>>> very light, and I haven’t

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

ia Marathon yet. What you >>> have stated here makes sense, I will look into doing this. >>> >>> Adding this info to the docs would be great. Is the appropriate action >>> to create an issue regarding improvement of the docs? For those of us who >>> ar

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

stated here makes sense, I will look into doing this. >>> >>> Adding this info to the docs would be great. Is the appropriate action to >>> create an issue regarding improvement of the docs? For those of us who are >>> gaining the experience having such a pointer is

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

e correct, my Mesos experience is >>>> very light, and I haven’t deployed anything via Marathon yet. What you >>>> have stated here makes sense, I will look into doing this. >>>> >>>> Adding this info to the docs would be great. Is the appropriate act

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

;> Adding this info to the docs would be great. Is the appropriate action >>> to create an issue regarding improvement of the docs? For those of us who >>> are gaining the experience having such a pointer is very helpful. >>> >>> Tom >>> >>>

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

helpful. >> >> Tom >> >> From: Tim Chen >> Date: Thursday, September 10, 2015 at 10:25 AM >> To: Tom Waterhouse >> Cc: "user@spark.apache.org" >> Subject: Re: Spark on Mesos with Jobs in Cluster Mode Documentation >> >>

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

issue regarding improvement of the docs? For those of us who are > gaining the experience having such a pointer is very helpful. > > Tom > > From: Tim Chen > Date: Thursday, September 10, 2015 at 10:25 AM > To: Tom Waterhouse > Cc: "user@spark.apache.org"

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: Spark on Mesos with Jobs in Cluster Mode Documentation Hi Tom, Sorry the documentation isn't really rich, since it's probably assuming users understands how Mesos and framework w

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

Hi Tom, Sorry the documentation isn't really rich, since it's probably assuming users understands how Mesos and framework works. First I need explain the rationale of why create the dispatcher. If you're not familiar with Mesos yet, each node in your datacenter is installed a Mesos slave where it

Spark on Mesos with Jobs in Cluster Mode Documentation

After spending most of yesterday scouring the Internet for sources of documentation for submitting Spark jobs in cluster mode to a Spark cluster managed by Mesos I was able to do just that, but I am not convinced that how I have things setup is correct. I used the Mesos published

Re: Can not allocate executor when running spark on mesos

;>> http://spark.apache.org/docs/latest/running-on-mesos.html#using-a-mesos-master-url >>> >>> Thanks >>> Best Regards >>> >>> On Tue, Sep 8, 2015 at 12:54 PM, canan chen wrote: >>> >>>> Hi all, >>>> >>>

Re: Can not allocate executor when running spark on mesos

Have you gone through this documentation already? >> http://spark.apache.org/docs/latest/running-on-mesos.html#using-a-mesos-master-url >> >> Thanks >> Best Regards >> >> On Tue, Sep 8, 2015 at 12:54 PM, canan chen wrote: >> >>> Hi all, >>> &

Re: Can not allocate executor when running spark on mesos

pache.org/docs/latest/running-on-mesos.html#using-a-mesos-master-url > > Thanks > Best Regards > > On Tue, Sep 8, 2015 at 12:54 PM, canan chen wrote: > >> Hi all, >> >> I try to run spark on mesos, but it looks like I can not allocate >> resources from mesos. I

Re: Can not allocate executor when running spark on mesos

wrote: > Hi all, > > I try to run spark on mesos, but it looks like I can not allocate > resources from mesos. I am not expert of mesos, but from the mesos log, it > seems spark always decline the offer from mesos. Not sure what's wrong, > maybe need some configuration change.

Can not allocate executor when running spark on mesos

Hi all, I try to run spark on mesos, but it looks like I can not allocate resources from mesos. I am not expert of mesos, but from the mesos log, it seems spark always decline the offer from mesos. Not sure what's wrong, maybe need some configuration change. Here's the mesos master log

spark on mesos with docker from private repository

Hi, My Spark set up is a cluster on top of mesos using docker containers. I want to pull the docker images from a private repository (currently gcr.io ), and I can't get the authentication to work. I know how to generate a .dockercfg file (running on GCE, using gcloud docker -a). My problem is th

Re: Running multiple batch jobs in parallel using Spark on Mesos

. Thanks Best Regards On Mon, Aug 3, 2015 at 2:25 PM, Akash Mishra wrote: > Hello *, > > We are trying to build some Batch jobs using Spark on Mesos. Mesos offer's > two main mode of deployment of Spark job. > > 1. Fine-grained > 2. Coarse-grained > > > When we

Running multiple batch jobs in parallel using Spark on Mesos

Hello *, We are trying to build some Batch jobs using Spark on Mesos. Mesos offer's two main mode of deployment of Spark job. 1. Fine-grained 2. Coarse-grained When we are running the spark jobs in fine grained mode then spark is using max amount of offers from Mesos and running th

Re: Spark on Mesos - Shut down failed while running spark-shell

Hi Haripriya, Your master has registered it's public ip to be 127.0.0.1:5050 which won't be able to be reached by the slave node. If mesos didn't pick up the right ip you can specifiy one yourself via the --ip flag. Tim On Mon, Jul 27, 2015 at 8:32 PM, Haripriya Ayyalasomayajula < aharipriy...@

Spark on Mesos - Shut down failed while running spark-shell

Hi all, I am running Spark 1.4.1 on mesos 0.23.0 While I am able to start spark-shell on the node with mesos-master running, it works fine. But when I try to start spark-shell on mesos-slave nodes, I'm encounter this error. I greatly appreciate any help. 15/07/27 22:14:44 INFO Utils: Successf

RE: Accessing Kerberos Secured HDFS Resources from Spark on Mesos

yone's feedback, thanks everyone. From: Steve Loughran [mailto:ste...@hortonworks.com] Sent: Monday, June 29, 2015 10:32 AM To: Dave Ariens Cc: Tim Chen; Marcelo Vanzin; Olivier Girardot; user@spark.apache.org Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos On 29 Jun

Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos

On 29 Jun 2015, at 14:18, Dave Ariens mailto:dari...@blackberry.com>> wrote: I'd like to toss out another idea that doesn't involve a complete end-to-end Kerberos implementation. Essentially, have the driver authenticate to Kerberos, instantiate a Hadoop file system, and serialize/cache it f

RE: Accessing Kerberos Secured HDFS Resources from Spark on Mesos

e.org Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos On 27 Jun 2015, at 07:56, Tim Chen mailto:t...@mesosphere.io>> wrote: Does YARN provide the token through that env variable you mentioned? Or how does YARN do this? Roughly: 1. client-side launcher create

Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos

setting the KRB5CCNAME env variable when starting the process. You can check the Hadoop sources for details. Not sure if there's another way. From: Marcelo Vanzin Sent: Friday, June 26, 2015 6:20 PM To: Dave Ariens Cc: Tim Chen; Olivier Girardot; user@spark.apache.org<mailto:user@spark.a

Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos

apache.org > *Subject: *Re: Accessing Kerberos Secured HDFS Resources from Spark on > Mesos > > On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens > wrote: > >> Would there be any way to have the task instances in the slaves call >> the UGI login with a principal/keytab pro

Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos

p > sources for details. Not sure if there's another way. > > > >> >> *From: *Marcelo Vanzin >> *Sent: *Friday, June 26, 2015 6:20 PM >> *To: *Dave Ariens >> *Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org >> *Subject: *Re: Accessing

Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos

when starting the process. You can check the Hadoop sources for details. Not sure if there's another way. > > *From: *Marcelo Vanzin > *Sent: *Friday, June 26, 2015 6:20 PM > *To: *Dave Ariens > *Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org > *Subject:

  1   2   >