To: Pat Ferrel
>> Cc: Felix Cheung ,
>> Marcelo Vanzin , user
>>
>> Subject: Re: spark.submit.deployMode: cluster
>>
>> Check out the Spark Jobs API... it sits behind a REST service...
>>
>>
>> On Thu, Mar 28, 2019 at 12:29 Pat Ferrel wrote:
Nerothin
> Reply: Jason Nerothin
> Date: March 28, 2019 at 2:53:05 PM
> To: Pat Ferrel
> Cc: Felix Cheung ,
> Marcelo
> Vanzin , user
>
> Subject: Re: spark.submit.deployMode: cluster
>
> Check out the Spark Jobs API... it sits behind a REST service...
>
nches trivially in test apps since most uses are as a lib.
>
>
> From: Felix Cheung
> Reply: Felix Cheung
>
> Date: March 28, 2019 at 9:42:31 AM
> To: Pat Ferrel , Marcelo
> Vanzin
> Cc: user
> Subject: Re: spark.submit.deployMode: cluster
>
> If an
ince most uses are as a lib.
>
>
> From: Felix Cheung
> Reply: Felix Cheung
>
> Date: March 28, 2019 at 9:42:31 AM
> To: Pat Ferrel , Marcelo
> Vanzin
> Cc: user
> Subject: Re: spark.submit.deployMode: cluster
>
> If anyone wants to improve docs please c
: user
Subject: Re: spark.submit.deployMode: cluster
If anyone wants to improve docs please create a PR.
lol
But seriously you might want to explore other projects that manage job
submission on top of spark instead of rolling your own with spark-submit.
--
*From
: Marcelo Vanzin
Cc: user
Subject: Re: spark.submit.deployMode: cluster
Ahh, thank you indeed!
It would have saved us a lot of time if this had been documented. I know, OSS
so contributions are welcome… I can also imagine your next comment; “If anyone
wants to improve docs see the Apache
Reply: Marcelo Vanzin
Date: March 26, 2019 at 1:59:36 PM
To: Pat Ferrel
Cc: user
Subject: Re: spark.submit.deployMode: cluster
If you're not using spark-submit, then that option does nothing.
If by "context creation API" you mean "new SparkContext()" or a
If you're not using spark-submit, then that option does nothing.
If by "context creation API" you mean "new SparkContext()" or an
equivalent, then you're explicitly creating the driver inside your
application.
On Tue, Mar 26, 2019 at 1:56 PM Pat Ferrel wrote:
>
> I have a server that starts a Sp
