I'm using org.apache.spark.deploy.yarn.Client object to run my spark job. I
guess this is what spark-submit wraps really.
- Amey
On Mon, Nov 3, 2014 at 5:25 PM, Tobias Pfeiffer wrote:
> Hi,
>
> On Mon, Nov 3, 2014 at 1:29 PM, Amey Chaugule wrote:
>
>> I thought that only applied when you're tr
Hi,
On Mon, Nov 3, 2014 at 1:29 PM, Amey Chaugule wrote:
> I thought that only applied when you're trying to run a job using
> spark-submit or in the shell...
>
And how are you starting your Yarn job, if not via spark-submit?
Tobias
I thought that only applied when you're trying to run a job using
spark-submit or in the shell...
On Sun, Nov 2, 2014 at 8:47 AM, Akhil Das
wrote:
> You can set HADOOP_CONF_DIR inside the spark-env.sh file
>
> Thanks
> Best Regards
>
> On Sat, Nov 1, 2014 at 4:14 AM, ameyc wrote:
>
>> How do i
You can set HADOOP_CONF_DIR inside the spark-env.sh file
Thanks
Best Regards
On Sat, Nov 1, 2014 at 4:14 AM, ameyc wrote:
> How do i setup hadoop_conf_dir correctly when I'm running my spark job on
> Yarn? My Yarn environment has the correct hadoop_conf_dir settings by the
> configuration that
How do i setup hadoop_conf_dir correctly when I'm running my spark job on
Yarn? My Yarn environment has the correct hadoop_conf_dir settings by the
configuration that I pull from sc.hadoopConfiguration() is incorrect.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nab
