Hi Arush,
I got the pre-build from https://spark.apache.org/downloads.html. When I
start spark-shell, it prompts:
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
So we don't have pre-build with YARN support? If so, how the spark-submit
work? I checked the YARN log, and job is really submitted and ran
successfully.
Thanks,
David
On Tue Feb 24 2015 at 6:35:38 PM Arush Kharbanda <ar...@sigmoidanalytics.com>
wrote:
> Hi
>
> Are you sure that you built Spark for Yarn.If standalone works, not sure
> if its build for Yarn.
>
> Thanks
> Arush
> On Tue, Feb 24, 2015 at 12:06 PM, Xi Shen <davidshe...@gmail.com> wrote:
>
>> Hi,
>>
>> I followed this guide,
>> http://spark.apache.org/docs/1.2.1/running-on-yarn.html, and tried to
>> start spark-shell with yarn-client
>>
>> ./bin/spark-shell --master yarn-client
>>
>>
>> But I got
>>
>> WARN ReliableDeliverySupervisor: Association with remote system
>> [akka.tcp://sparkYarnAM@10.0.2.15:38171] has failed, address is now gated
>> for [5000] ms. Reason is: [Disassociated].
>>
>> In the spark-shell, and other exceptions in they yarn log. Please see
>> http://stackoverflow.com/questions/28671171/spark-shell-cannot-connect-to-yarn
>> for more detail.
>>
>>
>> However, submitting to the this cluster works. Also, spark-shell as
>> standalone works.
>>
>>
>> My system:
>>
>> - ubuntu amd64
>> - spark 1.2.1
>> - yarn from hadoop 2.6 stable
>>
>>
>> Thanks,
>>
>> [image: --]
>> Xi Shen
>> [image: http://]about.me/davidshen
>> <http://about.me/davidshen?promo=email_sig>
>> <http://about.me/davidshen>
>>
>>
> --
>
> [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com>
>
> *Arush Kharbanda* || Technical Teamlead
>
> ar...@sigmoidanalytics.com || www.sigmoidanalytics.com
>
