In that case, fire-up a sparkshell and try the following:
scala>import org.apache.spark.streaming.{Seconds, StreamingContext}
> scala>import org.apache.spark.streaming.StreamingContext._
> scala>val ssc = new
> StreamingContext("spark://YOUR-SPARK-MASTER-URI","Streaming
> Job",Seconds(5),"/home/ak
Hi Akhil,
Thanks for your reply.
We are using CDH 5.1.3 and spark configuration is taken care by Cloudera
configuration. Please let me know if you would like to review the
configuration.
Regards,
Riyaz
On Wed, Oct 1, 2014 at 10:10 AM, Akhil Das
wrote:
> Looks like a configuration issue, can y
Looks like a configuration issue, can you paste your spark-env.sh on the
worker?
Thanks
Best Regards
On Wed, Oct 1, 2014 at 8:27 AM, Tathagata Das
wrote:
> It would help to turn on debug level logging in log4j and see the logs.
> Just looking at the error logs is not giving me any sense. :(
>
>
It would help to turn on debug level logging in log4j and see the logs.
Just looking at the error logs is not giving me any sense. :(
TD
On Tue, Sep 30, 2014 at 4:30 PM, Shaikh Riyaz wrote:
> Hi TD,
>
> Thanks for your reply.
>
> Attachment in previous email was from Master.
>
> Below is the lo
Hi TD,
Thanks for your reply.
Attachment in previous email was from Master.
Below is the log message from one of the worker.
---
2014-10-01 01:49:22,348 ERROR akka.remote.EndpointWriter: AssociationError
Is this the logs of the worker where the failure occurs? I think issues
similar to these have since been solved in later versions of Spark.
TD
On Tue, Sep 30, 2014 at 11:33 AM, Shaikh Riyaz wrote:
> Dear All,
>
> We are using Spark streaming version 1.0.0 in our Cloudea Hadoop cluster
> CDH 5.1
