Looks like a configuration issue, can you paste your spark-env.sh on the worker?
Thanks Best Regards On Wed, Oct 1, 2014 at 8:27 AM, Tathagata Das <[email protected]> wrote: > It would help to turn on debug level logging in log4j and see the logs. > Just looking at the error logs is not giving me any sense. :( > > TD > > On Tue, Sep 30, 2014 at 4:30 PM, Shaikh Riyaz <[email protected]> > wrote: > >> Hi TD, >> >> Thanks for your reply. >> >> Attachment in previous email was from Master. >> >> Below is the log message from one of the worker. >> >> ----------------------------------------------------------------------------------------------- >> 2014-10-01 01:49:22,348 ERROR akka.remote.EndpointWriter: >> AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> >> [akka.tcp://sparkExecutor@*<worker4>*:34010]: Error [Association failed >> with [akka.tcp://sparkExecutor@*<worker4>*:34010]] [ >> akka.remote.EndpointAssociationException: Association failed with >> [akka.tcp://sparkExecutor@*<worker4>*:34010] >> Caused by: >> akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: >> Connection refused: *<worker4>*:34010 >> ] >> 2014-10-01 02:14:54,868 ERROR akka.remote.EndpointWriter: >> AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> >> [akka.tcp://sparkExecutor@*<worker4>*:33184]: Error [Association failed >> with [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ >> akka.remote.EndpointAssociationException: Association failed with >> [akka.tcp://sparkExecutor@*<worker4>*:33184] >> Caused by: >> akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: >> Connection refused: *<worker4>*:33184 >> ] >> 2014-10-01 02:14:54,878 ERROR akka.remote.EndpointWriter: >> AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> >> [akka.tcp://sparkExecutor@*<worker4>*:33184]: Error [Association failed >> with [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ >> akka.remote.EndpointAssociationException: Association failed with >> [akka.tcp://sparkExecutor@*<worker4>*:33184] >> Caused by: >> akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: >> Connection refused: *<worker4>*:33184 >> ] >> 2014-10-01 02:14:54,887 ERROR akka.remote.EndpointWriter: >> AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> >> [akka.tcp://sparkExecutor@*<worker4>*:33184]: Error [Association failed >> with [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ >> akka.remote.EndpointAssociationException: Association failed with >> [akka.tcp://sparkExecutor@*<worker4>*:33184] >> Caused by: >> akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: >> Connection refused: *<worker4>*:33184 >> ] >> >> ----------------------------------------------------------------------------- >> >> Your support will be highly appreciated. >> >> Regards, >> Riyaz >> >> On Wed, Oct 1, 2014 at 1:16 AM, Tathagata Das < >> [email protected]> wrote: >> >>> Is this the logs of the worker where the failure occurs? I think issues >>> similar to these have since been solved in later versions of Spark. >>> >>> TD >>> >>> On Tue, Sep 30, 2014 at 11:33 AM, Shaikh Riyaz <[email protected]> >>> wrote: >>> >>>> Dear All, >>>> >>>> We are using Spark streaming version 1.0.0 in our Cloudea Hadoop >>>> cluster CDH 5.1.3. >>>> >>>> Spark streaming is reading messages from Kafka using >>>> https://github.com/dibbhatt/kafka-spark-consumer. >>>> >>>> We have allocated 4gb memory to executor and 5gb to each workers. We >>>> have total 6 workers spread across 6 machines. >>>> >>>> Please find the attach log file for detailed error messages. >>>> >>>> >>>> Thanks in advance. >>>> >>>> -- >>>> Regards, >>>> >>>> Riyaz >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>> >>> >> >> >> -- >> Regards, >> >> Riyaz >> >> >
