It would help to turn on debug level logging in log4j and see the logs. Just looking at the error logs is not giving me any sense. :(
TD On Tue, Sep 30, 2014 at 4:30 PM, Shaikh Riyaz <shaikh....@gmail.com> wrote: > Hi TD, > > Thanks for your reply. > > Attachment in previous email was from Master. > > Below is the log message from one of the worker. > > ----------------------------------------------------------------------------------------------- > 2014-10-01 01:49:22,348 ERROR akka.remote.EndpointWriter: AssociationError > [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ > *<worker4>*:34010]: Error [Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:34010]] [ > akka.remote.EndpointAssociationException: Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:34010] > Caused by: > akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: > Connection refused: *<worker4>*:34010 > ] > 2014-10-01 02:14:54,868 ERROR akka.remote.EndpointWriter: AssociationError > [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ > *<worker4>*:33184]: Error [Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ > akka.remote.EndpointAssociationException: Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:33184] > Caused by: > akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: > Connection refused: *<worker4>*:33184 > ] > 2014-10-01 02:14:54,878 ERROR akka.remote.EndpointWriter: AssociationError > [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ > *<worker4>*:33184]: Error [Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ > akka.remote.EndpointAssociationException: Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:33184] > Caused by: > akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: > Connection refused: *<worker4>*:33184 > ] > 2014-10-01 02:14:54,887 ERROR akka.remote.EndpointWriter: AssociationError > [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ > *<worker4>*:33184]: Error [Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ > akka.remote.EndpointAssociationException: Association failed with > [akka.tcp://sparkExecutor@*<worker4>*:33184] > Caused by: > akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: > Connection refused: *<worker4>*:33184 > ] > > ----------------------------------------------------------------------------- > > Your support will be highly appreciated. > > Regards, > Riyaz > > On Wed, Oct 1, 2014 at 1:16 AM, Tathagata Das <tathagata.das1...@gmail.com > > wrote: > >> Is this the logs of the worker where the failure occurs? I think issues >> similar to these have since been solved in later versions of Spark. >> >> TD >> >> On Tue, Sep 30, 2014 at 11:33 AM, Shaikh Riyaz <shaikh....@gmail.com> >> wrote: >> >>> Dear All, >>> >>> We are using Spark streaming version 1.0.0 in our Cloudea Hadoop cluster >>> CDH 5.1.3. >>> >>> Spark streaming is reading messages from Kafka using >>> https://github.com/dibbhatt/kafka-spark-consumer. >>> >>> We have allocated 4gb memory to executor and 5gb to each workers. We >>> have total 6 workers spread across 6 machines. >>> >>> Please find the attach log file for detailed error messages. >>> >>> >>> Thanks in advance. >>> >>> -- >>> Regards, >>> >>> Riyaz >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >> >> > > > -- > Regards, > > Riyaz > >
