Re: Spark-submit hangs indefinitely after job completion.

2016-05-24 Thread Pradeep Nayak
BTW, I am using a 6-node cluster with m4.2xlarge machines on amazon. I have tried with both yarn-cluster and spark's native cluster mode as well. On Tue, May 24, 2016 at 12:10 PM Mathieu Longtin wrote: > I have been seeing the same behavior in standalone with a master. > > > On Tue, May 24, 2016

Re: Spark-submit hangs indefinitely after job completion.

2016-05-24 Thread Mathieu Longtin
I have been seeing the same behavior in standalone with a master. On Tue, May 24, 2016 at 3:08 PM Pradeep Nayak wrote: > > > I have posted the same question of Stack Overflow: > http://stackoverflow.com/questions/37421852/spark-submit-continues-to-hang-after-job-completion > > I am trying to tes

Spark-submit hangs indefinitely after job completion.

2016-05-24 Thread Pradeep Nayak
I have posted the same question of Stack Overflow: http://stackoverflow.com/questions/37421852/spark-submit-continues-to-hang-after-job-completion I am trying to test spark 1.6 with hdfs in AWS. I am using the wordcount python example available in the examples folder. I submit the job with spark-s