Thanks Best Regards On Thu, Oct 30, 2014 at 1:43 PM, Akhil Das <ak...@sigmoidanalytics.com> wrote:
> Awesome. > > Thanks > Best Regards > > On Thu, Oct 30, 2014 at 1:30 PM, Naveen Kumar Pokala < > npok...@spcapitaliq.com> wrote: > >> Thanks Akhil, It is working. >> >> >> >> Regards, >> >> Naveen >> >> >> >> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com] >> *Sent:* Thursday, October 30, 2014 1:24 PM >> >> *To:* Naveen Kumar Pokala >> *Subject:* Re: Spark Debugging >> >> >> >> Yes you can, but the values might not give you a complete clue. Give it a >> try >> >> >> Thanks >> >> Best Regards >> >> >> >> On Thu, Oct 30, 2014 at 1:20 PM, Naveen Kumar Pokala < >> npok...@spcapitaliq.com> wrote: >> >> Do you mean can’t I debug line by line? >> >> >> >> Thanks and Regards >> >> Naveen >> >> >> >> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com] >> *Sent:* Thursday, October 30, 2014 1:19 PM >> >> >> *To:* Naveen Kumar Pokala >> *Subject:* Re: Spark Debugging >> >> >> >> Master machine. The execution will get pause in eclipse regardless of >> where its executing the program, you might not get a complete flow, but you >> can see whats happening. >> >> >> Thanks >> >> Best Regards >> >> >> >> On Thu, Oct 30, 2014 at 1:10 PM, Naveen Kumar Pokala < >> npok...@spcapitaliq.com> wrote: >> >> Hi Akhil, >> >> >> >> With below configuration I am successfully able to connect to Node >> A(Master node) where I am running the spark submit program, Spark may give >> driver program to another Node B also. >> >> >> >> Which machine IP should I use to debug the programs from eclipse. >> >> >> >> Thanks, >> >> Naveen >> >> >> >> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com] >> *Sent:* Thursday, October 30, 2014 12:35 PM >> *To:* Naveen Kumar Pokala >> *Subject:* Re: Spark Debugging >> >> >> >> Hi Naveen, >> >> >> >> You should be able to connect to 9999 port also from your windows machine. >> >> >> >> Thanks >> >> Best Regards >> >> >> >> On Thu, Oct 30, 2014 at 12:32 PM, Naveen Kumar Pokala < >> npok...@spcapitaliq.com> wrote: >> >> Hi Akhil, >> >> >> >> I have gone through the article. But I am not starting my java program >> with Java command, I am submitting through spark-submit and I have added >> like below is that ok? >> >> >> >> bin/spark-submit --class sample.spark.test.SparkJob --master yarn-cluster >> --driver-memory 4g --executor-memory --conf >> "spark.executor.extraJavaOptions=-Xdebug >> -agentlib:jdwp=transport=dt_socket,address=9999,server=y,suspend=n" >> /home/npokala/data/test.jar >> >> >> >> I am able to connect to cluster from my windows machine only through >> putty. I am able to access the cluster information and spark information >> from my browser. Is that sufficient.? >> >> >> >> >> >> Regards, >> >> Naveen. >> >> >> >> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com] >> *Sent:* Thursday, October 30, 2014 12:10 PM >> *To:* Naveen Kumar Pokala >> *Cc:* user@spark.apache.org >> *Subject:* Re: Spark Debugging >> >> >> >> Its called remote debugging. You can read this article >> <http://www.eclipse.org/jetty/documentation/current/enable-remote-debugging.html> >> for more information. You will have to make sure that the network between >> your cluster and windows machine can communicate with each other. >> >> >> Thanks >> >> Best Regards >> >> >> >> On Thu, Oct 30, 2014 at 12:06 PM, Naveen Kumar Pokala < >> npok...@spcapitaliq.com> wrote: >> >> Hi, >> >> >> >> I have installed 2 node hadoop cluster (For example, on Unix machines A >> and B. A master node and data node, B is data node) >> >> >> >> I am submitting my driver programs through SPARK 1.1.0 with >> bin/spark-submit from Putty Client from my Windows machine. >> >> >> >> I want to debug my program from Eclipse on my local machine. I am not >> able to find a way to debug. >> >> >> >> Please let me know the ways to debug my driver program as well as >> executor programs >> >> >> >> >> >> Regards, >> >> >> >> Naveen. >> >> >> >> >> >> >> >> >> > >
