I am not sure what you mean by some portion of source code of Hadoop. Because you are running on a fully distributed environment you need to make sure that eclipse is attached to the correct process. If you are trying to debug the data upload pipeline then you need to be sure that eclipse is attached to the proper data node. It is hard to guess which one because you setup has 10 to choose from. Your image showed you attached to the client, you have to use remote debugging to debug a different process.
The next thing you are likely to run into is timeout issues. Almost all of the code has a lot of timeouts, to be able to detect when another process has stopped responding. This is a bigger problem in MR where you will be walking through code and suddenly the process disappears because it was shot be the Task Tracker or Node Manager. I am not as familiar with the HDFS timeouts, but you probably want to be aware that they exist. --Bobby On 10/7/13 12:38 AM, "Yoonmin Nam" <rony...@dgist.ac.kr> wrote: >Sorry for this inconvenience. >I uploaded image file on our lab server. > >http://infolab.dgist.ac.kr/~ronymin/pictures/debug.png > >-----Original Message----- >From: haosdent [mailto:haosd...@gmail.com] >Sent: Monday, October 07, 2013 2:24 PM >To: hdfs-dev@hadoop.apache.org; 남윤민 >Subject: Re: About debugging HDFS and MapReduce... > >I still couldn't see the image. The page shows that "The page you >requested could not be found.". > >-- >Best Regards, >Haosong Huang >Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > >On Monday, October 7, 2013 at 12:38 PM, 남윤민 wrote: > >> Here is image I uploaded. >> >> Sorry for that! >> >> http://mail.dgist.ac.kr/design/common/image/attaches//ronymin_167_inli >> ne0_6806441608437847.png >> > --- Original Message --- >> > From : "Ted Yu"<yuzhih...@gmail.com (mailto:yuzhih...@gmail.com)> To >> > : hdfs-dev@hadoop.apache.org (mailto:hdfs-dev@hadoop.apache.org) >> > Cc : hdfs-dev@hadoop.apache.org (mailto:hdfs-dev@hadoop.apache.org) >> > Date : 2013/10/07 월요일 오후 1:22:45 >> > Subject : Re: About debugging HDFS and MapReduce... >> > >> > Your image didn't go through. >> > You may want to upload it to image sharing site. >> > >> > What version of Hadoop are you using ? >> > >> > Thanks >> > >> > On Oct 6, 2013, at 8:56 PM, 남윤민 <rony...@dgist.ac.kr >>(mailto:rony...@dgist.ac.kr)> wrote: >> > > Hello!! >> > > My name is Yoonmin Nam from South Korea. >> > > >> > > Currently, I am trying to debug namenode and datanode using >> > > eclipse and fully-distributed environemnt in my lab. (1 master and >> > > 10 nodes) >> > > >> > > However, I can do debug but not fully, just some portion of source >>code of hadoop. >> > > >> > > This means I cannot see how some daemon thread such as DataStreamer. >> > > Also, I want to see how NameNode thread and DataNode thread work >>but I can't. >> > > >> > > As you can see, there is Daemon thread (Thre ad-2) and it might be >>DataStreamer thread but.. I can't see. >> > > >> > > >> > > >> > > >> > > So, I think everyone in the hdfs-dev have very professional >>experience in developing HDFS and MapReduce and I want to ask you about >>how can I see all real daemons are working for academic reason. >> > > >> > > Even you have no idea about it, may I ask you that how you test >>your implementation of HDFS or MapReduce? >> > > >> > > >> > > Thanks!! >> > > >> > > >> > > >> > > >> > > >> > > // Yoonmin Nam >> > > >> >> >> // Yoonmin Nam >> >> > > > >
