Thanks Sanjay, I had a look, but I got some help doing it with eclipse, which worked great.
I've updated the wiki with that process. http://wiki.apache.org/hadoop/Hive/DeveloperGuide#Debugging_Hive_code Russell On Wed, 2010-11-17 at 13:16 +0530, Sanjay Sharma wrote: > Russell, > See if this can help you- > http://indoos.wordpress.com/2010/06/24/hive-remote-debugging/ > > > You would have to tweak the old scripts however for your Hadoop and Hive > versions > > Regards, > Sanjay Sharma > > > -----Original Message----- > From: Russell Melick [mailto:rmel...@hmc.edu] > Sent: Wednesday, November 17, 2010 1:04 PM > To: hive-...@hadoop.apache.org > Cc: j...@hmc.edu; mw...@hmc.edu > Subject: Debugging Hive > > Hi all, > > Apologies for the long email. I'm attempting to connect to the hive > code with a debugger to understand the optimizer. In this example, I > would like to pause at the beginning of the createIndex function in > DDLTask. After following the directions on the wiki, I'm stumped as to > why connecting remotely with jdb is not working. Any help is > appreciated. I've detailed my process below. > > Thanks, > Russell Melick > > > > First, I check out a fresh copy of trunk: > > > svn checkout http://svn.apache.org/repos/asf/hive/trunk hive > > Then, I compile it for debugging, as I found at > http://wiki.apache.org/hadoop/Hive/DeveloperGuide#Debugging_Hive_code: > > > cd hive > > ant -Djavac.debug=on package > > Here are the important variables before I run hive: > > echo $HIVE_DEBUG_PORT > > 8000 > > > echo $HIVE_DEBUG > > -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y > > > echo $HADOOP_OPTS > > -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y > > I then try to run one of the unit tests: > > > ant test -Dtestcase=TestCliDriver -Dqfile=index_compact_1.q > > The test then waits for a debugger to attach: > > [junit] Begin query: index_compact_1.q > > [junit] Listening for transport dt_socket at address: 8000 > > However, when I try to attach with jdb, it doesn't work. > > > jdb -attach 8000 > > Set uncaught java.lang.Throwable > > Set deferred uncaught java.lang.Throwable > > Initializing jdb ... > > > > > VM Started: No frames on the current call stack > > > > main[1] stop at DDLTask:317 > > Deferring breakpoint DDLTask:317. > > It will be set after the class is loaded. > > main[1] resume > > All threads resumed. > > > > > The application exited > > The unit test does not seem to recognize that it received the connection. > After jdb prints "application exited", the test says it is listening again, > but it didn't confirm the connection. > > > [junit] Begin query: index_compact_1.q > > [junit] Listening for transport dt_socket at address: 8000 > > [junit] Deleted > > file:/home/rmelick/hive/build/ql/test/data/warehouse/default__src_src_index__ > > [junit] Listening for transport dt_socket at address: 8000 > > [junit] Listening for transport dt_socket at address: 8000 > > I have also attempted to start hiveCli, but I have had no success getting the > CLI to start after attaching the debugger. > > This is the only output I get from the cli: > > build/dist/bin/hive > > Unable to determine Hadoop version information. > > 'hadoop version' returned: > > jdb doesn't seem to be connecting correctly > > jdb -attach 8000 > > Set uncaught java.lang.Throwable > > Set deferred uncaught java.lang.Throwable > > Initializing jdb ... > > > > > VM Started: No frames on the current call stack > > > > main[1] stop at DDLTask:317 > > Deferring breakpoint DDLTask:317. > > It will be set after the class is loaded. > > main[1] resume > > All threads resumed. > > > > > The application exited > > I've also tried connecting with Eclipse, after generating the eclipse files, > importing the project, and creating the MapRedTask configuration. It either > seems to similarly connect without the client recognizing it, or throws a > connection error. > > Again, thanks for any help. > > > > Impetus is a proud sponsor for ASCI Tour 2010 (Agile Software Community of > India) on Oct 30 in Noida, India. > > Meet Impetus at the Cloud Computing Expo from Nov 1-4 in Santa Clara. Our Sr. > Director of Engineering, Vineet Tyagi will be speaking about ‘Using Hadoop > for Deriving Intelligence from Large Data’. > > Click http://www.impetus.com/ to know more. Follow us on > www.twitter.com/impetuscalling > > NOTE: This message may contain information that is confidential, proprietary, > privileged or otherwise protected by law. The message is intended solely for > the named addressee. If received in error, please destroy and notify the > sender. Any use of this email is prohibited when received in error. Impetus > does not represent, warrant and/or guarantee, that the integrity of this > communication has been maintained nor that the communication is free of > errors, virus, interception or interference.
