Hi all,
Apologies for the long email. I'm attempting to connect to the hive
code with a debugger to understand the optimizer. In this example, I
would like to pause at the beginning of the createIndex function in
DDLTask. After following the directions on the wiki, I'm stumped as to
why connecting remotely with jdb is not working. Any help is
appreciated. I've detailed my process below.
Thanks,
Russell Melick
First, I check out a fresh copy of trunk:
> svn checkout http://svn.apache.org/repos/asf/hive/trunk hive
Then, I compile it for debugging, as I found at
http://wiki.apache.org/hadoop/Hive/DeveloperGuide#Debugging_Hive_code:
> cd hive
> ant -Djavac.debug=on package
Here are the important variables before I run hive:
> echo $HIVE_DEBUG_PORT
> 8000
> echo $HIVE_DEBUG
> -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y
> echo $HADOOP_OPTS
> -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y
I then try to run one of the unit tests:
> ant test -Dtestcase=TestCliDriver -Dqfile=index_compact_1.q
The test then waits for a debugger to attach:
> [junit] Begin query: index_compact_1.q
> [junit] Listening for transport dt_socket at address: 8000
However, when I try to attach with jdb, it doesn't work.
> jdb -attach 8000
> Set uncaught java.lang.Throwable
> Set deferred uncaught java.lang.Throwable
> Initializing jdb ...
> >
> VM Started: No frames on the current call stack
>
> main[1] stop at DDLTask:317
> Deferring breakpoint DDLTask:317.
> It will be set after the class is loaded.
> main[1] resume
> All threads resumed.
> >
> The application exited
The unit test does not seem to recognize that it received the connection.
After jdb prints "application exited", the test says it is listening again, but
it didn't confirm the connection.
> [junit] Begin query: index_compact_1.q
> [junit] Listening for transport dt_socket at address: 8000
> [junit] Deleted
> file:/home/rmelick/hive/build/ql/test/data/warehouse/default__src_src_index__
> [junit] Listening for transport dt_socket at address: 8000
> [junit] Listening for transport dt_socket at address: 8000
I have also attempted to start hiveCli, but I have had no success getting the
CLI to start after attaching the debugger.
This is the only output I get from the cli:
> build/dist/bin/hive
> Unable to determine Hadoop version information.
> 'hadoop version' returned:
jdb doesn't seem to be connecting correctly
> jdb -attach 8000
> Set uncaught java.lang.Throwable
> Set deferred uncaught java.lang.Throwable
> Initializing jdb ...
> >
> VM Started: No frames on the current call stack
>
> main[1] stop at DDLTask:317
> Deferring breakpoint DDLTask:317.
> It will be set after the class is loaded.
> main[1] resume
> All threads resumed.
> >
> The application exited
I've also tried connecting with Eclipse, after generating the eclipse files,
importing the project, and creating the MapRedTask configuration. It either
seems to similarly connect without the client recognizing it, or throws a
connection error.
Again, thanks for any help.
