Hi all,
I have been interested in getting the debug information and display it
to clients.
Specifically, when you submit a job to hive cli (on terminal say you
write "select * from somewhere ...) or through a thrift server, you will
see they hive displays some debug information on the screen (See below)
How I can I get this "live" information to the client while the
job is running so the client can see the stages of the job and how it
is going, see if the job gets killed, or fails... I am using thrift
server, and see this information on the thrift server's terminal and
wonder if I can transfer this info to thrift client. Or if it's
impossible to transfer this info to thrift client, how would one
direct this debug info somewhere to local file or mysql then the
information can be read from there?
Any ideas?
Thanks
------------------------------------------------------
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
---> getPaths = XXXXXXXXXXXX
Starting Job = XXXXXXXXXXXXXX, Tracking URL = XXXXXXXXXXXXXXXXXX
Kill Command = XXXXXXXXXXXXXXXXXXXX
Hadoop job information for Stage-1: number of mappers: 505; number of
reducers: 0
2012-07-04 01:52:42,145 Stage-1 map = 0%, reduce = 0%
2012-07-04 01:52:48,185 Stage-1 map = 5%, reduce = 0%, Cumulative CPU 42.92 sec
2012-07-04 01:52:49,196 Stage-1 map = 26%, reduce = 0%, Cumulative
CPU 213.03 sec
