[
https://issues.apache.org/jira/browse/HIVE-2347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079549#comment-13079549
]
[email protected] commented on HIVE-2347:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1296/
-----------------------------------------------------------
Review request for hive and Ning Zhang.
Summary
-------
I added a field for the Hadoop Job ID to the Task class. This will make it
accessible to the Driver and hence to the hooks for logging/debugging purposes.
By including it in the Task, we only need to check that the type of the task
is MAPRED, before getting the job ID.
I considered adding it to several places:
as separate fields in ExecDriver and BlockMergeTask: this would require
duplicating code, require conditions to determine the type of a task and
casting to either ExecDriver or BlockMergeTask in order to get the JobID from
them
in the MapRedWork: this would require modifying a field in MapRedWork in the
execute function, and I could not find a precedent for this
This addresses bug HIVE-2347.
https://issues.apache.org/jira/browse/HIVE-2347
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1153966
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1153966
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java
1153966
Diff: https://reviews.apache.org/r/1296/diff
Testing
-------
Ran the TestCliDriver and TestNegativeCliDriver test suites and verified they
passed.
Also, created a sample post exec hook which simply logged the JobID for every
map reduce task, and verified it.
Thanks,
Kevin
> Make Hadoop Job ID available after task finishes executing
> ----------------------------------------------------------
>
> Key: HIVE-2347
> URL: https://issues.apache.org/jira/browse/HIVE-2347
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
>
> After Map Reduce tasks finish the execute method (ExecDriver and
> BlockMergeTask) the Hadoop Job ID is inaccessible to the Driver, and hence
> the hooks it runs. Expose this information could help to improve logging,
> debugging, etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
