[
https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121304#comment-13121304
]
jirapos...@reviews.apache.org commented on HIVE-2479:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2167/#review2355
-----------------------------------------------------------
trunk/ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java
<https://reviews.apache.org/r/2167/#comment5419>
We may get NPE, so use code like this:
Counter ctr = null;
if(counters != null) {
ctr= counters.findCounter(...);
if (ctr != null) {
......
}
}
Or put this to a common method.
- Yongqiang
On 2011-10-05 17:58:04, Kevin Wilfong wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2167/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-05 17:58:04)
bq.
bq.
bq. Review request for hive, Ramkumar Vadali and Yongqiang He.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I added the counters mentioned in the task to the MapRedStats class, and
modified HadoopJobExecHelper to collect them.
bq.
bq. I got tired of writing the same code over and over again, so I modified
the way MapRedStats and HadoopJobExecHelper treat task counters. MapRedStats
now has an enum with all of the task counters we want to collect, it is a
subset of the enum in Task$Counter. Task is package private so the enum in it
is unavailable. MapRedStats now contains a map from the enum values to the
values of the counters, if they were set. HadoopJobExecHelper loops over the
enum values and tries to get a value for each counter. As long as the new
getter and setter methods are used the functionality is the same, in particular
for the getter, if a counter was set, it returns the value of the counter,
otherwise it returns -1.
bq.
bq.
bq. This addresses bug Hive-2479.
bq. https://issues.apache.org/jira/browse/Hive-2479
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 1178612
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
1178612
bq.
bq. Diff: https://reviews.apache.org/r/2167/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I ran some queries to verify the counters were being populated.
bq.
bq. I also ran a few of the unit test queries to verify I hadn't broken
anything.
bq.
bq.
bq. Thanks,
bq.
bq. Kevin
bq.
bq.
> Log more Hadoop task counter values in the MapRedStats class.
> -------------------------------------------------------------
>
> Key: HIVE-2479
> URL: https://issues.apache.org/jira/browse/HIVE-2479
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2479.1.patch.txt, HIVE-2479.2.patch.txt,
> HIVE-2479.3.patch.txt
>
>
> We should log more of the Hadoop task tracker counters in the MapRedStats
> class, in order to make them available to hooks and improve logging.
> We should make the counters that are logged configurable, so that if
> different Hadoop versions are used, different counters can be collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
