----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1909/#review1935 -----------------------------------------------------------
Ship it! - Ning On 2011-09-16 17:35:58, Robert Surówka wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/1909/ > ----------------------------------------------------------- > > (Updated 2011-09-16 17:35:58) > > > Review request for hive and Ning Zhang. > > > Summary > ------- > > The purpose of this change is to allow publication or storage of counters > while the job is running. > > Introduced two new variables to hive-default.xml and HiveConf.java: > "hive.client.stats.publishers" and "hive.client.stats.counters". First one > specifies classes names, whose instances will be executed by > HadoopJobExecHelper.java (similarly as hooks are) in its method > progress(ExecDriverTaskHandle): MapRedStats. Second one specifies list of > counters that any client stat publishers should publish or stored. Details > regarding format of this list is up to a specific deployment (it is up to > client stats publishers to parse it), yet it is required to use display names > of counter groups and counters. > > Added interface ClientStatsPublishers in org.apache.hadoop.hive.ql.stats > package, that must be implemented by all stats publishers. > > Added code to progress(ExecDriverTaskHandle): MapRedStats from > HadoopJobExecHelper.java that puts counters' values to a Java map and then > executes registered client stats publishers giving them that map and running > job id. Added two new methods to HadoopJobExecHelper: > extractAllCounterValues(Counters) and getClientStatsPublishers() that are > used by code from previous sentence. > > Made cosmetic changes in one other class > > > This addresses bug HIVE-2446. > https://issues.apache.org/jira/browse/HIVE-2446 > > > Diffs > ----- > > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1171283 > trunk/conf/hive-default.xml 1171283 > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java > 1171283 > trunk/ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistory.java > 1171283 > trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/ClientStatsPublisher.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/1909/diff > > > Testing > ------- > > Run some random tests, and still running the unit tests. > > > Thanks, > > Robert > >
