----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/664/ -----------------------------------------------------------
(Updated 2011-04-25 23:42:52.373740) Review request for hive. Changes ------- This patch has the following changes: 1) changed Utilities.executeWithRetry() to take a SQLCommand helper class and a PreparedStatement to allow customized functions to be included in the retry logic. 2) defined Utilities.connectWithRetry() and Utilities.prepareWithRetry() with the similar logic as executeWithRetry(). 3) The major change in the retry logic is that the Utilities.*withRetry() functions handles a subclass of SQLException: SQLTransientException. These are the exceptions that can be resolved when simply retrying after some time. Other types of SQLExceptions (SQLNonTransientException and SQLRecoverableException are thrown to the callers to handle). 4) the caller (JDBCStatsAggregator/Publisher) handles SQLRecoverableException in the similar fashion as Utilities.*withRetry(), but it will close the SQLConnection first and wait some time before reopen the connection and retry the SQL statements. Summary ------- The major changes are: 0) 2 parameters are introduced: hive.stats.retries.max (default 0) to be the maximum # of retries on SQLException failures, and hive.stats.retries.wait (default 3 sec) to be the base time window (explained below) to wait before the next retry. 1) introduced a couple of Utilities function to execute SQL queries with retries on failures. One Utilities function is to determine the wait time based on the number of failures and a base wait window (same as the one introduced in HDFS-767 for DFSClient to retry on BlockMissingExceptions). The actual wait time is determined by baseWindow * failues + baseWindow * (failure + 1) * (random number between [0.0,1.0]). 2) changed the JDBCStatsAggregator.java to use PreparedStatement to be able to use executeWithRetries(). 3) change the JDBCStatsPublisher.java and JDBCStasAggregator.java to use retries on SQL connections and SQL executions. Diffs (updated) ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1096632 trunk/conf/hive-default.xml 1096632 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1096632 trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/JDBCStatsAggregator.java 1096632 trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/JDBCStatsPublisher.java 1096632 Diff: https://reviews.apache.org/r/664/diff Testing ------- Running unit tests. Thanks, Ning
