----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/777/#review711 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java <https://reviews.apache.org/r/777/#comment1420> error code -101 is also used in TaskRunner.java to indicate OOM exception. We should define all these error code in a centralized place. ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java <https://reviews.apache.org/r/777/#comment1421> We should use interface in the declaration (Set<String>). ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java <https://reviews.apache.org/r/777/#comment1422> Also the return type should use interface rather than class implementation. ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java <https://reviews.apache.org/r/777/#comment1423> Print out error message related to the exception so that user know what goes wrong? ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java <https://reviews.apache.org/r/777/#comment1424> Some task log are very big. retrieving all may take a long time. It may be sufficient to just retrieve the last 8KB. ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java <https://reviews.apache.org/r/777/#comment1425> Do you have some numbers on how long it takes to get all the TaskCompletionEvents? There are cases that a job may have more than 10k tasks and all of them failed with the same error. If it takes too long you may want to consider adding a threshold to the time spent in getting all the TaskCompleteEvents. - Ning On 2011-05-24 04:29:32, Syed Albiz wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/777/ > ----------------------------------------------------------- > > (Updated 2011-05-24 04:29:32) > > > Review request for hive and John Sichi. > > > Summary > ------- > > - Add local error messages to point to job logs and provide TaskIDs > - Add a timeout to the fetching of task logs and errors > > > This addresses bug HIVE-2156. > https://issues.apache.org/jira/browse/HIVE-2156 > > > Diffs > ----- > > build-common.xml 00c3680 > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java dc96a1f > conf/hive-default.xml 159d825 > ql/build.xml 449b47a > ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 4717c25 > ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java 53769a0 > ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 691f038 > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 9cb407c > ql/src/test/queries/clientnegative/minimr_broken_pipe.q PRE-CREATION > ql/src/test/results/clientnegative/dyn_part3.q.out 5f4df65 > ql/src/test/results/clientnegative/minimr_broken_pipe.q.out PRE-CREATION > ql/src/test/results/clientnegative/script_broken_pipe1.q.out d33d2cc > ql/src/test/results/clientnegative/script_broken_pipe2.q.out afbaa44 > ql/src/test/results/clientnegative/script_broken_pipe3.q.out fe8f757 > ql/src/test/results/clientnegative/script_error.q.out c72d780 > ql/src/test/results/clientnegative/udf_reflect_neg.q.out f2082a3 > ql/src/test/results/clientnegative/udf_test_error.q.out 5fd9a00 > ql/src/test/results/clientnegative/udf_test_error_reduce.q.out ddc5e5b > ql/src/test/templates/TestNegativeCliDriver.vm ec13f79 > > Diff: https://reviews.apache.org/r/777/diff > > > Testing > ------- > > Tested TestNegativeCliDriver in both local and miniMR mode > > > Thanks, > > Syed > >
