----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24627/#review52632 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractFileMergeOperator.java <https://reviews.apache.org/r/24627/#comment91490> inputDepth or tmpPath? ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractFileMergeOperator.java <https://reviews.apache.org/r/24627/#comment91489> inputDepth perhaps? ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractFileMergeOperator.java <https://reviews.apache.org/r/24627/#comment91491> nit updated. ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractFileMergeOperator.java <https://reviews.apache.org/r/24627/#comment91502> nit: has slight difference w.r.t ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java <https://reviews.apache.org/r/24627/#comment91505> Usually done in the reverse way. The java imports come before the hadoop/hive imports. ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java <https://reviews.apache.org/r/24627/#comment91507> Don't really need it here. Can be moved down to line 582. ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java <https://reviews.apache.org/r/24627/#comment91506> HashMap only on the LHS. Don't use LinkedHashMap. ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java <https://reviews.apache.org/r/24627/#comment91508> Nice work! Although this section usually comes above the hadoop imports though. ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java <https://reviews.apache.org/r/24627/#comment91509> nit: comma after merge makes it easier to read. ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java <https://reviews.apache.org/r/24627/#comment91515> Nice work! ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeFileOutputFormat.java <https://reviews.apache.org/r/24627/#comment91517> What purpose does this class serve? Doesn't look like it does anything. Comments please. - Vikram Dixit Kumaraswamy On Sept. 6, 2014, 2:03 a.m., Prasanth_J wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/24627/ > ----------------------------------------------------------- > > (Updated Sept. 6, 2014, 2:03 a.m.) > > > Review request for hive and Gunther Hagleitner. > > > Bugs: HIVE-7704 > https://issues.apache.org/jira/browse/HIVE-7704 > > > Repository: hive-git > > > Description > ------- > > Currently tez falls back to MR task for merge file task. It will beneficial > to convert the merge file tasks to tez task to make use of the performance > gains from tez. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 54e2b18 > itests/src/test/resources/testconfiguration.properties 99049ca > > ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java > 6f23575 > ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractFileMergeOperator.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e076683 > ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 7477199 > ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 8946221 > ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/RCFileMergeOperator.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java 3d74459 > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 2d9b9c3 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 4ff568d1 > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/MergeFileRecordProcessor.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/MergeFileTezProcessor.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/RecordProcessor.java 994721f > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezProcessor.java 831e6a5 > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeFileInputFormat.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeFileMapper.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeFileOutputFormat.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeFileTask.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeFileWork.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeInputFormat.java > 4651920 > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeMapper.java 6c691b1 > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeOutputFormat.java > a3ce699 > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeTask.java c30476b > ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeWork.java 9efee3c > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileMergeMapper.java > 13ec642 > > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileStripeMergeInputFormat.java > a6c92fb > ql/src/java/org/apache/hadoop/hive/ql/io/orc/Writer.java c391b0e > ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java 195d60e > > ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileBlockMergeInputFormat.java > 6809c79 > > ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileMergeMapper.java > dee6b1c > ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 7129ed8 > ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 11a9419 > ql/src/java/org/apache/hadoop/hive/ql/plan/FileMergeDesc.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/OrcFileMergeDesc.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/RCFileMergeDesc.java > PRE-CREATION > ql/src/test/queries/clientpositive/list_bucket_dml_8.q 9e81b8d > ql/src/test/queries/clientpositive/orc_merge1.q ee65b98 > ql/src/test/queries/clientpositive/orc_merge5.q PRE-CREATION > ql/src/test/queries/clientpositive/orc_merge6.q PRE-CREATION > ql/src/test/queries/clientpositive/orc_merge7.q PRE-CREATION > ql/src/test/results/clientpositive/infer_bucket_sort_dyn_part.q.out ea37c36 > ql/src/test/results/clientpositive/list_bucket_dml_10.q.out e9367ac > ql/src/test/results/clientpositive/list_bucket_dml_4.q.out 99496d5 > ql/src/test/results/clientpositive/list_bucket_dml_6.q.out d5deadb > ql/src/test/results/clientpositive/list_bucket_dml_7.q.out 4aea4db > ql/src/test/results/clientpositive/list_bucket_dml_9.q.out f94a3cc > ql/src/test/results/clientpositive/merge_dynamic_partition4.q.out 0899648 > ql/src/test/results/clientpositive/merge_dynamic_partition5.q.out 0653469 > ql/src/test/results/clientpositive/orc_createas1.q.out b0c58dd > ql/src/test/results/clientpositive/orc_merge1.q.out fc3e206 > ql/src/test/results/clientpositive/orc_merge3.q.out 258f538 > ql/src/test/results/clientpositive/orc_merge5.q.out PRE-CREATION > ql/src/test/results/clientpositive/orc_merge6.q.out PRE-CREATION > ql/src/test/results/clientpositive/orc_merge7.q.out PRE-CREATION > ql/src/test/results/clientpositive/rcfile_createas1.q.out cdfa036 > ql/src/test/results/clientpositive/rcfile_merge1.q.out ac6a2bd > ql/src/test/results/clientpositive/rcfile_merge2.q.out d8a61f3 > ql/src/test/results/clientpositive/rcfile_merge3.q.out 5c717b5 > ql/src/test/results/clientpositive/tez/orc_merge1.q.out f79f878 > ql/src/test/results/clientpositive/tez/orc_merge5.q.out PRE-CREATION > ql/src/test/results/clientpositive/tez/orc_merge6.q.out PRE-CREATION > ql/src/test/results/clientpositive/tez/orc_merge7.q.out PRE-CREATION > ql/src/test/results/clientpositive/tez/orc_merge_incompat1.q.out > PRE-CREATION > ql/src/test/results/clientpositive/tez/orc_merge_incompat2.q.out > PRE-CREATION > ql/src/test/results/clientpositive/union_remove_10.q.out 74b9e68 > ql/src/test/results/clientpositive/union_remove_11.q.out e0bd498 > ql/src/test/results/clientpositive/union_remove_12.q.out 033475f > ql/src/test/results/clientpositive/union_remove_13.q.out 3c7f8fa > ql/src/test/results/clientpositive/union_remove_14.q.out 315c958 > ql/src/test/results/clientpositive/union_remove_16.q.out 721caa0 > ql/src/test/results/clientpositive/union_remove_9.q.out 5a9c5a2 > > Diff: https://reviews.apache.org/r/24627/diff/ > > > Testing > ------- > > > Thanks, > > Prasanth_J > >
