----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49766/ -----------------------------------------------------------
(Updated July 20, 2016, 3:55 p.m.) Review request for hive and Eugene Koifman. Changes ------- Updated the patch by rebasing with master. No additional code changes. Same as Patch #10 at https://issues.apache.org/jira/browse/HIVE-14035 Repository: hive-git Description ------- https://issues.apache.org/jira/browse/HIVE-14035 In current Hive version, delta files created by ACID transactions do not allow predicate pushdown if they contain any update/delete events. This is done to preserve correctness when following a multi-version approach during event collapsing, where an update event overwrites an existing insert event. This JIRA proposes to split an update event into a combination of a delete event followed by a new insert event, that can enable predicate push down to all delta files without breaking correctness. To support backward compatibility for this feature, this JIRA also proposes to add some sort of versioning to ACID that can allow different versions of ACID transactions to co-exist together. Diffs (updated) ----- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 66203a5 hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java 14f7316 hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java 974c6b8 metastore/if/hive_metastore.thrift 4d92b73 metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h ae14bd1 metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp f982bf2 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java 5a666f2 metastore/src/gen/thrift/gen-php/metastore/Types.php f505208 metastore/src/gen/thrift/gen-py/hive_metastore/constants.py d1c07a5 metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb eeccc84 metastore/src/java/org/apache/hadoop/hive/metastore/TransactionalValidationListener.java 3e74675 orc/src/java/org/apache/orc/impl/TreeReaderFactory.java c4a2093 ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java db6848a ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 57b6c67 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 23a13d6 ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java c150ec5 ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 945b828 ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 69d58d6 ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat.java b0f8c8b ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java e577961 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ef0bb3d ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 8cf261d ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 6caca98 ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java d48e441 ql/src/test/org/apache/hadoop/hive/ql/io/TestAcidUtils.java b83cea4 Diff: https://reviews.apache.org/r/49766/diff/ Testing ------- Tests for the feature are in ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java. These are mostly integration tests that test end-to-end insert/update/delete scenarios followed by compaction and cleaning. Thanks, Saket Saurabh
