> On Aug. 4, 2016, 2:29 p.m., Sergey Shelukhin wrote: > > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java, line 399 > > <https://reviews.apache.org/r/49766/diff/4/?file=1455576#file1455576line399> > > > > rowId is always -1 here. Intended?
@Sergey: Thanks for pointing this out. It was a bug and not intended. - Saket ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49766/#review144816 ----------------------------------------------------------- On Aug. 8, 2016, 5:53 p.m., Saket Saurabh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49766/ > ----------------------------------------------------------- > > (Updated Aug. 8, 2016, 5:53 p.m.) > > > Review request for hive and Eugene Koifman. > > > Repository: hive-git > > > Description > ------- > > https://issues.apache.org/jira/browse/HIVE-14035 > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 70816bd > > hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java > 14f7316 > > hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java > 974c6b8 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java > 731caa8 > metastore/if/hive_metastore.thrift a2e35b8 > metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h ae14bd1 > metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp f982bf2 > > metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java > 5a666f2 > metastore/src/gen/thrift/gen-php/metastore/Types.php d6f7f49 > metastore/src/gen/thrift/gen-py/hive_metastore/constants.py d1c07a5 > metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb eeccc84 > > metastore/src/java/org/apache/hadoop/hive/metastore/TransactionalValidationListener.java > 3e74675 > ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java db6848a > ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 57b6c67 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 26e6443 > ql/src/java/org/apache/hadoop/hive/ql/io/AcidOutputFormat.java dd90a95 > ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 449d889 > ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 945b828 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 334cb31 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java > efde2db > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 1a1af28 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 8cb5e8a > ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 8cf261d > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > 6caca98 > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java af192fb > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands3.java PRE-CREATION > ql/src/test/org/apache/hadoop/hive/ql/io/TestAcidUtils.java 556df18 > ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java > 6648829 > > Diff: https://reviews.apache.org/r/49766/diff/ > > > Testing > ------- > > Tests for the feature are in > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java. These are mostly > integration tests that test end-to-end insert/update/delete scenarios > followed by compaction and cleaning. > > > Thanks, > > Saket Saurabh > >
