-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71708/
-----------------------------------------------------------
(Updated Nov. 19, 2019, 4:25 p.m.)
Review request for hive, Gopal V, Jesús Camacho Rodríguez, Zoltan Haindrich,
and Rajesh Balamohan.
Bugs: HIVE-20148 and HIVE-22435
https://issues.apache.org/jira/browse/HIVE-20148
https://issues.apache.org/jira/browse/HIVE-22435
Repository: hive-git
Description
-------
Exception when using VectorTopNKeyOperator operator
===================================================
VectorTopNKeyOperator extends TopNKeyOperator and it calls it's
super.initializeOp method
https://github.com/apache/hive/blob/5c8392468cb581f53b6cb55d201fc933dca025e3/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java#L71
which is focusing on non-vectorized execution.
Fix: Derive VectorTopNKeyOperator from Oprator instead of TopNKeyOperator and
do the initialization
- Use VectorHashKeyWrapperBatch to store the current batch keys
- Add support of NULLS FIRST/LAST
Diffs (updated)
-----
itests/src/test/resources/testconfiguration.properties 2918a6852c
ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyFilter.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java d16500ef05
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java
c80bc804a2
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBatch.java
dd31991d03
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperGeneralComparator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
4cc02b4975
ql/src/test/queries/clientpositive/topnkey.q 283f426f18
ql/src/test/queries/clientpositive/topnkey_order_null.q PRE-CREATION
ql/src/test/queries/clientpositive/vector_topnkey.q e1b7d26afe
ql/src/test/results/clientpositive/llap/topnkey.q.out cd47e9d223
ql/src/test/results/clientpositive/llap/topnkey_order_null.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/vector_topnkey.q.out d859270ff0
ql/src/test/results/clientpositive/tez/topnkey.q.out 206c0c805d
ql/src/test/results/clientpositive/tez/topnkey_order_null.q.out PRE-CREATION
ql/src/test/results/clientpositive/tez/vector_topnkey.q.out b6760db156
ql/src/test/results/clientpositive/topnkey.q.out 66d0eca989
ql/src/test/results/clientpositive/topnkey_order_null.q.out PRE-CREATION
ql/src/test/results/clientpositive/vector_topnkey.q.out 3438be2dc0
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectComparator.java
PRE-CREATION
Diff: https://reviews.apache.org/r/71708/diff/7/
Changes: https://reviews.apache.org/r/71708/diff/6-7/
Testing
-------
run q test: vector_topnkey and limit_pushdown3 after applying the patch for
TopNKey pushdown
(https://issues.apache.org/jira/secure/attachment/12984389/HIVE-20150.15.patch)
Thanks,
Krisztian Kasa
