[
https://issues.apache.org/jira/browse/HIVE-5771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028377#comment-14028377
]
Ashutosh Chauhan commented on HIVE-5771:
----------------------------------------
[~tedxu] Can you create a Review Board request for your latest patch. I took a
cursorily look and have following observations:
* In few tests an extra (or in some cases 2) MR stages got added in the plan.
These tests were testing specific optimizations, so seems like those
optimizations got disabled now. Tests are :
groupby_sort_1.q,groupby_sort_skew_1.q
* Tests subquery_multiinsert.q,subquery_notin.q are generating in wrong results
* For test annotate_stats_filter.q plan changed from MR to fetch-only, which
seems like an improvement. But, not sure how plan got changed.
* Some join tests now print a warning about being getting converted into
cross-join, which will be performance degradation.
cluster.q,join38.q,join_literals.q,join_nullsafe.q,ppd2.q,ppd_clusterby.q,ppd_join4.q,ppd_outer_join5.q
* Test smb_mapjoin_25.q is failing with following stack trace:
{code}
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.getValueObjectInspectors(MapJoinOperator.java:135)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getJoinOutputObjectInspector(CommonJoinOperator.java:167)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:310)
at
org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:72)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.initializeOp(MapJoinOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420)
at
org.apache.hadoop.hive.ql.exec.HashTableDummyOperator.initializeOp(HashTableDummyOperator.java:40)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:145)
{code}
> Constant propagation optimizer for Hive
> ---------------------------------------
>
> Key: HIVE-5771
> URL: https://issues.apache.org/jira/browse/HIVE-5771
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Ted Xu
> Assignee: Ted Xu
> Attachments: HIVE-5771.1.patch, HIVE-5771.10.patch,
> HIVE-5771.11.patch, HIVE-5771.2.patch, HIVE-5771.3.patch, HIVE-5771.4.patch,
> HIVE-5771.5.patch, HIVE-5771.6.patch, HIVE-5771.7.patch, HIVE-5771.8.patch,
> HIVE-5771.9.patch, HIVE-5771.patch, HIVE-5771.patch.javaonly
>
>
> Currently there is no constant folding/propagation optimizer, all expressions
> are evaluated at runtime.
> HIVE-2470 did a great job on evaluating constants on UDF initializing phase,
> however, it is still a runtime evaluation and it doesn't propagate constants
> from a subquery to outside.
> It may reduce I/O and accelerate process if we introduce such an optimizer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
