> On Nov. 19, 2013, 6:45 p.m., Xuefu Zhang wrote: > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java, > > line 398 > > <https://reviews.apache.org/r/15561/diff/2/?file=386971#file386971line398> > > > > What is this fix for?
Because ColumnPruner is applied after constant propagation optimizer(ConstantPO), some expressions are already eliminated by ConstantPO. This statement here is to prevent ArrayOutOfBoundException in that cases. - Ted ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15561/#review29119 ----------------------------------------------------------- On Jan. 18, 2014, 2:54 p.m., Ted Xu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15561/ > ----------------------------------------------------------- > > (Updated Jan. 18, 2014, 2:54 p.m.) > > > Review request for hive. > > > Bugs: HIVE-5771 > https://issues.apache.org/jira/browse/HIVE-5771 > > > Repository: hive > > > Description > ------- > > Currently there is no constant folding/propagation optimizer, all expressions > are evaluated at runtime. > > HIVE-2470 did a great job on evaluating constants on UDF initializing phase, > however, it is still a runtime evaluation and it doesn't propagate constants > from a subquery to outside. > > It may reduce I/O and accelerate process if we introduce such an optimizer. > > > Diffs > ----- > > > http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/contrib/src/test/results/clientpositive/udf_example_add.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/contrib/src/test/results/clientpositive/udf_example_format.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/hbase-handler/src/test/results/positive/hbase_ppd_key_range.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/hbase-handler/src/test/results/positive/hbase_pushdown.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/hbase-handler/src/test/results/positive/ppd_key_ranges.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnInfo.java > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagate.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcCtx.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/alter_partition_coltype.q > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/annotate_stats_filter.q > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/annotate_stats_part.q > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/annotate_stats_select.q > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog1.q > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog2.q > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_dp.q > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_type.q > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/decimal_udf.q > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/pcr.q > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cast1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog1.q.out > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_dp.q.out > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_type.q.out > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_sort_1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_sort_skew_1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_empty.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_file_format.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_multiple.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_partitioned.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_update.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_stale.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_stale_partitioned.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/infer_const_type.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input18.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input23.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input26.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input38.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input6.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input_part4.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input_part6.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/insert1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_cond_pushdown_1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_cond_pushdown_3.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_view.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/lateral_view_ppd.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/literal_double.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/macro.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/mapjoin1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/multi_insert.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/multi_insert_move_tasks_share_dependencies.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/num_op_type_conv.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/orc_predicate_pushdown.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_constant_where.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_col.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/query_result_fileformat.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quote1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_partition.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/regex_col.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/regexp_extract.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/sample8.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/select_unquote_and.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/select_unquote_not.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/select_unquote_or.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/set_processor_namespaces.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/set_variable_sub.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_18.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/stats_empty_dyn_part.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subq_where_serialization.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_exists.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_exists_having.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_in.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_in_having.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_multiinsert.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_notexists.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_notexists_having.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_notin.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/subquery_notin_having.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/type_cast_1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/type_widening.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf1.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf4.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf5.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf6.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf7.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf9.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_10_trims.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_E.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_PI.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_abs.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_ascii.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_between.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_case.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_coalesce.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_degrees.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_elt.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_format_number.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_hash.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_hour.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_if.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_instr.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_isnull_isnotnull.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_like.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_locate.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_lower.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_lpad.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_minute.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_nvl.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_parse_url.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_printf.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_radians.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_reflect2.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_repeat.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_rpad.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_second.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_sign.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_space.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_when.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union33.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union_view.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/vectorization_div0.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/vectorization_short_regress.q.out > 1559345 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/vectorized_math_funcs.q.out > 1559345 > > Diff: https://reviews.apache.org/r/15561/diff/ > > > Testing > ------- > > > Thanks, > > Ted Xu > >
