[
https://issues.apache.org/jira/browse/HIVE-22893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042233#comment-17042233
]
Hive QA commented on HIVE-22893:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994117/HIVE-22893.09.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20776/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20776/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20776/
Messages:
{noformat}
**** This message was trimmed, see log for full details ****
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 6c3ee53 HIVE-21216: Write Parquet INT64 timestamp (Karen Coppage
via Marta Kuczora)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-02-21 23:24:50.444
+ rm -rf ../yetus_PreCommit-HIVE-Build-20776
+ mkdir ../yetus_PreCommit-HIVE-Build-20776
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20776
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20776/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: cannot apply binary patch to
'ql/src/test/results/clientpositive/llap/vector_udf1.q.out' without full index
line
Falling back to three-way merge...
error: cannot apply binary patch to
'ql/src/test/results/clientpositive/llap/vector_udf1.q.out' without full index
line
error: ql/src/test/results/clientpositive/llap/vector_udf1.q.out: patch does
not apply
Trying to apply the patch with -p1
error: src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in
index
error: src/test/results/clientpositive/udaf_example_group_concat.q.out: does
not exist in index
error: src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java: does not exist
in index
error: src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java: does not exist in
index
error: src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java: does not
exist in index
error: src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java:
does not exist in index
error: src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCase.java: does
not exist in index
error: src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java:
does not exist in index
error: src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java: does
not exist in index
error: src/test/queries/clientpositive/udf_coalesce.q: does not exist in index
error: src/test/results/clientpositive/cbo_rp_gby2_map_multi_distinct.q.out:
does not exist in index
error:
src/test/results/clientpositive/cbo_rp_groupby3_noskew_multi_distinct.q.out:
does not exist in index
error: src/test/results/clientpositive/constprog_when_case.q.out: does not
exist in index
error: src/test/results/clientpositive/count_dist_rewrite.q.out: does not exist
in index
error: src/test/results/clientpositive/groupby11.q.out: does not exist in index
error: src/test/results/clientpositive/groupby2_map.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby2_map_multi_distinct.q.out: does
not exist in index
error: src/test/results/clientpositive/groupby2_map_skew.q.out: does not exist
in index
error: src/test/results/clientpositive/groupby2_noskew.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby2_noskew_multi_distinct.q.out:
does not exist in index
error: src/test/results/clientpositive/groupby3_map.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby3_map_multi_distinct.q.out: does
not exist in index
error: src/test/results/clientpositive/groupby3_map_skew.q.out: does not exist
in index
error: src/test/results/clientpositive/groupby4.q.out: does not exist in index
error: src/test/results/clientpositive/groupby4_noskew.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby6.q.out: does not exist in index
error: src/test/results/clientpositive/groupby6_map.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby6_map_skew.q.out: does not exist
in index
error: src/test/results/clientpositive/groupby6_noskew.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby8_map_skew.q.out: does not exist
in index
error: src/test/results/clientpositive/groupby9.q.out: does not exist in index
error: src/test/results/clientpositive/groupby_map_ppr.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out:
does not exist in index
error: src/test/results/clientpositive/groupby_multi_single_reducer.q.out: does
not exist in index
error: src/test/results/clientpositive/groupby_multi_single_reducer2.q.out:
does not exist in index
error: src/test/results/clientpositive/groupby_nocolumnalign.q.out: does not
exist in index
error: src/test/results/clientpositive/groupby_position.q.out: does not exist
in index
error: src/test/results/clientpositive/groupby_ppr.q.out: does not exist in
index
error: src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out: does
not exist in index
error: src/test/results/clientpositive/infer_bucket_sort_dyn_part.q.out: does
not exist in index
error: src/test/results/clientpositive/list_bucket_dml_6.q.out: does not exist
in index
error: src/test/results/clientpositive/list_bucket_dml_7.q.out: does not exist
in index
error: src/test/results/clientpositive/list_bucket_dml_8.q.out: does not exist
in index
error: src/test/results/clientpositive/llap/count_dist_rewrite.q.out: does not
exist in index
error: src/test/results/clientpositive/llap/explainuser_2.q.out: does not exist
in index
error: src/test/results/clientpositive/llap/groupby2.q.out: does not exist in
index
error: src/test/results/clientpositive/llap/tez_union_multiinsert.q.out: does
not exist in index
error: src/test/results/clientpositive/llap/udf_coalesce.q.out: does not exist
in index
error: src/test/results/clientpositive/llap/vector_case_when_1.q.out: does not
exist in index
error: src/test/results/clientpositive/llap/vector_case_when_2.q.out: does not
exist in index
error: src/test/results/clientpositive/llap/vector_groupby4.q.out: does not
exist in index
error: src/test/results/clientpositive/llap/vector_groupby6.q.out: does not
exist in index
error: src/test/results/clientpositive/llap/vector_if_expr.q.out: does not
exist in index
error: src/test/results/clientpositive/llap/vector_nvl.q.out: does not exist in
index
error: src/test/results/clientpositive/llap/vector_udf1.q.out: does not exist
in index
error: src/test/results/clientpositive/llap/vectorized_string_funcs.q.out: does
not exist in index
error: src/test/results/clientpositive/llap/vectorized_timestamp_funcs.q.out:
does not exist in index
error: src/test/results/clientpositive/merge_dynamic_partition4.q.out: does not
exist in index
error: src/test/results/clientpositive/merge_dynamic_partition5.q.out: does not
exist in index
error: src/test/results/clientpositive/nullgroup4_multi_distinct.q.out: does
not exist in index
error: src/test/results/clientpositive/offset_limit_global_optimizer.q.out:
does not exist in index
error: src/test/results/clientpositive/perf/tez/constraints/query19.q.out: does
not exist in index
error: src/test/results/clientpositive/perf/tez/constraints/query79.q.out: does
not exist in index
error: src/test/results/clientpositive/perf/tez/constraints/query8.q.out: does
not exist in index
error: src/test/results/clientpositive/perf/tez/constraints/query85.q.out: does
not exist in index
error: src/test/results/clientpositive/perf/tez/constraints/query99.q.out: does
not exist in index
error: src/test/results/clientpositive/perf/tez/query19.q.out: does not exist
in index
error: src/test/results/clientpositive/perf/tez/query23.q.out: does not exist
in index
error: src/test/results/clientpositive/perf/tez/query79.q.out: does not exist
in index
error: src/test/results/clientpositive/perf/tez/query8.q.out: does not exist in
index
error: src/test/results/clientpositive/perf/tez/query85.q.out: does not exist
in index
error: src/test/results/clientpositive/perf/tez/query99.q.out: does not exist
in index
error: src/test/results/clientpositive/spark/union17.q.out: does not exist in
index
error: src/test/results/clientpositive/union17.q.out: does not exist in index
error: src/test/results/clientpositive/vector_case_when_1.q.out: does not exist
in index
error: src/test/results/clientpositive/vector_case_when_2.q.out: does not exist
in index
error: src/test/results/clientpositive/vector_groupby4.q.out: does not exist in
index
error: src/test/results/clientpositive/vector_groupby6.q.out: does not exist in
index
error: src/test/results/clientpositive/vector_if_expr.q.out: does not exist in
index
error: src/test/results/clientpositive/vector_nvl.q.out: does not exist in index
error: src/test/results/clientpositive/vectorization_multi_value.q.out: does
not exist in index
error: src/test/results/clientpositive/vectorized_string_funcs.q.out: does not
exist in index
error: src/test/results/clientpositive/vectorized_timestamp_funcs.q.out: does
not exist in index
Trying to apply the patch with -p2
error: java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index
error: test/results/clientpositive/udaf_example_group_concat.q.out: does not
exist in index
error: java/org/apache/hadoop/hive/ql/stats/StatsUtils.java: does not exist in
index
error: java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java: does not exist in
index
error: java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java: does not
exist in index
error: java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java: does
not exist in index
error: java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCase.java: does not
exist in index
error: java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java: does
not exist in index
error: java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java: does not
exist in index
error: test/queries/clientpositive/udf_coalesce.q: does not exist in index
error: test/results/clientpositive/cbo_rp_gby2_map_multi_distinct.q.out: does
not exist in index
error: test/results/clientpositive/cbo_rp_groupby3_noskew_multi_distinct.q.out:
does not exist in index
error: test/results/clientpositive/constprog_when_case.q.out: does not exist in
index
error: test/results/clientpositive/count_dist_rewrite.q.out: does not exist in
index
error: test/results/clientpositive/groupby11.q.out: does not exist in index
error: test/results/clientpositive/groupby2_map.q.out: does not exist in index
error: test/results/clientpositive/groupby2_map_multi_distinct.q.out: does not
exist in index
error: test/results/clientpositive/groupby2_map_skew.q.out: does not exist in
index
error: test/results/clientpositive/groupby2_noskew.q.out: does not exist in
index
error: test/results/clientpositive/groupby2_noskew_multi_distinct.q.out: does
not exist in index
error: test/results/clientpositive/groupby3_map.q.out: does not exist in index
error: test/results/clientpositive/groupby3_map_multi_distinct.q.out: does not
exist in index
error: test/results/clientpositive/groupby3_map_skew.q.out: does not exist in
index
error: test/results/clientpositive/groupby4.q.out: does not exist in index
error: test/results/clientpositive/groupby4_noskew.q.out: does not exist in
index
error: test/results/clientpositive/groupby6.q.out: does not exist in index
error: test/results/clientpositive/groupby6_map.q.out: does not exist in index
error: test/results/clientpositive/groupby6_map_skew.q.out: does not exist in
index
error: test/results/clientpositive/groupby6_noskew.q.out: does not exist in
index
error: test/results/clientpositive/groupby8_map_skew.q.out: does not exist in
index
error: test/results/clientpositive/groupby9.q.out: does not exist in index
error: test/results/clientpositive/groupby_map_ppr.q.out: does not exist in
index
error: test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out: does
not exist in index
error: test/results/clientpositive/groupby_multi_single_reducer.q.out: does not
exist in index
error: test/results/clientpositive/groupby_multi_single_reducer2.q.out: does
not exist in index
error: test/results/clientpositive/groupby_nocolumnalign.q.out: does not exist
in index
error: test/results/clientpositive/groupby_position.q.out: does not exist in
index
error: test/results/clientpositive/groupby_ppr.q.out: does not exist in index
error: test/results/clientpositive/groupby_ppr_multi_distinct.q.out: does not
exist in index
error: test/results/clientpositive/infer_bucket_sort_dyn_part.q.out: does not
exist in index
error: test/results/clientpositive/list_bucket_dml_6.q.out: does not exist in
index
error: test/results/clientpositive/list_bucket_dml_7.q.out: does not exist in
index
error: test/results/clientpositive/list_bucket_dml_8.q.out: does not exist in
index
error: test/results/clientpositive/llap/count_dist_rewrite.q.out: does not
exist in index
error: test/results/clientpositive/llap/explainuser_2.q.out: does not exist in
index
error: test/results/clientpositive/llap/groupby2.q.out: does not exist in index
error: test/results/clientpositive/llap/tez_union_multiinsert.q.out: does not
exist in index
error: test/results/clientpositive/llap/udf_coalesce.q.out: does not exist in
index
error: test/results/clientpositive/llap/vector_case_when_1.q.out: does not
exist in index
error: test/results/clientpositive/llap/vector_case_when_2.q.out: does not
exist in index
error: test/results/clientpositive/llap/vector_groupby4.q.out: does not exist
in index
error: test/results/clientpositive/llap/vector_groupby6.q.out: does not exist
in index
error: test/results/clientpositive/llap/vector_if_expr.q.out: does not exist in
index
error: test/results/clientpositive/llap/vector_nvl.q.out: does not exist in
index
error: test/results/clientpositive/llap/vector_udf1.q.out: does not exist in
index
error: test/results/clientpositive/llap/vectorized_string_funcs.q.out: does not
exist in index
error: test/results/clientpositive/llap/vectorized_timestamp_funcs.q.out: does
not exist in index
error: test/results/clientpositive/merge_dynamic_partition4.q.out: does not
exist in index
error: test/results/clientpositive/merge_dynamic_partition5.q.out: does not
exist in index
error: test/results/clientpositive/nullgroup4_multi_distinct.q.out: does not
exist in index
error: test/results/clientpositive/offset_limit_global_optimizer.q.out: does
not exist in index
error: test/results/clientpositive/perf/tez/constraints/query19.q.out: does not
exist in index
error: test/results/clientpositive/perf/tez/constraints/query79.q.out: does not
exist in index
error: test/results/clientpositive/perf/tez/constraints/query8.q.out: does not
exist in index
error: test/results/clientpositive/perf/tez/constraints/query85.q.out: does not
exist in index
error: test/results/clientpositive/perf/tez/constraints/query99.q.out: does not
exist in index
error: test/results/clientpositive/perf/tez/query19.q.out: does not exist in
index
error: test/results/clientpositive/perf/tez/query23.q.out: does not exist in
index
error: test/results/clientpositive/perf/tez/query79.q.out: does not exist in
index
error: test/results/clientpositive/perf/tez/query8.q.out: does not exist in
index
error: test/results/clientpositive/perf/tez/query85.q.out: does not exist in
index
error: test/results/clientpositive/perf/tez/query99.q.out: does not exist in
index
error: test/results/clientpositive/spark/union17.q.out: does not exist in index
error: test/results/clientpositive/union17.q.out: does not exist in index
error: test/results/clientpositive/vector_case_when_1.q.out: does not exist in
index
error: test/results/clientpositive/vector_case_when_2.q.out: does not exist in
index
error: test/results/clientpositive/vector_groupby4.q.out: does not exist in
index
error: test/results/clientpositive/vector_groupby6.q.out: does not exist in
index
error: test/results/clientpositive/vector_if_expr.q.out: does not exist in index
error: test/results/clientpositive/vector_nvl.q.out: does not exist in index
error: test/results/clientpositive/vectorization_multi_value.q.out: does not
exist in index
error: test/results/clientpositive/vectorized_string_funcs.q.out: does not
exist in index
error: test/results/clientpositive/vectorized_timestamp_funcs.q.out: does not
exist in index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20776
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12994117 - PreCommit-HIVE-Build
> Enhance data size estimation for fields computed by UDFs
> --------------------------------------------------------
>
> Key: HIVE-22893
> URL: https://issues.apache.org/jira/browse/HIVE-22893
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
> Reporter: Zoltan Haindrich
> Assignee: Zoltan Haindrich
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22893.01.patch, HIVE-22893.02.patch,
> HIVE-22893.03.patch, HIVE-22893.04.patch, HIVE-22893.05.patch,
> HIVE-22893.06.patch, HIVE-22893.07.patch, HIVE-22893.08.patch,
> HIVE-22893.09.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Right now if we have columnstat on a column ; we use that to estimate things
> about the column; - however if an UDF is executed on a column ; the resulting
> column is treated as unknown thing and defaults are assumed.
> An improvement could be to give wide estimation(s) in case of frequently used
> udf.
> For example; consider {{substr(c,1,1)}} ; no matter what the input; the
> output is at most a 1 long string
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
