[
https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674237#comment-16674237
]
Hive QA commented on HIVE-20842:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946765/HIVE-20842.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14734/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14734/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14734/
Messages:
{noformat}
**** This message was trimmed, see log for full details ****
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:21:26.758
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14734/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:21:26.762
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
ae1eb15..bb77751 master -> origin/master
+ git reset --hard HEAD
HEAD is now at ae1eb15 HIVE-20778: Join reordering may not be triggered if all
joins in plan are created by decorrelation logic (Vineet Garg via Jesus Camacho
Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at bb77751 HIVE-20815: JdbcRecordReader.next shall not eat
exception (Daniel Dai, reviewed by Thejas Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:21:27.781
+ rm -rf ../yetus_PreCommit-HIVE-Build-14734
+ mkdir ../yetus_PreCommit-HIVE-Build-14734
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14734
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14734/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java:
does not exist in index
error: a/ql/src/test/results/clientpositive/annotate_stats_groupby.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/check_constraint.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/llap/constraints_optimization.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer6.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/except_distinct.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/explainanalyze_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_4.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/intersect_all.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/intersect_distinct.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/intersect_merge.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/mrr.q.out: does not exist in
index
error:
a/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/parallel.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/llap/parallel_colstats.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/ptf.q.out: does not exist in
index
error:
a/ql/src/test/results/clientpositive/llap/reduce_deduplicate_distinct.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/reduce_deduplicate_extended.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/sharedworkext.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/sqlmerge_stats.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/subquery_in.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/subquery_in_having.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/subquery_multi.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/subquery_notin.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/subquery_select.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/subquery_views.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/tez_union2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/tez_union_multiinsert.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/unionDistinct_3.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/llap/vector_adaptor_usage_mode.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/vector_char_2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/vector_distinct_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/vector_groupby_3.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/vector_outer_reference_windowed.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/vector_windowing.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/vector_windowing_gby2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query11.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query23.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query33.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query34.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query38.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query4.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query49.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query58.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query64.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query73.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query74.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query75.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query78.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query79.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query83.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query87.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query95.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query97.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query11.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query14.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query33.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query34.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query38.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query4.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query45.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query49.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query5.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query58.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query64.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query70.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query73.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query74.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query78.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query79.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query80.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query83.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query87.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/perf/tez/query97.q.out: does not
exist in index
error: patch failed:
ql/src/test/results/clientpositive/llap/explainuser_1.q.out:2388
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/explainuser_1.q.out'
with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_in_having.q.out:655
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/subquery_in_having.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_multi.q.out:4051
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/subquery_multi.q.out'
with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_scalar.q.out:5636
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/subquery_scalar.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_views.q.out:243
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/subquery_views.q.out'
with conflicts.
Going to apply patch with: git apply -p1
/data/hiveptest/working/scratch/build.patch:7161: trailing whitespace.
PREHOOK: query: SELECT
/data/hiveptest/working/scratch/build.patch:7170: trailing whitespace.
FROM srcpart a JOIN srcpart b
/data/hiveptest/working/scratch/build.patch:7172: trailing whitespace.
AND a.key = b.key
/data/hiveptest/working/scratch/build.patch:7176: trailing whitespace.
select key, value
/data/hiveptest/working/scratch/build.patch:7186: trailing whitespace.
POSTHOOK: query: SELECT
error: patch failed:
ql/src/test/results/clientpositive/llap/explainuser_1.q.out:2388
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/explainuser_1.q.out'
with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_in_having.q.out:655
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/subquery_in_having.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_multi.q.out:4051
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/subquery_multi.q.out'
with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_scalar.q.out:5636
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/subquery_scalar.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/llap/subquery_views.q.out:243
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/subquery_views.q.out'
with conflicts.
U ql/src/test/results/clientpositive/llap/explainuser_1.q.out
U ql/src/test/results/clientpositive/llap/subquery_multi.q.out
U ql/src/test/results/clientpositive/llap/subquery_views.q.out
warning: squelched 3 whitespace errors
warning: 8 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-14734
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946765 - PreCommit-HIVE-Build
> Fix logic introduced in HIVE-20660 to estimate statistics for group by
> ----------------------------------------------------------------------
>
> Key: HIVE-20842
> URL: https://issues.apache.org/jira/browse/HIVE-20842
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 4.0.0
> Reporter: Vineet Garg
> Assignee: Vineet Garg
> Priority: Major
> Attachments: HIVE-20842.1.patch, HIVE-20842.2.patch,
> HIVE-20842.3.patch
>
>
> HIVE-20660 introduced better estimation for group by operator. But the logic
> did not account for Partial and Full group by separately.
> For partial group by parallelism (i.e. number of tasks) should be taken into
> account.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
