[
https://issues.apache.org/jira/browse/HIVE-9341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273284#comment-14273284
]
Hive QA commented on HIVE-9341:
-------------------------------
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12691593/HIVE-9341.1.patch.txt
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 7311 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_ptf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_ptf
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ptf
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ptf_streaming
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorized_ptf
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_windowing
org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2335/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2335/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2335/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12691593 - PreCommit-HIVE-TRUNK-Build
> Apply ColumnPrunning for noop PTFs
> ----------------------------------
>
> Key: HIVE-9341
> URL: https://issues.apache.org/jira/browse/HIVE-9341
> Project: Hive
> Issue Type: Improvement
> Components: PTF-Windowing
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-9341.1.patch.txt
>
>
> Currently, PTF disables CP optimization, which can make a huge burden. For
> example,
> {noformat}
> select p_mfgr, p_name, p_size,
> rank() over (partition by p_mfgr order by p_name) as r,
> dense_rank() over (partition by p_mfgr order by p_name) as dr,
> sum(p_retailprice) over (partition by p_mfgr order by p_name rows between
> unbounded preceding and current row) as s1
> from noop(on part
> partition by p_mfgr
> order by p_name
> );
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: part
> Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: p_mfgr (type: string), p_name (type: string)
> sort order: ++
> Map-reduce partition columns: p_mfgr (type: string)
> Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE
> Column stats: NONE
> value expressions: p_partkey (type: int), p_name (type:
> string), p_mfgr (type: string), p_brand (type: string), p_type (type:
> string), p_size (type: int), p_container (type: string), p_retailprice (type:
> double), p_comment (type: string), BLOCK__OFFSET__INSIDE__FILE (type:
> bigint), INPUT__FILE__NAME (type: string), ROW__ID (type:
> struct<transactionid:bigint,bucketid:int,rowid:bigint>)
> ...
> {noformat}
> There should be a generic way to discern referenced columns but before that,
> we know CP can be safely applied to noop functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
