[
https://issues.apache.org/jira/browse/HIVE-9341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287178#comment-14287178
]
Hive QA commented on HIVE-9341:
-------------------------------
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12693757/HIVE-9341.4.patch.txt
{color:green}SUCCESS:{color} +1 7348 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2473/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2473/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2473/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12693757 - PreCommit-HIVE-TRUNK-Build
> Apply ColumnPrunning for noop PTFs
> ----------------------------------
>
> Key: HIVE-9341
> URL: https://issues.apache.org/jira/browse/HIVE-9341
> Project: Hive
> Issue Type: Improvement
> Components: PTF-Windowing
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-9341.1.patch.txt, HIVE-9341.2.patch.txt,
> HIVE-9341.3.patch.txt, HIVE-9341.4.patch.txt
>
>
> Currently, PTF disables CP optimization, which can make a huge burden. For
> example,
> {noformat}
> select p_mfgr, p_name, p_size,
> rank() over (partition by p_mfgr order by p_name) as r,
> dense_rank() over (partition by p_mfgr order by p_name) as dr,
> sum(p_retailprice) over (partition by p_mfgr order by p_name rows between
> unbounded preceding and current row) as s1
> from noop(on part
> partition by p_mfgr
> order by p_name
> );
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: part
> Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: p_mfgr (type: string), p_name (type: string)
> sort order: ++
> Map-reduce partition columns: p_mfgr (type: string)
> Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE
> Column stats: NONE
> value expressions: p_partkey (type: int), p_name (type:
> string), p_mfgr (type: string), p_brand (type: string), p_type (type:
> string), p_size (type: int), p_container (type: string), p_retailprice (type:
> double), p_comment (type: string), BLOCK__OFFSET__INSIDE__FILE (type:
> bigint), INPUT__FILE__NAME (type: string), ROW__ID (type:
> struct<transactionid:bigint,bucketid:int,rowid:bigint>)
> ...
> {noformat}
> There should be a generic way to discern referenced columns but before that,
> we know CP can be safely applied to noop functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
