[
https://issues.apache.org/jira/browse/HIVE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125553#comment-13125553
]
binlijin commented on HIVE-2497:
--------------------------------
PredicatePushDown will push the predicate sub.tag_student > 0 down following
TableScanOperator. When partition pruning , try to iterate all partitions ,
the filter condition is :
case when (null or null)
then 1
else 0
> 0
The case when return 0, so all partitions are pruned.
> partition pruning prune some right partition under specific conditions
> -----------------------------------------------------------------------
>
> Key: HIVE-2497
> URL: https://issues.apache.org/jira/browse/HIVE-2497
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: binlijin
> Labels: patch
> Fix For: 0.9.0
>
> Attachments: hive-2497.patch
>
>
> create table src3(key string, value string) partitioned by (pt string)
> row format delimited fields terminated by ',';
> ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110911000000') ;
> ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110912000000') ;
> ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110913000000') ;
> explain extended
> select user_id
> from
> (
> select
> cast(key as int) as user_id
> ,case when (value like 'aaa%' or value like 'vvv%')
> then 1
> else 0 end as tag_student
> from src3
> ) sub
> where sub.tag_student > 0;
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> sub:src3
> TableScan
> alias: src3
> Filter Operator
> isSamplingPred: false
> predicate:
> expr: (CASE WHEN (((value like 'aaa%') or (value like
> 'vvv%'))) THEN (1) ELSE (0) END > 0)
> type: boolean
> Select Operator
> expressions:
> expr: UDFToInteger(key)
> type: int
> expr: CASE WHEN (((value like 'aaa%') or (value like
> 'vvv%'))) THEN (1) ELSE (0) END
> type: int
> outputColumnNames: _col0, _col1
> Filter Operator
> isSamplingPred: false
> predicate:
> expr: (_col1 > 0)
> type: boolean
> Select Operator
> expressions:
> expr: _col0
> type: int
> outputColumnNames: _col0
> File Output Operator
> compressed: false
> GlobalTableId: 0
> directory:
> hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-26-12_894_9085644225727185586/-ext-10001
> NumFilesPerFileSink: 1
> table:
> input format:
> org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> columns _col0
> columns.types int
> serialization.format 1
> TotalFiles: 1
> MultiFileSpray: false
> Needs Tagging: false
> Stage: Stage-0
> Fetch Operator
> limit: -1
> if we set hive.optimize.ppd=false;
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> sub:src3
> TableScan
> alias: src3
> Select Operator
> expressions:
> expr: UDFToInteger(key)
> type: int
> expr: CASE WHEN (((value like 'aaa%') or (value like
> 'vvv%'))) THEN (1) ELSE (0) END
> type: int
> outputColumnNames: _col0, _col1
> Filter Operator
> isSamplingPred: false
> predicate:
> expr: (_col1 > 0)
> type: boolean
> Select Operator
> expressions:
> expr: _col0
> type: int
> outputColumnNames: _col0
> File Output Operator
> compressed: false
> GlobalTableId: 0
> directory:
> hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-27-22_527_1729287213481398480/-ext-10001
> NumFilesPerFileSink: 1
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> columns _col0
> columns.types int
> serialization.format 1
> TotalFiles: 1
> MultiFileSpray: false
> Needs Tagging: false
> Path -> Alias:
> hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110911000000
> [sub:src3]
> hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110912000000
> [sub:src3]
> hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110913000000
> [sub:src3]
> Path -> Partition:
> hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110911000000
> Partition
> base file name: pt=20110911000000
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
