[
https://issues.apache.org/jira/browse/HIVE-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393571#comment-14393571
]
Gunther Hagleitner commented on HIVE-10175:
-------------------------------------------
[~gopalv] do you have more info about where the time is spent and what the
query is? I'm looking at the dynamic pruner and what you ask isn't easy to do.
The reason is that the synthetic predicate gets inserted by a logical opt early
on. The dynamic pruner has to find these and nuke them, because the synthetic
ones can't be executed. So now I'm looking for alternatives to speed this up.
I don't understand for instance how it can take 2 seconds to scan an expression
tree. Maybe something else bust?
> PartitionPruning lacks a fast-path exit for large IN() queries
> --------------------------------------------------------------
>
> Key: HIVE-10175
> URL: https://issues.apache.org/jira/browse/HIVE-10175
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Tez
> Affects Versions: 1.2.0
> Reporter: Gopal V
> Assignee: Gunther Hagleitner
> Priority: Minor
>
> TezCompiler::runDynamicPartitionPruning() & ppr.PartitionPruner() calls the
> graph walker even if all tables provided to the optimizer are unpartitioned
> (or temporary) tables.
> This makes it extremely slow as it will walk & inspect a large/complex
> FilterOperator later in the pipeline.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
