[
https://issues.apache.org/jira/browse/HIVE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568425#comment-13568425
]
Harish Butani commented on HIVE-896:
------------------------------------
Ashutosh,
So the plan looks like this:
... -> ReduceSink -> Extract -> PTFOp
We don't know what columns a PTF will access; the contract is it has access to
all columns in its input. So we don't want any Column Pruning to happen. So we
don't put a Select Op before the Reduce Sink. At translation time we see all
the Columns, including the VCs. It appears as though during optimization VCs
are carried forward only if required; so at runtime the ColumnExprNodeDescs are
referring to the wrong internalNames. Does this make sense? Is there a way to
carry forward the VCs when a PTF is present. The other option is (which we have
taken is) to say VCs are not available to PTFs.
Having said this, when the PTF is Windowing, we do know the columns being
referred; so we should put a Select Op in front of the ReduceSink.
> Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive.
> ---------------------------------------------------------------
>
> Key: HIVE-896
> URL: https://issues.apache.org/jira/browse/HIVE-896
> Project: Hive
> Issue Type: New Feature
> Components: OLAP, UDF
> Reporter: Amr Awadallah
> Priority: Minor
> Attachments: DataStructs.pdf, HIVE-896.1.patch.txt,
> Hive-896.2.patch.txt
>
>
> Windowing functions are very useful for click stream processing and similar
> time-series/sliding-window analytics.
> More details at:
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1006709
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007059
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007032
> -- amr
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
