[
https://issues.apache.org/jira/browse/HIVE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568018#comment-13568018
]
Ashutosh Chauhan commented on HIVE-896:
---------------------------------------
Did some further testing on my 1-node pseudo cluster. After getting past
dependency problems, I hit this OOM exception. This is with
{code}
-- 10. testOnlyPTF
select p_mfgr, p_name, p_size from noop(part distribute by p_mfgr sort by
p_name);
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row (tag=0)
{"key":{"reducesinkkey0":"Manufacturer#2","reducesinkkey1":"almond antique
violet chocolate turquoise"},"value":{"_col0":105685,"_col1":"almond antique
violet chocolate
turquoise","_col2":"Manufacturer#2","_col3":"Brand#22","_col4":"MEDIUM ANODIZED
COPPER","_col5":14,"_col6":"MED CAN","_col7":1690.68,"_col8":"ly pending
requ"},"alias":0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:274)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:518)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:419)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row (tag=0)
{"key":{"reducesinkkey0":"Manufacturer#2","reducesinkkey1":"almond antique
violet chocolate turquoise"},"value":{"_col0":105685,"_col1":"almond antique
violet chocolate
turquoise","_col2":"Manufacturer#2","_col3":"Brand#22","_col4":"MEDIUM ANODIZED
COPPER","_col5":14,"_col6":"MED CAN","_col7":1690.68,"_col8":"ly pending
requ"},"alias":0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:262)
... 7 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.hive.ql.exec.PTFPersistence.createList(PTFPersistence.java:47)
at
org.apache.hadoop.hive.ql.exec.PTFPartition.init(PTFPartition.java:42)
at
org.apache.hadoop.hive.ql.exec.PTFPartition.<init>(PTFPartition.java:35)
at
org.apache.hadoop.hive.ql.exec.PTFOperator.createFirstPartitionForChain(PTFOperator.java:371)
at
org.apache.hadoop.hive.ql.exec.PTFOperator.processOp(PTFOperator.java:133)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:521)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:849)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:521)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:253)
... 7 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.hive.ql.exec.PTFPersistence.createList(PTFPersistence.java:43)
... 16 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.hadoop.hive.ql.exec.PTFPersistence$ByteBasedList.<init>(PTFPersistence.java:77)
at
org.apache.hadoop.hive.ql.exec.PTFPersistence$PartitionedByteBasedList.addPartition(PTFPersistence.java:407)
at
org.apache.hadoop.hive.ql.exec.PTFPersistence$PartitionedByteBasedList.<init>(PTFPersistence.java:386)
... 21 more
{code}
I tried with other queries as well and all of them failed there. Note that
these queries are from ptf_general_queries.q which when run via ant for unit
tests, passes fine.
> Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive.
> ---------------------------------------------------------------
>
> Key: HIVE-896
> URL: https://issues.apache.org/jira/browse/HIVE-896
> Project: Hive
> Issue Type: New Feature
> Components: OLAP, UDF
> Reporter: Amr Awadallah
> Priority: Minor
> Attachments: DataStructs.pdf, HIVE-896.1.patch.txt,
> Hive-896.2.patch.txt
>
>
> Windowing functions are very useful for click stream processing and similar
> time-series/sliding-window analytics.
> More details at:
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1006709
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007059
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007032
> -- amr
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
