[
https://issues.apache.org/jira/browse/HIVE-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Pivovarov updated HIVE-7738:
--------------------------------------
Attachment: HIVE-7738.patch
> tez select sum(decimal) from union all of decimal and null throws NPE
> ---------------------------------------------------------------------
>
> Key: HIVE-7738
> URL: https://issues.apache.org/jira/browse/HIVE-7738
> Project: Hive
> Issue Type: Bug
> Components: Tez
> Affects Versions: 0.13.1
> Reporter: Alexander Pivovarov
> Attachments: HIVE-7738.patch
>
>
> if run this query using tez engine then hive will throw NPE
> {code}
> select sum(a) from (
> select cast(1.1 as decimal) a from dual
> union all
> select cast(null as decimal) a from dual
> ) t;
> {code}
> hive> select sum(a) from (
> > select cast(1.1 as decimal) a from dual
> > union all
> > select cast(null as decimal) a from dual
> > ) t;
> Query ID = apivovarov_20140814200909_438385b2-4147-47bc-98a0-a01567bbb5c5
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: application_1407388228332_5616)
> Map 1: -/- Map 4: -/- Reducer 3: 0/1
> Map 1: 0/1 Map 4: 0/1 Reducer 3: 0/1
> Map 1: 0/1 Map 4: 0/1 Reducer 3: 0/1
> Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1
> Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1
> Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1
> Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1
> Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1407388228332_5616_1_02,
> diagnostics=[Task failed, taskId=task_1407388228332_5616_1_02_000000,
> diagnostics=[AttemptID:attempt_1407388228332_5616_1_02_000000_0 Info:Error:
> java.lang.RuntimeException: java.lang.RuntimeException: Map operator
> initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> at
> org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> at
> org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> ... 7 more
> Container released by application,
> AttemptID:attempt_1407388228332_5616_1_02_000000_1 Info:Error:
> java.lang.RuntimeException: java.lang.RuntimeException: Map operator
> initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> at
> org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> at
> org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> ... 7 more
> Container released by application,
> AttemptID:attempt_1407388228332_5616_1_02_000000_2 Info:Error:
> java.lang.RuntimeException: java.lang.RuntimeException: Map operator
> initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> at
> org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> at
> org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> ... 7 more
> Container released by application,
> AttemptID:attempt_1407388228332_5616_1_02_000000_3 Info:Error:
> java.lang.RuntimeException: java.lang.RuntimeException: Map operator
> initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> at
> org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> at
> org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> ... 7 more
> ], Vertex failed as one or more tasks failed. failedTasks:1]
> Vertex killed, vertexName=Reducer 3, vertexId=vertex_1407388228332_5616_1_01,
> diagnostics=[Vertex received Kill while in RUNNING state., Vertex killed as
> other vertex failed. failedTasks:0]
> DAG failed due to vertex failure. failedVertices:1 killedVertices:1
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask
--
This message was sent by Atlassian JIRA
(v6.2#6252)
