Another occurrence of the issue:
{code:java}
0: jdbc:hive2://tesths2-merge-ks-1.openstackl> merge into lineitem using (select * from lineitem_stage WHERE L_ORDERKEY%2=1) sub on sub.L_ORDERKEY = lineitem.L_ORDERKEY when matched then delete;
INFO : Session is already open
INFO : Dag name: merge into lineitem using (select *...delete(Stage-1)
INFO : Setting tez.task.scale.memory.reserve-fraction to 0.30000001192092896
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1485398058799_0109)
INFO : Map 1: -/- Map 3: 0/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+1)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+2)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+3)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+4)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+6)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+7)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+9)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 0(+10)/10 Reducer 2: 0/2
INFO : Map 1: 0/5 Map 3: 1(+9)/10 Reducer 2: 0/2
INFO : Map 1: 0(+1)/5 Map 3: 1(+9)/10 Reducer 2: 0/2
INFO : Map 1: 0(+1)/5 Map 3: 1(+9)/10 Reducer 2: 0/2
INFO : Map 1: 0(+1)/5 Map 3: 2(+8)/10 Reducer 2: 0/2
INFO : Map 1: 0(+2)/5 Map 3: 2(+8)/10 Reducer 2: 0/2
INFO : Map 1: 0(+2)/5 Map 3: 2(+8)/10 Reducer 2: 0/2
INFO : Map 1: 0(+2)/5 Map 3: 2(+8)/10 Reducer 2: 0/2
INFO : Map 1: 0(+2)/5 Map 3: 2(+8)/10 Reducer 2: 0/2
INFO : Map 1: 0(+2)/5 Map 3: 2(+8)/10 Reducer 2: 0/2
INFO : Map 1: 0(+2)/5 Map 3: 4(+6)/10 Reducer 2: 0/2
INFO : Map 1: 0(+4)/5 Map 3: 4(+6)/10 Reducer 2: 0/2
INFO : Map 1: 0(+4)/5 Map 3: 6(+4)/10 Reducer 2: 0/2
INFO : Map 1: 0(+5)/5 Map 3: 6(+4)/10 Reducer 2: 0/2
INFO : Map 1: 0(+5)/5 Map 3: 7(+3)/10 Reducer 2: 0/2
INFO : Map 1: 0(+5)/5 Map 3: 8(+2)/10 Reducer 2: 0/2
INFO : Map 1: 0(+5)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 0(+5)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 0(+5)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 1(+4)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 2(+3)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 2(+2)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 3(+2)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 3(+2)/5 Map 3: 10/10 Reducer 2: 0/2
INFO : Map 1: 4(+1)/5 Map 3: 10/10 Reducer 2: 0/1
INFO : Map 1: 4(+1)/5 Map 3: 10/10 Reducer 2: 0(+1)/1
INFO : Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1)/1
INFO : Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1,-1)/1
INFO : Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1,-2)/1
INFO : Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1,-3)/1
ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Reducer 2, vertexId=vertex_1485398058799_0109_3_02, diagnostics=[Task failed, taskId=task_1485398058799_0109_3_02_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":38,"bucketid":1,"rowid":0}},"value":null}
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":38,"bucketid":1,"rowid":0}},"value":null}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:266)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":38,"bucketid":1,"rowid":0}},"value":null}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
... 16 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:780)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)
at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
... 17 more
{code}
Hs2 logs:
{code:java}
2017-01-26 22:13:20,924 INFO [HiveServer2-Background-Pool: Thread-648]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=TezRunVertex.Map 1 start=1485468754798 end=1485468800924 duration=46126 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor>
2017-01-26 22:13:20,925 INFO [HiveServer2-Background-Pool: Thread-648]: SessionState (SessionState.java:printInfo(971)) - Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1)/1
2017-01-26 22:13:22,744 INFO [HiveServer2-Background-Pool: Thread-648]: SessionState (SessionState.java:printInfo(971)) - Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1,-1)/1
2017-01-26 22:13:24,369 INFO [HiveServer2-Background-Pool: Thread-648]: SessionState (SessionState.java:printInfo(971)) - Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1,-2)/1
2017-01-26 22:13:26,194 INFO [HiveServer2-Background-Pool: Thread-648]: SessionState (SessionState.java:printInfo(971)) - Map 1: 5/5 Map 3: 10/10 Reducer 2: 0(+1,-3)/1
2017-01-26 22:13:28,221 ERROR [HiveServer2-Background-Pool: Thread-648]: SessionState (SessionState.java:printError(980)) - Status: Failed
2017-01-26 22:13:28,222 ERROR [HiveServer2-Background-Pool: Thread-648]: SessionState (SessionState.java:printError(980)) - Vertex failed, vertexName=Reducer 2, vertexId=vertex_1485398058799_0108_2_02, diagnostics=[Task failed, taskId=task_1485398058799_0108_2_02_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":38,"bucketid":1,"rowid":0}},"value":null}
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":38,"bucketid":1,"rowid":0}},"value":null}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:266)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":38,"bucketid":1,"rowid":0}},"value":null}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
... 16 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:780)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)
at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
... 17 more
{code}
|
