Hi All, I'm having a problem in Hive 0.13.0 using INSERT OVERWRITE with dynamic partitioning, selecting from an ORC table to another ORC table (I don't think the target table being ORC is significant). The insert is generating an java.lang.ArrayIndexOutOfBoundsException. Has anyone seen this error before? Is it fixed in a later version? I've included reproduction steps below.
Thanks - Dave *Create a sample text file* echo "a1,b1" > part *Create a textfile table and load the data into it* CREATE TABLE t1 (a STRING, b STRING) PARTITIONED BY (c STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE; LOAD DATA LOCAL INPATH 'part' OVERWRITE INTO TABLE t1 PARTITION (c = 'c1'); *Create an ORC table and load data from the textfile table into it* CREATE TABLE t2 (a STRING, b STRING) PARTITIONED BY (c STRING) STORED AS ORC; INSERT OVERWRITE TABLE t2 PARTITION (c = 'c1') SELECT a, b FROM t1; *Create another ORC table and load from the first ORC table - works fine* CREATE TABLE t3 (a STRING, b STRING) PARTITIONED BY (c STRING) STORED AS ORC; INSERT OVERWRITE TABLE t3 PARTITION (c = 'c1') SELECT a, b FROM t2; *Now enable dynamic partitioning* set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict; *Insert into the new ORC table from the textfile table - works fine* INSERT OVERWRITE TABLE t3 PARTITION (c) SELECT a, b, c FROM t1; *Insert into the new ORC table from the first ORC table - fails with the below stack trace* INSERT OVERWRITE TABLE t3 PARTITION (c) SELECT a, b, c FROM t2; Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177) ... 8 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcSerde.serialize(VectorizedOrcSerde.java:73) at org.apache.hadoop.hive.ql.io.orc.OrcSerde.serializeVector(OrcSerde.java:148) at org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.processOp(VectorFileSinkOperator.java:79) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:793) at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:793) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:793) at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43) ... 9 more
