[
https://issues.apache.org/jira/browse/HIVE-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13699750#comment-13699750
]
Hudson commented on HIVE-3253:
------------------------------
Integrated in Hive-trunk-hadoop2 #270 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2/270/])
HIVE-3253 : ArrayIndexOutOfBounds exception for deeply nested structs
(Thejas Nair via Ashutosh Chauhan) (Revision 1499450)
Result = FAILURE
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1499450
Files :
* /hive/trunk/data/files/nested_complex.txt
*
/hive/trunk/hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java
* /hive/trunk/ql/src/test/queries/clientnegative/nested_complex_neg.q
* /hive/trunk/ql/src/test/queries/clientpositive/nested_complex.q
* /hive/trunk/ql/src/test/results/clientnegative/nested_complex_neg.q.out
* /hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucket_map_join_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucket_map_join_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
* /hive/trunk/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out
* /hive/trunk/ql/src/test/results/clientpositive/combine2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/combine2_hadoop20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_skew_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input23.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input42.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join_filters_overlap.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_9.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/list_bucket_query_multiskew_1.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/list_bucket_query_oneskew_1.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/list_bucket_query_oneskew_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/macro.q.out
* /hive/trunk/ql/src/test/results/clientpositive/metadataonly1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nested_complex.q.out
* /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_join_filter.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out
* /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/regexp_extract.q.out
* /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sample10.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sample6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sample8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sample9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_15.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/truncate_column_list_bucket.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_explode.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_java_method.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_reflect.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_reflect2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udtf_explode.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union24.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java
*
/hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
*
/hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyArrayMapStruct.java
> ArrayIndexOutOfBounds exception for deeply nested structs
> ---------------------------------------------------------
>
> Key: HIVE-3253
> URL: https://issues.apache.org/jira/browse/HIVE-3253
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.9.0, 0.10.0
> Reporter: Swarnim Kulkarni
> Assignee: Thejas M Nair
> Fix For: 0.12.0
>
> Attachments: HIVE-3253.2.patch, HIVE-3253.3.patch,
> HIVE-3253_moar_nesting.1.patch, jsonout.hive
>
>
> It was observed that while creating table with deeply nested structs might
> throw this exception:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 9
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:281)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> at
> org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyStructInspector(LazyFactory.java:354)
> {code}
> The reason being that currently the separators array has been hardcoded to be
> of size 8 in the LazySimpleSerde.
> {code}
> // Read the separators: We use 8 levels of separators by default, but we
> // should change this when we allow users to specify more than 10 levels
> // of separators through DDL.
> serdeParams.separators = new byte[8];
> {code}
> If possible, we should increase this size or at least make it configurable to
> properly handle deeply nested structs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
