[
https://issues.apache.org/jira/browse/HIVE-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16858860#comment-16858860
]
Yongzhi Chen commented on HIVE-21796:
-------------------------------------
Each field's index in the list is stored in the field object itself
(StructFieldImpl index), if it is a valid list, the index in the list must
match the value in StructFieldImpl. So two same size valid lists with the same
set of value entries should have the same order.
Double check the code, fields and the map are all calculated in
ArrayWritableObjectInspector, and there are no other ways to change the map, so
equal fieldsName combined with equal fields should create the equal map.
You change look fine +1
> ArrayWritableObjectInspector.equals can take O(2^nesting_depth) time
> --------------------------------------------------------------------
>
> Key: HIVE-21796
> URL: https://issues.apache.org/jira/browse/HIVE-21796
> Project: Hive
> Issue Type: Bug
> Reporter: Csaba Ringhofer
> Assignee: Zoltan Matyus
> Priority: Major
> Attachments: HIVE-21796.1.patch, HIVE-21796.2.patch,
> HIVE-21796.3.patch, HIVE-21796.patch, jstack.txt
>
>
> The issue came up during an Impala test when we tried to run it with Hive
> 3.1. The a query hanged: it tried to insert 1 row from a Parquet file with a
> 99 level nested column to a similar Orc file, and spent its time in
> ArrayWritableObjectInspector.equals() according to jstack.
> The problem seems to be that equals() calls both fields.equals(that.fields)
> and fieldsByName.equals(that.fieldsByName), and both try to compare all
> nested fields recursively, which leads to the O(2^nesting_depth) complexity.
> The commit that introduced this behavior:
> https://github.com/apache/hive/commit/98a25f2d831ab27e174bc99792047eaa8ec08b82#diff-8c6363e90d442f239bc252a104f1bfed
> The Impala test:
> https://github.com/apache/impala/blob/9ee4a5e1940afa47227a92e0f6fba6d4c9909f63/tests/query_test/test_nested_types.py#L612
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
