Daniel Becker has uploaded a new patch set (#3). (
http://gerrit.cloudera.org:8080/21105 )
Change subject: IMPALA-12845: Crash with DESCRIBE on a complex type from an
Iceberg table
......................................................................
IMPALA-12845: Crash with DESCRIBE on a complex type from an Iceberg table
A DESCRIBE statement on a complex column contained in an Iceberg table
runs into a DCHECK and crashes Impala. An example with an array:
describe functional_parquet.iceberg_resolution_test_external.phone
Note that this also happens with Iceberg metadata tables, for example:
describe functional_parquet.iceberg_query_metadata.\
entries.readable_metrics;
With non-Iceberg tables there is no error.
The problem is that for Iceberg tables, the DESCRIBE statement returns
four columns: "name", "type", "comment" and "nullable" (only Iceberg and
Kudu tables have "nullable"). However, the DESCRIBE statement response
for complex types only contains the first three columns as they are
always nullable. But as the table is an Iceberg table, the 'metadata_'
field of HS2ColumnarResultSet is still populated with four columns.
The DCHECK in HS2ColumnarResultSet::AddOneRow() expects the number of
columns to be the same in the DESCRIBE statement response and the
'metadata_' field.
This commit solves the problem by only adding the "nullable" column to
the 'metadata_' field if the target of the DESCRIBE statement is a
table, not a complex type.
Note that Kudu tables do not support complex types so this issue does
not arise there.
This change also addresses a minor issue: DescribeTableStmt::analyze()
did not check whether the statement was already analyzed and did not set
the 'analyzer_' field which would indicate that analysis had already
been done. This is now corrected.
Testing:
- added tests in describe-path.test for arrays, maps and structs from
regular Iceberg tables and metadata tables.
Change-Id: I5eda21a41167cc1fda183aa16fd6276a6a16f5d3
---
M fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M testdata/workloads/functional-query/queries/QueryTest/describe-path.test
3 files changed, 87 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/21105/3
--
To view, visit http://gerrit.cloudera.org:8080/21105
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5eda21a41167cc1fda183aa16fd6276a6a16f5d3
Gerrit-Change-Number: 21105
Gerrit-PatchSet: 3
Gerrit-Owner: Daniel Becker <[email protected]>
Gerrit-Reviewer: Daniel Becker <[email protected]>
Gerrit-Reviewer: Gabor Kaszab <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
