[
https://issues.apache.org/jira/browse/IMPALA-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17985043#comment-17985043
]
ASF subversion and git services commented on IMPALA-14154:
----------------------------------------------------------
Commit 892b33f35d1cab01d91d9b5eaac079ec8ba236bb in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=892b33f35 ]
IMPALA-14154: IllegalStateException with Iceberg table with DELETE
Planning a query on an Iceberg table runs on IllegalStateException in
the following case:
- the table has delete files for each data file (i.e. no data file
without deletes)
AND
- there is an anti-join on top of the Iceberg delete operation
(IcebergDeleteNode or HashJoinNode).
The exception is thrown by a Preconditions check in
SingleNodePlanner.createJoinNode() because there is no null-matching EQ
operator. This happens because the conjunct that should be there is
discarded, ultimately in Analyzer.canEvalAntiJoinedConjunct(). The
reason is that the tuple ids passed to that function include the delete
part of the Iceberg scan, but the conjunct (correctly) only refers to
the tuple of the data files.
Note that this does not happen if we have two regular anti-joins on top
of each other because in that case the getTblRefIds() method of the
bottom anti-join returns a single TableRefId, the one corresponding to
the (inline) view containing it, not the two TableRefIds corresponding
to its two child nodes. The conjunct references this TupleId, so
Analyzer.canEvalAntiJoinedConjunct() returns true. This is not the case
with Iceberg delete operations because there is no (inline) view
involved.
This commit solves the issue by setting the TableRefIds of the node
corresponding to the Iceberg delete operation (IcebergDeleteNode or
HashJoinNode) to only the table ref that corresponds to the data files,
not the delete files.
Testing:
- added a test in iceberg-v2-read-position-deletes.test that
reproduces the issue.
Change-Id: If2c03fe3da44dc0516ebdf32430416a1059d37b2
Reviewed-on: http://gerrit.cloudera.org:8080/23051
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> IllegalStateException with Iceberg table with DELETE
> ----------------------------------------------------
>
> Key: IMPALA-14154
> URL: https://issues.apache.org/jira/browse/IMPALA-14154
> Project: IMPALA
> Issue Type: Bug
> Reporter: Daniel Becker
> Assignee: Daniel Becker
> Priority: Major
> Labels: impala-iceberg
>
> The following SQL returns an IllegalStateException:
> {code:java}
> DROP DATABASE IF EXISTS db CASCADE;
> CREATE DATABASE db;
> CREATE TABLE db.tbl1 (
> uuid1 BIGINT
> );
> CREATE TABLE db.tbl2 (
> uuid2 BIGINT
> ) STORED AS ICEBERG TBLPROPERTIES ('format-version'='2');
> INSERT INTO db.tbl2 VALUES
> (1005),
> (1006);
> DELETE FROM db.tbl2 WHERE uuid2 = 1005;
> SELECT uuid2
> FROM db.tbl2
> WHERE uuid2 NOT IN
> (SELECT uuid1
> FROM db.tbl1);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
