[
https://issues.apache.org/jira/browse/HIVE-30235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30235:
----------------------------------
Description:
With hive.txn.ext.locking.enabled=true, HiveIcebergStorageHandler.getLockType
rejects DML on a table with engine.hive.lock-enabled=true before any lock is
taken. AcidUtils.makeLockComponents doesn't consult the storage handler for
DDL_EXCLUSIVE, though, so DDL (ADD/REPLACE/CHANGE COLUMNS, UNSET TBLPROPERTIES,
TRUNCATE, DROP PARTITION) takes its EXCLUSIVE Hive lock on db.table. The meta
hook then requests the Iceberg commit lock on the same table (the same HMS
resource since HIVE-30231) and waits on the statement's own lock until
iceberg.hive.lock-timeout-ms (3 min), failing with "Timed out ... waiting for
lock on <db>.<table>". Without the hook's lock, the Iceberg commit would wait
the same way.
Proposed: have makeLockComponents ask the storage handler for DDL_EXCLUSIVE
of non-native tables too, so the DDL is rejected like DML; handlers overriding
getLockType (Iceberg, Kafka) return EXCLUSIVE for it
was:
With hive.txn.ext.locking.enabled=true, HiveIcebergStorageHandler.getLockType
rejects DML on a table with
engine.hive.lock-enabled=true before any lock is taken.
AcidUtils.makeLockComponents doesn't consult the storage
handler for DDL_EXCLUSIVE, though, so DDL (ADD/REPLACE/CHANGE COLUMNS, UNSET
TBLPROPERTIES, TRUNCATE, DROP
PARTITION) takes its EXCLUSIVE Hive lock on db.table. The meta hook then
requests the Iceberg commit lock on the
same table (the same HMS resource since HIVE-30231) and waits on the
statement's own lock until
iceberg.hive.lock-timeout-ms (3 min), failing with "Timed out ... waiting for
lock on <db>.<table>". Without the
hook's lock, the Iceberg commit would wait the same way.
Proposed: have makeLockComponents ask the storage handler for DDL_EXCLUSIVE
of non-native tables too, so the DDL is
rejected like DML; handlers overriding getLockType (Iceberg, Kafka) return
EXCLUSIVE for it
> Iceberg: exclusive DDL on a lock-enabled table waits on its own Hive lock
> under external locking
> -------------------------------------------------------------------------------------------------
>
> Key: HIVE-30235
> URL: https://issues.apache.org/jira/browse/HIVE-30235
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration, Locking
> Affects Versions: 4.2.0, 4.2.1
> Reporter: Denys Kuzmenko
> Priority: Major
>
> With hive.txn.ext.locking.enabled=true, HiveIcebergStorageHandler.getLockType
> rejects DML on a table with engine.hive.lock-enabled=true before any lock is
> taken. AcidUtils.makeLockComponents doesn't consult the storage handler for
> DDL_EXCLUSIVE, though, so DDL (ADD/REPLACE/CHANGE COLUMNS, UNSET
> TBLPROPERTIES, TRUNCATE, DROP PARTITION) takes its EXCLUSIVE Hive lock on
> db.table. The meta hook then requests the Iceberg commit lock on the same
> table (the same HMS resource since HIVE-30231) and waits on the statement's
> own lock until iceberg.hive.lock-timeout-ms (3 min), failing with "Timed out
> ... waiting for lock on <db>.<table>". Without the hook's lock, the Iceberg
> commit would wait the same way.
>
>
>
>
>
> Proposed: have makeLockComponents ask the storage handler for DDL_EXCLUSIVE
> of non-native tables too, so the DDL is rejected like DML; handlers
> overriding getLockType (Iceberg, Kafka) return EXCLUSIVE for it
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
