[
https://issues.apache.org/jira/browse/HIVE-26882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824352#comment-17824352
]
Rui Li commented on HIVE-26882:
-------------------------------
I run our test again, with MariaDB at SERIALIZABLE level and the results are
correct this time. So there must be something wrong with our previous test,
sorry about that :(
Also enabled audit log in MariaDB and verified the alter table process is
similar to the test in my last comment, and the underlying error for commit
conflict is:
{noformat}
Caused by: org.datanucleus.store.rdbms.exceptions.MappedDatastoreException:
UPDATE `TABLE_PARAMS` SET `PARAM_VALUE` = ? WHERE `TBL_ID`=? AND `PARAM_KEY`=?
at
org.datanucleus.store.rdbms.scostore.JoinMapStore.internalUpdate(JoinMapStore.java:1020)
at
org.datanucleus.store.rdbms.scostore.JoinMapStore.put(JoinMapStore.java:304)
... 39 more
Caused by: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException:
Deadlock found when trying to get lock; try restarting transaction
at
com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:123)
{noformat}
IMO {{REPEATABLE_READ}} cannot guarantee the semantics we need here. [~pvary]
do you think we can change the level to SERIALIZABLE? Or do you think MariaDB
doesn't implement {{REPEATABLE_READ}} correctly, in which case we might want to
use different levels for different DBMS vendor?
> Allow transactional check of Table parameter before altering the Table
> ----------------------------------------------------------------------
>
> Key: HIVE-26882
> URL: https://issues.apache.org/jira/browse/HIVE-26882
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Reporter: Peter Vary
> Assignee: Peter Vary
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.10, 4.0.0-beta-1
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> We should add the possibility to transactionally check if a Table parameter
> is changed before altering the table in the HMS.
> This would provide an alternative, less error-prone and faster way to commit
> an Iceberg table, as the Iceberg table currently needs to:
> - Create an exclusive lock
> - Get the table metadata to check if the current snapshot is not changed
> - Update the table metadata
> - Release the lock
> After the change these 4 HMS calls could be substituted with a single alter
> table call.
> Also we could avoid cases where the locks are left hanging by failed processes
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
