Re: Odp: Re: hive 4.0.0-alpha2 external table locks

DbTxnManager doesn't have support for explicit locking. You could use HMSClient to issue an explicit lock request or create a lock record manually in the backend DB `HIVE_LOCKS` table. Maybe you could temporarily restrict access to those tables via Ranger or drop them from the HMS while performi

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

Any other options with DbTxnManager to prevent the access from hive queries to files on hdfs while performing maintenance of external tables ?

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

Unfortunately no. DummyTxnManager doesn't support DB locks, only Zookeeper and in-memory. So you won't be able to transparently lock the external table from other non-DummyTxtManager's.

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

Would it force non-DummyTxtManager instances to not access external tables during exclusive locks? Dnia 1 lutego 2023 15:55 Denys Kuzmenko napisał(a): DbTxnManager comes with DbLockManager implementation that doesn't support explicit lock/unlock functionality. That's

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

DbTxnManager comes with DbLockManager implementation that doesn't support explicit lock/unlock functionality. It was designed to be used only for ACID tables, however, due to code refactor challenges is reused for external tables as well. This comes with few drawbacks like suboptimal behavior (

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

We are using external tables on early stages of ETL. Next we switch to managed tables as we don't need to worry about partition management. With DummyTxnManager we had problems dealing with managed tables. Regards, Wojtek Dnia 1 lutego 2023 10:31 Denys Kuzmenko nap

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

If you are using external tables only, you could try switching to DummyTxnManager. It supports explicit lock requests.

Odp: Re: hive 4.0.0-alpha2 external table locks

My settings regarding transactions are: hive.txn.acid.dir.cache.duration=120   | | hive.txn.acid.metrics.delta.num.threshold=100  | | hive.txn.acid.metrics.delta.pct.threshold=0.01 | | hive.txn.acid.metrics.max.cache.size=100   | | hive.txn.acid.metrics.obsolet

Re: hive 4.0.0-alpha2 external table locks

Hi Wojtek Meler, Have you tried setting the below config before running the queries? > set hive.txn.ext.locking.enabled=true On Fri, Jan 27, 2023 at 1:48 PM Wojtek Meler wrote: > I've enabled external tables locking with [HIVE-26258] Provide an option > for enable locking of external tables -

hive 4.0.0-alpha2 external table locks

I've enabled external tables locking with  issues.apache.org [HIVE-26258] Provide an option for enable locking of external tables - ASF JIRA (apache.org)  but still locking with HQL is not posible: org.apache.hadoop.hive.ql.metadata.HiveException: Current transaction manager does not support