Shushant,
Creating a patched jar that would include the lock functionality you
want is unlikely to work. Wouldn't the following workflow work for you:
1. Writer locks the table explicitly via LOCK TABLE
2. Writer inserts
3. Writer unlocks the table explicitly via UNLOCK TABLE
If you're using ZK for your locking I think the client dying (as opposed
to ending the session) should cause the lock to expire. If not, you may
have to assure the unlock happens in your application. Hope that helps.
Alan.
Shushant Arora <mailto:shushantaror...@gmail.com>
September 20, 2014 at 8:00
Hi Alan
I have 0.10 version of hive deployed in my org's cluster, I cannot
update that because of org's policy.
How can I achieve exclusive lock functionality while inserting in
dynamic partition on hive 0.10 ?
Does calling hive scripts via some sort of java api with patched jar
included will help ?
Moreover hive does not release locks in 0.10 when hive session is
killed . User has to explicitly unlock a table.
Can i specify any sort of max expiry time while taking a lock.
Thanks
Shushant
Alan Gates <mailto:ga...@hortonworks.com>
September 20, 2014 at 7:41
Up until Hive 0.13 locks in Hive were really advisory only, since as
you note any user can remove any other user's lock. In Hive 0.13 a
new type of locking was introduced, see
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-LockManager
This new locking is automatic and ignores both LOCK and UNLOCK
commands. Note that it is off by default, you have to configure Hive
to use the new DbTxnManager to get turn on this locking. In 0.13 it
still has the bug you describe as far as acquiring the wrong lock for
dynamic partitioning, but I believe I've fixed that in 0.14.
Alan.
Shushant Arora <mailto:shushantaror...@gmail.com>
September 20, 2014 at 5:39
Hive version 0.9 and later has a bug
While inserting in a hive table Hive takes an exclusive lock. But if
table is partitioned , and insert is in dynamic partition , it will
take shared lock on table but if all partitions are static then hive
takes exclusive lock on partitions in which data is being inserted
and shared lock on table.
https://issues.apache.org/jira/browse/HIVE-3509
1.What if I want to take exclusive lock on table while inserting in
dynamic partition ?
I tried to take explicit lock using :
LOCK TABLE tablename EXCLUSIVE;
But it made table to be disabled.
I cannot even read from table anymore even is same session until I do
unlock table tablename in another session;
2. moreover whats lock level in hive , I mean any user can remove any
other users lock. that too seems buggy.
Thanks
Shushant
--
Sent with Postbox <http://www.getpostbox.com>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
