Shushant - What I believe what Stephen is sarcastically trying to say is that some organizational education may be in order here. Hive itself is not even at version 1.0, those of us who use Hive in production know this, and have to accept that there will be bugs like the one you are trying to address. There MAY be a workaround, that takes more hours and introduces other bugs into your environment, alternatively, taking the time to explain why moving forward form Hive 0.10 to Hive 0.14 really is in the best interest of your organization. Perhaps there can be a way where you can do a proof of concept using Hive 0.14, i.e. copy the metastore to another SQL server, and try moving the data the table to another location so you can prove out the fix of your issue. Also, perhaps there can be a way to test the current workflows that work on 0.10 in 0.14 so you can show that this change really is right way to move.
Being at this level in an open source project has huge benefits, but challenges as well. On one hand you can be much more nimble in your environment because open source is fluid, but if you are trying to do this within an environment that doesn't allow you to move like you need, it may be losing a long term war while winning short term battles. I guess, what I am saying is the similar to Stephen, but I highly recommend you work with team that sets the policy and develop a new way to address how Hive and other similar projects live within your change management policies. You will benefit greatly in the long run. John On Sun, Sep 21, 2014 at 1:26 AM, Shushant Arora <shushantaror...@gmail.com> wrote: > Hi Stephen > > We have cloudera setup deployed in our cluster, which we cannot update due > to orgs policy. > Till the time its not updated to version 0.14, How can I achieve the > locking feature please suggest. > > > On Sun, Sep 21, 2014 at 10:40 AM, Stephen Sprague <sprag...@gmail.com> > wrote: > >> great policy. install open source software that's not even version 1.0 >> into production and then not allow the ability to improve it (but of course >> reap all the rewards of its benefits.) so instead of actually fixing the >> problem the right way introduce a super-hack work-around cuz, you know, >> that's much more "stable." >> >> Gotta luv it. Good luck. >> >> On Sat, Sep 20, 2014 at 8:00 AM, Shushant Arora < >> shushantaror...@gmail.com> wrote: >> >>> Hi Alan >>> >>> I have 0.10 version of hive deployed in my org's cluster, I cannot >>> update that because of org's policy. >>> How can I achieve exclusive lock functionality while inserting in >>> dynamic partition on hive 0.10 ? >>> Does calling hive scripts via some sort of java api with patched jar >>> included will help ? >>> Moreover hive does not release locks in 0.10 when hive session is killed >>> . User has to explicitly unlock a table. >>> Can i specify any sort of max expiry time while taking a lock. >>> >>> Thanks >>> Shushant >>> >>> On Sat, Sep 20, 2014 at 8:11 PM, Alan Gates <ga...@hortonworks.com> >>> wrote: >>> >>>> Up until Hive 0.13 locks in Hive were really advisory only, since as >>>> you note any user can remove any other user's lock. In Hive 0.13 a new >>>> type of locking was introduced, see >>>> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-LockManager >>>> This new locking is automatic and ignores both LOCK and UNLOCK commands. >>>> Note that it is off by default, you have to configure Hive to use the new >>>> DbTxnManager to get turn on this locking. In 0.13 it still has the bug you >>>> describe as far as acquiring the wrong lock for dynamic partitioning, but I >>>> believe I've fixed that in 0.14. >>>> >>>> Alan. >>>> >>>> Shushant Arora <shushantaror...@gmail.com> >>>> September 20, 2014 at 5:39 >>>> >>>> Hive version 0.9 and later has a bug >>>> >>>> >>>> >>>> While inserting in a hive table Hive takes an exclusive lock. But if >>>> table is partitioned , and insert is in dynamic partition , it will take >>>> shared lock on table but if all partitions are static then hive takes >>>> exclusive lock on partitions in which data is being inserted >>>> >>>> and shared lock on table. >>>> >>>> https://issues.apache.org/jira/browse/HIVE-3509 >>>> >>>> >>>> 1.What if I want to take exclusive lock on table while inserting in >>>> dynamic partition ? >>>> >>>> >>>> I tried to take explicit lock using : >>>> >>>> LOCK TABLE tablename EXCLUSIVE; >>>> >>>> >>>> But it made table to be disabled. >>>> >>>> I cannot even read from table anymore even is same session until I do >>>> >>>> unlock table tablename in another session; >>>> >>>> >>>> 2. moreover whats lock level in hive , I mean any user can remove any >>>> other users lock. that too seems buggy. >>>> >>>> >>>> Thanks >>>> >>>> Shushant >>>> >>>> >>>> >>>> >>>> -- >>>> Sent with Postbox <http://www.getpostbox.com> >>>> >>>> CONFIDENTIALITY NOTICE >>>> NOTICE: This message is intended for the use of the individual or >>>> entity to which it is addressed and may contain information that is >>>> confidential, privileged and exempt from disclosure under applicable law. >>>> If the reader of this message is not the intended recipient, you are hereby >>>> notified that any printing, copying, dissemination, distribution, >>>> disclosure or forwarding of this communication is strictly prohibited. If >>>> you have received this communication in error, please contact the sender >>>> immediately and delete it from your system. Thank You. >>> >>> >>> >> >
