Hi Mark It’s my understanding that when you do an INSERT OVERWRITE into a partition, Hive will take out an exclusive lock on the partition and a shared lock on the table itself. This blocks are read and write operations on the partition, and allows reads against the other partitions to proceed.
I am assuming that you have hive.txn.strict.locking.mode set to its default value of true, and the table is non-ACID. With hive.txn.strict.locking.mode=false and a non-ACID table, then the lock is shared, i.e. concurrent writes to the same table are allowed. Hopefully that is this information you were looking for? Apologies if not. Thanks Austin > On 11 Jan 2021, at 16:44, Mark Norkin <markinno...@gmail.com> wrote: > > Hello Hive users, > > We are using AWS Glue as Hive compatible metastore when running queries on > EMR. For Hive external tables we are using AWS S3. > > After looking at the docs we didn't find a conclusive answer on whether an > Insert Overwrite table partition is an atomic operation, maybe we've missed > it and it is documented somewhere, or maybe someone knows from their > experience? > > If it's an atomic operation, is there any difference whether the table is > external or a managed one? > > Thank you, > > Mark
