Thanks for your reply.
To clarify, do you mean that the problem is solved already because Hive
ACID looks at the '_orc_acid_version' in the directory before assuming a
directory is ready to read? Or do you mean that it *could have *looked at
the file before deciding to read it and that would have
>Oh but s3Guard will not solve the atomicity problem, right?
S3Guard does solve the atomicity problem, because compactors don't just rename
directories.
The basic consistency needed for ACID is - list after delete and list after
create (which S3 does not have).
They also place a file nam
Oh but s3Guard will not solve the atomicity problem, right?
Reference:
https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/bk_cloud-data-access/content/ch03s07s01.html
*What S3Guard Cannot Do*
*...*
*Mimic the "directory rename is a single atomic transaction" behavior of a
filesystem like H
>To me it looks like this problem will be solved by
>https://issues.apache.org/jira/browse/HIVE-20823, but until then, is this
>broken or I have missed a crucial detail?
Yes, S3Guard.
https://www.slideshare.net/hortonworks/s3guard-whats-in-your-consistency-model
However, that's anoth
Hi,
I have a question on using Hive ACID on Hive 3.x against cloud blob stores,
would be much obliged if someone could answer the same.
As I understand it, the results of a compaction(major or minor) need to be
atomically visible, so that when there are uncompacted and compacted
directories prese
