[
https://issues.apache.org/jira/browse/HIVE-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769209#comment-13769209
]
Roshan Naik commented on HIVE-5138:
-----------------------------------
Thanks [~ekoifman] for the comments:
h5. On Pt 1.
Thanks. I need to take a closer look at this.
h5. On Pt 2.
I think you mean 'safe to invoke concurrently' instead of 'atomic', since the
intermediate states are going to be visible when an operation spans both file
system and meta store. Here is a summary of the reasons why each operation is
concurrency safe:
- *streamingStatus* : Readonly metastore operation
- *chunkGet* : This is an atomic metastore operation
- *chunkAbort* : Just deletes a file. So no concurrency issues here.
- *chunkCommit* : Just renames a file. So only one of concurrent operations
will succeed.
- *disableStreaming* : This is an atomic metastore operation
- *enableStreaming* : Does a couple of mkdirs (for setup) followed by an
atomic metastore operation. mkdirs() is idempotent, so all concurrent calls
succeed. All concurrent invocations enter a transaction to do the metastore
update atomically...only one should update metastore.
- *partitionRoll* : Creates empty dir for the new current partition & then
atomically updates metastore as follows:
-# Make note of this new current partition dir
-# Do an addPartition() on the previous current partition.
- If concurrent partitionRoll() invocations use same arguments, the
addPartition() step will fail on all but one. If arguments are not same in
concurrent invocations, they all succeed and updates made by the last
invocation to exit the metastore transaction would override the others.
> Streaming - Web HCat API
> -------------------------
>
> Key: HIVE-5138
> URL: https://issues.apache.org/jira/browse/HIVE-5138
> Project: Hive
> Issue Type: Sub-task
> Components: HCatalog, Metastore, WebHCat
> Reporter: Roshan Naik
> Assignee: Roshan Naik
> Attachments: HIVE-4196.v2.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
