[
https://issues.apache.org/jira/browse/HIVE-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527166#comment-15527166
]
Eugene Koifman commented on HIVE-14778:
---------------------------------------
It depends on what you mean by multiple threads. TransactionBatches are made
from StreamingConnection objects. So the model is that if you want to write to
the same HiveEndPoint in parallel, you create different StreamingConnection
objects but from any given StreamingConnection you open/close
TransactionBatches sequentially. (The exception is that you can heartbeat any
given TransactionBatch using a separate thread).
This seems like a reasonable model. For example, JDBC (usually) has the same
model. You can create any number of connections but operations on a given
Connection are expected to be sequential.
> document threading model of Streaming API
> -----------------------------------------
>
> Key: HIVE-14778
> URL: https://issues.apache.org/jira/browse/HIVE-14778
> Project: Hive
> Issue Type: Bug
> Components: HCatalog, Transactions
> Affects Versions: 0.14.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Attachments: HIVE-14778.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The model is not obvious and needs to be documented properly.
> A StreamingConnection internally maintains 2 MetaStoreClient objects (each
> has 1 Thrift client for actual RPC). Let's call them "primary" and
> "heartbeat". Each TransactionBatch created from a given StreamingConnection,
> gets a reference to both of these MetaStoreClients.
> So the model is that there is at most 1 outstanding (not closed)
> TransactionBatch for any given StreamingConnection and for any given
> TransactionBatch there can be at most 2 threads accessing it concurrently. 1
> thread calling TransactionBatch.heartbeat() (and nothing else) and the other
> calling all other methods.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
