[
https://issues.apache.org/jira/browse/HIVE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207696#comment-15207696
]
Eugene Koifman commented on HIVE-11388:
---------------------------------------
1. The purpose is to run Select For Update. So if the key is already there,
the 1st "rs = stmt.executeQuery(sqlStmt);" will do it.
2. we do need this. Since you may have several Cleaner processes running, they
will each accumulate state in these data structures. But you don't know which
instance will end up actually cleaning files so if you remove data from these
structures you'll have a memory leak.
3. What would that confirm? If the counts are off at this point, it means the
2nd thread somehow ran ahead and thus it will see its counts being different.
> Allow ACID Compactor components to run in multiple metastores
> -------------------------------------------------------------
>
> Key: HIVE-11388
> URL: https://issues.apache.org/jira/browse/HIVE-11388
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 1.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
> Attachments: HIVE-11388.2.patch, HIVE-11388.4.patch,
> HIVE-11388.5.patch, HIVE-11388.6.patch, HIVE-11388.7.patch, HIVE-11388.patch
>
>
> (this description is no loner accurate; see further comments)
> org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs
> inside the metastore service to manage compactions of ACID tables. There
> should be exactly 1 instance of this thread (even with multiple Thrift
> services).
> This is documented in
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
> but not enforced.
> Should add enforcement, since more than 1 Initiator could cause concurrent
> attempts to compact the same table/partition - which will not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
