[
https://issues.apache.org/jira/browse/HIVE-12353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110009#comment-15110009
]
Eugene Koifman commented on HIVE-12353:
---------------------------------------
bq. info.highestTxnId = rs.wasNull() ? null : highestTxnId;
ResultSet.getLong() returns 0 when value was actually NULL so I thought it
would be more consistent with other places in the code that use 0 as invalid
txn id
bq. PreparedStatement
cc_meta_info is a binary field - I don't think there is way to set it w/o
prepared statement. (Even though it isn't set yet but will be soon)
bq. A concern on the history stuff
my thought here was that if I'm looking at 10 failed compactions, it would be
useful for me to know when the last successful run was. Allowing retention
value to be set per type allows that.
bq. checkFailedCompactions
The idea in this loop is to look at the last "failedThreshold" elements. If
they are all "failed" - don't start another one. If not all "failed" - that's
the intervening success.
bq. It also seems users could unwittingly mess things
Yes, that is why there is CompactionTxnHandler.getFailedCompactionRetention()
to make sure this doesn't happen
> When Compactor fails it calls CompactionTxnHandler.markedCleaned(). it
> should not.
> -----------------------------------------------------------------------------------
>
> Key: HIVE-12353
> URL: https://issues.apache.org/jira/browse/HIVE-12353
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 1.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Blocker
> Attachments: HIVE-12353.2.patch, HIVE-12353.3.patch,
> HIVE-12353.4.patch, HIVE-12353.6.patch, HIVE-12353.7.patch, HIVE-12353.patch
>
>
> One of the things that this method does is delete entries from TXN_COMPONENTS
> for partition that it was trying to compact.
> This causes Aborted transactions in TXNS to become empty according to
> CompactionTxnHandler.cleanEmptyAbortedTxns() which means they can now be
> deleted.
> Once they are deleted, data that belongs to these txns is deemed committed...
> We should extend COMPACTION_QUEUE state with 'f' and 's' (failed, success)
> states. We should also not delete then entry from markedCleaned()
> We'll have separate process that cleans 'f' and 's' records after X minutes
> (or after > N records for a given partition exist).
> This allows SHOW COMPACTIONS to show some history info and how many times
> compaction failed on a given partition (subject to retention interval) so
> that we don't have to call markCleaned() on Compactor failures at the same
> time preventing Compactor to constantly getting stuck on the same bad
> partition/table.
> Ideally we'd want to include END_TIME field.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
