[
https://issues.apache.org/jira/browse/HIVE-30026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-30026:
----------------------------------
Labels: pull-request-available (was: )
> (ACID Compaction) Extend the query for finding potential compactions with
> ordering
> ----------------------------------------------------------------------------------
>
> Key: HIVE-30026
> URL: https://issues.apache.org/jira/browse/HIVE-30026
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Marta Kuczora
> Assignee: Marta Kuczora
> Priority: Major
> Labels: pull-request-available
>
> After HIVE-27444, if the number of candidates is bigger than the
> compactor.fetch.size value, it can happen that the Initiator doesn't schedule
> all tables for compaction.
> For example: we have 4 tables, which are all eligible for compaction, but the
> fetch size is 2.
> In the first initiator run, it will pick two of these tables. It would be
> expected that during the second run of the initiator it would pick the other
> two tables, so all 4 tables have been scheduled for compaction.
> Since there is no ordering when selecting the tables for compaction, this was
> not always the case. It could happen that some tables never got compacted.
>
> Also if the number of candidates is bigger than the compactor.fetch.size
> limit, the aborted txn candidates will never be picked up by the initiator.
> Because the "remaining budget" will always be 0 in the
> FindPotentialCompactionsFunction.
> {code:java}
> Set<CompactionInfo> candidates = new HashSet<>(jdbcResource.execute(new
> CompactionCandidateHandler(lastChecked, fetchSize)));
> int remaining = fetchSize - candidates.size();
> if (collectAbortedTxns && remaining > 0) {
> candidates.addAll(jdbcResource.execute(new
> AbortedTxnHandler(abortedTimeThreshold, abortedThreshold, remaining)));
> }{code}
> This affects only the cases when the aborted txn clean-up is done by the
> compaction cycle and not just the cleaner.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
