[
https://issues.apache.org/jira/browse/HIVE-25746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage resolved HIVE-25746.
----------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Committed to master branch. Thanks for your contribution [~vcsomor]!
> Compaction Failure Counter counted incorrectly
> ----------------------------------------------
>
> Key: HIVE-25746
> URL: https://issues.apache.org/jira/browse/HIVE-25746
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 4.0.0
> Reporter: Viktor Csomor
> Assignee: Viktor Csomor
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The count of the below metrics counted incorrectly upon an exception.
> - {{compaction_initator_failure_counter}}
> - {{compaction_cleaner_failure_counter}}
> Reasoning:
> In the {{Initator}}/{{Cleaner}} class creates a list of {{CompletableFuture}}
> which {{Runnable}} core exception is being wrapped to {{RuntimeExceptions}}.
> The below code-snippet waits all cleaners to complete (Initiators does it
> similarly).
> {code:java}
> try {
> ....
> for (CompactionInfo compactionInfo : readyToClean) {
>
> cleanerList.add(CompletableFuture.runAsync(CompactorUtil.ThrowingRunnable.unchecked(()
> ->
> clean(compactionInfo, cleanerWaterMark,
> metricsEnabled)), cleanerExecutor));
> }
> CompletableFuture.allOf(cleanerList.toArray(new
> CompletableFuture[0])).join();
> }
> } catch (Throwable t) {
> // the lock timeout on AUX lock, should be ignored.
> if (metricsEnabled && handle != null) {
> failuresCounter.inc();
> }
> {code}
> If the {{CompleteableFututre#join}} throws an Exception then the failure
> counter is incremented.
> Docs:
> {code}
> /**
> * Returns the result value when complete, or throws an
> * (unchecked) exception if completed exceptionally. To better
> * conform with the use of common functional forms, if a
> * computation involved in the completion of this
> * CompletableFuture threw an exception, this method throws an
> * (unchecked) {@link CompletionException} with the underlying
> * exception as its cause.
> *
> * @return the result value
> * @throws CancellationException if the computation was cancelled
> * @throws CompletionException if this future completed
> * exceptionally or a completion computation threw an exception
> */
> public T join() {
> Object r;
> return reportJoin((r = result) == null ? waitingGet(false) : r);
> }
> {code}
> (!) Let's suppose we have 10 cleaners and the 2nd throws an exception. The
> {{catch}} block will be initiated and the {{failuresCounter}} will be
> incremented. If there is any consecutive error amongst the remaining cleaners
> the counter won't be incremented.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
