Viktor Csomor created HIVE-25746:
------------------------------------
Summary: Compaction Failure Counter counted incorrectly
Key: HIVE-25746
URL: https://issues.apache.org/jira/browse/HIVE-25746
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 4.0.0
Reporter: Viktor Csomor
Assignee: Viktor Csomor
The count of the below metrics counted incorrectly upon an exception.
- {{compaction_initator_failure_counter}}
- {{compaction_cleaner_failure_counter}}
Reasoning:
In the {{Initator}}/{{Cleaner}} class creates a list of {{CompletableFuture}}
which {{Runnable}} core exception is being wrapped to {{RuntimeExceptions}}.
The below code-snippet waits all cleaners to complete (Initiators does it
similarly).
{code:java}
try {
....
for (CompactionInfo compactionInfo : readyToClean) {
cleanerList.add(CompletableFuture.runAsync(CompactorUtil.ThrowingRunnable.unchecked(()
->
clean(compactionInfo, cleanerWaterMark, metricsEnabled)),
cleanerExecutor));
}
CompletableFuture.allOf(cleanerList.toArray(new
CompletableFuture[0])).join();
}
} catch (Throwable t) {
// the lock timeout on AUX lock, should be ignored.
if (metricsEnabled && handle != null) {
failuresCounter.inc();
}
{code}
If the {{CompleteableFututre#join}} throws an Exception then the failure
counter is incremented.
Let's suppose we have 10 cleaners and the 2nd throws an exception. The
{{catch}} block will be initiated and the {{failuresCounter}} will be
incremented. If there is any consecutive error amongst the remaining cleaners
the counter won't be incremented.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
