[
https://issues.apache.org/jira/browse/HIVE-26471?focusedWorklogId=815511&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-815511
]
ASF GitHub Bot logged work on HIVE-26471:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Oct/22 08:07
Start Date: 11/Oct/22 08:07
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3645:
URL: https://github.com/apache/hive/pull/3645#discussion_r991965326
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metrics/CompactionMetricData.java:
##########
@@ -67,34 +70,53 @@ static CompactionMetricData
of(List<ShowCompactResponseElement> compacts) {
private void init() {
final Map<String, ShowCompactResponseElement> lastElements = new
HashMap<>();
- poolCount = new HashMap<>();
+ initiatedPoolCount = new HashMap<>();
+ workingPoolCount = new HashMap<>();
+ oldestEnqueueTimePerPool = new HashMap<>();
+ oldestWorkingTimePerPool = new HashMap<>();
oldestEnqueueTime = OLDEST_TIME_NO_VALUE;
oldestWorkingTime = OLDEST_TIME_NO_VALUE;
oldestCleaningTime = OLDEST_TIME_NO_VALUE;
+
+ long currentTime = System.currentTimeMillis();
for (ShowCompactResponseElement element : compacts) {
final String key = element.getDbname() + "/" + element.getTablename() +
(element.getPartitionname() != null ? "/" +
element.getPartitionname() : "");
// If new key, add the element, if there is an existing one, change to
the element if the element.id is greater than old.id
lastElements.compute(key, (k, old) -> (old == null) ? element :
(element.getId() > old.getId() ? element : old));
- // find the oldest elements with initiated and working states
String state = element.getState();
- if (TxnStore.INITIATED_RESPONSE.equals(state) && (oldestEnqueueTime >
element.getEnqueueTime())) {
- oldestEnqueueTime = element.getEnqueueTime();
- poolCount.compute(element.getPoolName(), (k, old) -> (old == null) ? 1
: old + 1);
+ if (TxnStore.INITIATED_RESPONSE.equals(state)) {
+ final int enqueueSeconds = (int) ((currentTime -
element.getEnqueueTime()) / 1000);
+ oldestWorkingTimePerPool.compute(element.getPoolName(), (k, old) ->
(old == null) ? enqueueSeconds : Math.max(enqueueSeconds, old));
Review Comment:
good catch, I accidentally swapped the variables. I also changed the names
to longestEnqueueDurationPerPool and longestWorkingDurationPerPool (+ getters).
Issue Time Tracking
-------------------
Worklog Id: (was: 815511)
Time Spent: 4.5h (was: 4h 20m)
> New metric for Compaction pooling
> ---------------------------------
>
> Key: HIVE-26471
> URL: https://issues.apache.org/jira/browse/HIVE-26471
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: László Végh
> Assignee: László Végh
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> To be able to properly supervise the pool based compaction, a new metric is
> required:
> Number of 'Initiated' compaction requests per compaction pool.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
