[
https://issues.apache.org/jira/browse/HIVE-26666?focusedWorklogId=823928&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-823928
]
ASF GitHub Bot logged work on HIVE-26666:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Nov/22 14:22
Start Date: 07/Nov/22 14:22
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3712:
URL: https://github.com/apache/hive/pull/3712#discussion_r1015436368
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactOperation.java:
##########
@@ -127,14 +128,10 @@ private void waitForCompactionToFinish(CompactionResponse
resp) throws HiveExcep
break;
}
- //this could be expensive when there are a lot of compactions....
- //todo: update to search by ID once HIVE-13353 is done
- ShowCompactResponse allCompactions = context.getDb().showCompactions();
+ ShowCompactRequest request = new ShowCompactRequest();
+ request.setId(resp.getId());
+ ShowCompactResponse allCompactions =
context.getDb().showCompactions(request);
for (ShowCompactResponseElement compaction :
allCompactions.getCompacts()) {
Review Comment:
Please remove the for loop. Since you searching by id you should rather
check if you have exactly one result and throw an exception (with the
appropriate error message) if not.
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactOperation.java:
##########
@@ -127,14 +128,10 @@ private void waitForCompactionToFinish(CompactionResponse
resp) throws HiveExcep
break;
}
- //this could be expensive when there are a lot of compactions....
- //todo: update to search by ID once HIVE-13353 is done
- ShowCompactResponse allCompactions = context.getDb().showCompactions();
+ ShowCompactRequest request = new ShowCompactRequest();
+ request.setId(resp.getId());
+ ShowCompactResponse allCompactions =
context.getDb().showCompactions(request);
Review Comment:
Please rename the variable since it is no longer holding all the compactions.
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java:
##########
@@ -345,23 +344,17 @@ private void recoverFailedCompactions(boolean remoteOnly)
throws MetaException {
HiveConf.ConfVars.HIVE_COMPACTOR_WORKER_TIMEOUT,
TimeUnit.MILLISECONDS));
}
- private boolean foundCurrentOrFailedCompactions(ShowCompactResponse
compactions, CompactionInfo ci) throws MetaException {
- if (compactions.getCompacts() == null) {
+ private boolean foundCurrentOrFailedCompactions(CompactionInfo ci) throws
MetaException {
+
+ ShowCompactRequest request = new ShowCompactRequest();
+ request.setDbname(ci.dbname);
+ request.setTablename(ci.tableName);
+ request.setPartitionname(ci.partName);
+ final ShowCompactResponse currentCompactions =
txnHandler.showCompact(request);
Review Comment:
I'm not sure this will be faster. Now you are filtering by DB, table, and
partition, but on the other hand executing showcompact multiple times which has
both serious DB and network (thrift) overhead. Did you do any measurements to
check if this improves performance or not?
Issue Time Tracking
-------------------
Worklog Id: (was: 823928)
Time Spent: 1h 20m (was: 1h 10m)
> Filter out compactions by id to minimise expense of db operations
> ------------------------------------------------------------------
>
> Key: HIVE-26666
> URL: https://issues.apache.org/jira/browse/HIVE-26666
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 4.0.0
> Reporter: KIRTI RUGE
> Assignee: KIRTI RUGE
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> At present we use below operations while filtering out compactions in classes
> like
> AlterTableCompactOperation
> cleaner
> Use show compaction filter option provided afterÂ
> https://issues.apache.org/jira/browse/HIVE-13353
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
