[
https://issues.apache.org/jira/browse/HIVE-26026?focusedWorklogId=747232&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-747232
]
ASF GitHub Bot logged work on HIVE-26026:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Mar/22 13:38
Start Date: 24/Mar/22 13:38
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on a change in pull request #3126:
URL: https://github.com/apache/hive/pull/3126#discussion_r834237496
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java
##########
@@ -442,31 +439,24 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
// Don't start compaction or cleaning if not necessary
if (isDynPartAbort(t, ci)) {
- msc.markCompacted(CompactionInfo.compactionInfoToStruct(ci));
+ markWithState(ci, null, CompactionState.Compacted);
compactionTxn.wasSuccessful();
return false;
}
AcidDirectory dir = getAcidStateForWorker(ci, sd, tblValidWriteIds);
if (!isEnoughToCompact(ci.isMajorCompaction(), dir, sd)) {
if (needsCleaning(dir, sd)) {
- msc.markCompacted(CompactionInfo.compactionInfoToStruct(ci));
+ markWithState(ci, null, CompactionState.Compacted);
} else {
- // do nothing
- msc.markCleaned(CompactionInfo.compactionInfoToStruct(ci));
+ markWithState(ci, "None of the compaction thresholds hit, compaction
request is refused!", CompactionState.Refused);
Review comment:
Since the compaction will be rescheduled when the table changes, REFUSED
can be thought of 'The current state of the table is refused.' The main
difference is whether we want to keep the original request and process it once
possible, or just refuse it and process request only which fulfill all criteria
at the time of checking.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java
##########
@@ -442,31 +439,24 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
// Don't start compaction or cleaning if not necessary
if (isDynPartAbort(t, ci)) {
- msc.markCompacted(CompactionInfo.compactionInfoToStruct(ci));
+ markWithState(ci, null, CompactionState.Compacted);
compactionTxn.wasSuccessful();
return false;
}
AcidDirectory dir = getAcidStateForWorker(ci, sd, tblValidWriteIds);
if (!isEnoughToCompact(ci.isMajorCompaction(), dir, sd)) {
if (needsCleaning(dir, sd)) {
- msc.markCompacted(CompactionInfo.compactionInfoToStruct(ci));
+ markWithState(ci, null, CompactionState.Compacted);
} else {
- // do nothing
- msc.markCleaned(CompactionInfo.compactionInfoToStruct(ci));
+ markWithState(ci, "None of the compaction thresholds hit, compaction
request is refused!", CompactionState.Refused);
Review comment:
Since the compaction will be rescheduled when the table changes, REFUSED
can be thought of 'The current state of the table is refused.' The main
difference is whether we want to keep the original request and process it once
possible, or just refuse it and process only those requests which fulfill all
criterias at the time of checking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 747232)
Time Spent: 5h 50m (was: 5h 40m)
> Use the new "REFUSED" compaction state where it makes sense
> -----------------------------------------------------------
>
> Key: HIVE-26026
> URL: https://issues.apache.org/jira/browse/HIVE-26026
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: László Végh
> Assignee: László Végh
> Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> The
> org.apache.hadoop.hive.ql.txn.compactor.Worker#findNextCompactionAndExecute
> method does several checks (The table/partition exists, is not sorted, there
> are enough files to compact, etc.) before it actually executes the compaction
> request. If the compaction request fails on any of these checks, it is put to
> "SUCCEEDED" state which is often misleading for users. SHOW COMPACTIONS will
> show these requests as succeeded without an error, while the table is not
> compacted at all.
> For these cases, the state should be "REFUSED" instead of "SUCCEEDED" among
> with the appropriate error message.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
