[
https://issues.apache.org/jira/browse/HIVE-25740?focusedWorklogId=688576&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-688576
]
ASF GitHub Bot logged work on HIVE-25740:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Dec/21 10:45
Start Date: 01/Dec/21 10:45
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2817:
URL: https://github.com/apache/hive/pull/2817#discussion_r760063403
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java
##########
@@ -233,39 +234,36 @@ void gatherStats() {
private final CompactionTxn compactionTxn;
private final String tableName;
private final HiveConf conf;
- private final long txnTimeout;
+ private final AtomicBoolean shouldLogError;
public CompactionHeartbeater(CompactionTxn compactionTxn, String
tableName, HiveConf conf) {
this.tableName = Objects.requireNonNull(tableName);
this.compactionTxn = Objects.requireNonNull(compactionTxn);
this.conf = Objects.requireNonNull(conf);
-
- this.txnTimeout = MetastoreConf.getTimeVar(conf,
MetastoreConf.ConfVars.TXN_TIMEOUT, TimeUnit.MILLISECONDS);
+ this.shouldLogError = new AtomicBoolean(true);
setDaemon(true);
setPriority(MIN_PRIORITY);
setName("CompactionHeartbeater-" + compactionTxn.getTxnId());
}
+ public void shouldLogError(boolean shouldLogError) {
+ this.shouldLogError.set(shouldLogError);
+ }
+
@Override
public void run() {
LOG.debug("Heartbeating compaction transaction id {} for table: {}",
compactionTxn, tableName);
-
IMetaStoreClient msc = null;
try {
// Create a metastore client for each thread since it is not thread
safe
msc = HiveMetaStoreUtils.getHiveMetastoreClient(conf);
Review comment:
That makes sense, and I was debating that internally as well. However,
it would complicate the cleanup of the thread quite a bit. Since the thread
runs infrequently (every 2.5 minutes or so) I think the cost of client creation
should be pretty low, and not worth the extra complexity of storing it in a
field and worry about clean up in the various scenarios. Let me know if that
makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 688576)
Time Spent: 2h (was: 1h 50m)
> Handle race condition between compaction txn abort/commit and heartbeater
> -------------------------------------------------------------------------
>
> Key: HIVE-25740
> URL: https://issues.apache.org/jira/browse/HIVE-25740
> Project: Hive
> Issue Type: Bug
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> This issue is the following: once the compaction worker finishes,
> commitTxn/abortTxn is invoked first, and the heartbeater thread is only
> interrupted after that. This can lead to race conditions where the txn has
> already been deleted from the backend DB via commit/abort, but the
> concurrently running heartbeater thread still attempts to send a last
> heartbeat after that, but the txn id won't be found in the DB, leading to
> {{{}NoSuchTxnException{}}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
