[
https://issues.apache.org/jira/browse/HIVE-26265?focusedWorklogId=783601&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-783601
]
ASF GitHub Bot logged work on HIVE-26265:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Jun/22 22:49
Start Date: 21/Jun/22 22:49
Worklog Time Spent: 10m
Work Description: cmunkey commented on code in PR #3365:
URL: https://github.com/apache/hive/pull/3365#discussion_r903131112
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/events/AbortTxnHandler.java:
##########
@@ -39,6 +48,19 @@ public void handle(Context withinContext) throws Exception {
if (!ReplUtils.includeAcidTableInDump(withinContext.hiveConf)) {
return;
}
+
+ if (ReplUtils.filterTransactionOperations(withinContext.hiveConf)) {
+ String contextDbName =
StringUtils.normalizeIdentifier(withinContext.replScope.getDbName());
+ GetTxnWriteIdsRequest request = new
GetTxnWriteIdsRequest(eventMessage.getTxnId());
+ request.setDbName(contextDbName);
+ GetTxnWriteIdsResponse response =
withinContext.db.getMSC().getTxnWriteIds(request);
Review Comment:
Ok, I changed how this is done.
TxnHandler.abortTxn() will check if the txn had allocated writeids, and
attach the databases to the AbortTxnEvent. During REPL DUMP, when processing
the AbortTxnEvent, add the event to the dump only if the AbortTxnEvent affected
the database being dumped.
Issue Time Tracking
-------------------
Worklog Id: (was: 783601)
Time Spent: 1h (was: 50m)
> REPL DUMP should filter out OpenXacts and unneeded CommitXact/Abort.
> --------------------------------------------------------------------
>
> Key: HIVE-26265
> URL: https://issues.apache.org/jira/browse/HIVE-26265
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Reporter: francis pang
> Assignee: francis pang
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> REPL DUMP is replication all OpenXacts, even when they are from other non
> replicated databases. This wastes space in the dump, and ends up opening
> unneeded transactions during REPL LOAD.
>
> Add a config property for replication that filters out OpenXact events during
> REPL DUMP. During REPL LOAD, the txns can be implicitly opened when the
> ALLOC_WRITE_ID is processed. For CommitTxn and AbortTxn, dump only if WRITE
> ID was allocated.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
