priyeshkaratha commented on code in PR #11307:
URL: https://github.com/apache/ozone/pull/11307#discussion_r4312642004
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/block/TestDeletedBlockLog.java:
##########
@@ -412,6 +412,51 @@ public void testCommitTransactions() throws Exception {
assertEquals(0, blocks.size());
}
+ @Test
+ public void testCommitTransactionsDefersPurgeForReplicaAddedDuringCommit()
Review Comment:
Added
TestKeyValueContainer#testExportCarriesOnlyDeletionsRecordedBeforeExport. A
copy exported before the deletion is recorded has no delete record and keeps
all its blocks. A copy exported after it carries the deletion.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/SCMDeletedBlockTransactionStatusManager.java:
##########
@@ -619,6 +623,18 @@ public void
commitTransactions(List<DeleteBlockTransactionResult> transactionRes
transactionResult.getTxID(), e);
}
}
+ // Re-check replica membership immediately before the durable removal. A
container
+ // copy (ReplicationManager) can add a new replica after the ack-set check
above but
+ // before this point; purging then would leave that replica's blocks
undeleted with
+ // no way to resend the transaction. Transactions that fail this recheck
stay tracked.
+ txIDsToBeDeleted.removeIf(txID -> !isSafeToPurge(txID,
purgeCandidateContainers.get(txID)));
Review Comment:
Thanks. Taking another snapshot isn't the fix. What prevents this is
ordering. A copy can only miss the deletion if it was exported before the
source recorded it, because export and recording take the same container lock
on the datanode. That copy was scheduled before the ACK, so its ADD is still
pending when A is checked and A is deferred. A copy scheduled later carries the
delete record, so it deletes the blocks itself. I've documented this in
commitTransactions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
