taklwu commented on code in PR #7222:
URL: https://github.com/apache/hbase/pull/7222#discussion_r2274529332
##########
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/replication/ContinuousBackupReplicationEndpoint.java:
##########
@@ -298,11 +301,21 @@ private void backupWalEntries(long day, List<WAL.Entry>
walEntries) throws IOExc
try {
FSHLogProvider.Writer walWriter = walWriters.computeIfAbsent(day,
this::createWalWriter);
+ List<Path> bulkLoadFiles =
BulkLoadProcessor.processBulkLoadFiles(walEntries);
+
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("{} Processed {} bulk load files for WAL entries",
Utils.logPeerId(peerId),
+ bulkLoadFiles.size());
+ LOG.trace("{} Bulk load files: {}", Utils.logPeerId(peerId),
+
bulkLoadFiles.stream().map(Path::toString).collect(Collectors.joining(", ")));
+ }
for (WAL.Entry entry : walEntries) {
walWriter.append(entry);
}
+
walWriter.sync(true);
+ uploadBulkLoadFiles(day, bulkLoadFiles);
Review Comment:
I put my questions in the JIRA, let me
[link](https://issues.apache.org/jira/browse/HBASE-29519?focusedCommentId=18013771&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-18013771)
here as well, it's mainly about if we do any async upload or upload a list of
files at the end of the WAL closing, we will need to be very careful and may
have a different implementation. so having current approach may be the best for
the moment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
