keith-turner commented on code in PR #5341:
URL: https://github.com/apache/accumulo/pull/5341#discussion_r2010801428
##########
server/manager/src/main/java/org/apache/accumulo/manager/tableOps/bulkVer2/LoadFiles.java:
##########
@@ -366,15 +369,35 @@ static long loadFiles(Loader loader, BulkInfo bulkInfo,
Path bulkDir,
ImportTimingStats importTimingStats = new ImportTimingStats();
Timer timer = Timer.startNew();
- try (TabletsMetadata tabletsMetadata =
factory.newTabletsMetadata(startRow)) {
+ KeyExtent prevLastExtent = null; // KeyExtent of last tablet from prior
loadMapEntry
+ TabletsMetadata tabletsMetadata = factory.newTabletsMetadata(startRow);
+ try {
Iterator<TabletMetadata> tabletIter = tabletsMetadata.iterator();
while (lmi.hasNext()) {
loadMapEntry = lmi.next();
+ // If the user set the TABLE_BULK_SKIP_THRESHOLD property, then only
look
+ // at the next skipDistance tablets before recreating the iterator
+ if (skipDistance > 0) {
+ final KeyExtent loadMapKey = loadMapEntry.getKey();
+ if (prevLastExtent != null &&
!loadMapKey.isPreviousExtent(prevLastExtent)) {
+ final KeyExtent search = prevLastExtent;
+ SearchResults results =
+ lmi.advanceTo(e -> e.getKey().isPreviousExtent(search),
skipDistance);
Review Comment:
Seems like this should advance the tablet iterator and not the load mapping
iterator. Would need to wrap the tablet iterator w/ a PeekingIterator.
```suggestion
SearchResults results =
tabletIter.advanceTo(currTablet ->
PREV_COMP.compare(currTablet.getPrevEndRow(), loadMapKey.prevEndRow()) < 0,
skipDistance);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
