snleee commented on code in PR #11537:
URL: https://github.com/apache/pinot/pull/11537#discussion_r1318893468
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java:
##########
@@ -131,13 +130,19 @@ public void process()
}
}
- // Create/modify/remove star-trees if required.
- processStarTrees(indexDir);
-
- // Perform post-cleanup operations on the index handlers. This should be
called after processing the startrees
+ // Perform post-cleanup operations on the index handlers.
for (IndexHandler handler : indexHandlers) {
handler.postUpdateIndicesCleanup(segmentWriter);
}
+ segmentWriter.save();
+ }
+
+ // Startree creation will load the segment again, so we need to close and
re-open the segment writer to make sure
+ // that the other required indices (e.g. forward index) are up-to-date.
+ try (SegmentDirectory.Writer segmentWriter =
_segmentDirectory.createWriter()) {
+ // Create/modify/remove star-trees if required.
+ processStarTrees(indexDir);
+ _segmentDirectory.reloadMetadata();
// Add min/max value to column metadata according to the prune mode.
// For star-tree index, because it can only increase the range, so
min/max value can still be used in pruner.
Review Comment:
Good question. I was simply keeping the same order. I will try out moving
the min/max update to happen along with the index generation. I think that
startree index should be independent of column based indices and not impact to
min/max metadata. I will follow up on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
