jpountz commented on code in PR #13196:
URL: https://github.com/apache/lucene/pull/13196#discussion_r1535326207
##########
lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInputProvider.java:
##########
@@ -92,10 +110,22 @@ private final MemorySegment[] map(
}
if (preload) {
segment.load();
+ } else if (segSize > 0L && advice.isPresent()) { // not when preloading!
+ nativeAccess.madvise(segment, advice.getAsInt());
}
segments[segNr] = segment;
startOffset += segSize;
}
return segments;
}
+
+ private OptionalInt mapContextToMadvise(IOContext context) {
+ if (context.randomAccess) {
+ return OptionalInt.of(NativeAccess.POSIX_MADV_RANDOM);
+ }
+ if (context.readOnce || context.context == Context.MERGE) {
Review Comment:
HNSW merging is fine because the first thing that merging does it to write
all vectors to a temporary file. And then it builds the graph on top of this
temporary file. So it's ok for the input segments to read sequentially, and we
could update the merging logic to open this temporary file with
`IOContext.RANDOM` since it's the one that will have a random access pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
