Hi MMap is most important for searching. Indexing bypasses the cache by using direct IO.
I have noticed slow real time get on Solr 8.x during atomic update myself. Would be interesting with a comparison with profiling. RTG gets the document from transaction log I believe? Could there be some RTG changes in 8.x that caused such slowdown? Jan Høydahl > 31. mai 2023 kl. 16:57 skrev Rahul Goswami <rahul196...@gmail.com>: > > Thanks for the response Shawn. We are using Windows server with pretty huge > indexes (multiple TB cores). With Mmap, I have observed that the machine > just completely freezes with high CPU and memory usage to a point where it > becomes impossible to even connect to it. SimpleFS works out well for us in > this case. > > As noted in my first email, even with SimpleFS, Solr 7 completes the crawl > in nearly 1/5th the time taken in Solr 8. Hence there should be something > OUTSIDE the directory factory in the code which is causing this. > > Thanks, > Rahul > > >> On Tue, May 30, 2023 at 10:47 PM Shawn Heisey <apa...@elyograg.org> wrote: >> >>> On 5/30/23 15:34, Rahul Goswami wrote: >>> Environment details: - Java 11 on Windows server - Xms1536m Xmx3072m - >>> Indexing client code running 15 parallel threads indexing in batches of >>> 1000 - using SimpleFSDirectoryFactory (since Mmap doesn't quite work >>> well on Windows for our index sizes which commonly run north of 1 TB) >> >> Don't change the directoryFactory. You *WANT* Solr to use MMAP for your >> indexes. Not using MMAP is likely to slow things down considerably. >> MMAP should work just fine on 64-bit Windows with 64-bit Java. Which of >> course requires 64-bit hardware. >> >> 32 bit systems and software cannot properly deal with data larger than >> about 2GB. >> >> Thanks, >> Shawn >>
