I would love some profiling as well. I know 8.8 or 8.9 had some performance problems with atomic update but this was later addressed. I cant find the jira atm though. Also I am on 8.11.1 and atomic update is not an issue for me.
By the way, do you happen to have nested docs? On Wed, May 31, 2023, 11:20 Jan Høydahl <jan....@cominvent.com> wrote: > Hi > > MMap is most important for searching. Indexing bypasses the cache by using > direct IO. > > I have noticed slow real time get on Solr 8.x during atomic update myself. > Would be interesting with a comparison with profiling. RTG gets the > document from transaction log I believe? Could there be some RTG changes in > 8.x that caused such slowdown? > > Jan Høydahl > > > 31. mai 2023 kl. 16:57 skrev Rahul Goswami <rahul196...@gmail.com>: > > > > Thanks for the response Shawn. We are using Windows server with pretty > huge > > indexes (multiple TB cores). With Mmap, I have observed that the machine > > just completely freezes with high CPU and memory usage to a point where > it > > becomes impossible to even connect to it. SimpleFS works out well for us > in > > this case. > > > > As noted in my first email, even with SimpleFS, Solr 7 completes the > crawl > > in nearly 1/5th the time taken in Solr 8. Hence there should be something > > OUTSIDE the directory factory in the code which is causing this. > > > > Thanks, > > Rahul > > > > > >> On Tue, May 30, 2023 at 10:47 PM Shawn Heisey <apa...@elyograg.org> > wrote: > >> > >>> On 5/30/23 15:34, Rahul Goswami wrote: > >>> Environment details: - Java 11 on Windows server - Xms1536m Xmx3072m - > >>> Indexing client code running 15 parallel threads indexing in batches of > >>> 1000 - using SimpleFSDirectoryFactory (since Mmap doesn't quite work > >>> well on Windows for our index sizes which commonly run north of 1 TB) > >> > >> Don't change the directoryFactory. You *WANT* Solr to use MMAP for your > >> indexes. Not using MMAP is likely to slow things down considerably. > >> MMAP should work just fine on 64-bit Windows with 64-bit Java. Which of > >> course requires 64-bit hardware. > >> > >> 32 bit systems and software cannot properly deal with data larger than > >> about 2GB. > >> > >> Thanks, > >> Shawn > >> >
