On 8/4/23 11:43, Jayesh Shende wrote:
The NRTCachingDirectoryFactory is trying to cache as much segments as possible into the memory, but the queries are for different collections and are varying (less of repetitive query terms), so thinking this cached segments are not actually very useful here, and RAM (apart from JVM assigned) is not enough to cache even 10% of the index, for each Solr node running.
Solr does NOT proactively cache data from the index files. It leaves that to the operating system.
If a certain piece of data is never accessed, it will never end up in the cache. Same is true of the on-heap caches that Solr generates and maintains ... data that is never accessed will NOT be cached.
Also it is an existing Solr, trying to improve performance, and as we know NIO is better than IO in java and I can increase IOPS and throughput for disk, so was gathering how will it affect?
When resources are properly sized, MMAP is the most efficient option for accessing file data available on ANY operating system.
The best way to improve Solr performance is to add memory. But if you're in a cloud-based setup, that is quite expensive.
From the other response, I gather that for memory-starved setups, switching the directory implementation can improve performance. But I offer this prediction: If you continue to run without sufficient memory, you're eventually going to hit a performance wall that can only be fixed by adding memory.
Thanks, Shawn
