Hi Roman,
Do you really need a 31gb heap size ? Use gceasy.io in order to analyse
your heap usage.
I thing server free system memory is to small for efficiente caching. I
suggest to increase server memory and reduce heap size if possible.
Don’t forget to disable os swap by setting swapiness to 1.
Regards
Dominique
Le mer. 8 sept. 2021 à 03:02, Roman Voronin <flpf...@gmail.com> a écrit :
> Hi All,
>
> Recently one of our solr nodes thrown an error:
>
> ...
> 2021-09-07 00:15:03.258 ERROR (qtp1278677872-985164)
> [c:userRequestResults_00000000-4444-0000-8888-000000000001 s:shard1
> r:core_node7
>
> x:userRequestResults_00000000-4444-0000-8888-000000000001_shard1_replica_n4]
> o.a.s.c.SolrCore java.lang.IllegalArgumentException: Unknown directory:
> NRTCachingDirectory(MMapDirectory@
> /data/solrhome/userRequestResults_00000000-4444-0000-8888-000000000001_shard1_replica_n4/data/snapshot_metadata
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@4489a5c4;
> maxCacheMB=48.0 maxMergeSizeMB=4.0) {}
> at
>
> org.apache.solr.core.CachingDirectoryFactory.release(CachingDirectoryFactory.java:427)
> at org.apache.solr.core.SolrCore.close(SolrCore.java:1654)
> at org.apache.solr.servlet.HttpSolrCall.destroy(HttpSolrCall.java:654)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:442)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
> at
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
> ...
>
> Further work with "unknown" collection was not possible until the node
> restarts.
>
> This error doesn't new to us. Some time ago, we tried to use the /export
> handler, and while export about 200-300K docs per query from time to time
> faced this error on a random node. But until recently, queries with /select
> handlers were always executed without problems.
>
> We are quite newbies in solr usage, and suppose it's a consequence of
> incorrect solr configuration. Can you help with understanding this problem?
> According to search results, MMapDirectory requires as much as possible
> free RAM, but attempts to increase free memory (+10Gb) do nothing
>
> Our cluster specs:
> 8 machines with 40Gb RAM, 500 SSD.
> Solr version 8.3.1
> Solr heap 31Gb
> Docs count ~15M
> Collections ~350
> Total index size per machine ~150Gb
> One part of the collections is configured with 4 shards and 2 replicas and
> the other with 1 shard and 8 replicas (we do join operation with them).
>
> Any help you could provide would be much appreciated,
> Roman
>
