Hi All,
Recently one of our solr nodes thrown an error:
...
2021-09-07 00:15:03.258 ERROR (qtp1278677872-985164)
[c:userRequestResults_00000000-4444-0000-8888-000000000001 s:shard1
r:core_node7
x:userRequestResults_00000000-4444-0000-8888-000000000001_shard1_replica_n4]
o.a.s.c.SolrCore java.lang.IllegalArgumentException: Unknown directory:
NRTCachingDirectory(MMapDirectory@/data/solrhome/userRequestResults_00000000-4444-0000-8888-000000000001_shard1_replica_n4/data/snapshot_metadata
lockFactory=org.apache.lucene.store.NativeFSLockFactory@4489a5c4;
maxCacheMB=48.0 maxMergeSizeMB=4.0) {}
at
org.apache.solr.core.CachingDirectoryFactory.release(CachingDirectoryFactory.java:427)
at org.apache.solr.core.SolrCore.close(SolrCore.java:1654)
at org.apache.solr.servlet.HttpSolrCall.destroy(HttpSolrCall.java:654)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:442)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
...
Further work with "unknown" collection was not possible until the node
restarts.
This error doesn't new to us. Some time ago, we tried to use the /export
handler, and while export about 200-300K docs per query from time to time
faced this error on a random node. But until recently, queries with /select
handlers were always executed without problems.
We are quite newbies in solr usage, and suppose it's a consequence of
incorrect solr configuration. Can you help with understanding this problem?
According to search results, MMapDirectory requires as much as possible
free RAM, but attempts to increase free memory (+10Gb) do nothing
Our cluster specs:
8 machines with 40Gb RAM, 500 SSD.
Solr version 8.3.1
Solr heap 31Gb
Docs count ~15M
Collections ~350
Total index size per machine ~150Gb
One part of the collections is configured with 4 shards and 2 replicas and
the other with 1 shard and 8 replicas (we do join operation with them).
Any help you could provide would be much appreciated,
Roman
