Helo,
We tried migrating to solr 8.7 (from 7.2). We use solr cloud. We have a total
of 5 collections.
Migration Steps:
- Take back up of collections on 7.2 using collection APIs with command backup
- Restore the backup on 8.7 using collection APIs with command restore
The cluster starts fine and is able to handle traffic. But after sometime (tens
of minutes
to a few hours) only one of the collections (sellers) gets into an an error
state.
All the other collections are fine.
Please find below the entries in the log. We see that the searcher is being
closed, but fails to
create a new one. This happens on different replicas (of the same collection)
at different points in time.
Any help apprciated.
2021-06-26 18:56:36.866 INFO (qtp1539947037-32) [c:catalog s:shard1
r:core_node4 x:catalog_shard1_replica_n3]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.sellers.shard1.replica_n3 tag=SolrCore@705ab99d
2021-06-26 18:56:36.866 INFO (qtp1539947037-32) [c:catalog s:shard1
r:core_node4 x:catalog_shard1_replica_n3]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.collection.sellers.shard1.leader tag=SolrCore@705ab99d
2021-06-26 18:56:36.866 WARN (qtp1539947037-32) [c:catalog s:shard1
r:core_node4 x:catalog_shard1_replica_n3]
o.a.s.c.ZkController Was asked to remove OnReconnect listener
ZkIndexSchemaReader: /configs/sellers/managed-schema, uniqueCoreId:
sellers_shard1_replica_n3:5034137196427355, but remove operation did not find
it in the list of registered listeners.
2021-06-26 18:56:36.866 WARN (qtp1539947037-32) [c:catalog s:shard1
r:core_node4 x:catalog_shard1_replica_n3]
o.a.s.c.ZkController /configs/sellers has no more registered listeners, but a
live one attempted to unregister!
2021-06-26 18:56:36.869 ERROR (qtp1539947037-32) [c:catalog s:shard1
r:core_node4 x:catalog_shard1_replica_n3]
o.a.s.c.SolrCore java.lang.IllegalArgumentException: Unknown directory:
NRTCachingDirectory(MMapDirectory@/var/solr/data/sellers_
shard1_replica_n3/data/snapshot_metadatalockFactory=org.apache.lucene.store.NativeFSLockFactory@35b829da;
maxCacheMB=48
.....
**2021-06-26 18:56:36.872 ERROR (qtp1539947037-32) [c:catalog s:shard1
r:core_node4 x:catalog_shard1_replica_n3]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error opening
new searcher**
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2205)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2329)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2059)
at
org.apache.solr.search.JoinQuery$JoinQueryWeight.<init>(JoinQParserPlugin.java:345)
at
org.apache.solr.search.JoinQuery.createWeight(JoinQParserPlugin.java:304)
at
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726)
at org.apache.lucene.search.BooleanWeight.<init>(BooleanWeight.java:63)
at
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:231)
at
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445)
at
org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:145)
....
**Caused by: org.apache.lucene.store.AlreadyClosedException: Already closed
at
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:343)**
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:400)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2085)
Regards.
@shankerbalan
