[
https://issues.apache.org/jira/browse/SOLR-17462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18043056#comment-18043056
]
Hakan Özler commented on SOLR-17462:
------------------------------------
Hi [~gerlowskija] ,
I've recently got around to looking at this problem again. The bug you referred
to seems to be the exact issue we encountered, because after fiddling with the
parameters below, we managed to take a backup of our largest collection on
9.10. Apparently, some internal async tasks were lost during the longer backup
process and the default parameters introduced in 9.7 were enough to make our
backup tasks fail.
{code:java}
-Dsolr.admin.async.timeout.minutes=
-Dsolr.admin.async.timeout.completed.minutes={code}
We can consider this task resolved. Thank you for the help!
> Collection RESTORE: Invalid Status Request Error
> -------------------------------------------------
>
> Key: SOLR-17462
> URL: https://issues.apache.org/jira/browse/SOLR-17462
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 9.7, 9.6.1
> Reporter: Hakan Özler
> Priority: Major
> Attachments: logs.png
>
>
> We're unable to restore a S3 backup taken from 9.6.1 to 9.7.0. There are no
> helpful log messages from Solr during the restore apart from "Invalid status
> request for requestId: 'xxxx168686616873218' - 'notfound'. Retried 6 times" .
> Interestingly, although we received this error message once, the data was
> successfully downloaded from S3. We even noticed this message "Successfully
> restored to the backup index" a couple of times. There is more; once we have
> the full data with the error message, the search completely failed, showing
> `Cannot invoke "java.util.Map.values()" because "rb.resultIds" is null`.
> However, we were able to retrieve the data using the `shards` parameter with
> all the shard names e.g. `shards=shard1,shard2,shard3,...,shardN`.
> {code:java}
> org.apache.solr.common.SolrException: Invalid status request for requestId:
> 'xxxxxx168686616873218' - 'notfound'. Retried 6 times
> at
> org.apache.solr.cloud.api.collections.CollectionHandlingUtils.waitForCoreAdminAsyncCallToComplete(CollectionHandlingUtils.java:549)
> at
> org.apache.solr.cloud.api.collections.CollectionHandlingUtils$ShardRequestTracker.waitForAsyncCallsToComplete(CollectionHandlingUtils.java:705)
> at
> org.apache.solr.cloud.api.collections.CollectionHandlingUtils$ShardRequestTracker.processResponses(CollectionHandlingUtils.java:694)
> at
> org.apache.solr.cloud.api.collections.CollectionHandlingUtils$ShardRequestTracker.processResponses(CollectionHandlingUtils.java:665)
> at
> org.apache.solr.cloud.api.collections.RestoreCmd.requestReplicasToRestore(RestoreCmd.java:139)
> at
> org.apache.solr.cloud.api.collections.RestoreCmd$RestoreOnANewCollection.process(RestoreCmd.java:298)
> at
> org.apache.solr.cloud.api.collections.RestoreCmd.call(RestoreCmd.java:109)
> at
> org.apache.solr.cloud.api.collections.CollApiCmds$TraceAwareCommand.call(CollApiCmds.java:225)
> at
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:130)
> at
> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:564)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> {code}
> !logs.png|width=997,height=176!
> {code:java}
> java.lang.NullPointerException: Cannot invoke "java.util.Map.values()"
> because "rb.resultIds" is null
> at
> org.apache.solr.handler.component.QueryComponent.createRetrieveDocs(QueryComponent.java:1246)
> at
> org.apache.solr.handler.component.QueryComponent.regularDistributedProcess(QueryComponent.java:629)
> at
> org.apache.solr.handler.component.QueryComponent.distributedProcess(QueryComponent.java:578)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:508)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2880)
> at
> org.apache.solr.servlet.HttpSolrCall.executeCoreRequest(HttpSolrCall.java:890)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:576)
> at
> org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:251)
> at
> org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:208)
> at
> org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:243)
> at
> org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:213)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)
> at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:149)
> at
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:228)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
> at org.eclipse.jetty.server.Server.handle(Server.java:563)
> at
> org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
> at
> org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
> at
> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
> at
> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
> at
> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
> at
> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
> at java.base/java.lang.Thread.run(Unknown Source)
> {code}
> It appears that the cluster used to restore the backup should be the same
> Solr version as the one, where the backup was taken, because the restore went
> smoothly.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
