[
https://issues.apache.org/jira/browse/SOLR-17433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17878662#comment-17878662
]
Eric Pugh commented on SOLR-17433:
----------------------------------
Interesting! I hope some other folks with more expertise can weigh in. It
seems like in the case of streaming, versus having any specific hard update,
you would reset the timeout everytime more data was accessed? Maybe that is
how it works and you are pointing out that the pause limit should be longer
than 60 seconds?
> SolrStream by default creates a Http2SolrClient that can only stream for 60
> seconds
> -----------------------------------------------------------------------------------
>
> Key: SOLR-17433
> URL: https://issues.apache.org/jira/browse/SOLR-17433
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
> Affects Versions: 9.4, 9.5, 9.6
> Reporter: Daniel Lowe
> Priority: Major
>
> This behaviour changed in SolrJ 9.4, in earlier versions a {{HttpSolrClient}}
> was directly instantiated {{{}in SolrStream's open() method, while from SolrJ
> 9.4 a client is instantiated via a SolrClientCache{}}}.
> I believe the expected behaviour is that the client that the SolrClientCache
> instantiates should have appropriate timeout settings for streaming usage. I
> think the 60 second timeout arises from {{{}SolrClientCache{}}}'s MIN_TIMEOUT
> of 60 seconds. I note that that this is inconsistent with
> HttpClientUtil.DEFAULT_SO_TIMEOUT (600 seconds) which is used by default if
> you manually instantiate a {{Http2SolrClient or HttpJdkSolrClient.}}
> Given what I think the typical use case is for SolrStream ideally I don't
> think there should even be a timeout assuming the stream is still returning
> content.
> I think you can restore the original behaviour by using:
>
> {code:java}
> context.setSolrClientCache(new SolrClientCache(new
> HttpSolrClient.Builder(url).build().getHttpClient())); {code}
>
>
> Example code:
>
> {code:java}
> ModifiableSolrParams params = new ModifiableSolrParams();
> params.set("expr", "search(<your_collection_name>,qt=\"/export\",q=\"" +
> someLongRunningQuery + "\", fl=\"id\", sort=\"id asc\")");
> params.set("qt", "/stream");
> try (TupleStream solrStream = new SolrStream(url, params)) {
> StreamContext context = new StreamContext();
> solrStream.setStreamContext(context);
> solrStream.open();
> Tuple tuple;
> while (!(tuple = solrStream.read()).EOF) {
> }
> }
> {code}
>
>
> {code:java}
> Caused by: java.util.concurrent.TimeoutException: Total timeout 60000 ms
> elapsed at
> org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:342)
> at
> org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:322)
> at
> org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:110)
> at
> org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:197)
> at
> org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:294)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
