[
https://issues.apache.org/jira/browse/SOLR-17433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908861#comment-17908861
]
Alex Deparvu commented on SOLR-17433:
-------------------------------------
there are 2 timeouts that can be set on the SolrClientCache that affect the
streaming request.
* One is the idle timeout (equivalent to the socket timeout in the other client
implementation). this is set per client (some docs here
https://jetty.org/docs/jetty/12/programming-guide/client/io-arch.html#transport).
I think this will throw an `TimeoutException "Idle timeout expired..."`.
* the other one is the request timeout. this is set per each request and
represents `the total timeout for the request/response conversation`. (some
docs
https://jetty.org/docs/jetty/12/programming-guide/client/http.html#blocking).
going over this limit looks like it will throw a `TimeoutException "Total
timeout "` same as your error sample.
You can set both of these limits via system properties if the defaults are not
sufficient for your usecase:
* idle timeout can be set via "socketTimeout" system property
* request timeout can be set via "connTimeout" system property
both timeouts are set by default at 60s and I am more inclined to say the
second one is causing issues here due to long overall duration of streaming
queries.
> SolrStream by default creates a Http2SolrClient that can only stream for 60
> seconds
> -----------------------------------------------------------------------------------
>
> Key: SOLR-17433
> URL: https://issues.apache.org/jira/browse/SOLR-17433
> Project: Solr
> Issue Type: Bug
> Components: SolrJ, streaming expressions
> Affects Versions: 9.4, 9.5, 9.6
> Reporter: Daniel Lowe
> Priority: Major
>
> This behaviour changed in SolrJ 9.4, in earlier versions a {{HttpSolrClient}}
> was directly instantiated {{{}in SolrStream's open() method, while from SolrJ
> 9.4 a client is instantiated via a SolrClientCache{}}}.
> I believe the expected behaviour is that the client that the SolrClientCache
> instantiates should have appropriate timeout settings for streaming usage. I
> think the 60 second timeout arises from {{{}SolrClientCache{}}}'s MIN_TIMEOUT
> of 60 seconds. I note that that this is inconsistent with
> HttpClientUtil.DEFAULT_SO_TIMEOUT (600 seconds) which is used by default if
> you manually instantiate a {{Http2SolrClient or HttpJdkSolrClient.}}
> Given what I think the typical use case is for SolrStream ideally I don't
> think there should even be a timeout assuming the stream is still returning
> content.
> I think you can restore the original behaviour by using:
>
> {code:java}
> context.setSolrClientCache(new SolrClientCache(new
> HttpSolrClient.Builder(url).build().getHttpClient())); {code}
>
>
> Example code:
>
> {code:java}
> ModifiableSolrParams params = new ModifiableSolrParams();
> params.set("expr", "search(<your_collection_name>,qt=\"/export\",q=\"" +
> someLongRunningQuery + "\", fl=\"id\", sort=\"id asc\")");
> params.set("qt", "/stream");
> try (TupleStream solrStream = new SolrStream(url, params)) {
> StreamContext context = new StreamContext();
> solrStream.setStreamContext(context);
> solrStream.open();
> Tuple tuple;
> while (!(tuple = solrStream.read()).EOF) {
> }
> }
> {code}
>
>
> {code:java}
> Caused by: java.util.concurrent.TimeoutException: Total timeout 60000 ms
> elapsed at
> org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:342)
> at
> org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:322)
> at
> org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:110)
> at
> org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:197)
> at
> org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:294)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
