I tried the exact same procedure with a solr:8.11.1 container - generally,
the query response comes back within 2 seconds. When I reduce timeAllowed in
steps, it doesn't have much impact - e.g. with both timeAllowed=1000 and
timeAllowed=500, .debug.timing.process.query remains in the 1100-1400
region. If I reduce timeAllowed down to 200, sometimes I get results in a
similar response time, but often the timeout works - ExitableDirectoryReader
warnings show up in the log (which I understand is the legacy
implementation inside lucene, replaced/refactored for solr 9.x) and I get
zero results, with .responseHeader.partialResults set to true as expected.
2024-11-08 09:15:50.229 WARN (qtp1344697180-25) [ x:landreg]
o.a.s.s.SolrIndexSearcher Query: [*:*]; =>
org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException: The
request took too long to iterate over doc values. Timeout: timeoutAt:
66169929699299 (System.nanoTime(): 66169929948466),
DocValues=org.apache.lucene.index.SingletonSortedNumericDocValues@3ac7c42b
at
org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader.checkAndThrow(ExitableDirectoryReader.java:319)
{
"responseHeader":{
"partialResults":true,
"status":0,
"QTime":916,
"params":{
"q":"*:*",
"debug":"timing",
"fl":"uuid",
"start":"940000",
"sort":"transfer_date desc",
"timeAllowed":"200"}},
"response":{"numFound":0,"start":0,"numFoundExact":true,"docs":[]
},
On Thu, 7 Nov 2024 at 19:25, Gus Heck <gus.h...@gmail.com> wrote:
> Did you also run this against 8.11 and get shorter times? It has always
> been possible for timeAllowed to overrun somewhat (the code has to
> periodically check if limits have been exceeded, and the work between
> checks isn't going to stop), but if how much it overruns has changed
> drastically we do want to look into that.
>
> On Thu, Nov 7, 2024 at 11:46 AM Darren Foreman <dar...@adzuna.com.invalid>
> wrote:
>
> > Hi all,
> >
> > I also have little java expertise, but I seem able to replicate the
> problem
> > locally like this:
> >
> > 1. Downloaded a large CSV: the UK government's "price paid data" CSV
> for
> > an entire year:
> >
> >
> http://prod.publicdata.landregistry.gov.uk.s3-website-eu-west-1.amazonaws.com/pp-2019.csv
> > 2. Save the above in a directory called solrdata, and edit it
> > 1. swap the first two lines around (so that type inference works
> when
> > importing)
> > 2. add a header to it
> >
> >
> >
> >
> uuid,price,transfer_date,postcode,property_type,old_new,duration,paon,saon,street,locality,town_city,district,county,ppd_category,record_status
> >
> > 3. Run solr in a container with constrained resources - 1 CPU,
> 512mb
> > memory
> >
> > docker run --rm --cpus 1 --memory 536870912 -v
> "$PWD/solrdata:/var/solr"
> > -p 8983:8983 --name solr solr:9.7.0
> >
> > 4. Create a core that will infer the schema from data
> >
> > docker exec -it solr ./bin/solr create -c landreg -n
> > data_driven_schema_configs
> >
> > 5. Import the CSV. I called mine landreg-data.csv:
> >
> > docker exec -it solr ./bin/post -c landreg /var/solr/landreg-data.csv
> >
> > 6. Query for a deep-paged set of results after applying a sort,
> > supplying timeAllowed:
> >
> > curl
> >
> >
> 'localhost:8983/solr/landreg/select?fl=uuid&q=*:*&sort=transfer_date+desc&start=940000&debug=timing&timeAllowed=200'
> >
> >
> > For me, the timeAllowed parameter is almost always seemingly ignored i.e.
> > exceeded (especially if low) - here I ask for 200ms, or even 2000ms:
> >
> > $ curl -s
> >
> >
> 'localhost:8983/solr/landreg/select?fl=uuid&q=*:*&sort=transfer_date+desc&start=940000&debug=timing&timeAllowed=200'
> > \
> > > | jq .debug.timing.process.query
> > {
> > "time": 3954.0
> > }
> > $ curl -s
> >
> >
> 'localhost:8983/solr/landreg/select?fl=uuid&q=*:*&sort=transfer_date+desc&start=940000&debug=timing&timeAllowed=2000'
> > | jq .debug.timing.process.query
> > {
> > "time": 3645.0
> > }
> >
> >
> > On 2024/11/06 15:34:58 Gus Heck wrote:
> > > If you have access to a test instance where the problem can be
> > reproduced,
> > > attaching a profiler would be one way. Another cruder method is to use
> > > jstack to dump all the threads.
> > >
> > > Another way to tackle this is to help us reproduce your problem. Can
> you
> > > share details about your query? Obviously, please don't post anything
> > your
> > > company wouldn't want public, but if you can share some details that
> > would
> > > be a start.
> > >
> > > The ideal thing would be to provide a minimum working example of the
> > > problem you are experiencing.
> > >
> > > On Wed, Nov 6, 2024 at 9:55 AM Dominic Humphries
> > <do...@adzuna.com.invalid
> > >
> > > wrote:
> > >
> > > > I've tried both timeAllowed and cpuAllowed and neither are
> restricting
> > the
> > > > amount of time the queries take to run. I have a test query that's
> > reliably
> > > > taking 20-30 seconds, if there's any useful debug params or such I
> can
> > run
> > > > to provide the information you want I'm happy to run them - I'm not
> > sure
> > > > how to usefully interrogate solr for where its time is being spent,
> > sorry
> > > >
> > > > Thanks
> > > >
> > > > On Wed, 6 Nov 2024 at 14:25, Gus Heck <gu...@gmail.com> wrote:
> > > >
> > > > > There are unit tests that seem to suggest that timeAllowed still
> > works,
> > > > can
> > > > > you provide some more information about your use case? Particularly
> > > > > important is any information about where (what code) your queries
> are
> > > > > spending a lot of time in if you have it.
> > > > >
> > > > > On Wed, Nov 6, 2024 at 6:18 AM Dominic Humphries
> > > > > <do...@adzuna.com.invalid>
> > > > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > we're testing Solr 9.7 to upgrade our existing 8.11 stack. We're
> > > > seeing a
> > > > > > problem with long requests: we send `timeAllowed=4900` which
> works
> > fine
> > > > > on
> > > > > > the existing 8.11 and keeps requests to just a few seconds.
> > > > > >
> > > > > > With 9.7, however, the flag is basically ignored - requests can
> > take
> > > > over
> > > > > > 30 seconds whether the flag is present or not, which is causing
> > higher
> > > > > CPU
> > > > > > load and slowing response times.
> > > > > >
> > > > > > I've tried setting the flag suggested in
> > > > > >
> > > > > >
> > > > >
> > > >
> >
> >
> https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-9.html#use-of-timeallowed
> > > > > > - but even with solr.useExitableDirectoryReader set we still
> don't
> > get
> > > > > the
> > > > > > desired behaviour.
> > > > > >
> > > > > > Is there anything else I can try to get the old behaviour back?
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > http://www.needhamsoftware.com (work)
> > > > > https://a.co/d/b2sZLD9 (my fantasy fiction book)
> > > > >
> > > >
> > >
> > >
> > > --
> > > http://www.needhamsoftware.com (work)
> > > https://a.co/d/b2sZLD9 (my fantasy fiction book)
> > >
> >
>
>
> --
> http://www.needhamsoftware.com (work)
> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>
--
*Darren Foreman*
*Senior Perl Developer*
*M:* 07973 953 581 <%23>
www.adzuna.co.uk
<https://en-gb.facebook.com/adzuna/> <https://twitter.com/adzuna>
<https://www.linkedin.com/company/adzuna> <https://www.adzuna.co.uk/blog/>
