Hi all, I also have little java expertise, but I seem able to replicate the problem locally like this:
1. Downloaded a large CSV: the UK government's "price paid data" CSV for an entire year: http://prod.publicdata.landregistry.gov.uk.s3-website-eu-west-1.amazonaws.com/pp-2019.csv 2. Save the above in a directory called solrdata, and edit it 1. swap the first two lines around (so that type inference works when importing) 2. add a header to it uuid,price,transfer_date,postcode,property_type,old_new,duration,paon,saon,street,locality,town_city,district,county,ppd_category,record_status 3. Run solr in a container with constrained resources - 1 CPU, 512mb memory docker run --rm --cpus 1 --memory 536870912 -v "$PWD/solrdata:/var/solr" -p 8983:8983 --name solr solr:9.7.0 4. Create a core that will infer the schema from data docker exec -it solr ./bin/solr create -c landreg -n data_driven_schema_configs 5. Import the CSV. I called mine landreg-data.csv: docker exec -it solr ./bin/post -c landreg /var/solr/landreg-data.csv 6. Query for a deep-paged set of results after applying a sort, supplying timeAllowed: curl 'localhost:8983/solr/landreg/select?fl=uuid&q=*:*&sort=transfer_date+desc&start=940000&debug=timing&timeAllowed=200' For me, the timeAllowed parameter is almost always seemingly ignored i.e. exceeded (especially if low) - here I ask for 200ms, or even 2000ms: $ curl -s 'localhost:8983/solr/landreg/select?fl=uuid&q=*:*&sort=transfer_date+desc&start=940000&debug=timing&timeAllowed=200' \ > | jq .debug.timing.process.query { "time": 3954.0 } $ curl -s 'localhost:8983/solr/landreg/select?fl=uuid&q=*:*&sort=transfer_date+desc&start=940000&debug=timing&timeAllowed=2000' | jq .debug.timing.process.query { "time": 3645.0 } On 2024/11/06 15:34:58 Gus Heck wrote: > If you have access to a test instance where the problem can be reproduced, > attaching a profiler would be one way. Another cruder method is to use > jstack to dump all the threads. > > Another way to tackle this is to help us reproduce your problem. Can you > share details about your query? Obviously, please don't post anything your > company wouldn't want public, but if you can share some details that would > be a start. > > The ideal thing would be to provide a minimum working example of the > problem you are experiencing. > > On Wed, Nov 6, 2024 at 9:55 AM Dominic Humphries <do...@adzuna.com.invalid > > wrote: > > > I've tried both timeAllowed and cpuAllowed and neither are restricting the > > amount of time the queries take to run. I have a test query that's reliably > > taking 20-30 seconds, if there's any useful debug params or such I can run > > to provide the information you want I'm happy to run them - I'm not sure > > how to usefully interrogate solr for where its time is being spent, sorry > > > > Thanks > > > > On Wed, 6 Nov 2024 at 14:25, Gus Heck <gu...@gmail.com> wrote: > > > > > There are unit tests that seem to suggest that timeAllowed still works, > > can > > > you provide some more information about your use case? Particularly > > > important is any information about where (what code) your queries are > > > spending a lot of time in if you have it. > > > > > > On Wed, Nov 6, 2024 at 6:18 AM Dominic Humphries > > > <do...@adzuna.com.invalid> > > > wrote: > > > > > > > Hi folks, > > > > > > > > we're testing Solr 9.7 to upgrade our existing 8.11 stack. We're > > seeing a > > > > problem with long requests: we send `timeAllowed=4900` which works fine > > > on > > > > the existing 8.11 and keeps requests to just a few seconds. > > > > > > > > With 9.7, however, the flag is basically ignored - requests can take > > over > > > > 30 seconds whether the flag is present or not, which is causing higher > > > CPU > > > > load and slowing response times. > > > > > > > > I've tried setting the flag suggested in > > > > > > > > > > > > > https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-9.html#use-of-timeallowed > > > > - but even with solr.useExitableDirectoryReader set we still don't get > > > the > > > > desired behaviour. > > > > > > > > Is there anything else I can try to get the old behaviour back? > > > > > > > > Thanks > > > > > > > > > > > > > -- > > > http://www.needhamsoftware.com (work) > > > https://a.co/d/b2sZLD9 (my fantasy fiction book) > > > > > > > > -- > http://www.needhamsoftware.com (work) > https://a.co/d/b2sZLD9 (my fantasy fiction book) >
