Are you using query-time multi-term synonyms or WordDelimiter[Graph]Filter? -- these can trigger "graph phrase" queries, which are handled _quite_ differently in Solr 8.11 vs 6.5 (and although unlikely to directly cause the performance issues you're observing, might well explain the performance discrepancy). If you're _not_ using either of those, then the rest of this message is likely irrelevant.
One thing to possibly keep an eye out for (in addition to gathering more evidence, as Mike Drob suggests): 6.5 started using span queries for "graph phrase" queries (LUCENE-7699), but the resulting phrase queries were completely ignored in Solr (bug) until 7.6 (SOLR-12243). Completely ignoring complex phrase queries did however greatly reduce latency and CPU load on 6.5! 7.6 started paying attention to these queries again (SOLR-12243), but also went back to "fully-enumerated" combinatoric approach to phrase queries when `ps` (phrase slop) is greater than 0 (LUCENE-8531). Some parameters you could tweak, assuming you're using edismax: 1. set `pf=` (phrase field empty string), disabling implicit phrase query building. This would help give a sense of whether phrase queries are involved in the performance issues you're seeing. 2. set `ps=0` (phrase slop 0), this should allow span queries to be built, which should generally be more efficient than analogous non-span-query approach (basically this would make the change introduced by LUCENE-8531 irrelevant); tangentially: the special case building span queries for `ps=0` is removed as of Lucene 9.0 (will be removed as of Solr 9.0 -- not directly relevant to this issue though). Michael On Sat, Mar 26, 2022 at 8:26 AM Mike Drob <md...@mdrob.com> wrote: > Can you provide more details on what they CPU time is spent on? Maybe look > at some JFR profiles or collect several jstacks to see where they > bottlenecks are. > > On Sat, Mar 26, 2022 at 3:49 AM Modassar Ather <modather1...@gmail.com> > wrote: > > > Hi, > > > > We are trying to migrate to Solr-8.11.0 from Solr-6.5.1. Following are > the > > details of Solr installation. > > > > Server : EC2 instance with 32 CPUs and 521 GB > > Storage : EBS Volume. General Purpose SSD (gp3) with 3000/5000 IOPS > > Ubuntu :Ubuntu 20.04.3 LTS > > Java : openjdk 11.0.14 > > SolrCloud : 12 shards having a total 4+ TB index. Each node has a 30GB > max > > memory limit. > > GC setting on Solr : G1GC > > Solr query timeout : 5 minutes > > > > During testing we observed a high CPU utilisation and few of the queries > > with wildcard queries are timing out. These queries are getting executed > > completely on Solr-6.5.1. > > After tuning a few of the parameters of GC settings the CPU utilisation > > came down but it is still high when compared with Solr-6.5.1 and some > > queries with wildcard queries are still failing. > > > > Kindly provide your suggestions. > > > > Thanks, > > Modassar > > >
