Hi Michael,
1. set `pf=` (phrase field empty string), disabling implicit phrase query > building. This would help give a sense of whether phrase queries are > involved in the performance issues you're seeing. We are also in the process of moving from standalone 6.6 to 8.7 Solr cloud, We also noticed a huge response time increase (91 ms To 170 ms +). We tried applying the tweak of disabling the pf field and response time was back to normal. So somehow pf was responsible for increased response time. We are using both query-time multi-term synonyms and WordDelimiter[Graph]Filter. What should we do next from here as we can't disable the pf field? Cluster Configuration: 3 Solr Nodes: 5 CPU, 42 GB Ram (Each) 3 Zookeeper Nodes: 1 CPU, 2 GB Ram (Each) 3 Shards: 42m Documents, 42 GB (Each) Heap: 8 GB There are no deleted documents in the cluster and no updates going on. We are trying to match the performance first. On Sat, Mar 26, 2022 at 9:42 PM Michael Gibney <mich...@michaelgibney.net> wrote: > Are you using query-time multi-term synonyms or WordDelimiter[Graph]Filter? > -- these can trigger "graph phrase" queries, which are handled _quite_ > differently in Solr 8.11 vs 6.5 (and although unlikely to directly cause > the performance issues you're observing, might well explain the performance > discrepancy). If you're _not_ using either of those, then the rest of this > message is likely irrelevant. > > One thing to possibly keep an eye out for (in addition to gathering more > evidence, as Mike Drob suggests): 6.5 started using span queries for "graph > phrase" queries (LUCENE-7699), but the resulting phrase queries were > completely ignored in Solr (bug) until 7.6 (SOLR-12243). Completely > ignoring complex phrase queries did however greatly reduce latency and CPU > load on 6.5! > > 7.6 started paying attention to these queries again (SOLR-12243), but also > went back to "fully-enumerated" combinatoric approach to phrase queries > when `ps` (phrase slop) is greater than 0 (LUCENE-8531). > > Some parameters you could tweak, assuming you're using edismax: > 1. set `pf=` (phrase field empty string), disabling implicit phrase query > building. This would help give a sense of whether phrase queries are > involved in the performance issues you're seeing. > 2. set `ps=0` (phrase slop 0), this should allow span queries to be built, > which should generally be more efficient than analogous non-span-query > approach (basically this would make the change introduced by LUCENE-8531 > irrelevant); tangentially: the special case building span queries for > `ps=0` is removed as of Lucene 9.0 (will be removed as of Solr 9.0 -- not > directly relevant to this issue though). > > Michael > > On Sat, Mar 26, 2022 at 8:26 AM Mike Drob <md...@mdrob.com> wrote: > > > Can you provide more details on what they CPU time is spent on? Maybe > look > > at some JFR profiles or collect several jstacks to see where they > > bottlenecks are. > > > > On Sat, Mar 26, 2022 at 3:49 AM Modassar Ather <modather1...@gmail.com> > > wrote: > > > > > Hi, > > > > > > We are trying to migrate to Solr-8.11.0 from Solr-6.5.1. Following are > > the > > > details of Solr installation. > > > > > > Server : EC2 instance with 32 CPUs and 521 GB > > > Storage : EBS Volume. General Purpose SSD (gp3) with 3000/5000 IOPS > > > Ubuntu :Ubuntu 20.04.3 LTS > > > Java : openjdk 11.0.14 > > > SolrCloud : 12 shards having a total 4+ TB index. Each node has a 30GB > > max > > > memory limit. > > > GC setting on Solr : G1GC > > > Solr query timeout : 5 minutes > > > > > > During testing we observed a high CPU utilisation and few of the > queries > > > with wildcard queries are timing out. These queries are getting > executed > > > completely on Solr-6.5.1. > > > After tuning a few of the parameters of GC settings the CPU utilisation > > > came down but it is still high when compared with Solr-6.5.1 and some > > > queries with wildcard queries are still failing. > > > > > > Kindly provide your suggestions. > > > > > > Thanks, > > > Modassar > > > > > >
