Thanks, Michael. I missed that mail thread among all responses. I will check that too.
On Thu, May 5, 2022 at 6:26 PM Michael Gibney <mich...@michaelgibney.net> wrote: > Did you yet dig through the mailing list thread link that I posted earlier > in this thread? It explains in more depth, suggests a number of possible > mitigations, and has a bunch of links to jira issues that provide extra > context. Off the cuff, I'd say that setting `enableGraphQueries=false` may > be most immediately helpful in terms of restoring performance. > > (As an aside: from my perspective though, even if you can restore > performance, it would be at the expense of nuances of functionality. Longer > term I'd really like to help solve this properly, involving some > combination of the issues linked to in the above thread ...) > > Michael > > On Thu, May 5, 2022 at 3:01 AM Satya Nand <satya.n...@indiamart.com > .invalid> > wrote: > > > Hi Michael, > > > > > > 1. set `pf=` (phrase field empty string), disabling implicit phrase query > > > building. This would help give a sense of whether phrase queries are > > > involved in the performance issues you're seeing. > > > > > > We are also in the process of moving from standalone 6.6 to 8.7 Solr > cloud, > > We also noticed a huge response time increase (91 ms To 170 ms +). > > > > We tried applying the tweak of disabling the pf field and response time > was > > back to normal. So somehow pf was responsible for increased response > time. > > > > We are using both query-time multi-term synonyms and > > WordDelimiter[Graph]Filter. > > > > What should we do next from here as we can't disable the pf field? > > > > Cluster Configuration: > > > > 3 Solr Nodes: 5 CPU, 42 GB Ram (Each) > > 3 Zookeeper Nodes: 1 CPU, 2 GB Ram (Each) > > 3 Shards: 42m Documents, 42 GB (Each) > > Heap: 8 GB > > > > > > There are no deleted documents in the cluster and no updates going on. We > > are trying to match the performance first. > > > > > > > > > > > > > > > > On Sat, Mar 26, 2022 at 9:42 PM Michael Gibney < > mich...@michaelgibney.net> > > wrote: > > > > > Are you using query-time multi-term synonyms or > > WordDelimiter[Graph]Filter? > > > -- these can trigger "graph phrase" queries, which are handled _quite_ > > > differently in Solr 8.11 vs 6.5 (and although unlikely to directly > cause > > > the performance issues you're observing, might well explain the > > performance > > > discrepancy). If you're _not_ using either of those, then the rest of > > this > > > message is likely irrelevant. > > > > > > One thing to possibly keep an eye out for (in addition to gathering > more > > > evidence, as Mike Drob suggests): 6.5 started using span queries for > > "graph > > > phrase" queries (LUCENE-7699), but the resulting phrase queries were > > > completely ignored in Solr (bug) until 7.6 (SOLR-12243). Completely > > > ignoring complex phrase queries did however greatly reduce latency and > > CPU > > > load on 6.5! > > > > > > 7.6 started paying attention to these queries again (SOLR-12243), but > > also > > > went back to "fully-enumerated" combinatoric approach to phrase queries > > > when `ps` (phrase slop) is greater than 0 (LUCENE-8531). > > > > > > Some parameters you could tweak, assuming you're using edismax: > > > 1. set `pf=` (phrase field empty string), disabling implicit phrase > query > > > building. This would help give a sense of whether phrase queries are > > > involved in the performance issues you're seeing. > > > 2. set `ps=0` (phrase slop 0), this should allow span queries to be > > built, > > > which should generally be more efficient than analogous non-span-query > > > approach (basically this would make the change introduced by > LUCENE-8531 > > > irrelevant); tangentially: the special case building span queries for > > > `ps=0` is removed as of Lucene 9.0 (will be removed as of Solr 9.0 -- > not > > > directly relevant to this issue though). > > > > > > Michael > > > > > > On Sat, Mar 26, 2022 at 8:26 AM Mike Drob <md...@mdrob.com> wrote: > > > > > > > Can you provide more details on what they CPU time is spent on? Maybe > > > look > > > > at some JFR profiles or collect several jstacks to see where they > > > > bottlenecks are. > > > > > > > > On Sat, Mar 26, 2022 at 3:49 AM Modassar Ather < > modather1...@gmail.com > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > We are trying to migrate to Solr-8.11.0 from Solr-6.5.1. Following > > are > > > > the > > > > > details of Solr installation. > > > > > > > > > > Server : EC2 instance with 32 CPUs and 521 GB > > > > > Storage : EBS Volume. General Purpose SSD (gp3) with 3000/5000 IOPS > > > > > Ubuntu :Ubuntu 20.04.3 LTS > > > > > Java : openjdk 11.0.14 > > > > > SolrCloud : 12 shards having a total 4+ TB index. Each node has a > > 30GB > > > > max > > > > > memory limit. > > > > > GC setting on Solr : G1GC > > > > > Solr query timeout : 5 minutes > > > > > > > > > > During testing we observed a high CPU utilisation and few of the > > > queries > > > > > with wildcard queries are timing out. These queries are getting > > > executed > > > > > completely on Solr-6.5.1. > > > > > After tuning a few of the parameters of GC settings the CPU > > utilisation > > > > > came down but it is still high when compared with Solr-6.5.1 and > some > > > > > queries with wildcard queries are still failing. > > > > > > > > > > Kindly provide your suggestions. > > > > > > > > > > Thanks, > > > > > Modassar > > > > > > > > > > > > > > >
