q=cats&defType=edismax&qf=keywords&fl=id&rows=10
&distrib=false&cache=false&debug=timing
For query with distrib=false and cache=false, most shards have Qtime ~120
and few of them have QTime 0. It is the same with cache=true or without
cache param in the query.
debug time response from one shard is as following - there is only single
entry under process time and it is query time same as QTime
"process": {"time": 107,"query": {"time": 107
Same query with distrib=true, QTime avg is ~315. Is this expected Qtime for
an aggregated query? And why is per shard latency so high? (~120 ms)
On Fri, Nov 17, 2023 at 4:31 PM Mike Drob <md...@mdrob.com> wrote:
> Maybe also experiment with cache=false
>
> On Fri, Nov 17, 2023 at 2:38 PM Mikhail Khludnev <m...@apache.org> wrote:
>
> > > What causes this issue?
> > You may try to find an answer with distrib=false and debug=timing
> >
> >
> > On Fri, Nov 17, 2023 at 8:49 PM rajani m <rajinima...@gmail.com> wrote:
> >
> > > Hi again,
> > >
> > > Thank you for all the pointers, they were very helpful. After
> digging
> > in
> > > enough, I figured that it is a certain text field that matches a large
> > set
> > > of docs for a given query. And it is the one adding to the latency.
> > > Appreciate any suggestions to optimize it.
> > >
> > > An example query that matches a large set of documents.
> > >
> > > q=cats&defType=edismax&qf=keywords description title&fl=id&rows=10 |
> > Qtime
> > > = 531
> > >
> > > The query fields are the same field type(text_en). I tried the
> following
> > > variants and have included their respective QTime notes.
> > > 1. rows=0 | Qtime = 518
> > > 2. qf=keywords | Qtime = 323 - This is the field that matches 85% of
> > total
> > > docs and is adding to the overall latency of the queries
> > > 3. qf=description | Qtime= 144 - This field matches 25% of total docs
> > > 4. qf=title | Qtime = 17 - This field matches 5% of total docs
> > >
> > > It is clear that whenever a query matches a large number of docs from
> any
> > > query fields, which is often the keywords field in this case, it has
> high
> > > latency. What causes this issue? What are my options to optimize such
> > > queries latency?
> > >
> > > Thank you,
> > > Rajani
> > >
> > > On Thu, Nov 2, 2023 at 12:30 PM Mikhail Khludnev <m...@apache.org>
> > wrote:
> > >
> > > > On Thu, Nov 2, 2023 at 5:01 AM rajani m <rajinima...@gmail.com>
> wrote:
> > > >
> > > > > Sorry, it took too long to get back to this one.
> > > > >
> > > > > The search query "http://host:8983/solr/v9/select?&q=*&rows=10";
> > > > > consistently
> > > > > took ~500 ms. With "distrib=false" all the 96 shards have QTime
> 0-25
> > > ms.
> > > > > Does this mean aggregation of results from all the shards is taking
> > > ~475
> > > > > ms? I also tried shards.rows=5 and it still returned in ~475 ms
> query
> > > > time.
> > > > >
> > > >
> > > > Giving that. Don't you have any limits for ShardHandlerFactory set in
> > > > solr.xml?
> > > > debugQuery=true or debug=track can produce verbose shard request
> > tracing
> > > > info. Perhaps it might get some clue.
> > > > Full fledged approach is
> > > >
> > > >
> > >
> >
> https://solr.apache.org/guide/solr/latest/deployment-guide/distributed-tracing.html
> > > > for sure.
> > > > Might it be a problem of slow dns names resolution?
> > > > Does it still slow on rows=0?
> > > > --
> > > > Sincerely yours
> > > > Mikhail Khludnev
> > > >
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
