Oh yeah, glad you found that. It's not really a bug but something to document. The collapse filter can be conceptually thought of as something that occurs last, and thus "sees" documents in the order of your sort parameter. Thus don't specify redundant sort options inside the collapse because it's redundant work.
~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Wed, Mar 17, 2021 at 11:19 AM Florin Babes <babesflo...@gmail.com> wrote: > Yes. It seems that if you specify sort="score desc" the collapser > recomputes the score of the documents as far as I could understand from the > solr's code. > Thanks, > > Florin Babeş > PHP Developer > > babesflo...@gmail.com > > > În mie., 17 mar. 2021 la 15:52, Joel Bernstein <joels...@gmail.com> a > scris: > > > That is an interesting finding. The default behavior is quite fast > compared > > to the sort approach. > > > > > > > > Joel Bernstein > > http://joelsolr.blogspot.com/ > > > > > > On Wed, Mar 17, 2021 at 3:42 AM Florin Babes <babesflo...@gmail.com> > > wrote: > > > > > @Gajendra we used a sort="score desc" even if we only use the head of > the > > > group. By removing this sort and leaving the default behavior > (selecting > > > the head with the highest score) our response time dropped with 42% in > > our > > > synthetic load tests. > > > > > > > > > În joi, 11 mar. 2021 la 20:40 Gajendra Dadheech <gajju3...@gmail.com> > a > > > scris: > > > > > > > @florin > > > > > > > > Great advice. Null key for unique documents is really helpful. Any > > other > > > > such tricks that you are using to improve collapse performance ? > > > > > > > > On Tue, Mar 9, 2021, 2:45 PM Parshant Kumar > > > > <parshant.ku...@indiamart.com.invalid> wrote: > > > > > > > > > Hi Joel, > > > > > > > > > > 1) What are the response times for both methods. Saying one is > faster > > > is > > > > > not specific enough. > > > > > > > > > > Response time for the grouped method is 167 ms for 0.65 million > > > requests. > > > > > Response time for the collapsed method is 177 ms for 0.65 million > > > > requests. > > > > > > > > > > 2) What is the cardinality of the collapse field, saying it's high > is > > > not > > > > > specific enough. What is the actual cardinality? > > > > > > > > > > Cardinality of the collapse field is around 6.2 Million > > > > > > > > > > [image: image.png] > > > > > 3) Is ngroups used in the grouping query > > > > > > > > > > Yes, ngroups is used in grouping query. > > > > > > > > > > Thanks > > > > > Parshant Kumar > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 9, 2021 at 12:30 AM Joel Bernstein <joels...@gmail.com > > > > > > wrote: > > > > > > > > > >> Collapse is designed to outperform grouping in the following > > scenario: > > > > >> > > > > >> There is high cardinality on the group field and group.ngroups is > > > > needed. > > > > >> If either of these conditions is not satisfied grouping will > > typically > > > > be > > > > >> faster. > > > > >> > > > > >> You will need to provide some more information about your setup to > > get > > > > an > > > > >> answer to the collapse performance question. > > > > >> > > > > >> 1) What are the response times for both methods. Saying one is > > faster > > > is > > > > >> not specific enough. > > > > >> 2) What is the cardinality of the collapse field, saying it's high > > is > > > > not > > > > >> specific enough. What is the actual cardinality? > > > > >> 3) Is ngroups used in the grouping query. > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> Joel Bernstein > > > > >> http://joelsolr.blogspot.com/ > > > > >> > > > > >> > > > > >> On Mon, Mar 8, 2021 at 11:30 AM Gajendra Dadheech < > > > gajju3...@gmail.com> > > > > >> wrote: > > > > >> > > > > >> > @prashant Florin means to put null for parentglusrid in > documents > > > > where > > > > >> > this field-value is only present in one document [Group has only > > one > > > > >> > document]. and then use nullPolicy to include/expand. > > > > >> > > > > > >> > > > > > >> > > > > > >> > On Mon, Mar 8, 2021 at 6:55 PM Parshant Kumar > > > > >> > <parshant.ku...@indiamart.com.invalid> wrote: > > > > >> > > > > > >> > > client should set to null the field if it's unique. > > > > >> > > > > > > >> > > @florin @Gajendra can you please explain more .I am not clear > > how > > > to > > > > >> > > perform this. > > > > >> > > > > > > >> > > On Mon, Mar 8, 2021 at 6:09 PM Florin Babes < > > > babesflo...@gmail.com> > > > > >> > wrote: > > > > >> > > > > > > >> > > > @Gajendra Our response time dropped by 36% and our rps > > increased > > > > >> with > > > > >> > > 27%. > > > > >> > > > > > > > >> > > > You have to reindex the core and the client should set to > null > > > the > > > > >> > field > > > > >> > > if > > > > >> > > > it's unique. > > > > >> > > > > > > > >> > > > În lun., 8 mar. 2021 la 13:18, Parshant Kumar > > > > >> > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > >> > > > > > > > >> > > > > How can we make group_field null? Using nullPolicy=expand > ? > > > > >> > > > > > > > > >> > > > > On Mon, Mar 8, 2021 at 4:41 PM Florin Babes < > > > > >> babesflo...@gmail.com> > > > > >> > > > wrote: > > > > >> > > > > > > > > >> > > > > > We improved the performance of collapse by making the > > > > >> group_field > > > > >> > > null > > > > >> > > > > for > > > > >> > > > > > the documents that have an unique value for group_field. > > > This > > > > >> might > > > > >> > > > help/ > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > În lun., 8 mar. 2021 la 12:40, Parshant Kumar > > > > >> > > > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > >> > > > > > > > > > >> > > > > > > yes,group_field is having high cardinality. > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > Thanks > > > > >> > > > > > > Parshant Kumar > > > > >> > > > > > > > > > > >> > > > > > > On Mon, Mar 8, 2021 at 4:06 PM Florin Babes < > > > > >> > babesflo...@gmail.com > > > > >> > > > > > > > >> > > > > > wrote: > > > > >> > > > > > > > > > > >> > > > > > > > Your group_field has a high cardinality? > > > > >> > > > > > > > Thanks, > > > > >> > > > > > > > Florin Babes > > > > >> > > > > > > > > > > > >> > > > > > > > În lun., 8 mar. 2021 la 10:35, Parshant Kumar > > > > >> > > > > > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > >> > > > > > > > > > > > >> > > > > > > > > Hi florin, > > > > >> > > > > > > > > > > > > >> > > > > > > > > I am using below. > > > > >> > > > > > > > > > > > > >> > > > > > > > > 1) fq={!collapse field=parentglusrid} > > > > >> > > > > > > > > 2) expand.rows=4 > > > > >> > > > > > > > > 3) expand=true > > > > >> > > > > > > > > > > > > >> > > > > > > > > Size of index is around 100GB. > > > > >> > > > > > > > > Solr version is 6.5 > > > > >> > > > > > > > > > > > > >> > > > > > > > > On Mon, Mar 8, 2021 at 1:46 PM Florin Babes < > > > > >> > > > babesflo...@gmail.com > > > > >> > > > > > > > > > >> > > > > > > > wrote: > > > > >> > > > > > > > > > > > > >> > > > > > > > > > Hello, > > > > >> > > > > > > > > > First let's call the field you collapse on > > > group_field > > > > >> > > > > > > > > > If group_field has a high cardinality you should > > > make > > > > >> > > > group_field > > > > >> > > > > > > null > > > > >> > > > > > > > > for > > > > >> > > > > > > > > > those documents that have a unique group_field > and > > > set > > > > >> > > > > > > > nullPolicy=expand. > > > > >> > > > > > > > > > By doing that solr will use less memory for it's > > > > >> internal > > > > >> > > maps > > > > >> > > > > (so > > > > >> > > > > > > > faster > > > > >> > > > > > > > > > gc) and the head selecting will be faster. > > > > >> > > > > > > > > > What is your head selecting strategy? Can you > > share > > > > >> your fq > > > > >> > > > which > > > > >> > > > > > you > > > > >> > > > > > > > use > > > > >> > > > > > > > > > for collapsing? > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > Thanks, > > > > >> > > > > > > > > > Florin Babes > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > În lun., 8 mar. 2021 la 06:44, Parshant Kumar > > > > >> > > > > > > > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > anyone please help > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > On Wed, Mar 3, 2021 at 4:55 PM Parshant Kumar > < > > > > >> > > > > > > > > > > parshant.ku...@indiamart.com> > > > > >> > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > Hi all, > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > We have implemented collapse queries in > place > > of > > > > >> > grouped > > > > >> > > > > > queries > > > > >> > > > > > > on > > > > >> > > > > > > > > our > > > > >> > > > > > > > > > > > production solr. As mentioned in solr > > > > documentation > > > > >> > > > collapse > > > > >> > > > > > > > queries > > > > >> > > > > > > > > > are > > > > >> > > > > > > > > > > > recommended in place of grouped queries in > > terms > > > > of > > > > >> > > > > > performance . > > > > >> > > > > > > > But > > > > >> > > > > > > > > > > after > > > > >> > > > > > > > > > > > switching to collapsed queries from grouped > > > > queries > > > > >> > > > response > > > > >> > > > > > time > > > > >> > > > > > > > of > > > > >> > > > > > > > > > > > queries have increased. This is unexpected > > > > >> behaviour, > > > > >> > the > > > > >> > > > > > > response > > > > >> > > > > > > > > time > > > > >> > > > > > > > > > > > should have been improved but results are > > > > opposites. > > > > >> > > > > > > > > > > > Please someone help why response time is > > > increased > > > > >> for > > > > >> > > > > > collapsed > > > > >> > > > > > > > > > queries. > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > Thanks > > > > >> > > > > > > > > > > > Parshant Kumar > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > -- > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > -- > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > -- > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > -- > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > -- > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
