Hi Joel, 1) What are the response times for both methods. Saying one is faster is not specific enough.
Response time for the grouped method is 167 ms for 0.65 million requests. Response time for the collapsed method is 177 ms for 0.65 million requests. 2) What is the cardinality of the collapse field, saying it's high is not specific enough. What is the actual cardinality? Cardinality of the collapse field is around 6.2 Million [image: image.png] 3) Is ngroups used in the grouping query Yes, ngroups is used in grouping query. Thanks Parshant Kumar On Tue, Mar 9, 2021 at 12:30 AM Joel Bernstein <joels...@gmail.com> wrote: > Collapse is designed to outperform grouping in the following scenario: > > There is high cardinality on the group field and group.ngroups is needed. > If either of these conditions is not satisfied grouping will typically be > faster. > > You will need to provide some more information about your setup to get an > answer to the collapse performance question. > > 1) What are the response times for both methods. Saying one is faster is > not specific enough. > 2) What is the cardinality of the collapse field, saying it's high is not > specific enough. What is the actual cardinality? > 3) Is ngroups used in the grouping query. > > > > > > > > > > > > > > > > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Mon, Mar 8, 2021 at 11:30 AM Gajendra Dadheech <gajju3...@gmail.com> > wrote: > > > @prashant Florin means to put null for parentglusrid in documents where > > this field-value is only present in one document [Group has only one > > document]. and then use nullPolicy to include/expand. > > > > > > > > On Mon, Mar 8, 2021 at 6:55 PM Parshant Kumar > > <parshant.ku...@indiamart.com.invalid> wrote: > > > > > client should set to null the field if it's unique. > > > > > > @florin @Gajendra can you please explain more .I am not clear how to > > > perform this. > > > > > > On Mon, Mar 8, 2021 at 6:09 PM Florin Babes <babesflo...@gmail.com> > > wrote: > > > > > > > @Gajendra Our response time dropped by 36% and our rps increased with > > > 27%. > > > > > > > > You have to reindex the core and the client should set to null the > > field > > > if > > > > it's unique. > > > > > > > > În lun., 8 mar. 2021 la 13:18, Parshant Kumar > > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > > > > > > How can we make group_field null? Using nullPolicy=expand ? > > > > > > > > > > On Mon, Mar 8, 2021 at 4:41 PM Florin Babes <babesflo...@gmail.com > > > > > > wrote: > > > > > > > > > > > We improved the performance of collapse by making the group_field > > > null > > > > > for > > > > > > the documents that have an unique value for group_field. This > might > > > > help/ > > > > > > > > > > > > > > > > > > În lun., 8 mar. 2021 la 12:40, Parshant Kumar > > > > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > > > > > > > > > > yes,group_field is having high cardinality. > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > Parshant Kumar > > > > > > > > > > > > > > On Mon, Mar 8, 2021 at 4:06 PM Florin Babes < > > babesflo...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Your group_field has a high cardinality? > > > > > > > > Thanks, > > > > > > > > Florin Babes > > > > > > > > > > > > > > > > În lun., 8 mar. 2021 la 10:35, Parshant Kumar > > > > > > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > > > > > > > > > > > > > > Hi florin, > > > > > > > > > > > > > > > > > > I am using below. > > > > > > > > > > > > > > > > > > 1) fq={!collapse field=parentglusrid} > > > > > > > > > 2) expand.rows=4 > > > > > > > > > 3) expand=true > > > > > > > > > > > > > > > > > > Size of index is around 100GB. > > > > > > > > > Solr version is 6.5 > > > > > > > > > > > > > > > > > > On Mon, Mar 8, 2021 at 1:46 PM Florin Babes < > > > > babesflo...@gmail.com > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > First let's call the field you collapse on group_field > > > > > > > > > > If group_field has a high cardinality you should make > > > > group_field > > > > > > > null > > > > > > > > > for > > > > > > > > > > those documents that have a unique group_field and set > > > > > > > > nullPolicy=expand. > > > > > > > > > > By doing that solr will use less memory for it's internal > > > maps > > > > > (so > > > > > > > > faster > > > > > > > > > > gc) and the head selecting will be faster. > > > > > > > > > > What is your head selecting strategy? Can you share your > fq > > > > which > > > > > > you > > > > > > > > use > > > > > > > > > > for collapsing? > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Florin Babes > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > În lun., 8 mar. 2021 la 06:44, Parshant Kumar > > > > > > > > > > <parshant.ku...@indiamart.com.invalid> a scris: > > > > > > > > > > > > > > > > > > > > > anyone please help > > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 3, 2021 at 4:55 PM Parshant Kumar < > > > > > > > > > > > parshant.ku...@indiamart.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > We have implemented collapse queries in place of > > grouped > > > > > > queries > > > > > > > on > > > > > > > > > our > > > > > > > > > > > > production solr. As mentioned in solr documentation > > > > collapse > > > > > > > > queries > > > > > > > > > > are > > > > > > > > > > > > recommended in place of grouped queries in terms of > > > > > > performance . > > > > > > > > But > > > > > > > > > > > after > > > > > > > > > > > > switching to collapsed queries from grouped queries > > > > response > > > > > > time > > > > > > > > of > > > > > > > > > > > > queries have increased. This is unexpected behaviour, > > the > > > > > > > response > > > > > > > > > time > > > > > > > > > > > > should have been improved but results are opposites. > > > > > > > > > > > > Please someone help why response time is increased > for > > > > > > collapsed > > > > > > > > > > queries. > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > Parshant Kumar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > --
