Re: Increase in response time in case of collapse queries.

2021-03-17 Thread Florin Babes 
@Gajendra we used a sort="score desc" even if we only use the head of the
group. By removing this sort and leaving the default behavior (selecting
the head with the highest score) our response time dropped with 42% in our
synthetic load tests.


În joi, 11 mar. 2021 la 20:40 Gajendra Dadheech  a
scris:

> @florin
>
> Great advice. Null key for unique documents is really helpful. Any other
> such tricks that you are using to improve collapse performance ?
>
> On Tue, Mar 9, 2021, 2:45 PM Parshant Kumar
>  wrote:
>
> > Hi Joel,
> >
> > 1) What are the response times for both methods. Saying one is faster is
> > not specific enough.
> >
> > Response time for the grouped method is 167 ms for 0.65 million requests.
> > Response time for the collapsed method is 177 ms for 0.65 million
> requests.
> >
> > 2) What is the cardinality of the collapse field, saying it's high is not
> > specific enough. What is the actual cardinality?
> >
> > Cardinality of the collapse field is around 6.2 Million
> >
> > [image: image.png]
> > 3) Is ngroups used in the grouping query
> >
> > Yes, ngroups is used in grouping query.
> >
> > Thanks
> > Parshant Kumar
> >
> >
> >
> >
> > On Tue, Mar 9, 2021 at 12:30 AM Joel Bernstein 
> wrote:
> >
> >> Collapse is designed to outperform grouping in the following scenario:
> >>
> >> There is high cardinality on the group field and group.ngroups is
> needed.
> >> If either of these conditions is not satisfied grouping will typically
> be
> >> faster.
> >>
> >> You will need to provide some more information about your setup to get
> an
> >> answer to the collapse performance question.
> >>
> >> 1) What are the response times for both methods. Saying one is faster is
> >> not specific enough.
> >> 2) What is the cardinality of the collapse field, saying it's high is
> not
> >> specific enough. What is the actual cardinality?
> >> 3) Is ngroups used in the grouping query.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Joel Bernstein
> >> http://joelsolr.blogspot.com/
> >>
> >>
> >> On Mon, Mar 8, 2021 at 11:30 AM Gajendra Dadheech 
> >> wrote:
> >>
> >> > @prashant Florin means to put null for parentglusrid in documents
> where
> >> > this field-value is only present in one document [Group has only one
> >> > document]. and then use nullPolicy to include/expand.
> >> >
> >> >
> >> >
> >> > On Mon, Mar 8, 2021 at 6:55 PM Parshant Kumar
> >> >  wrote:
> >> >
> >> > > client should set to null the field if it's unique.
> >> > >
> >> > > @florin @Gajendra can you please explain more .I am not clear how to
> >> > > perform this.
> >> > >
> >> > > On Mon, Mar 8, 2021 at 6:09 PM Florin Babes 
> >> > wrote:
> >> > >
> >> > > > @Gajendra Our response time dropped by 36% and our rps increased
> >> with
> >> > > 27%.
> >> > > >
> >> > > > You have to reindex the core and the client should set to null the
> >> > field
> >> > > if
> >> > > > it's unique.
> >> > > >
> >> > > > În lun., 8 mar. 2021 la 13:18, Parshant Kumar
> >> > > >  a scris:
> >> > > >
> >> > > > > How can we make group_field null? Using nullPolicy=expand ?
> >> > > > >
> >> > > > > On Mon, Mar 8, 2021 at 4:41 PM Florin Babes <
> >> babesflo...@gmail.com>
> >> > > > wrote:
> >> > > > >
> >> > > > > > We improved the performance of collapse by making the
> >> group_field
> >> > > null
> >> > > > > for
> >> > > > > > the documents that have an unique value for group_field. This
> >> might
> >> > > > help/
> >> > > > > >
> >> > > > > >
> >> > > > > > În lun., 8 mar. 2021 la 12:40, Parshant Kumar
> >> > > > > >  a scris:
> >> > > > > >
> >> > > > > > > yes,group_field is having high cardinality.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Thanks
> >> > > > > > > Parshant Kumar
> >> > > > > > >
> >> > > > > > > On Mon, Mar 8, 2021 at 4:06 PM Florin Babes <
> >> > babesflo...@gmail.com
> >> > > >
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > Your group_field has a high cardinality?
> >> > > > > > > > Thanks,
> >> > > > > > > > Florin Babes
> >> > > > > > > >
> >> > > > > > > > În lun., 8 mar. 2021 la 10:35, Parshant Kumar
> >> > > > > > > >  a scris:
> >> > > > > > > >
> >> > > > > > > > > Hi florin,
> >> > > > > > > > >
> >> > > > > > > > > I am using below.
> >> > > > > > > > >
> >> > > > > > > > > 1) fq={!collapse field=parentglusrid}
> >> > > > > > > > > 2) expand.rows=4
> >> > > > > > > > > 3) expand=true
> >> > > > > > > > >
> >> > > > > > > > > Size of index is around 100GB.
> >> > > > > > > > > Solr version is 6.5
> >> > > > > > > > >
> >> > > > > > > > > On Mon, Mar 8, 2021 at 1:46 PM Florin Babes <
> >> > > > babesflo...@gmail.com
> >> > > > > >
> >> > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Hello,
> >> > > > > > > > > > First let's call the field you collapse on group_field
> >> > > > > > > > > > If group_field has a high cardinality you should make
> >> > > > group_field
> >> > > > > > > null
> >> > > > > > > > > for
> >> > > > > > > > > > those doc

Building suggester on multiple nodes for solr-cloud; failes due to forward

2021-03-17 Thread Martin Knoller 
Hi there

We have some issues with our solr-cloud setup and the suggester feature.

We run solr 8.8.0 and have three nodes so far. We create collections in an 
import environment with a local docker solr, where we create a backup and 
restore the collection on the production system. Along the collection itself we 
need the suggester index too for our functionality. Since this is not part of 
the backup/restore we have to build on the production system. And since this is 
not build on every node automatically we call each node to build it. This 
already feels wrong but has been working for years by now.

We build by calling the api with the following params: 
'suggest.build=true','suggest=true', 'q=*:*'.. . The problem is, that some 
nodes do forward this request, visible by the reponse ( 
responseHeader.params._forwardedCoung == 1) and so our restore/suggest-build 
process ends up with having the suggest feature enabled on 2 nodes instead of 
three.

It feels like we do too much workaround and there should be another way. If 
not, how can we force the suggest build on a node that does not want to?


Best Regards

Martin Knoller Stocker

Re: documentCache vs IO Cache?

2021-03-17 Thread Karl Stoney 
OK this makes sense, thanks for the reply Shawn.

On 13/03/2021, 21:34, "Shawn Heisey"  wrote:

On 3/13/2021 11:36 AM, Karl Stoney wrote:
> Apologies if this is a silly question, I just can't find anything 
explaining the benefits online.
>
> Would anyone be able to tell me why you would use a documentCache, if you 
have sufficient RAM on your machine that the OS disk cache is effectively 
caching all the documents anyway?

I'm pretty sure that the document cache stores uncompressed data.  With
the disk cache, the decompression step would be required, using CPU
resources and taking a little bit of time.  Reading from the document
cache would be faster and not hit the CPU as hard.

Since I think version 4.1, Solr (Lucene, really) writes stored fields in
compressed format.

Thanks,
Shawn

Unless expressly stated otherwise in this email, this e-mail is sent on behalf 
of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, 
Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited 
is part of the Auto Trader Group Plc group. This email and any files 
transmitted with it are confidential and may be legally privileged, and 
intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error please notify the sender. 
This email message has been swept for the presence of computer viruses.

Re: Increase in response time in case of collapse queries.

2021-03-17 Thread Joel Bernstein 
That is an interesting finding. The default behavior is quite fast compared
to the sort approach.



Joel Bernstein
http://joelsolr.blogspot.com/


On Wed, Mar 17, 2021 at 3:42 AM Florin Babes  wrote:

> @Gajendra we used a sort="score desc" even if we only use the head of the
> group. By removing this sort and leaving the default behavior (selecting
> the head with the highest score) our response time dropped with 42% in our
> synthetic load tests.
>
>
> În joi, 11 mar. 2021 la 20:40 Gajendra Dadheech  a
> scris:
>
> > @florin
> >
> > Great advice. Null key for unique documents is really helpful. Any other
> > such tricks that you are using to improve collapse performance ?
> >
> > On Tue, Mar 9, 2021, 2:45 PM Parshant Kumar
> >  wrote:
> >
> > > Hi Joel,
> > >
> > > 1) What are the response times for both methods. Saying one is faster
> is
> > > not specific enough.
> > >
> > > Response time for the grouped method is 167 ms for 0.65 million
> requests.
> > > Response time for the collapsed method is 177 ms for 0.65 million
> > requests.
> > >
> > > 2) What is the cardinality of the collapse field, saying it's high is
> not
> > > specific enough. What is the actual cardinality?
> > >
> > > Cardinality of the collapse field is around 6.2 Million
> > >
> > > [image: image.png]
> > > 3) Is ngroups used in the grouping query
> > >
> > > Yes, ngroups is used in grouping query.
> > >
> > > Thanks
> > > Parshant Kumar
> > >
> > >
> > >
> > >
> > > On Tue, Mar 9, 2021 at 12:30 AM Joel Bernstein 
> > wrote:
> > >
> > >> Collapse is designed to outperform grouping in the following scenario:
> > >>
> > >> There is high cardinality on the group field and group.ngroups is
> > needed.
> > >> If either of these conditions is not satisfied grouping will typically
> > be
> > >> faster.
> > >>
> > >> You will need to provide some more information about your setup to get
> > an
> > >> answer to the collapse performance question.
> > >>
> > >> 1) What are the response times for both methods. Saying one is faster
> is
> > >> not specific enough.
> > >> 2) What is the cardinality of the collapse field, saying it's high is
> > not
> > >> specific enough. What is the actual cardinality?
> > >> 3) Is ngroups used in the grouping query.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Joel Bernstein
> > >> http://joelsolr.blogspot.com/
> > >>
> > >>
> > >> On Mon, Mar 8, 2021 at 11:30 AM Gajendra Dadheech <
> gajju3...@gmail.com>
> > >> wrote:
> > >>
> > >> > @prashant Florin means to put null for parentglusrid in documents
> > where
> > >> > this field-value is only present in one document [Group has only one
> > >> > document]. and then use nullPolicy to include/expand.
> > >> >
> > >> >
> > >> >
> > >> > On Mon, Mar 8, 2021 at 6:55 PM Parshant Kumar
> > >> >  wrote:
> > >> >
> > >> > > client should set to null the field if it's unique.
> > >> > >
> > >> > > @florin @Gajendra can you please explain more .I am not clear how
> to
> > >> > > perform this.
> > >> > >
> > >> > > On Mon, Mar 8, 2021 at 6:09 PM Florin Babes <
> babesflo...@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > > @Gajendra Our response time dropped by 36% and our rps increased
> > >> with
> > >> > > 27%.
> > >> > > >
> > >> > > > You have to reindex the core and the client should set to null
> the
> > >> > field
> > >> > > if
> > >> > > > it's unique.
> > >> > > >
> > >> > > > În lun., 8 mar. 2021 la 13:18, Parshant Kumar
> > >> > > >  a scris:
> > >> > > >
> > >> > > > > How can we make group_field null? Using nullPolicy=expand ?
> > >> > > > >
> > >> > > > > On Mon, Mar 8, 2021 at 4:41 PM Florin Babes <
> > >> babesflo...@gmail.com>
> > >> > > > wrote:
> > >> > > > >
> > >> > > > > > We improved the performance of collapse by making the
> > >> group_field
> > >> > > null
> > >> > > > > for
> > >> > > > > > the documents that have an unique value for group_field.
> This
> > >> might
> > >> > > > help/
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > În lun., 8 mar. 2021 la 12:40, Parshant Kumar
> > >> > > > > >  a scris:
> > >> > > > > >
> > >> > > > > > > yes,group_field is having high cardinality.
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > Thanks
> > >> > > > > > > Parshant Kumar
> > >> > > > > > >
> > >> > > > > > > On Mon, Mar 8, 2021 at 4:06 PM Florin Babes <
> > >> > babesflo...@gmail.com
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > > Your group_field has a high cardinality?
> > >> > > > > > > > Thanks,
> > >> > > > > > > > Florin Babes
> > >> > > > > > > >
> > >> > > > > > > > În lun., 8 mar. 2021 la 10:35, Parshant Kumar
> > >> > > > > > > >  a scris:
> > >> > > > > > > >
> > >> > > > > > > > > Hi florin,
> > >> > > > > > > > >
> > >> > > > > > > > > I am using below.
> > >> > > > > > > > >
> > >> > > > > > > > > 1) fq={!collapse field=parentglusrid}
> > >> > > > > > > > > 2) expand.rows=4
> > >> > > > > > > > > 3) expand=true
>

Re: Increase in response time in case of collapse queries.

2021-03-17 Thread Florin Babes 
Yes. It seems that if you specify sort="score desc" the collapser
recomputes the score of the documents as far as I could understand from the
solr's code.
Thanks,

Florin Babeş
PHP Developer

babesflo...@gmail.com


În mie., 17 mar. 2021 la 15:52, Joel Bernstein  a scris:

> That is an interesting finding. The default behavior is quite fast compared
> to the sort approach.
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Wed, Mar 17, 2021 at 3:42 AM Florin Babes 
> wrote:
>
> > @Gajendra we used a sort="score desc" even if we only use the head of the
> > group. By removing this sort and leaving the default behavior (selecting
> > the head with the highest score) our response time dropped with 42% in
> our
> > synthetic load tests.
> >
> >
> > În joi, 11 mar. 2021 la 20:40 Gajendra Dadheech  a
> > scris:
> >
> > > @florin
> > >
> > > Great advice. Null key for unique documents is really helpful. Any
> other
> > > such tricks that you are using to improve collapse performance ?
> > >
> > > On Tue, Mar 9, 2021, 2:45 PM Parshant Kumar
> > >  wrote:
> > >
> > > > Hi Joel,
> > > >
> > > > 1) What are the response times for both methods. Saying one is faster
> > is
> > > > not specific enough.
> > > >
> > > > Response time for the grouped method is 167 ms for 0.65 million
> > requests.
> > > > Response time for the collapsed method is 177 ms for 0.65 million
> > > requests.
> > > >
> > > > 2) What is the cardinality of the collapse field, saying it's high is
> > not
> > > > specific enough. What is the actual cardinality?
> > > >
> > > > Cardinality of the collapse field is around 6.2 Million
> > > >
> > > > [image: image.png]
> > > > 3) Is ngroups used in the grouping query
> > > >
> > > > Yes, ngroups is used in grouping query.
> > > >
> > > > Thanks
> > > > Parshant Kumar
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Mar 9, 2021 at 12:30 AM Joel Bernstein 
> > > wrote:
> > > >
> > > >> Collapse is designed to outperform grouping in the following
> scenario:
> > > >>
> > > >> There is high cardinality on the group field and group.ngroups is
> > > needed.
> > > >> If either of these conditions is not satisfied grouping will
> typically
> > > be
> > > >> faster.
> > > >>
> > > >> You will need to provide some more information about your setup to
> get
> > > an
> > > >> answer to the collapse performance question.
> > > >>
> > > >> 1) What are the response times for both methods. Saying one is
> faster
> > is
> > > >> not specific enough.
> > > >> 2) What is the cardinality of the collapse field, saying it's high
> is
> > > not
> > > >> specific enough. What is the actual cardinality?
> > > >> 3) Is ngroups used in the grouping query.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> Joel Bernstein
> > > >> http://joelsolr.blogspot.com/
> > > >>
> > > >>
> > > >> On Mon, Mar 8, 2021 at 11:30 AM Gajendra Dadheech <
> > gajju3...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > @prashant Florin means to put null for parentglusrid in documents
> > > where
> > > >> > this field-value is only present in one document [Group has only
> one
> > > >> > document]. and then use nullPolicy to include/expand.
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Mon, Mar 8, 2021 at 6:55 PM Parshant Kumar
> > > >> >  wrote:
> > > >> >
> > > >> > > client should set to null the field if it's unique.
> > > >> > >
> > > >> > > @florin @Gajendra can you please explain more .I am not clear
> how
> > to
> > > >> > > perform this.
> > > >> > >
> > > >> > > On Mon, Mar 8, 2021 at 6:09 PM Florin Babes <
> > babesflo...@gmail.com>
> > > >> > wrote:
> > > >> > >
> > > >> > > > @Gajendra Our response time dropped by 36% and our rps
> increased
> > > >> with
> > > >> > > 27%.
> > > >> > > >
> > > >> > > > You have to reindex the core and the client should set to null
> > the
> > > >> > field
> > > >> > > if
> > > >> > > > it's unique.
> > > >> > > >
> > > >> > > > În lun., 8 mar. 2021 la 13:18, Parshant Kumar
> > > >> > > >  a scris:
> > > >> > > >
> > > >> > > > > How can we make group_field null? Using nullPolicy=expand ?
> > > >> > > > >
> > > >> > > > > On Mon, Mar 8, 2021 at 4:41 PM Florin Babes <
> > > >> babesflo...@gmail.com>
> > > >> > > > wrote:
> > > >> > > > >
> > > >> > > > > > We improved the performance of collapse by making the
> > > >> group_field
> > > >> > > null
> > > >> > > > > for
> > > >> > > > > > the documents that have an unique value for group_field.
> > This
> > > >> might
> > > >> > > > help/
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > În lun., 8 mar. 2021 la 12:40, Parshant Kumar
> > > >> > > > > >  a scris:
> > > >> > > > > >
> > > >> > > > > > > yes,group_field is having high cardinality.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > Thanks
> > > >> > > > > > > Parshant Kumar
> > > >> > > > > > >
> > > >> > > > > > > On Mon, Mar 8, 2021 at 4:06 PM Florin Babes <

Re: Difference in response times between direct to shard vs random shard with route param

2021-03-17 Thread yasoobhaider 
Hi. Any help would be great



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Rule Based Authorization

2021-03-17 Thread Jason Gerlowski 
Hi,

Solr's Rule-Based Authz is complex to configure but this should be
possible.  If you share a security.json configuration you attempted, I
can suggest tweaks from there to get you what you need.

In case you haven't seen them, the docs here might be helpful:
https://solr.apache.org/guide/8_8/rule-based-authorization-plugin.html

Best,

Jason

On Thu, Feb 25, 2021 at 8:31 AM Subhajit Das  wrote:
>
> Hi There,
>
> I am trying to create a rule based authorization for two types of user.
>
> Role : Access
> -
> power-user : Everything except data change in collections/cores
> ui-user : UI access to view all data. But no edit access except data change 
> in collections/cores
>
> How to implement this?
>
> No predefined permissions for this. Tried by adding all read permissions to 
> this ui-user. But dosent wok. One of failing APIs is “/admin/info/system”. 
> Cant match this url, with custom permission also.
>
> Please help.
>
> Thanks in advance.
>

Disable commits during a REINDEXCOLLECTION

2021-03-17 Thread Karl Stoney 
Hi all,
We're wanting to use REINDEXCOLLECTION, but our config has a relatively 
aggressive autoCommit interval configured by default (intentionally).

Ideally I'd like to be able to disable hard commits for the duration of the 
reindex, but can't see a way to do that without pushing a whole new config and 
reloading that collection.

Does anyone know of a ninja way at runtime to disable autoCommits on a 
collection (solr cloud)?

Thanks
Unless expressly stated otherwise in this email, this e-mail is sent on behalf 
of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, 
Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited 
is part of the Auto Trader Group Plc group. This email and any files 
transmitted with it are confidential and may be legally privileged, and 
intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error please notify the sender. 
This email message has been swept for the presence of computer viruses.

REINDEXCOLLECTION gradually slows

2021-03-17 Thread Karl Stoney 
Hey,
So we're trying to use REINDEXCOLLECTION again (solr 8.8) and similar to the 
last time we tried it (8.1) we see it gradually slow down; as you can see from 
the logs below.

I terminated it early but you can see the trend in the processingRatePerSecond.

I'm wondering if anyone has any idea's why this might happen? 🤷‍♂️. The machine 
isn't CPU or Memory constrained, so it makes reindexing larger collections 
really hard (we've had to code a separate app to do it).

Thanks



```
{"timestamp":"2021-03-17T19:04:44.703Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":156000,"targetDocs":787176,"percentComplete":20,"durationSeconds":125,"processingRatePerSecond":1248}
{"timestamp":"2021-03-17T19:04:49.719Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":156000,"targetDocs":787176,"percentComplete":20,"durationSeconds":130,"processingRatePerSecond":1200}
{"timestamp":"2021-03-17T19:04:54.736Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":172000,"targetDocs":787176,"percentComplete":22,"durationSeconds":135,"processingRatePerSecond":1274}
{"timestamp":"2021-03-17T19:04:59.751Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":172000,"targetDocs":787176,"percentComplete":22,"durationSeconds":140,"processingRatePerSecond":1229}
{"timestamp":"2021-03-17T19:05:04.765Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":188000,"targetDocs":787176,"percentComplete":24,"durationSeconds":145,"processingRatePerSecond":1297}
{"timestamp":"2021-03-17T19:05:09.782Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":188000,"targetDocs":787176,"percentComplete":24,"durationSeconds":150,"processingRatePerSecond":1253}
{"timestamp":"2021-03-17T19:05:14.798Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":202000,"targetDocs":787176,"percentComplete":26,"durationSeconds":155,"processingRatePerSecond":1303}
{"timestamp":"2021-03-17T19:05:19.816Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":202000,"targetDocs":787176,"percentComplete":26,"durationSeconds":161,"processingRatePerSecond":1255}
{"timestamp":"2021-03-17T19:05:24.832Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":212000,"targetDocs":787176,"percentComplete":27,"durationSeconds":166,"processingRatePerSecond":1277}
{"timestamp":"2021-03-17T19:05:29.850Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":212000,"targetDocs":787176,"percentComplete":27,"durationSeconds":171,"processingRatePerSecond":1240}
{"timestamp":"2021-03-17T19:05:34.866Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":22,"targetDocs":787176,"percentComplete":28,"durationSeconds":176,"processingRatePerSecond":1250}
{"timestamp":"2021-03-17T19:05:39.884Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":22,"targetDocs":787176,"percentComplete":28,"durationSeconds":181,"processingRatePerSecond":1215}
{"timestamp":"2021-03-17T19:05:44.899Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","configName":"at-uk-002-509","rows":2000,"processedDocs":228000,"targetDocs":787176,"percentComplete":29,"durationSeconds":186,"processingRatePerSecond":1226}
{"timestamp":"2021-03-17T19:05:49.916Z","level":"info","module":"solr","message":"reindex
 still 
running","sourceCollection":"at-uk-002","destinationCollection":"karl-test","confi

Re: Increase in response time in case of collapse queries.

2021-03-17 Thread David Smiley 
Oh yeah, glad you found that.  It's not really a bug but something to
document.  The collapse filter can be conceptually thought of as something
that occurs last, and thus "sees" documents in the order of your sort
parameter.  Thus don't specify redundant sort options inside the collapse
because it's redundant work.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Mar 17, 2021 at 11:19 AM Florin Babes  wrote:

> Yes. It seems that if you specify sort="score desc" the collapser
> recomputes the score of the documents as far as I could understand from the
> solr's code.
> Thanks,
>
> Florin Babeş
> PHP Developer
>
> babesflo...@gmail.com
>
>
> În mie., 17 mar. 2021 la 15:52, Joel Bernstein  a
> scris:
>
> > That is an interesting finding. The default behavior is quite fast
> compared
> > to the sort approach.
> >
> >
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> >
> > On Wed, Mar 17, 2021 at 3:42 AM Florin Babes 
> > wrote:
> >
> > > @Gajendra we used a sort="score desc" even if we only use the head of
> the
> > > group. By removing this sort and leaving the default behavior
> (selecting
> > > the head with the highest score) our response time dropped with 42% in
> > our
> > > synthetic load tests.
> > >
> > >
> > > În joi, 11 mar. 2021 la 20:40 Gajendra Dadheech 
> a
> > > scris:
> > >
> > > > @florin
> > > >
> > > > Great advice. Null key for unique documents is really helpful. Any
> > other
> > > > such tricks that you are using to improve collapse performance ?
> > > >
> > > > On Tue, Mar 9, 2021, 2:45 PM Parshant Kumar
> > > >  wrote:
> > > >
> > > > > Hi Joel,
> > > > >
> > > > > 1) What are the response times for both methods. Saying one is
> faster
> > > is
> > > > > not specific enough.
> > > > >
> > > > > Response time for the grouped method is 167 ms for 0.65 million
> > > requests.
> > > > > Response time for the collapsed method is 177 ms for 0.65 million
> > > > requests.
> > > > >
> > > > > 2) What is the cardinality of the collapse field, saying it's high
> is
> > > not
> > > > > specific enough. What is the actual cardinality?
> > > > >
> > > > > Cardinality of the collapse field is around 6.2 Million
> > > > >
> > > > > [image: image.png]
> > > > > 3) Is ngroups used in the grouping query
> > > > >
> > > > > Yes, ngroups is used in grouping query.
> > > > >
> > > > > Thanks
> > > > > Parshant Kumar
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Mar 9, 2021 at 12:30 AM Joel Bernstein  >
> > > > wrote:
> > > > >
> > > > >> Collapse is designed to outperform grouping in the following
> > scenario:
> > > > >>
> > > > >> There is high cardinality on the group field and group.ngroups is
> > > > needed.
> > > > >> If either of these conditions is not satisfied grouping will
> > typically
> > > > be
> > > > >> faster.
> > > > >>
> > > > >> You will need to provide some more information about your setup to
> > get
> > > > an
> > > > >> answer to the collapse performance question.
> > > > >>
> > > > >> 1) What are the response times for both methods. Saying one is
> > faster
> > > is
> > > > >> not specific enough.
> > > > >> 2) What is the cardinality of the collapse field, saying it's high
> > is
> > > > not
> > > > >> specific enough. What is the actual cardinality?
> > > > >> 3) Is ngroups used in the grouping query.
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> Joel Bernstein
> > > > >> http://joelsolr.blogspot.com/
> > > > >>
> > > > >>
> > > > >> On Mon, Mar 8, 2021 at 11:30 AM Gajendra Dadheech <
> > > gajju3...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > @prashant Florin means to put null for parentglusrid in
> documents
> > > > where
> > > > >> > this field-value is only present in one document [Group has only
> > one
> > > > >> > document]. and then use nullPolicy to include/expand.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Mon, Mar 8, 2021 at 6:55 PM Parshant Kumar
> > > > >> >  wrote:
> > > > >> >
> > > > >> > > client should set to null the field if it's unique.
> > > > >> > >
> > > > >> > > @florin @Gajendra can you please explain more .I am not clear
> > how
> > > to
> > > > >> > > perform this.
> > > > >> > >
> > > > >> > > On Mon, Mar 8, 2021 at 6:09 PM Florin Babes <
> > > babesflo...@gmail.com>
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > > @Gajendra Our response time dropped by 36% and our rps
> > increased
> > > > >> with
> > > > >> > > 27%.
> > > > >> > > >
> > > > >> > > > You have to reindex the core and the client should set to
> null
> > > the
> > > > >> > field
> > > > >> > > if
> > > > >> > > > it's unique.
> > > > >> > > >
> > > > >> > > > În lun., 8 mar. 2021 la 13:18, Parshant Kumar
> > > > >> > > >  a scris:
> > > > >> > > >
> > > > >> > > > > How can we make group_field null? Using nullPolicy=expand
> ?
> > > >

Re: Disable commits during a REINDEXCOLLECTION

2021-03-17 Thread David Smiley 
Hi Karl,

Look into the "config apI".  Let us know how it goes!
Ideally, this feature would do this automatically.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
 wrote:

> Hi all,
> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
> aggressive autoCommit interval configured by default (intentionally).
>
> Ideally I'd like to be able to disable hard commits for the duration of
> the reindex, but can't see a way to do that without pushing a whole new
> config and reloading that collection.
>
> Does anyone know of a ninja way at runtime to disable autoCommits on a
> collection (solr cloud)?
>
> Thanks
> Unless expressly stated otherwise in this email, this e-mail is sent on
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
> Trader Limited is part of the Auto Trader Group Plc group. This email and
> any files transmitted with it are confidential and may be legally
> privileged, and intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this email in error please
> notify the sender. This email message has been swept for the presence of
> computer viruses.
>

Re: NRT Real time Get with documentCache

2021-03-17 Thread David Smiley 
I agree with Erick that it shouldn't matter at all.  The freshness is
indifferent to the presence of that cache, since Solr is smart enough to
know when it's dirty.
The document cache is used by RTG and thus will play a role in performance
of it.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Feb 4, 2020 at 2:31 AM Karl Stoney
 wrote:

> Great stuff thank you Erick
>
> On 04/02/2020, 00:17, "Erick Erickson"  wrote:
>
> The documentCache shouldn’t matter at all. RTG should return the
> latest doc by maintaining a pointer into the tlogs and returning that
> version.
>
> > On Feb 3, 2020, at 6:43 PM, Karl Stoney <
> karl.sto...@autotrader.co.uk.INVALID> wrote:
> >
> > Hi,
> > Could anyone let me know if a real time get would return a cached,
> up to date version of a document if we enabled documentCache?
> >
> > Thanks
> > Karl
> > This e-mail is sent on behalf of Auto Trader Group Plc, Registered
> Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in
> England No. 9439967). This email and any files transmitted with it are
> confidential and may be legally privileged, and intended solely for the use
> of the individual or entity to whom they are addressed. If you have
> received this email in error please notify the sender. This email message
> has been swept for the presence of computer viruses.
>
>
>
> This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office:
> 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England
> No. 9439967). This email and any files transmitted with it are confidential
> and may be legally privileged, and intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> email in error please notify the sender. This email message has been swept
> for the presence of computer viruses.
>

Re: Disable commits during a REINDEXCOLLECTION

2021-03-17 Thread Karl Stoney 
Can you believe I had never come across that before!  Thanks!

Every day’s a school day

Get Outlook for iOS

From: David Smiley 
Sent: Wednesday, March 17, 2021 8:48:23 PM
To: users@solr.apache.org 
Subject: Re: Disable commits during a REINDEXCOLLECTION

Hi Karl,

Look into the "config apI".  Let us know how it goes!
Ideally, this feature would do this automatically.

~ David Smiley
Apache Lucene/Solr Search Developer
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7C3bd0a3557c6841f4a91008d8e9860bbe%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516109216986721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JcNE3AWytx2Ny5lyrH82EvS4hzI9eiCBImlZi0Az86M%3D&reserved=0


On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
 wrote:

> Hi all,
> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
> aggressive autoCommit interval configured by default (intentionally).
>
> Ideally I'd like to be able to disable hard commits for the duration of
> the reindex, but can't see a way to do that without pushing a whole new
> config and reloading that collection.
>
> Does anyone know of a ninja way at runtime to disable autoCommits on a
> collection (solr cloud)?
>
> Thanks
> Unless expressly stated otherwise in this email, this e-mail is sent on
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
> Trader Limited is part of the Auto Trader Group Plc group. This email and
> any files transmitted with it are confidential and may be legally
> privileged, and intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this email in error please
> notify the sender. This email message has been swept for the presence of
> computer viruses.
>
Unless expressly stated otherwise in this email, this e-mail is sent on behalf 
of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, 
Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited 
is part of the Auto Trader Group Plc group. This email and any files 
transmitted with it are confidential and may be legally privileged, and 
intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error please notify the sender. 
This email message has been swept for the presence of computer viruses.

Re: Disable commits during a REINDEXCOLLECTION

2021-03-17 Thread Vivaldi 
I thought the REINDEXCOLLECTION puts the source collection in read-only mode, 
am I wrong? Doesn’t that alsı disable commits?

Sent from my iPhone

> On 18 Mar 2021, at 00:28, Karl Stoney  
> wrote:
> 
> Can you believe I had never come across that before!  Thanks!
> 
> Every day’s a school day
> 
> Get Outlook for iOS
> 
> From: David Smiley 
> Sent: Wednesday, March 17, 2021 8:48:23 PM
> To: users@solr.apache.org 
> Subject: Re: Disable commits during a REINDEXCOLLECTION
> 
> Hi Karl,
> 
> Look into the "config apI".  Let us know how it goes!
> Ideally, this feature would do this automatically.
> 
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7C3bd0a3557c6841f4a91008d8e9860bbe%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516109216986721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JcNE3AWytx2Ny5lyrH82EvS4hzI9eiCBImlZi0Az86M%3D&reserved=0
> 
> 
> On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
>  wrote:
> 
>> Hi all,
>> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
>> aggressive autoCommit interval configured by default (intentionally).
>> 
>> Ideally I'd like to be able to disable hard commits for the duration of
>> the reindex, but can't see a way to do that without pushing a whole new
>> config and reloading that collection.
>> 
>> Does anyone know of a ninja way at runtime to disable autoCommits on a
>> collection (solr cloud)?
>> 
>> Thanks
>> Unless expressly stated otherwise in this email, this e-mail is sent on
>> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
>> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
>> Trader Limited is part of the Auto Trader Group Plc group. This email and
>> any files transmitted with it are confidential and may be legally
>> privileged, and intended solely for the use of the individual or entity to
>> whom they are addressed. If you have received this email in error please
>> notify the sender. This email message has been swept for the presence of
>> computer viruses.
>> 
> Unless expressly stated otherwise in this email, this e-mail is sent on 
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, 
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto 
> Trader Limited is part of the Auto Trader Group Plc group. This email and any 
> files transmitted with it are confidential and may be legally privileged, and 
> intended solely for the use of the individual or entity to whom they are 
> addressed. If you have received this email in error please notify the sender. 
> This email message has been swept for the presence of computer viruses.

Re: Disable commits during a REINDEXCOLLECTION

2021-03-17 Thread Karl Stoney 
That's the source collection, I'm referring to the destination.

@David - seeing as REINDEX creates the target collection, I had to start the 
process and then attempt to set the config on the destination.
However setting the destination config kills the REINDEX:

```
22:18:40.389 
[DaemonStream-karl-test-20015-thread-1-processing-n:solr-0.search-solr.svc.cluster.local:80_solr
 x:at-uk-002_shard1_replica_n1 c:at-uk-002 s:shard1 r:core_node2] ERROR 
org.apache.solr.client.solrj.io.stream.DaemonStream - Err
or in DaemonStream: karl-test
java.io.IOException: org.apache.solr.common.SolrException: Could not find a 
healthy node to handle the request.
at 
org.apache.solr.client.solrj.io.stream.TopicStream.persistCheckpoints(TopicStream.java:472)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.TopicStream.close(TopicStream.java:342) 
~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.PushBackStream.close(PushBackStream.java:75)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.UpdateStream.close(UpdateStream.java:147)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.CommitStream.close(CommitStream.java:155)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.DaemonStream$StreamRunner.stream(DaemonStream.java:380)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
```

If i remove the set-config; it's fine.

From: Vivaldi 
Sent: 17 March 2021 21:59
To: users@solr.apache.org 
Subject: Re: Disable commits during a REINDEXCOLLECTION

I thought the REINDEXCOLLECTION puts the source collection in read-only mode, 
am I wrong? Doesn’t that alsı disable commits?

Sent from my iPhone

> On 18 Mar 2021, at 00:28, Karl Stoney  
> wrote:
>
> Can you believe I had never come across that before!  Thanks!
>
> Every day’s a school day
>
> Get Outlook for 
> iOS
> 
> From: David Smiley 
> Sent: Wednesday, March 17, 2021 8:48:23 PM
> To: users@solr.apache.org 
> Subject: Re: Disable commits during a REINDEXCOLLECTION
>
> Hi Karl,
>
> Look into the "config apI".  Let us know how it goes!
> Ideally, this feature would do this automatically.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=7fZ17ijzSk3rs34K01tJ4SjP4kV4XU6Md8yzpcnu1jw%3D&reserved=0
>
>
> On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
>  wrote:
>
>> Hi all,
>> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
>> aggressive autoCommit interval configured by default (intentionally).
>>
>> Ideally I'd like to be able to disable hard commits for the duration of
>> the reindex, but can't see a way to do that without pushing a whole new
>> config and reloading that collection.
>>
>> Does anyone know of a ninja way at runtime to disable autoCommits on a
>> collection (solr cloud)?
>>
>> Thanks
>> Unless expressly stated otherwise in this email, this e-mail is sent on
>> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
>> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
>> Trader Limited is part of the Auto Trader Group Plc group. This email and
>> any files transmitted with it are confidential and may be legally
>> privileged, and intended solely for the use of the individual or entity to
>> whom they are addressed. If you have received this email in error please
>> notify the sender. This email message has been swept for the presence of
>> computer viruses.
>>
> Unless expressly stated otherwise in this email, this e-mail is sent on 
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, 
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto 
> Trader Limited is part of the Auto Trader Group Plc group. This email and

Re: Disable commits during a REINDEXCOLLECTION

2021-03-17 Thread Karl Stoney 
Bit of a hacky workaround but:


  *   create a collection from the target config name before reindex
  *   set property on that (which writes overlay to zookeeper)
  *   delete the collection
  *   run reindex
  *   unset-property on the created collection


From: Karl Stoney 
Sent: 17 March 2021 22:20
To: users@solr.apache.org 
Subject: Re: Disable commits during a REINDEXCOLLECTION

That's the source collection, I'm referring to the destination.

@David - seeing as REINDEX creates the target collection, I had to start the 
process and then attempt to set the config on the destination.
However setting the destination config kills the REINDEX:

```
22:18:40.389 
[DaemonStream-karl-test-20015-thread-1-processing-n:solr-0.search-solr.svc.cluster.local:80_solr
 x:at-uk-002_shard1_replica_n1 c:at-uk-002 s:shard1 r:core_node2] ERROR 
org.apache.solr.client.solrj.io.stream.DaemonStream - Err
or in DaemonStream: karl-test
java.io.IOException: org.apache.solr.common.SolrException: Could not find a 
healthy node to handle the request.
at 
org.apache.solr.client.solrj.io.stream.TopicStream.persistCheckpoints(TopicStream.java:472)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.TopicStream.close(TopicStream.java:342) 
~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.PushBackStream.close(PushBackStream.java:75)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.UpdateStream.close(UpdateStream.java:147)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.CommitStream.close(CommitStream.java:155)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
at 
org.apache.solr.client.solrj.io.stream.DaemonStream$StreamRunner.stream(DaemonStream.java:380)
 ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 
2021-02-16 15:52:04]
```

If i remove the set-config; it's fine.

From: Vivaldi 
Sent: 17 March 2021 21:59
To: users@solr.apache.org 
Subject: Re: Disable commits during a REINDEXCOLLECTION

I thought the REINDEXCOLLECTION puts the source collection in read-only mode, 
am I wrong? Doesn’t that alsı disable commits?

Sent from my iPhone

> On 18 Mar 2021, at 00:28, Karl Stoney  
> wrote:
>
> Can you believe I had never come across that before!  Thanks!
>
> Every day’s a school day
>
> Get Outlook for 
> iOS
> 
> From: David Smiley 
> Sent: Wednesday, March 17, 2021 8:48:23 PM
> To: users@solr.apache.org 
> Subject: Re: Disable commits during a REINDEXCOLLECTION
>
> Hi Karl,
>
> Look into the "config apI".  Let us know how it goes!
> Ideally, this feature would do this automatically.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=7fZ17ijzSk3rs34K01tJ4SjP4kV4XU6Md8yzpcnu1jw%3D&reserved=0
>
>
> On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
>  wrote:
>
>> Hi all,
>> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
>> aggressive autoCommit interval configured by default (intentionally).
>>
>> Ideally I'd like to be able to disable hard commits for the duration of
>> the reindex, but can't see a way to do that without pushing a whole new
>> config and reloading that collection.
>>
>> Does anyone know of a ninja way at runtime to disable autoCommits on a
>> collection (solr cloud)?
>>
>> Thanks
>> Unless expressly stated otherwise in this email, this e-mail is sent on
>> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
>> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
>> Trader Limited is part of the Auto Trader Group Plc group. This email and
>> any files transmitted with it are confidential and may be legally
>> privileged, and intended solely for the use of the individual or entity to
>> whom they are addressed. If you have received this