Re: Urgent help needed on Solr cloud(cont)

2021-06-22 Thread SayantiGmail
Hi This seems to be a bug in Solr 8.4 .Will this get resolved in higher versions or we need to update the stall time configuration as a workaround. > On 12 Apr 2021, at 22:41, Carlos .Sponchiado wrote: > > 1. I found a similar issue in this version of Solr here > https://github.com/clarin-e

Re: LTR on child documents

2021-06-22 Thread Alessandro Benedetti
Hi Roopa, can you elaborate a bit better? Child documents are documents nonetheless, so you can query them, rank them and re0rank them. So, what are you trying to do? Are you using any block join related query parsers? Do you want to combine it with learning to rank? How? Let us know and we can try

Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Stephen Lewis Bianamara
Hi SOLR Community, I've been investigating SOLR 8 and recovery behavior. From what I can tell, older SOLR versions (6 and before at least) required a solrdata drive with at least 2x the space of the index; so an index which could be up to 100GB on one shard would require a disk with at least 200GB

Re: LTR on child documents

2021-06-22 Thread Roopa Rao
Hi Alessandro, Thanks for the response. I am using version Solr 6.6 Basically, I have parent documents and multiple child documents for each parent. I am trying to create features and get feature scores on parent attributes (which is straightforward) and child attributes (which is what I am look

Re: Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Shawn Heisey
On 6/22/2021 11:45 AM, Stephen Lewis Bianamara wrote: However, SOLR 8 looks to have a different behavior wherein the index is perhaps updated in place, and thus a 100GB / shard index might only need a bit more headroom (call it 110GB say). Is this always the case with recovery on SOLR 8+? Or are

Re: Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Stephen Lewis Bianamara
Thanks Shawn! That is really helpful to know. Can you say more about what circumstance might cause an index to triple in size? Is it connected with bulk operations like "optimize" which can be avoided, or is it inherent to situations like merging segments? And if so, can this requirement be adjuste

Re: Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Shawn Heisey
On 6/22/2021 2:24 PM, Stephen Lewis Bianamara wrote: Thanks Shawn! That is really helpful to know. Can you say more about what circumstance might cause an index to triple in size? Is it connected with bulk operations like "optimize" which can be avoided, or is it inherent to situations like mergi

Re: Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Dave
The 3x index size has been around for a long time. Usually it’s for a full optimize. When this happens the original index stays in place, 1x, and is being reconstructed, 2x, then merged into the replacement 3x, once it’s all done you are back to less than 1x but you need the space or the optimi

Re: Excessive query expansion when using WordDelimiterGraphFilter

2021-06-22 Thread Francis Crimmins
Hi Michael: Thanks for your detailed response, which is much appreciated. We’ll take a look at some of your suggestions. We know that if we drop back to using the old WordDelimiterFilter we do not see this behaviour, but we would prefer not to use a deprecated class if possible. It would be gr

Re: Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Stephen Lewis Bianamara
In my experience, disks are not always cheap :) Running in AWS I have found several contexts which require local storage for cost effective performance of SOLR, but that does require scaling the instance as a whole to increase capacity (hence the particular motivation for this question). Generally

Re: Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Shawn Heisey
On 6/22/2021 5:02 PM, Stephen Lewis Bianamara wrote: The merging considerations are certainly interesting and naunced. Has there been any investigation into a "minimum number of segments" setting which could force a minimum number of segments (say 5 or 10) so that no one segment operation could i

Re: Is 2x drive size necessary for recovery on SOLR 8?

2021-06-22 Thread Stephen Lewis Bianamara
Thanks Shawn! I will definitely be interested to explore this space cautiously in that case. This was a ton of great info I needed, and more than I initially knew to ask :) Your first response seems to imply an answer to my original question, but I wanted to follow up to be as sure as I can. In th