Solr nodes crashing

2021-06-25 Thread Jon Morisi
Hi everyone, I'm running solr 7.4.0 and have a collection running on 4 nodes (2 shards, replication factor =2). I'm experiencing an issue where random nodes will crash when I submit large batches to be indexed (>500,000 documents). I've been successful in keeping things running if I keep an ey

RE: Solr nodes crashing

2021-06-29 Thread Jon Morisi
processes to 65,000. Yours are set to 1024 and 4096, respectively. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jun 25, 2021, at 11:35 AM, Jon Morisi wrote: > > Hi everyone, > I'm running solr 7.4.0 and have a collection runn

RE: Solr nodes crashing

2021-07-01 Thread Jon Morisi
ptor Count”. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jun 29, 2021, at 2:04 PM, Jon Morisi wrote: > > Thanks for the response Walter. Upon further review it looks like my solr > service account has: > open files

RE: Solr nodes crashing

2021-07-21 Thread Jon Morisi
e timeout? Any other advice to get this performing quicker is appreciated. Thanks again, Jon -Original Message- From: Shawn Heisey Sent: Thursday, July 1, 2021 6:48 PM To: users@solr.apache.org Subject: Re: Solr nodes crashing On 7/1/2021 4:23 PM, Jon Morisi wrote: > I've had

RE: Solr nodes crashing

2021-07-22 Thread Jon Morisi
rst searches and new searches queries you can configure in your Solr config, those would be able to warm your caches for you if that is the case. Mike On Wed, Jul 21, 2021 at 11:06 AM Jon Morisi wrote: > Thanks for the help Shawn and Walter. After increasing the open files > setting to

RE: Solr nodes crashing

2021-07-22 Thread Jon Morisi
then configure the maxCharsForDocValues to 4096. Is this a bad idea, or am I on the right track? Is there another way to enable docValues for a pipe delimited string of tokens? -Original Message- From: Jon Morisi Sent: Thursday, July 22, 2021 8:45 AM To: users@solr.apache.org Subject: RE

RE: Solr nodes crashing

2021-07-22 Thread Jon Morisi
RE Shawn and Michael, I am just looking for a way to speed it up. Mike Drob had mentioned docvalues, which is why I was researching that route. I am running my search tests from solr admin, no facets, no sorting. I am using Dsolr.directoryFactory=HdfsDirectoryFactory URL: . /select?q=ptokens:

RE: Solr nodes crashing

2021-07-23 Thread Jon Morisi
Thanks Shawn and Dave, some very helpful info in your emails. I'll continue testing. It's a bit of a tough one because 2nd run queries run fast, once they're cached. I found using the fq vs. q, to skip the scoring, interesting. What does the query in the below email do to improve performance?