Re: Solr nodes crashing

2021-07-23 Thread Shawn Heisey
On 7/23/2021 1:29 PM, Jon Morisi wrote: Thanks Shawn and Dave, some very helpful info in your emails. I'll continue testing. It's a bit of a tough one because 2nd run queries run fast, once they're cached. I found using the fq vs. q, to skip the scoring, interesting. What does the query in th

RE: Solr nodes crashing

2021-07-23 Thread Jon Morisi
ove performance? My first thought was that wouldn't a "SolrRangeQuery" make it take longer? Thanks again, Jon -Original Message- From: Shawn Heisey Sent: Friday, July 23, 2021 8:20 AM To: users@solr.apache.org Subject: Re: Solr nodes crashing On 7/23/2021 8:14 AM, Shaw

Re: Solr nodes crashing

2021-07-23 Thread Dave
By “get one” I meant a local ssd disk btw, for example https://www.amazon.com/SK-hynix-Gold-NAND-Internal/dp/B07SNHB4RC/ref=mp_s_a_1_3?dchild=1&keywords=server+ssd&qid=1627050778&sr=8-3 And you’re all set unless I’m missing the reason why you would ever want to use a network drive aside from hav

Re: Solr nodes crashing

2021-07-23 Thread Dave
For me personally on any system that swap has been disabled makes a much better situation, as well as setting the jvm xms and xmx to the exact same value, being 31gb, NOT higher than that as more actually makes gc slower. Also yeah, solr on a network disk is going to be slow unless it’s on an ss

Re: Solr nodes crashing

2021-07-23 Thread Shawn Heisey
On 7/23/2021 8:14 AM, Shawn Heisey wrote: Or you could turn the query into a range query, and it would work much better -- Point types are EXCELLENT for range queries. You might not know how to do this.  Try this query string: ptokens:[243796009 TO 243796009] AND ptokens:[410512000 TO 41051200

Re: Solr nodes crashing

2021-07-23 Thread Shawn Heisey
On 7/22/2021 11:53 AM, Jon Morisi wrote: RE Shawn and Michael, I am just looking for a way to speed it up. Mike Drob had mentioned docvalues, which is why I was researching that route. I am running my search tests from solr admin, no facets, no sorting. I am using Dsolr.directoryFactory=Hdf

Re: Solr nodes crashing

2021-07-23 Thread Michael Gibney
.7% > 377.39 GB > 368.77 GB > > Swap Space 4.7% > 4.00 GB > 193.25 MB > > File Descriptor Count 0.2% > 128000 > 226 > > JVM-Memory 22.7% > 15.33 GB > 15.33 GB > > Thanks for looking, > Jon > > > -Original Message- > From: Sha

RE: Solr nodes crashing

2021-07-22 Thread Jon Morisi
% 15.33 GB 15.33 GB Thanks for looking, Jon -Original Message- From: Shawn Heisey Sent: Thursday, July 22, 2021 11:26 AM To: users@solr.apache.org Subject: Re: Solr nodes crashing On 7/22/2021 10:39 AM, Michael Gibney wrote: > ps- wrt requesting a "literal, complete search url" to a

Re: Solr nodes crashing

2021-07-22 Thread Shawn Heisey
On 7/22/2021 10:39 AM, Michael Gibney wrote: ps- wrt requesting a "literal, complete search url" to aid troubleshooting: facets, `sort`, `offset`, and `rows` params would all be of particular interest. One way to get everything we are after for the query is to add "echoParams=all" to the quer

Re: Solr nodes crashing

2021-07-22 Thread Michael Gibney
ocValues? >> I would then configure the maxCharsForDocValues to 4096. >> >> Is this a bad idea, or am I on the right track? >> Is there another way to enable docValues for a pipe delimited string of >> tokens? >> >> -Original Message- >&g

Re: Solr nodes crashing

2021-07-22 Thread Michael Gibney
a, or am I on the right track? > Is there another way to enable docValues for a pipe delimited string of > tokens? > > -Original Message- > From: Jon Morisi > Sent: Thursday, July 22, 2021 8:45 AM > To: users@solr.apache.org > Subject: RE: Solr nodes crashing > >

RE: Solr nodes crashing

2021-07-22 Thread Jon Morisi
rom: Mike Drob Sent: Wednesday, July 21, 2021 12:36 PM To: users@solr.apache.org Subject: Re: Solr nodes crashing You may want to look into enabling docvalues for your fields in your scheme, if not already enabled. That often helps with memory usage during query, but requires a reindex of your d

RE: Solr nodes crashing

2021-07-22 Thread Jon Morisi
esday, July 21, 2021 12:36 PM To: users@solr.apache.org Subject: Re: Solr nodes crashing You may want to look into enabling docvalues for your fields in your scheme, if not already enabled. That often helps with memory usage during query, but requires a reindex of your data. There are also fi

Re: Solr nodes crashing

2021-07-21 Thread Mike Drob
essage- > From: Shawn Heisey > Sent: Thursday, July 1, 2021 6:48 PM > To: users@solr.apache.org > Subject: Re: Solr nodes crashing > > On 7/1/2021 4:23 PM, Jon Morisi wrote: > > I've had an indexing job running for 24+ hours. I'm importing 100m+ > documents

RE: Solr nodes crashing

2021-07-21 Thread Jon Morisi
e timeout? Any other advice to get this performing quicker is appreciated. Thanks again, Jon -Original Message- From: Shawn Heisey Sent: Thursday, July 1, 2021 6:48 PM To: users@solr.apache.org Subject: Re: Solr nodes crashing On 7/1/2021 4:23 PM, Jon Morisi wrote: > I've had

Re: Solr nodes crashing

2021-07-01 Thread Shawn Heisey
On 7/1/2021 4:23 PM, Jon Morisi wrote: I've had an indexing job running for 24+ hours. I'm importing 100m+ documents. After about 8 hours both of the replica nodes crashed but the primary nodes have continued to run and index. There's a common misconception. Java programs, including Solr,

RE: Solr nodes crashing

2021-07-01 Thread Jon Morisi
cas keep up? Thanks! -Original Message- From: Walter Underwood Sent: Tuesday, June 29, 2021 3:18 PM To: users@solr.apache.org Subject: Re: Solr nodes crashing The dashboard page in the admin UI has a graph that shows the current number of open files and the max. It is called “File Descri

Re: Solr nodes crashing

2021-06-29 Thread Walter Underwood
in account), or does it run under the service account? > > -Original Message- > From: Walter Underwood > Sent: Friday, June 25, 2021 12:39 PM > To: users@solr.apache.org > Subject: Re: Solr nodes crashing > > That is not correctly configured. The open files setting is

RE: Solr nodes crashing

2021-06-29 Thread Jon Morisi
under the service account? -Original Message- From: Walter Underwood Sent: Friday, June 25, 2021 12:39 PM To: users@solr.apache.org Subject: Re: Solr nodes crashing That is not correctly configured. The open files setting is too small. The documentation you reference says to set files and

Re: Solr nodes crashing

2021-06-25 Thread Walter Underwood
That is not correctly configured. The open files setting is too small. The documentation you reference says to set files and processes to 65,000. Yours are set to 1024 and 4096, respectively. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jun 25,