Re: [PATCH] Re: Solr connection timeout hardwired to 60s

On 4/14/2019 7:59 AM, John Fawcett via dovecot wrote: From dovecot point of view I can see the following as potentially useful features: 1) a configurable batch size would enable to tune the number of emails per request and help stay under the 60 seconds hard coded http request timeout. A config

Re: Solr connection timeout hardwired to 60s

On 14/04/2019 17:55, John Fawcett via dovecot wrote: > The solr server is a small test virtual machine with 0.2 (shared) vCPU > and 0.6MB of memory and non SSD storage. It can index around 2000 emails > per minute when there is no other activity. Average email size is about > 45Kb. I'm not indexing

Re: Solr connection timeout hardwired to 60s

On 14/04/2019 17:16, Peter Mogensen via dovecot wrote: > sorry... I got distracted half way and forgot to put a meaningfull > subject so the archive could figure out the thread. - resending. > > On 4/14/19 4:04 PM, dovecot-requ...@dovecot.org wrote: > >>> Solr ships with autoCommit set to 15 second

Re: [PATCH] Re: Solr connection timeout hardwired to 60s

On 14/04/2019 16:04, Aki Tuomi via dovecot wrote: > >> On 14 April 2019 16:59 John Fawcett via dovecot < dovecot@dovecot.org >> > wrote: >> >> >> On 13/04/2019 17:16, Shawn Heisey via dovecot wrote: >>> On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: If this v

Re: Solr connection timeout hardwired to 60s

sorry... I got distracted half way and forgot to put a meaningfull subject so the archive could figure out the thread. - resending. On 4/14/19 4:04 PM, dovecot-requ...@dovecot.org wrote: >> Solr ships with autoCommit set to 15 seconds and openSearcher set to >> false on the autoCommit.? The aut

Re: [PATCH] Re: Solr connection timeout hardwired to 60s

On 14 April 2019 16:59 John Fawcett via dovecot < dovecot@dovecot.org> wrote: On 13/04/2019 17:16, Shawn Heisey via dovecot wrote: On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: If th

[PATCH] Re: Solr connection timeout hardwired to 60s

On 13/04/2019 17:16, Shawn Heisey via dovecot wrote: > On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: >> If this value was made configurable people could set it to what they >> want. However the underlying problem is likely on solr configuration. > > The Jetty that is included in Solr has it

Re: Solr connection timeout hardwired to 60s

On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: If this value was made configurable people could set it to what they want. However the underlying problem is likely on solr configuration. The Jetty that is included in Solr has its idle timeout set to 50 seconds. But in practice, I have n

Re: Solr connection timeout hardwired to 60s

On 12/04/2019 12:09, Peter Mogensen via dovecot wrote: > Looking further at tcpdumps of the Dovecot->Solr traffic and Solr > metrics it doesn't seem like there's anything suspicious apart from the > TCP windows running full and Dovecot backing of ... until it times out > and close the connection. >

Re: Solr connection timeout hardwired to 60s

Looking further at tcpdumps of the Dovecot->Solr traffic and Solr metrics it doesn't seem like there's anything suspicious apart from the TCP windows running full and Dovecot backing of ... until it times out and close the connection. >From my understanding of how Dovecot operates towards Solr i

Re: Solr connection timeout hardwired to 60s

On 4/4/19 6:57 PM, Peter Mogensen wrote: > > > On 4/4/19 6:47 PM, dovecot-requ...@dovecot.org wrote: >> For a typical Solr index, 60 seconds is an eternity. Most people aim >> for query times of 100 milliseconds or less, and they often achieve >> that goal. > > I'm pretty sure I get these wh

Re: Solr connection timeout hardwired to 60s

On 4/4/2019 6:42 PM, M. Balridge via dovecot wrote: What is a general rule of thumb for RAM and SSD disk requirements as a fraction of indexed document hive size to keep query performance at 200ms or less? How do people deal with the JAVA GC world-stoppages, other than simply doubling or tripling

Re: Solr connection timeout hardwired to 60s

> I'm a denizen of the solr-u...@lucene.apache.org mailing list. > [...] > Here's a wiki page that I wrote about that topic. This wiki is going > away next month, but for now you can still access it: > > https://wiki.apache.org/solr/SolrPerformanceProblems That's a great resource, Shawn. I am

Re: Solr connection timeout hardwired to 60s

On 4/4/19 6:47 PM, dovecot-requ...@dovecot.org wrote: > For a typical Solr index, 60 seconds is an eternity. Most people aim > for query times of 100 milliseconds or less, and they often achieve > that goal. I'm pretty sure I get these while indexing, not querying. Apr 04 16:44:50 host doveco

Re: Solr connection timeout hardwired to 60s

On 4/4/2019 2:21 AM, Peter Mogensen via dovecot wrote: What's the recommended way to handling timeouts on large mailboxes given the hardwired request timeout of 60s in solr-connection.c: http_set.request_timeout_msecs = 60*1000; I'm a denizen of the solr-u...@lucene.apache.org mailing list

Re: Solr connection timeout hardwired to 60s

Hi Shawn Am 04.04.19 um 16:12 schrieb Shawn Heisey via dovecot: > On 4/4/2019 2:21 AM, Peter Mogensen via dovecot wrote: > Here's a wiki page that I wrote about that topic.  This wiki is going > away next month, but for now you can still access it: > > https://wiki.apache.org/solr/SolrPerformanc

Re: Solr connection timeout hardwired to 60s

On 4/4/2019 2:21 AM, Peter Mogensen via dovecot wrote: What's the recommended way to handling timeouts on large mailboxes given the hardwired request timeout of 60s in solr-connection.c: http_set.request_timeout_msecs = 60*1000; I'm a denizen of the solr-u...@lucene.apache.org mailing list