Re: IMAP hibernate and scalability in general

2017-04-30 Thread Timo Sirainen
On 30 Apr 2017, at 13.51, Timo Sirainen wrote: > >> New processes aren't created until client_limit is reached in all the >> existing processes. When there are multiple processes they're all listening >> for new connections and whichever happens to be fastest gets it. Related to >> this, I'm t

Re: IMAP hibernate and scalability in general

2017-04-30 Thread Timo Sirainen
On 24 Apr 2017, at 15.41, Timo Sirainen wrote: > > On 24 Apr 2017, at 4.04, Christian Balzer > wrote: >> >> >> Hello, >> >> Just to follow up on this, we've hit over 16k (default client limit here) >> hibernated sessions: >> --- >> dovecot 119157 0.1 0.0 63404 56140

Re: IMAP hibernate and scalability in general

2017-04-24 Thread Timo Sirainen
On 24 Apr 2017, at 4.04, Christian Balzer wrote: > > > Hello, > > Just to follow up on this, we've hit over 16k (default client limit here) > hibernated sessions: > --- > dovecot 119157 0.1 0.0 63404 56140 ?SApr01 62:05 > dovecot/imap-hibernate [11291 connections] > dovecot

Re: IMAP hibernate and scalability in general

2017-04-23 Thread Christian Balzer
Hello, Just to follow up on this, we've hit over 16k (default client limit here) hibernated sessions: --- dovecot 119157 0.1 0.0 63404 56140 ?SApr01 62:05 dovecot/imap-hibernate [11291 connections] dovecot 877825 0.2 0.0 28512 21224 ?SApr23 1:34 dovecot/imap

Re: IMAP hibernate and scalability in general

2017-04-12 Thread Timo Sirainen
> On 12 Apr 2017, at 3.33, Christian Balzer wrote: > >> Should be. In general I haven't heard of installations hitting CPU limits in >> proxies. The problem so far has always been related to getting enough >> outgoing sockets without errors, which is a server-wide problem. 2.2.29 has >> one t

Re: IMAP hibernate and scalability in general

2017-04-11 Thread Christian Balzer
On Mon, 10 Apr 2017 23:11:24 +0300 Timo Sirainen wrote: > On 10 Apr 2017, at 21.49, Mark Moseley wrote: > > > > Timo, any sense on where (if any) the point is where there are so many > > connections on a given login process that it would get too busy to keep up? > > I.e. where the sheer amount o

Re: IMAP hibernate and scalability in general

2017-04-10 Thread Timo Sirainen
On 10 Apr 2017, at 21.49, Mark Moseley wrote: > > Timo, any sense on where (if any) the point is where there are so many > connections on a given login process that it would get too busy to keep up? > I.e. where the sheer amount of stuff the login process has to do outweighs > the CPU savings of

Re: IMAP hibernate and scalability in general

2017-04-10 Thread Mark Moseley
On Thu, Apr 6, 2017 at 9:22 PM, Christian Balzer wrote: > > Hello, > > On Thu, 6 Apr 2017 22:13:07 +0300 Timo Sirainen wrote: > > > On 6 Apr 2017, at 21.14, Mark Moseley wrote: > > > > > >> > > >> imap-hibernate processes are similar to imap-login processes in that > they > > >> should be able t

Re: IMAP hibernate and scalability in general

2017-04-06 Thread Christian Balzer
Hello, On Thu, 6 Apr 2017 22:13:07 +0300 Timo Sirainen wrote: > On 6 Apr 2017, at 21.14, Mark Moseley wrote: > > > >> > >> imap-hibernate processes are similar to imap-login processes in that they > >> should be able to handle thousands or even tens of thousands of connections > >> per proc

Re: IMAP hibernate and scalability in general

2017-04-06 Thread Timo Sirainen
On 6 Apr 2017, at 21.14, Mark Moseley wrote: > >> >> imap-hibernate processes are similar to imap-login processes in that they >> should be able to handle thousands or even tens of thousands of connections >> per process. >> > > TL;DR: In a director/proxy setup, what's a good client_limit for

Re: IMAP hibernate and scalability in general

2017-04-06 Thread Mark Moseley
On Thu, Apr 6, 2017 at 3:10 AM, Timo Sirainen wrote: > On 6 Apr 2017, at 9.56, Christian Balzer wrote: > > > >> For no particular reason besides wanting to start conservatively, we've > got > >> client_limit set to 50 on the hibernate procs (with 1100 total > hibernated > >> connections on the b

Re: IMAP hibernate and scalability in general

2017-04-06 Thread Christian Balzer
On Thu, 6 Apr 2017 13:10:03 +0300 Timo Sirainen wrote: > On 6 Apr 2017, at 9.56, Christian Balzer wrote: > > > >> For no particular reason besides wanting to start conservatively, we've got > >> client_limit set to 50 on the hibernate procs (with 1100 total hibernated > >> connections on the b

Re: IMAP hibernate and scalability in general

2017-04-06 Thread Timo Sirainen
On 6 Apr 2017, at 9.56, Christian Balzer wrote: > >> For no particular reason besides wanting to start conservatively, we've got >> client_limit set to 50 on the hibernate procs (with 1100 total hibernated >> connections on the box I'm looking at). At only a little over a meg each, >> I'm fine wi

Re: IMAP hibernate and scalability in general

2017-04-05 Thread Christian Balzer
Hello, On Wed, 5 Apr 2017 23:45:33 -0700 Mark Moseley wrote: > We've been using hibernate for about half a year with no ill effects. There > were various logged errors in earlier versions of dovecot, but even with > those, we never heard a reported customer-side error (almost always when > trans

Re: IMAP hibernate and scalability in general

2017-04-05 Thread Mark Moseley
We've been using hibernate for about half a year with no ill effects. There were various logged errors in earlier versions of dovecot, but even with those, we never heard a reported customer-side error (almost always when transitioning from hibernate back to regular imap; in the case of those error

Re: IMAP hibernate and scalability in general

2017-04-05 Thread Aki Tuomi
On 06.04.2017 06:15, Christian Balzer wrote: > Hello, > > as some may remember, we're running very dense IMAP cluster here, in > excess of 50k IMAP sessions per node (current record holder is 68k, design > is for 200k+). > > The first issue we ran into was that the dovecot master process (which i