Re: IMAP hibernate and scalability in general

On 30 Apr 2017, at 13.51, Timo Sirainen wrote: > >> New processes aren't created until client_limit is reached in all the >> existing processes. When there are multiple processes they're all listening >> for new connections and whichever happens to be fastest gets it. Related to >> this, I'm t

Re: IMAP hibernate and scalability in general

On 24 Apr 2017, at 15.41, Timo Sirainen wrote: > > On 24 Apr 2017, at 4.04, Christian Balzer > wrote: >> >> >> Hello, >> >> Just to follow up on this, we've hit over 16k (default client limit here) >> hibernated sessions: >> --- >> dovecot 119157 0.1 0.0 63404 56140

Re: IMAP hibernate and scalability in general

On 24 Apr 2017, at 4.04, Christian Balzer wrote: > > > Hello, > > Just to follow up on this, we've hit over 16k (default client limit here) > hibernated sessions: > --- > dovecot 119157 0.1 0.0 63404 56140 ?SApr01 62:05 > dovecot/imap-hibernate [11291 connections] > dovecot

Re: IMAP hibernate and scalability in general

Hello, Just to follow up on this, we've hit over 16k (default client limit here) hibernated sessions: --- dovecot 119157 0.1 0.0 63404 56140 ?SApr01 62:05 dovecot/imap-hibernate [11291 connections] dovecot 877825 0.2 0.0 28512 21224 ?SApr23 1:34 dovecot/imap

Re: IMAP hibernate and scalability in general

> On 12 Apr 2017, at 3.33, Christian Balzer wrote: > >> Should be. In general I haven't heard of installations hitting CPU limits in >> proxies. The problem so far has always been related to getting enough >> outgoing sockets without errors, which is a server-wide problem. 2.2.29 has >> one t

Re: IMAP hibernate and scalability in general

On Mon, 10 Apr 2017 23:11:24 +0300 Timo Sirainen wrote: > On 10 Apr 2017, at 21.49, Mark Moseley wrote: > > > > Timo, any sense on where (if any) the point is where there are so many > > connections on a given login process that it would get too busy to keep up? > > I.e. where the sheer amount o

Re: IMAP hibernate and scalability in general

On 10 Apr 2017, at 21.49, Mark Moseley wrote: > > Timo, any sense on where (if any) the point is where there are so many > connections on a given login process that it would get too busy to keep up? > I.e. where the sheer amount of stuff the login process has to do outweighs > the CPU savings of

Re: IMAP hibernate and scalability in general

On Thu, Apr 6, 2017 at 9:22 PM, Christian Balzer wrote: > > Hello, > > On Thu, 6 Apr 2017 22:13:07 +0300 Timo Sirainen wrote: > > > On 6 Apr 2017, at 21.14, Mark Moseley wrote: > > > > > >> > > >> imap-hibernate processes are similar to imap-login processes in that > they > > >> should be able t

Re: IMAP hibernate and scalability in general

Hello, On Thu, 6 Apr 2017 22:13:07 +0300 Timo Sirainen wrote: > On 6 Apr 2017, at 21.14, Mark Moseley wrote: > > > >> > >> imap-hibernate processes are similar to imap-login processes in that they > >> should be able to handle thousands or even tens of thousands of connections > >> per proc

Re: IMAP hibernate and scalability in general

On 6 Apr 2017, at 21.14, Mark Moseley wrote: > >> >> imap-hibernate processes are similar to imap-login processes in that they >> should be able to handle thousands or even tens of thousands of connections >> per process. >> > > TL;DR: In a director/proxy setup, what's a good client_limit for

Re: IMAP hibernate and scalability in general

On Thu, Apr 6, 2017 at 3:10 AM, Timo Sirainen wrote: > On 6 Apr 2017, at 9.56, Christian Balzer wrote: > > > >> For no particular reason besides wanting to start conservatively, we've > got > >> client_limit set to 50 on the hibernate procs (with 1100 total > hibernated > >> connections on the b

Re: IMAP hibernate and scalability in general

On Thu, 6 Apr 2017 13:10:03 +0300 Timo Sirainen wrote: > On 6 Apr 2017, at 9.56, Christian Balzer wrote: > > > >> For no particular reason besides wanting to start conservatively, we've got > >> client_limit set to 50 on the hibernate procs (with 1100 total hibernated > >> connections on the b

Re: IMAP hibernate and scalability in general

On 6 Apr 2017, at 9.56, Christian Balzer wrote: > >> For no particular reason besides wanting to start conservatively, we've got >> client_limit set to 50 on the hibernate procs (with 1100 total hibernated >> connections on the box I'm looking at). At only a little over a meg each, >> I'm fine wi

Re: IMAP hibernate and scalability in general

Hello, On Wed, 5 Apr 2017 23:45:33 -0700 Mark Moseley wrote: > We've been using hibernate for about half a year with no ill effects. There > were various logged errors in earlier versions of dovecot, but even with > those, we never heard a reported customer-side error (almost always when > trans

Re: IMAP hibernate and scalability in general

We've been using hibernate for about half a year with no ill effects. There were various logged errors in earlier versions of dovecot, but even with those, we never heard a reported customer-side error (almost always when transitioning from hibernate back to regular imap; in the case of those error

Re: IMAP hibernate and scalability in general

On 06.04.2017 06:15, Christian Balzer wrote: > Hello, > > as some may remember, we're running very dense IMAP cluster here, in > excess of 50k IMAP sessions per node (current record holder is 68k, design > is for 200k+). > > The first issue we ran into was that the dovecot master process (which i

IMAP hibernate and scalability in general

Hello, as some may remember, we're running very dense IMAP cluster here, in excess of 50k IMAP sessions per node (current record holder is 68k, design is for 200k+). The first issue we ran into was that the dovecot master process (which is single thread and thus a bottleneck) was approaching 100