Hello, On Wed, 5 Apr 2017 23:45:33 -0700 Mark Moseley wrote:
> We've been using hibernate for about half a year with no ill effects. There > were various logged errors in earlier versions of dovecot, but even with > those, we never heard a reported customer-side error (almost always when > transitioning from hibernate back to regular imap; in the case of those > errors, presumably the mail client just reconnected silently). > This is my impression as well (silent reconnect w/o anything really bad happening) with regard to the bug I just reported/saw. > For no particular reason besides wanting to start conservatively, we've got > client_limit set to 50 on the hibernate procs (with 1100 total hibernated > connections on the box I'm looking at). At only a little over a meg each, > I'm fine with those extra processes. > Yeah, but 50 would be a tad too conservative for our purposes here. I'll keep an eye on it and see how it goes, first checkpoint would be at 1k hibernated sessions. ^_^ Christian > On Wed, Apr 5, 2017 at 11:17 PM, Aki Tuomi <aki.tu...@dovecot.fi> wrote: > > > > > > > On 06.04.2017 06:15, Christian Balzer wrote: > > > Hello, > > > > > > as some may remember, we're running very dense IMAP cluster here, in > > > excess of 50k IMAP sessions per node (current record holder is 68k, > > design > > > is for 200k+). > > > > > > The first issue we ran into was that the dovecot master process (which is > > > single thread and thus a bottleneck) was approaching 100% CPU usage (aka > > > using a full core) when trying to spawn off new IMAP processes. > > > > > > This was rectified by giving IMAP a service count of 200 to create a pool > > > of "idling" processes eventually, reducing the strain for the master > > > process dramatically. That of course required generous cranking up > > > ulimits, FDs in particular. > > > > > > The next issues of course is (as mentioned before) the memory usage of > > all > > > those IMAP processes and the fact that quite a few things outside of > > > dovecote (ps, etc) tend to get quite sedate when dealing with tens of > > > thousands of processes. > > > > > > We just started to deploy a new mailbox cluster pair with 2.2.27 and > > > having IMAP hibernate configured. > > > Getting this work is a PITA though with regards to ownership and access > > > rights to the various sockets, this part could definitely do with some > > > better (I know, difficult) defaults or at least more documentation (none > > > besides the source and this ML). > > > > > > Initial results are very promising, depending on what your clients are > > > doing (are they well behaved, are your users constantly looking at other > > > folders, etc) the vast majority of IDLE processes will be in hibernated > > > at any given time and thus not only using a fraction of the RAM otherwise > > > needed but also freeing up process space. > > > > > > Real life example: > > > 240 users, 86 imap processes (80% of those not IDLE) and: > > > dovecot 119157 0.0 0.0 10452 3236 ? S Apr01 0:21 > > dovecot/imap-hibernate [237 connections] > > > That's 237 hibernated connections and thus less processes than otherwise. > > > > > > > > > I assume that given the silence on the ML what we are going to be the > > > first hibernate users where the term "large scale" does apply. > > > Despite that I have some questions, clarifications/confirmations: > > > > > > Our current default_client_limit is 16k, which can be seen by having 5 > > > config processes on our 65k+ session node. ^_- > > > > > > This would also apply to imap-hibernate, one wonders if that's fine > > > (config certainly has no issues) or if something smaller would be > > > appropriate here? > > > > > > Since we have idling IMAP processes around most of the time, the strain > > of > > > re-spawning proper processes from imap-hibernate should be just as > > reduced > > > as for the dovecot master, correct? > > > > > > > > > I'll keep reporting our experiences here, that is if something blows up > > > spectacularly. ^o^ > > > > > > Christian > > > > Hi! > > > > We have customers using it in larger deployments. A good idea is to have > > as much of your clients hibernating as possible, as the hibernation > > process is much smaller than actual IMAP process. > > > > You should probably also look at reusing the processes, as this will > > probably help your performance, > > https://wiki.dovecot.org/PerformanceTuning and > > https://wiki.dovecot.org/LoginProcess are probably a good starting > > point, although I suspect you've read these already. > > > > If you are running a dense server, cranking up various limits is rather > > expected. > > > > Aki > > > -- Christian Balzer Network/Systems Engineer ch...@gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/
