Hi,
After upgrading Dovecot from version 2.2.27 to 2.3.9.2, we are
sporadically seeing lots of these errors in the error log on many of our
servers:
imap-login: Error: master(imap): net_connect_unix(imap) failed:
Resource temporarily unavailable -
http://wiki2.dovecot.org/SocketUnavailable
The issue is causing significant delays and/or timeouts on login.
From what I can tell, this happens because the imap-login service is
unable to connect to the imap service socket, however I am unable to
identify the root cause of the issue.
We have increased the client_limit for the stats service, as described
in the upgrade document at (https://wiki2.dovecot.org/Upgrading/2.3). It
is set to the same value as the process_limit for imap service.
System CPU usage does not appear to be saturated. The CPU usage sits at
around 50% when these errors appear.
We have doubled the number of file descriptors after upgrading from
2.2.27. It was previously set to 16392 (which worked fine):
~# cat /proc/`pidof dovecot`/limits | grep "open files"
Max open files 30000 30000 files
We have tried increasing default_process_limit and default_client_limit
from the default 1000 to 3000, but this has no effect.
Current configuration (with irrelevant parts removed):
~# doveconf -n
default_client_limit = 3000 # these were raised after upgrading in
attempt to remedy
default_process_limit = 3000
default_vsz_limit = 512 M
[...]
service imap-login {
client_limit = 1250
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
process_limit = 16
process_min_avail = 16
service_count = 0
vsz_limit = 512 M
}
service imap {
client_limit = 1
process_limit = 20000
}
service stats {
client_limit = 20000
}
[...]
We appear to be nowhere near the 20000 process_limit set for the imap
service. We are also not seeing any warnings being logged with regards
to the process_limit.
~# doveadm who -1 | wc -l
8765
~# doveadm process status | grep "^imap " | wc -l
7583
~# ps aux | grep "dovecot/imap" | wc -l
7449
~# doveadm process status | grep -v "^imap "
name pid available_count total_count idle_start
last_status_update last_kill_sent
stats 19875 12384 582387 0 1578305489
0
pop3-login 19868 1000 0 0 1578061504
0
pop3-login 19867 1000 0 0 1578061503
0
pop3-login 19866 1000 0 0 1578061502
0
pop3-login 19865 1000 0 0 1578061503
0
pop3-login 19864 1000 0 0 1578061503
0
pop3-login 19853 1000 0 0 1578061504
0
pop3-login 19852 1000 0 0 1578061504
0
pop3-login 19851 1000 0 0 1578061504
0
pop3-login 19849 1000 0 0 1578061503
0
pop3-login 19847 1000 0 0 1578061503
0
pop3-login 19846 1000 0 0 1578061503
0
pop3-login 19845 1000 0 0 1578061503
0
pop3-login 19844 1000 0 0 1578061503
0
pop3-login 19843 1000 0 0 1578061503
0
pop3-login 19842 1000 0 0 1578061503
0
pop3-login 19839 1000 0 0 1578061502
0
log 19841 2973 27 0 1578061502
0
imap-login 19873 666 14942 0 1578305488
0
imap-login 19872 523 32792 0 1578305489
0
imap-login 19871 316 62876 0 1578305489
0
imap-login 19870 167 78332 0 1578305489
0
imap-login 19869 118 97989 0 1578305489
0
imap-login 19863 0 184726 0 1578305489
0
imap-login 19862 0 193094 0 1578305489
0
imap-login 19861 0 186800 0 1578305487
0
imap-login 19860 675 17806 0 1578305487
0
imap-login 19859 0 169446 0 1578305489
0
imap-login 19858 0 143517 0 1578305489
0
imap-login 19857 0 119215 0 1578305488
0
imap-login 19856 0 151958 0 1578305489
0
imap-login 19855 483 47036 0 1578305488
0
imap-login 19854 1 185240 0 1578305489
0
imap-login 19840 567 23859 0 1578305485
0
config 19874 2967 124482 0 1578305489
0
auth 19885 2967 124185 0 1578305489
0
anvil 10017 967 7567 0 1578305463
0
--
Eirik
