Broken auth-* sockets.
Hello.
I am migrating my servers to Ubuntu and have been having an issue with
Dovecot authenticator service.
Exim is set up to use dovecot-auth, anticipating the question whether
exim binary was compiled with support of Dovecot authenticator, it was:
---
# exim -bV
Exim version 4.86_2 #1 built 05-Apr-2016 12:21:41
Copyright (c) University of Cambridge, 1995 - 2015
(c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007
- 2015
Berkeley DB: Berkeley DB 5.3.28: (September 9, 2013)
Support for: crypteq iconv() IPv6 PAM Perl Expand_dlfunc GnuTLS
move_frozen_messages Content_Scanning DKIM Old_Demime DNSSEC PRDR OCSP
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm
dbmjz dbmnz dnsdb dsearch ldap ldapdn ldapm mysql nis nis0 passwd pgsql
sqlite
Authenticators: cram_md5 cyrus_sasl dovecot plaintext spa tls
Routers: accept dnslookup ipliteral iplookup manualroute queryprogram
redirect
Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp
Fixed never_users: 0
Size of off_t: 8
Configuration file is /etc/exim4/exim4.conf
---
The problem is sockets Dovecot creates are somehow broken. Exim reports
something like:
---
2016-09-21 14:45:26 dovecot_plain authenticator failed for *** ([***])
[***]: 435 Unable to authenticate at present: authentication socket
connection error
2016-09-21 14:45:26 dovecot_login authenticator failed for *** ([***])
[***]: 435 Unable to authenticate at present: authentication socket
connection error
---
and I initially thought it was wrong permissions for the socket, triple
checked and they are 100% right. More than that, I temporarily chmodded
it "rw" for the world as follows:
---
# ls -l /var/dovecot/auth-*
srw-rw-rw- 1 Debian-exim root 0 Aug 17 21:20 /var/dovecot/auth-client
srw--- 1 dovecot root 0 Aug 17 21:20 /var/dovecot/auth-login
srw-rw-rw- 1 Debian-exim root 0 Aug 17 21:20 /var/dovecot/auth-master
-rw--- 1 rootroot 32 Aug 15 19:35
/var/dovecot/auth-token-secret.dat
srw-rw-rw- 1 dovecot root 0 Aug 17 21:20 /var/dovecot/auth-userdb
srw--- 1 dovecot root 0 Aug 17 21:20 /var/dovecot/auth-worker
---
and Exim reports the same problem.
What makes me think that it's Dovecot's and not Exim's problem? The fact
that the socket is really broken, on the server where it's not working:
---
# socat - UNIX-CONNECT:/var/dovecot/auth-client
2016/09/22 17:58:27 socat[15192] E connect(5, AF=1
"/var/dovecot/auth-client", 26): Connection refused
---
on the server where it is working (version 2.2.13 is installed there),
precisely same command:
---
# socat - UNIX-CONNECT:/var/dovecot/auth-client
VERSION 1 1
MECHPLAIN plaintext
MECHLOGIN plaintext
MECHCRAM-MD5dictionary active
MECHDIGEST-MD5 dictionary active mutual-auth
MECHAPOPprivate dictionary active
SPID535
CUID880
COOKIE 0311e84ed191fb63334819b1fc3bf2e3
DONE
---
with a different result!!!
The system:
---
uname -a
Linux *** 4.4.0-38-generic #57-Ubuntu SMP Tue Sep 6 15:41:41 UTC 2016
i686 i686 i686 GNU/Linux
---
Doveconf:
---
# dovecot -n
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-38-generic i686 Ubuntu 16.04.1 LTS ext3
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login cram-md5 digest-md5 apop
debug_log_path = /var/log/dovecot-debug.log
default_internal_user = dovenull
disable_plaintext_auth = no
first_valid_uid = 114
log_path = /var/log/dovecot.log
login_greeting = IMAP/POP3 server is ready.
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c
mail_location = maildir:/var/mail/exim/%d/%n
mail_log_prefix = "%Us(%u): "
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
protocols = imap pop3
service auth {
unix_listener auth-client {
mode = 0660
user = Debian-exim
}
unix_listener auth-master {
mode = 0600
user = Debian-exim
}
user = root
}
service imap-login {
chroot = login
client_limit = 8
inet_listener imap {
address = *
port = 143
}
inet_listener imaps {
address = *
port = 10143
}
process_limit = 8
process_min_avail = 5
service_count = 1
user = dovenull
vsz_limit = 64 M
}
service imap {
drop_priv_before_exec = yes
process_limit = 64
vsz_limit = 2 G
}
service pop3-login {
chroot = login
client_limit = 8
inet_listener pop3 {
address = *
port = 110
}
inet_listener pop3s {
address = *
port = 10110
}
process_limit = 8
process_min_avail = 5
service_count = 1
user = dovenull
vsz_limit = 64 M
}
service pop3 {
drop_priv_before_ex
Re: Broken auth-* sockets.
On 2016-09-23 01:47, Aki Tuomi wrote: On September 23, 2016 at 3:16 AM ygris...@pyramidheadgroup.ca wrote: Hello. I am migrating my servers to Ubuntu and have been having an issue with Dovecot authenticator service. Exim is set up to use dovecot-auth, anticipating the question whether exim binary was compiled with support of Dovecot authenticator, it was: --- I thought that dovecot sockets are under /var/run/dovecot, not /var/dovecot, by default on Ubuntu 16.04. Anyways, to make sure dovecot actually is configured to listen on those sockets, you could stop dovecot, rm the socket files, and start dovecot to see if those sockets get created. Aki Ah, that's precisely right! Leftover sockets from previous configuration. Thank you very much, glad I asked.
Dict quota calculation errors "remote disconnected"/"broken pipe" on 2.22.
Keywords: dovecot, dict, quota, postgre sql, broken pipe, remote
disconnected
Having Dovecot 2.2.22 (fe789d2) with Postgre SQL 9.5
(9.5.5-0ubuntu0.16.04) as the backend. I do not understand why quota
service is not working, not seeing it as a configuration error at least.
My quotas are DICT/SQL based.
OS: Ubuntu 16.0.4.1 32-bit (Linux XXX 4.4.0-59-generic #80-Ubuntu SMP
Fri Jan 6 17:36:54 UTC 2017 i686 i686 i686 GNU/Linux)
dovecot --build-options:
***
Build options: ioloop=epoll notify=inotify ipv6 openssl
io_block_size=8192
Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw
fail
SQL driver plugins: mysql postgresql sqlite
Passdb: checkpassword ldap pam passwd passwd-file shadow sql
Userdb: checkpassword ldap(plugin) nss passwd prefetch passwd-file sql
/etc/dovecot/conf.d/10-master.conf:
***
service quota-warning {
executable = script /etc/dovecot/some-script.sh
unix_listener quota-warning {
user = Debian-exim
mode = 0660
}
}
service dict {
unix_listener dict {
mode = 0660
user = Debian-exim
group = Debian-exim
}
}
/etc/dovecot/conf.d/90-quota.conf:
**
plugin {
quota = dict:user_quota::proxy::sqlquota
quota_rule2 = Trash:storage=+10%%
quota_rule3 = Junk:storage=+10%%
quota_grace = 10%%
quota_warning = storage=100%% quota-exceeded 100 %u
quota_warning2 = storage=95%% quota-warning 95 %u
quota_warning3 = storage=90%% quota-warning 90 %u
quota_warning4 = storage=75%% quota-warning 75 %u
}
dict {
sqlquota = pgsql:/etc/dovecot/dovecot-dict-sql-user.conf
}
/etc/dovecot/dovecot-dict-sql-user.conf:
connect = host=A.B.C.D dbname=db user=DDD password=YYY
map {
pattern = priv/quota/storage
table = quota2
username_field = username
value_field = bytes
}
map {
pattern = priv/quota/messages
table = quota2
username_field = username
value_field = messages
}
I will not be able to provide full "doveconf -n" output unfortunately.
Logging ALL incoming DB queries:
...
2017-02-04 12:03:12 MST [29500-10] DDD@db LOG: statement: SELECT
password FROM mailbox WHERE local_part = 'YYY' AND domain = 'XXX' AND
active ='t' LIMIT 1;
2017-02-04 12:03:12 MST [29501-10] DDD@db LOG: statement: SELECT 111 AS
uid, 222 AS gid, '/var/mail/AAA/' || 'BBB' || '/' || 'YYY' AS home,
'*:bytes=' || mailbox.quota AS quota_rule FROM mailbox WHERE local_part
= 'YYY' AND active ='t' LIMIT 1;
And seeing that BOTH dict statements are missing: SELECT and UPDATE.
dovecot-lda-erros.log:
**
Feb 04 14:23:33 lda(testuser@XXX): Error: read(/var/run/dovecot/dict)
failed: Remote disconnected
Feb 04 14:23:33 lda(testuser@XXX): Error: Internal quota calculation
error
Feb 04 14:23:33 lda(testuser@XXX): Error: Internal quota calculation
error
dovecot.log:
Feb 04 13:57:06 imap(YYY@XXX): Error: write(/var/run/dovecot/dict)
failed: Broken pipe
Feb 04 13:57:06 imap(YYY@XXX): Error: write(/var/run/dovecot/dict)
failed: Broken pipe
...
Feb 04 13:57:07 imap(YYY@XXX): Error: write(/var/run/dovecot/dict)
failed: Broken pipe
...
Feb 04 13:57:10 imap(YYY@XXX): Error: write(/var/run/dovecot/dict)
failed: Broken pipe
I have tried stopping dovecot, removing /var/run/dovecot/dict manually
and starting dovecot. This has not changed the behavior.
dovecot-debug.log:
**
Feb 04 13:18:12 lda(YYY@XXX): Error: read(/var/run/dovecot/dict) failed:
Remote disconnected
Feb 04 13:18:12 lda(YYY@XXX): Error: dict quota: Quota update failed,
it's now desynced
Feb 04 13:57:07 lda(testuser@XXX): Error: write(/var/run/dovecot/dict)
failed: Broken pipe
Feb 04 13:57:07 lda(testuser@XXX): Error: Internal quota calculation
error
Feb 04 13:57:07 lda(testuser@XXX): Error: Internal quota calculation
error
Running 'user_query' manually on the server:
db=> SELECT 111 AS uid, 222 AS gid, '/var/mail/AAA/' || 'XXX' || '/' ||
'testuser' AS home, '*:bytes=' || mailbox.quota AS quota_rule
FROM mailbox WHERE local_part = 'testuser' AND active ='t' LIMIT 1;
uid | gid | home|quota_rule
-+-+---+--
111 | 222 | /var/mail/AAA/BBB/testuser | *:bytes=10485760
(1 row)
The identity had been granted 'all' privilege for 'quota2' table:
*
db=> SELECT table_catalog, table_schema, table_name, privilege_type
db-> FROM information_schema.table_privileges
db-> WHERE grantee='DDD';
table_catalog | table_schema | table_name | privilege_type
---+--++
mail | public | quota2 | INSERT
mail | public | quota2 | SELECT
mail | public | quota2 | UPDATE
mail | public |
Re: Dict quota calculation errors "remote disconnected"/"broken pipe" on 2.22.
On 2017-02-08 00:10, Steffen Kaiser wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Sun, 5 Feb 2017, ygrishin-li...@mail2.ca wrote:
service dict {
unix_listener dict {
mode = 0660
user = Debian-exim
group = Debian-exim
}
}
dovecot-lda-erros.log:
**
Feb 04 14:23:33 lda(testuser@XXX): Error: read(/var/run/dovecot/dict)
failed: Remote disconnected
dovecot.log:
Feb 04 13:57:06 imap(YYY@XXX): Error: write(/var/run/dovecot/dict)
failed: Broken pipe
...
dovecot-debug.log:
**
Feb 04 13:18:12 lda(YYY@XXX): Error: read(/var/run/dovecot/dict)
failed: Remote disconnected
Feb 04 13:18:12 lda(YYY@XXX): Error: dict quota: Quota update failed,
it's now desynced
Feb 04 13:57:07 lda(testuser@XXX): Error: write(/var/run/dovecot/dict)
failed: Broken pipe
Does a process listens on /var/run/dovecot/dict ?
It certainly does:
#lsof /var/run/dovecot/dict
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dovecot 1140 root 42u unix 0xc6fe2300 0t0 15861
/var/run/dovecot/dict type=STREAM
$ ls -l /var/run/dovecot/dict
srw-rw 1 Debian-exim Debian-exim 0 Feb 12 03:53
/var/run/dovecot/dict
The socket is accessable by Debian-exim:Debian-exim only (0660). As
what user and group does the LDA and imap service run as?
LDA works as dovecot:Debian-exim:
*
lda:
driver = pipe
...
group = Debian-exim
...
(without specifying the user explicitly).
Yuriy
Re: Dict quota calculation errors "remote disconnected"/"broken pipe" on 2.22.
Solved the problem, reporting back to the community. /etc/dovecot/dovecot-dict-sql-user.conf had been lacking dovecot group permissions. It was 700/root:root. However why it wasn't reported by Dovecot explicitly in the log -- the greatest mystery to me. Now, after deleting dovecot and all its packages via 'apt' and installing afresh makes "lacking permissions for the file" entry to appear in the log. I can confirm that Dict-quota works perfectly well with Dovecot 2.2.22.
