Cannot Authenticate user with Kerberos/GSSAPI
My last message probably contained too much information. This one is more succient. I have a user, 'mark', who has been running a Thunderbird client on Windows to Dovecot server with Kerberos/GSSAPI authentication for over a year. I created a new Tbird account on a new Linux workstation for 'mark', also with Kerberos/GSSAPI and that worked just fine. I have another user, 'dsmith', who has been running a Thunderbird client on Windows to Dovecot server with Kerberos/GSSAPI authentication for over a year as well, no problems. I created a new Tbird account on the same new Linux workstation as above for 'dsmith', also with KerberosGSSAPI and that DID NOT WORK! I get the message in Thunderbird: "The Kerberos/GSSAPI ticket was not accepted by the IMAP server ... please check that you are logged into the Kerberos/GSSAPI realm." I created/recreated the smith account numerous time with slightly different settings hoping something will work, but I always get the same message. Why? I need to figure this out ASAP. Here is the dovecot log when user dsmith attempts to connect to dovecot from the Tbird client: Jul 11 19:29:43 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Jul 11 19:29:43 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Jul 11 19:29:43 auth: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Jul 11 19:29:43 auth: Debug: Read auth token secret from /usr/local/var/run/dovecot/auth-token-secret.dat Jul 11 19:29:43 auth: Debug: auth client connected (pid=1578) Jul 11 19:29:46 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write key exchange A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client certificate A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read certificate verify A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read finished A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write session ticket A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write finished A [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [192.168.0.57] Jul 11 19:29:46 imap-login: Debug: SSL alert: close notify [192.168.0.57] Jul 11 19:29:46 imap-login: Info: Disconnected (no auth attempts in 3 secs): user=<>, rip=192.168.0.57, lip=192.168.0.2, TLS, session= Jul 11 19:30:17 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Jul 11 19:30:17 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Jul 11 19:30:17 auth: Debug: auth client connected (pid=3148) Jul 11 19:30:17 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [192.168.0.57] Jul 11 19:30:17 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [192.168.0.57] Jul 11 19:30:17 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A [192.168.0.57] Jul 11 19:30:17 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A [192.168.0.57] Jul 11 19:30:17 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A [192.168.0.57] Jul 11 19:30:17 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate
Large jumps in dovecot-uidlist
Since upgrading to Debian Wheezy I have been observing large jumps in
the value stored in dovecot-uidlist. The effect of these jumps is to
confuse some mail clients (Thunderbird, Opera Mail) causing them not to
display messages from a (random?) point in time onwards in the affected
folder.
I have been unable to determine the root cause of these jumps.
What I do know:
1. I never observed the problem while running Debian Squeeze.
2. The only way I have found to fix the issue is to remove all the
dovecot* files from the affected folder and restart dovecot. I have
tried removing various subsets of the dovecot* files but haven't yet
found a subset that works.
3. I first tried upgrading from Squeeze to Wheezy within a few weeks of
the Wheezy release. At that point the problem occurred so frequently
that I couldn't sync my mail client with my mail box. Multiple folders
exhibited this problem and while I was fixing them the problem appeared
in other folders. I was unable to reach a point where the system was
stable so I reverted back to Squeeze.
4. Some time later (I think a year but I am not sure) I tried the
upgrade to Wheezy again. This time the system was stable. After a few
days I noticed that the problem was still occurring but less frequently.
Since then I have been fixing the problem as I notice it and
experimenting with various settings (mainly locking related) without
success. I now have a cron job that looks for problematic folders and
fixes them when found.
5. I have updated Dovecot packages to those from Jessie via
wheezy-backports and the problem still occurs.
6. Dovecot is installed along with postfix, amavisd-new, clamav-daemon,
procmail and fetchmail.
7. The mail server is running on a eSATA SheevaPlug that boots of an
external USB drive. All user home dirs (and associated mailboxes) are
located on a LUKS/ext3 encrypted partition.
8. The output of dovecot -n may be found at the end of this mail.
9. Uid values for the three most recent instances of this problem are:
3801596527, 3371927249, 3443181615
10. Some folders seem more susceptible to this problem but I have not
been able to identify any common factors/triggers.
11. The problem occurs - on average - once a day.
I am happy to provide any further information, configure any debug
logging and/or test suggested configuration settings that may help track
down the root cause of this problem.
Any help gratefully appreciated.
Mark
# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-kirkwood armv5tel Debian 7.8
mail_debug = yes
mail_location = maildir:~/Maildir
mail_max_userip_connections = 100
namespace {
inbox = yes
location =
mailbox {
special_use = \Drafts
name = Drafts
}
mailbox {
special_use = \Junk
name = Junk
}
mailbox {
special_use = \Sent
name = Sent
}
mailbox {
special_use = \Sent
name = Sent Messages
}
mailbox {
special_use = \Trash
name = Trash
}
prefix =
name = inbox
}
passdb {
driver = pam
}
protocols = " imap"
service replication-notify-fifo {
name = aggregator
}
service anvil-auth-penalty {
name = anvil
}
service auth-worker {
name = auth-worker
}
service {
unix_listener {
group = postfix
mode = 0666
user = postfix
path = /var/spool/postfix/private/auth-client
}
name = auth
}
service config {
name = config
}
service dict {
name = dict
}
service login/proxy-notify {
name = director
}
service dns-client {
name = dns_client
}
service doveadm-server {
name = doveadm
}
service {
inet_listener {
port = 0
name = imap
}
inet_listener {
port = 993
ssl = yes
name = imaps
}
process_min_avail = 5
name = imap-login
}
service imap-urlauth {
name = imap-urlauth-login
}
service imap-urlauth-worker {
name = imap-urlauth-worker
}
service token-login/imap-urlauth {
name = imap-urlauth
}
service {
executable = imap postlogin
name = imap
}
service indexer-worker {
name = indexer-worker
}
service indexer {
name = indexer
}
service ipc {
name = ipc
}
service lmtp {
name = lmtp
}
service log-errors {
name = log
}
service pop3 {
name = pop3-login
}
service login/pop3 {
name = pop3
}
service {
executable = script-login -d rawlog
name = postlogin
}
service replicator-doveadm {
name = replicator
}
service login/ssl-params {
name = ssl-params
}
service stats-mail {
name = stats
}
ssl_cert =
Re: Large jumps in dovecot-uidlist
On 15/06/2015 17:46, Felix Zielcke wrote: > Am Sonntag, den 14.06.2015, 10:46 +0100 schrieb Mark: >> Since upgrading to Debian Wheezy > ... >> # 2.2.13: /etc/dovecot/dovecot.conf >> # OS: Linux 3.2.0-4-kirkwood armv5tel Debian 7.8 > > So it's dovecot from the backports. > Didn't you notice that since about 2 months there's now jessie out? I did. Based on my previous experience of upgrading Debian as soon as the new release was announced and having to roll everything back because of this problem I'm taking a more cautious approach this time. > That has 2.2.13 too but maybe it's an issue outside of dovecot in some > library. Or just a bug in the backport. Possibly. I'm not against doing the full upgrade to Jessie to see what effect that has. I've been running the backport for a while and the problem certainly hasn't got worse. I'll see if I can find some time in the next few days to try the upgrade. > I have no problem at all with my dovecot under jessie. But it's > currently only responsible for my own private mails. So it hasn't much > to do. This install supports three users who access via IMAP using various combinations of Thunderbird, Opera Mail, native iPhone client, native Android client and K9. It seems to be the case that the mobile clients handle the UID jump gracefully whereas the desktop clients can't handle the larger UIDs. > Oh and now by looking again at your full mail: I mainly use IMAP + > Evolution or on my Android Smartphone K-9 Mail to access them. Thanks, Mark
Re: Large jumps in dovecot-uidlist
On 15/06/2015 18:04, Mark wrote: > On 15/06/2015 17:46, Felix Zielcke wrote: >> Am Sonntag, den 14.06.2015, 10:46 +0100 schrieb Mark: >>> Since upgrading to Debian Wheezy >> ... >>> # 2.2.13: /etc/dovecot/dovecot.conf >>> # OS: Linux 3.2.0-4-kirkwood armv5tel Debian 7.8 >> >> So it's dovecot from the backports. >> Didn't you notice that since about 2 months there's now jessie out? > > I did. Based on my previous experience of upgrading Debian as soon as > the new release was announced and having to roll everything back because > of this problem I'm taking a more cautious approach this time. > >> That has 2.2.13 too but maybe it's an issue outside of dovecot in some >> library. Or just a bug in the backport. > > Possibly. I'm not against doing the full upgrade to Jessie to see what > effect that has. I've been running the backport for a while and the > problem certainly hasn't got worse. I'll see if I can find some time in > the next few days to try the upgrade. I haven't found the time to do the upgrade to Jessie yet but I have tweaked my checking script to run more frequently. That gave me a narrower window to look at in the logs and these messages appear at the same time as the jump is introduced into the UID list: Jun 19 17:00:57 server02 dovecot: imap(mark): Error: Log synchronization error at seq=2,offset=152 for /secure/home/mark/Maildir/.Apache.Misc.committers/dovecot.index: Append with UID 2427206830, but next_uid = 2685375011 Jun 19 17:00:57 server02 dovecot: imap(mark): Warning: fscking index file /secure/home/mark/Maildir/.Apache.Misc.committers/dovecot.index I've done some Googling but haven't (yet) found anything that might match up. I'm hoping that the above might provide enough information to someone more familiar with Dovecot than me to suggest what I might be able to do to fix this. If not, I'll see what happens post Jessie upgrade. Thanks in advance, Mark
Re: Large jumps in dovecot-uidlist
On 19/06/2015 18:41, Mark wrote: > On 15/06/2015 18:04, Mark wrote: >> On 15/06/2015 17:46, Felix Zielcke wrote: >>> Am Sonntag, den 14.06.2015, 10:46 +0100 schrieb Mark: >>>> Since upgrading to Debian Wheezy >>> ... >>>> # 2.2.13: /etc/dovecot/dovecot.conf >>>> # OS: Linux 3.2.0-4-kirkwood armv5tel Debian 7.8 >>> >>> So it's dovecot from the backports. >>> Didn't you notice that since about 2 months there's now jessie out? >> >> I did. Based on my previous experience of upgrading Debian as soon as >> the new release was announced and having to roll everything back because >> of this problem I'm taking a more cautious approach this time. >> >>> That has 2.2.13 too but maybe it's an issue outside of dovecot in some >>> library. Or just a bug in the backport. >> >> Possibly. I'm not against doing the full upgrade to Jessie to see what >> effect that has. I've been running the backport for a while and the >> problem certainly hasn't got worse. I'll see if I can find some time in >> the next few days to try the upgrade. > > I haven't found the time to do the upgrade to Jessie yet but I have > tweaked my checking script to run more frequently. That gave me a > narrower window to look at in the logs and these messages appear at the > same time as the jump is introduced into the UID list: > > Jun 19 17:00:57 server02 dovecot: imap(mark): Error: Log synchronization > error at seq=2,offset=152 for > /secure/home/mark/Maildir/.Apache.Misc.committers/dovecot.index: Append > with UID 2427206830, but next_uid = 2685375011 > Jun 19 17:00:57 server02 dovecot: imap(mark): Warning: fscking index > file /secure/home/mark/Maildir/.Apache.Misc.committers/dovecot.index > > I've done some Googling but haven't (yet) found anything that might > match up. > > I'm hoping that the above might provide enough information to someone > more familiar with Dovecot than me to suggest what I might be able to do > to fix this. If not, I'll see what happens post Jessie upgrade. One more information point. I still see the same symptoms after upgrading to Jessie. I haven't been able to see a pattern in the errors yet but I'll keep looking. Spending some time trying to isolate a test case is on my TODO list but until I find the time to do that any other suggestions welcome. Mark
Re: stats module
On Fri, Nov 3, 2017 at 9:35 AM, Jeff Abrahamson wrote:
> Sorry, Aki, I don't follow you. Did I do it wrong in the file 91-stats
> that I shared in my original mail (attached here)?
>
> Jeff
>
>
> On 03/11/17 16:50, Aki Tuomi wrote:
> > You need to add the stats listener, by yourself.
> >
> > Aki
> >
> >> On November 3, 2017 at 5:19 PM Jeff Abrahamson wrote:
> >>
> >>
> >> Thanks for your suggestions, Steffen.
> >>
> >> Running doveconf -n shows no errors and also, sadly, no mention of the
> >> stats listener:
> >>
> >> ╭╴ (master=)╶╮
> >> ╰ [T] jeff@nantes-1:p27 $ doveconf -n
> >> # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
> >> # Pigeonhole version 0.4.13 (7b14904)
> >> # OS: Linux 4.4.0-97-generic x86_64 Ubuntu 16.04.3 LTS
> >> auth_mechanisms = plain login
> >> auth_socket_path = /var/run/dovecot/auth-userdb
> >> mail_location = maildir:~/Maildir
> >> managesieve_notify_capability = mailto
> >> managesieve_sieve_capability = fileinto reject envelope
> >> encoded-character vacation subaddress comparator-i;ascii-numeric
> >> relational regex imap4flags copy include variables body enotify
> >> environment mailbox date index ihave duplicate mime foreverypart
> >> extracttext
> >> namespace inbox {
> >> inbox = yes
> >> location =
> >> mailbox Drafts {
> >> special_use = \Drafts
> >> }
> >> mailbox Junk {
> >> special_use = \Junk
> >> }
> >> mailbox Sent {
> >> special_use = \Sent
> >> }
> >> mailbox "Sent Messages" {
> >> special_use = \Sent
> >> }
> >> mailbox Trash {
> >> special_use = \Trash
> >> }
> >> prefix =
> >> }
> >> passdb {
> >> driver = pam
> >> }
> >> plugin {
> >> sieve = ~/.dovecot.sieve
> >> sieve_dir = ~/sieve
> >> }
> >> protocols = imap sieve
> >> service auth {
> >> unix_listener /var/spool/postfix/private/auth {
> >> group = postfix
> >> mode = 0666
> >> user = postfix
> >> }
> >> unix_listener /var/spool/postfix/private/dovecot-auth {
> >> group = postfix
> >> mode = 0660
> >> user = postfix
> >> }
> >> }
> >> service imap-login {
> >> inet_listener imaps {
> >> port = 993
> >> ssl = yes
> >> }
> >> }
> >> ssl_cert = >> ssl_cipher_list =
> >> EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:
> EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!
> aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!
> ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
> >> ssl_key = >> ssl_protocols = !SSLv2 !SSLv3
> >> userdb {
> >> driver = passwd
> >> }
> >> protocol lda {
> >> deliver_log_format = msgid=%m: %$
> >> mail_plugins = sieve
> >> postmaster_address = postmaster
> >> quota_full_tempfail = yes
> >> rejection_reason = Your message to <%t> was automatically
> >> rejected:%n%r
> >> }
> >> protocol imap {
> >> imap_client_workarounds = delay-newmail
> >> mail_max_userip_connections = 20
> >> }
> >> protocol pop3 {
> >> mail_max_userip_connections = 10
> >> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
> >> }
> >> ╭╴ (master=)╶╮
> >> ╰ [T] jeff@nantes-1:p27 $
> >>
> >> Here I have a tail -f /var/log/mail.log and mail.err running in the
> >> background so we can see the results of the restart:
> >>
> >> [T] jeff@nantes-1:conf.d $ ls -l
> >> total 136
> >> -rw-r--r-- 1 root root 5301 Aug 25 15:26 10-auth.conf
> >> -rw-r--r-- 1 root root 1893 Mar 16 2016 10-director.conf
> >> -rw-r--r-- 1 root root 2805 Mar 16 2016 10-logging.conf
> >> -rw-r--r-- 1 root root 16172 Aug 25 15:35 10-mail.conf
> >> -rw-r--r-- 1 root root 3547 Aug 25 15:40 10-master.conf
> >> -rw-r--r-- 1 root root 2307 Aug 25 16:27 10-ssl.conf
> >> -rw-r--r-- 1 root root 291 Apr 11 2017 10-tcpwrapper.conf
> >> -rw-r--r-- 1 root root 1668 Mar 16 2016 15-lda.conf
> >> -rw-r--r-- 1 root root 2808 Mar 16 2016 15-mailboxes.conf
> >> -rw-r--r-- 1 root root 3295 Mar 16 2016 20-imap.conf
> >> -rw-r--r-- 1 root root 2398 Apr 11 2017 20-managesieve.conf
> >> -rw-r--r-- 1 root root 4109 Aug 25 15:28 20-pop3.conf
> >> -rw-r--r-- 1 root root 676 Mar 16 2016 90-acl.conf
> >> -rw-r--r-- 1 root root 292 Mar 16 2016 90-plugin.conf
> >> -rw-r--r-- 1 root root 2502 Mar 16 2016 90-quota.conf
> >> -rw-r--r-- 1 root root 6822 Apr 11 2017 90-sieve.conf
> >> -rw-r--r-- 1 root root 1829 Apr 11 2017 90-sieve-extprograms.conf
> >> -rw-r--r-- 1 root root 1856 Nov 3 16:11 91-stats
> >> -rw-r--r-- 1 root root 1430 Oct 31 16:33
> 99-mail-stack-delivery.conf
> >> -rw-r--r-- 1 root root 499 Mar 16 2016
> auth-checkpassword.conf.ext
> >
Upgrade to 2.2.32 from 2.2.15 failed
I have a problem. I have been running Dovecot 2.2.15 and I'd like to upgrade. My distro (Slackware) has dovecot 2.2.32 available. I downloaded and installed that, but it didn't work. No one was able to get messages from the dovecot server on their workstations. The following is the entire dovecot log file from startup to the last message generated. No more messages went into the logfile after line 76, even with clients trying to connect. The 174.233.134.88 IP is from an external user connecting from his iPhone. The normal successful message from this user are shown at bottom. I'm suspecting something to do with line 18 where is says "Auth process broken." If anyone has any insight I'd deeply appreciate it as I'd love to upgrade. THX -- Mark 1 Nov 24 19:22:24 master: Info: Dovecot v2.2.32 (dfbe293d4) starting up for imap (core dumps disabled) 2 Nov 24 19:22:24 ssl-params: Info: Generating SSL parameters 3 Nov 24 19:22:26 ssl-params: Info: SSL parameters regeneration completed 4 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [174.233.134.88] 5 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [174.233.134.88] 6 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A [174.233.134.88] 7 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A [174.233.134.88] 8 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A [174.233.134.88] 9 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write key exchange A [174.233.134.88] 10 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A [174.233.134.88] 11 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [174.233.134.88] 12 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client certificate A [174.233.134.88] 13 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A [174.233.134.88] 14 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A [174.233.134.88] 15 Nov 24 19:23:02 auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth 16 Nov 24 19:23:02 auth: Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so 17 Nov 24 19:23:02 auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth 18 Nov 24 19:23:02 imap-login: Info: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=174.233.134.88, lip=98.102.63.107, TLS handshaking, session= 19 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [174.233.134.88] 20 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [174.233.134.88] 21 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A [174.233.134.88] 22 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A [174.233.134.88] 23 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A [174.233.134.88] 24 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write key exchange A [174.233.134.88] 25 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A [174.233.134.88] 26 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [174.233.134.88] 27 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client certificate A [174.233.134.88] 28 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A [174.233.134.88] 29 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A [174.233.134.88] 30 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A [174.233.134.88] 31 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read certificate verify A [174.233.134.88] 32 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read finished A [174.233.134.88] 33 Nov 24 19:23:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read finished A [174.233.134.88] 34 Nov 24 19:23:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A [174.233.134.88] 35 Nov 24 19:23:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write finished A [174.233.134.88] 36 Nov 24 19:23:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [174.233.134.88] 37 Nov 24 19:23:03 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [174.233.134.88] 38 Nov 24 19:23:03 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [174.233.134.88] 39 Nov 24 19:23:04 au
Re: Upgrade to 2.2.32 from 2.2.15 failed
No, is that something that would make a difference between 2.2.15 and 2.2.32? --Mark On Fri, 24 Nov 2017 21:37:47 -0800 Gary wrote: > Out of curiosity, do you do a !SSLv3 in the conf file? > > > Original Message >> From: mfo...@ohprs.org >> Sent: November 24, 2017 9:04 PM >> To: dovecot@dovecot.org >> Subject: Upgrade to 2.2.32 from 2.2.15 failed >> >> I have a problem. I have been running Dovecot 2.2.15 and I'd like to >> upgrade. My distro >> (Slackware) has dovecot 2.2.32 available. I downloaded and installed that, >> but it didn't work. >> No one was able to get messages from the dovecot server on their >> workstations. The following is >> the entire dovecot log file from startup to the last message generated. No >> more messages went >> into the logfile after line 76, even with clients trying to connect. The >> 174.233.134.88 IP is >> from an external user connecting from his iPhone. The normal successful >> message from this user >> are shown at bottom. >> >> I'm suspecting something to do with line 18 where is says "Auth process >> broken." If anyone has >> any insight I'd deeply appreciate it as I'd love to upgrade. >> >> THX -- Mark >> >> 1 Nov 24 19:22:24 master: Info: Dovecot v2.2.32 (dfbe293d4) starting up for >> imap (core dumps disabled) >> 2 Nov 24 19:22:24 ssl-params: Info: Generating SSL parameters >> 3 Nov 24 19:22:26 ssl-params: Info: SSL parameters regeneration completed >> 4 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x10, ret=1: before/accept >> initialization [174.233.134.88] >> 5 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: >> before/accept initialization [174.233.134.88] >> 6 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read >> client hello A [174.233.134.88] >> 7 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> server hello A [174.233.134.88] >> 8 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> certificate A [174.233.134.88] >> 9 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> key exchange A [174.233.134.88] >> 10 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> server done A [174.233.134.88] >> 11 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush >> data [174.233.134.88] >> 12 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read >> client certificate A [174.233.134.88] >> 13 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read >> client key exchange A [174.233.134.88] >> 14 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read >> client key exchange A [174.233.134.88] >> 15 Nov 24 19:23:02 auth: Debug: Loading modules from directory: >> /usr/lib64/dovecot/auth >> 16 Nov 24 19:23:02 auth: Debug: Module loaded: >> /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so >> 17 Nov 24 19:23:02 auth: Debug: Loading modules from directory: >> /usr/lib64/dovecot/auth >> 18 Nov 24 19:23:02 imap-login: Info: Disconnected: Auth process broken >> (disconnected before auth was ready, waited 0 secs): user=<>, >> rip=174.233.134.88, lip=98.102.63.107, TLS handshaking, >> session= >> 19 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x10, ret=1: before/accept >> initialization [174.233.134.88] >> 20 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: >> before/accept initialization [174.233.134.88] >> 21 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read >> client hello A [174.233.134.88] >> 22 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> server hello A [174.233.134.88] >> 23 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> certificate A [174.233.134.88] >> 24 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> key exchange A [174.233.134.88] >> 25 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write >> server done A [174.233.134.88] >> 26 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush >> data [174.233.134.88] >> 27 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read >> client certificate A [174.233.134.88] >> 28 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read >> client key exchange A [174.233.134.88] >> 29 Nov 24 19:23:02 imap-login: Debug: SSL: where=0x2002, re
Re: Mark message as read when moved to Trash
See: https://forum.kde.org/viewtopic.php?f=215&t=55940 --Mark Felix Rubio Dalmau wrote: > Hi all, > > When I send a message to trash, without previously reading it (just > with the subject is enough to say I do not want to read it), it remains as > "unread". Then, clients (I am using Kmail) report there are unread message, > when all of them are in Trash. The question, then is: Is there any way to > automatically mark a message as read, when that message is moved to Trash? > > Thank you! > Felix
Lua Auth
Just happened to be surfing the docs and saw this. This is beyond awesome: https://wiki2.dovecot.org/AuthDatabase/Lua Any words of wisdom on using it? I'd be putting a bunch of mysql logic in it. Any horrible gotchas there? When it says 'blocking', should I assume that means that a auth worker process will *not* accept any new auth lookups until both auth_passdb_lookup() and auth_userdb_lookup() have completed (in which I'd be doing several mysql calls)? If that's the case, I assume that the number of auth workers should be bumped up. And is a 2.3 release fairly imminent?
Unable to build sieve plugin
I'm wanting to experiment with sieve processing for the first time. Having some trouble getting started. I googled to page, https://wiki2.dovecot.org/Pigeonhole/Sieve, went to the "Download and Installation" link, then the "Pigeonhole download page" link and downloaded dovecot-2.2-pigeonhole-0.4.21.tar.gz (I have Dovecot version 2.2.15). I untarred, ran ./configure (which appeared to run OK), then `make` and got the following erro: make[4]: Entering directory '/user/util/src/dovecot/dovecot-2.2-pigeonhole-0.4.21/src/lib-sieve/util' /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I/usr/local/include/dovecot -DMODULEDIR=\""/usr/local/lib/dovecot"\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I../../.. -MT edit-mail.lo -MD -MP -MF .deps/edit-mail.Tpo -c -o edit-mail.lo edit-mail.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I/usr/local/include/dovecot -DMODULEDIR=\"/usr/local/lib/dovecot\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I../../.. -MT edit-mail.lo -MD -MP -MF .deps/edit-mail.Tpo -c edit-mail.c -fPIC -DPIC -o .libs/edit-mail.o edit-mail.c: In function 'edit_mail_get_special': edit-mail.c:1592:8: error: 'MAIL_FETCH_STORAGE_ID' undeclared (first use in this function) case MAIL_FETCH_STORAGE_ID: ^ edit-mail.c:1592:8: note: each undeclared identifier is reported only once for each function it appears in This was followed by several more errors and the make failed. What did I do wrong? --Mark
iPhone no longer authenticating
I've switched a user to being an active directory user. That user's email
client authorizes
just fine with dovecot using GSSAPI. However, now his iPhone won't authorize.
In the dovecot
log file I get:
Dec 01 14:27:28 auth: Debug: client in: AUTH1 PLAIN service=imap
secured session=q4n3W0xfggBiZj9slip=98.102.63.107 rip=98.102.63.108
lport=993 rport=49538 resp=AG1wcmVzcwBEaW5va3JvbndhbGw0NQ==
(previous base64 data may contain sensitive data)
Dec 01 14:27:32 auth-worker(5988): Debug: shadow(mpress,98.102.xx.yyy): lookup
Dec 01 14:27:32 auth-worker(5988): Info: shadow(mpress,98.102.xx.yyy): unknown
user (given password: ***)
Dec 01 14:27:34 auth: Debug: client passdb out: FAIL1 user=mpress
Dec 01 14:27:34 imap-login: Info: Aborted login (auth failed, 1 attempts in 6
secs): user=, method=PLAIN, rip=98.102.xx.yyy, lip=98.102.63.107, TLS,
session=
Dec 01 14:27:34 imap-login: Debug: SSL alert: close notify [98.102.xx.yyy]
This same user will authenticate OK from his local domain workstation:
Dec 01 14:28:52 auth: Debug: master userdb out: USER1948516353 mpress
system_groups_user=HPRS\mpress uid=10005gid=1
home=/home/HPRS/mpress auth_token=ce3050035718ed0996af698400c4de1be453ec06
auth_user=mpress@HPRS.LOCAL
Dec 01 14:28:52 imap-login: Info: Login: user=, method=GSSAPI,
rip=192.168.0.54, lip=192.168.0.2, mpid=9755, TLS, session=<6MT1YExftwDAqAA2>
I'm pretty sure the reason has to do with Active Directory authenication
locally, but of course
his iPhone is not a member of the domain, and he is no longer in
/etc/passwd/shadow.
So, what is the best way to get the iPhone to authenticate?
Here's my current config:
> doveconf -n
# 2.2.15: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 4.4.88 x86_64 Slackware 14.2
auth_debug = yes
auth_debug_passwords = yes
auth_gssapi_hostname = $ALL
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = plain login gssapi
auth_use_winbind = yes
auth_username_format = %n
auth_verbose = yes
auth_verbose_passwords = plain
disable_plaintext_auth = no
info_log_path = /var/log/dovecot_info
mail_location = maildir:~/Maildir
passdb {
driver = shadow
}
protocols = imap
ssl_cert =
Re: Lua Auth
On Thu, Nov 30, 2017 at 5:26 AM, Stephan Bosch wrote: > > > Op 29-11-2017 om 6:17 schreef Aki Tuomi: > >> On November 29, 2017 at 4:37 AM Mark Moseley >>> wrote: >>> >>> >>> Just happened to be surfing the docs and saw this. This is beyond >>> awesome: >>> >>> https://wiki2.dovecot.org/AuthDatabase/Lua >>> >>> Any words of wisdom on using it? I'd be putting a bunch of mysql logic in >>> it. Any horrible gotchas there? When it says 'blocking', should I assume >>> that means that a auth worker process will *not* accept any new auth >>> lookups until both auth_passdb_lookup() and auth_userdb_lookup() have >>> completed (in which I'd be doing several mysql calls)? If that's the >>> case, >>> I assume that the number of auth workers should be bumped up. >>> >>> And is a 2.3 release fairly imminent? >>> >> Hi! >> >> This feature was added very recently, and there is very little >> operational experience on it. As the docs should say, blocking=yes means >> that an auth worker is used, and yes, it will block each auth worker during >> authentication, but what we tried, it should perform rather nicely. >> >> The most important gotcha is to always test your lua code rigorously, >> because there is not much we can do to save you. >> >> It should be present in master branch, so if someone feels like trying it >> out, please let us know if you find any bugs or strangeness. It's not >> present in nightlies yet. >> >> We are planning on releasing 2.3.0 this year. >> > > The Xi package builder has this feature enabled since yesterday. It is > available in the dovecot-lua package; the first Xi package that doesn't > have an official Debian equivalent (yet anyway). > > > I've been playing with Lua auth and so far no issues. I was previously putting together a very ugly MySQL stored procedure. Using Lua would be a lot easier (esp when it comes to returning an arbitrary number of columns). I'd love to see any test Lua code that the dovecot team has been playing around with (and realize it's not remotely production-ready, so don't worry about caveats I did have a couple of questions though: 1) Is the data returned by Lua auth not cacheable? I've got the following settings (and I'm just using Lua in the userdb lookup, not passdb -- passdb is doing a lightweight SQL lookup for username/password): auth_cache_negative_ttl = 1 mins auth_cache_size = 10 M auth_cache_ttl = 10 mins but I notice that every time I auth, it'll redo all the queries in my Lua code. I'd have expected that data to be served out of cache till the 10min TTL is up 2) Is there an appropriate way to return data with spaces in it (or presumably other non-alphanum chars. My quota name had a space in it, which somehow got interpreted as 'yes' , i.e.: imap: Error: Failed to initialize quota: Invalid quota root quota: Unknown quota backend: yes I simply changed the space to an underscore as a workaround, but I'm curious if there's a better way. I tried various quoting without success. Didn't try escaping yet. 3) Can you elaborate on the "auth_request#response_from_template(template)" and "auth_request#var_expand(template)" functions? Specifically how to use them. I'm guessing that I could've used one of them to work around #2 (that it would have done the escaping for me) Thanks!
Howto authenticate smartPhone via Active Directory
I have a Samba4 Active Directory server. Dovecot authenticates AD Users with
domain credentials
using GSSAPI (Thunderbird client). I believe I have Dovecot set to attempt
authentication via
shadow first and. failing that, it does authenticate via GSSAPI.
Smartphones connect to Dovecot via port 143 and SSL. They are not domain
members so if the
shadow authentication fails, no other methods are tried and no connection is
made.
What can I do with my dovecot config to fix this?
> doveconf -n
# 2.2.15: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 4.4.88 x86_64 Slackware 14.2
auth_debug = yes
auth_debug_passwords = yes
auth_gssapi_hostname = $ALL
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = plain login gssapi
auth_use_winbind = yes
auth_username_format = %n
auth_verbose = yes
auth_verbose_passwords = plain
disable_plaintext_auth = no
info_log_path = /var/log/dovecot_info
mail_location = maildir:~/Maildir
passdb {
driver = shadow
}
protocols = imap
ssl_cert =
Re: Upgrade to 2.2.32 from 2.2.15 failed
On Sat, 25 Nov 2017 10:13:58 +0200 (EET) Aki Tuomi wrote:
>
> > On November 25, 2017 at 7:04 AM Mark Foley wrote:
> >
> > I have a problem. I have been running Dovecot 2.2.15 and I'd like to
> > upgrade. My distro
> > (Slackware) has dovecot 2.2.32 available. I downloaded and installed that,
> > but it didn't work.
> > No one was able to get messages from the dovecot server on their
> > workstations. The following is
> > the entire dovecot log file from startup to the last message generated. No
> > more messages went
> > into the logfile after line 76, even with clients trying to connect. The
> > 174.233.134.88 IP is
> > from an external user connecting from his iPhone. The normal successful
> > message from this user
> > are shown at bottom.
> >
> > I'm suspecting something to do with line 18 where is says "Auth process
> > broken." If anyone has
> > any insight I'd deeply appreciate it as I'd love to upgrade.
> >
> > THX -- Mark
> >
>
> Can you try adding
>
> service auth {
> executable = strace -o /tmp/auth.trace /usr/libexec/dovecot/auth
> }
>
> and see if it gives any insight why it dies?
>
> Aki
>
The problem was that I did an install from sbopkg which downloads and installs
the package in
the SlackBuilds repository. This mechanism does not easily allow setting
options. I needed to
have the --with-gssapi=yes option set.
So, I just downloaded directly from
http://www.dovecot.org/releases/2.2/dovecot-2.2.33.2.tar.gz
and did:
./configure --with-gssapi=yes
make
make install
and everything appears to be working fine!
--Mark
Re: Howto authenticate smartPhone via Active Directory
Yes, you are right. This link:
https://www.redips.net/linux/android-email-postfix-auth/#section2
shows:
passdb pam {
}
used for authenticating Android. Problem #1 is that Slackware does not ship
with PAM and the
AD/DC Samba4 does not use it. It is used on Slackware for a domain member, but
I'm not sure I
should try configuring PAM on the AD/DC.
Is there some otherway I can get authentication using domain credentials
besides pam? the phone
can send user and password.
--Mark
-Original Message-
> Date: Sun, 03 Dec 2017 15:22:56 +0200
> Subject: Re: Howto authenticate smartPhone via Active Directory
> From: Aki Tuomi
> To: Mark Foley , dovecot@dovecot.org
>
> Actually you are authenticating gssapi clients from ad and everyone else from
> shadow. maybe you need to configure pam module?
> ---Aki TuomiDovecot oy
>
> Original message
> From: Mark Foley
> Date: 03/12/2017 06:03 (GMT+02:00)
> To: dovecot@dovecot.org
> Subject: Howto authenticate smartPhone via Active Directory
> I have a Samba4 Active Directory server. Dovecot authenticates AD Users with
> domain credentials
> using GSSAPI (Thunderbird client). I believe I have Dovecot set to attempt
> authentication via
> shadow first and. failing that, it does authenticate via GSSAPI.
>
> Smartphones connect to Dovecot via port 143 and SSL. They are not domain
> members so if the
> shadow authentication fails, no other methods are tried and no connection is
> made.
>
> What can I do with my dovecot config to fix this?
>
> > doveconf -n
> # 2.2.15: /usr/local/etc/dovecot/dovecot.conf
> # OS: Linux 4.4.88 x86_64 Slackware 14.2
> auth_debug = yes
> auth_debug_passwords = yes
> auth_gssapi_hostname = $ALL
> auth_krb5_keytab = /etc/dovecot/dovecot.keytab
> auth_mechanisms = plain login gssapi
> auth_use_winbind = yes
> auth_username_format = %n
> auth_verbose = yes
> auth_verbose_passwords = plain
> disable_plaintext_auth = no
> info_log_path = /var/log/dovecot_info
> mail_location = maildir:~/Maildir
> passdb {
> driver = shadow
> }
> protocols = imap
> ssl_cert =
> ssl_key = userdb {
> driver = passwd
> }
> verbose_ssl = yes
>
> Thanks, Mark
Re: Howto authenticate smartPhone via Active Directory
Unfortunately, I tried for weeks to figure out passdb ldap without success. I
guess I'm just
not knowledgeable enough about how to use ldap and Active Directory. The
dovecot wiki
https://wiki2.dovecot.org/AuthDatabase/LDAPm doesn't help me much. All it says
is:
Active Directory
When connecting to AD, you may need to use port 3268. Then again, not all LDAP
fields are
available in port 3268. Use whatever works.
http://technet.microsoft.com/en-us/library/cc978012.aspx
I have not been able to find an example of someone using Dovecot and ldap with
AD.
However, I have had some success with CheckPassword
(https://wiki2.dovecot.org/AuthDatabase/CheckPassword). Using a program I
wrote to do
ntlm_auth, I am able to authenticate the smartPhone user and pass the required
parameters back
to Dovecot. My auth-checkpasswd.conf.ext is the as-shipped standard except
pointing to my
checkpassword executable.
passdb {
driver = checkpassword
args = /user/util/bin/checkpassword
}
userdb {
driver = prefetch
}
The one issue I have with this at the moment is that dovecot runs checkpassword
for every user,
smartphone or otherwise:
Dec 03 18:56:32 auth-worker(14903): Info:
shadow(charmaine,192.168.0.52,): unknown user - trying the
next passdb
Dec 03 18:56:32 auth: Debug:
checkpassword(charmaine,192.168.0.52,): execute:
/user/util/bin/checkpassword /usr/local/libexec/dovecot/checkpassword-reply
Dec 03 18:56:32 auth: Debug:
checkpassword(charmaine,192.168.0.52,): Received input:
Dec 03 18:56:32 auth: Debug:
checkpassword(charmaine,192.168.0.52,): exit_status=1
Dec 03 18:56:32 auth: Debug:
checkpassword(charmaine,192.168.0.52,): Credentials:
Dec 03 18:56:32 auth: Debug: client passdb out: OK 1 user=charmaine
original_user=charmaine@HPRS.LOCAL
Dec 03 18:56:32 auth: Debug: master in: REQUEST 1884160001 14902 1
586863e54c57c999ee5731906a59257csession_pid=14907 request_auth_token
Dec 03 18:56:32 auth-worker(14903): Debug:
passwd(charmaine,192.168.0.52,): lookup
Dec 03 18:56:32 auth-worker(14903): Debug:
passwd(charmaine,192.168.0.52,): username changed charmaine
-> HPRS\charmaine
Dec 03 18:56:32 auth: Debug: master userdb out: USER1884160001
HPRS\charmaine system_groups_user=HPRS\charmaineuid=10003gid=1
home=/home/HPRS/charmaine
auth_token=d8d39ec4cc71923806ca7f539427e8aac44e90f7
auth_user=charmaine@HPRS.LOCAL
Dec 03 18:56:32 imap-login: Info: Login: user=, method=GSSAPI,
rip=192.168.0.52, lip=192.168.0.2, mpid=14907, TLS, session=
Dec 03 18:56:50 auth: Debug: auth client connected (pid=14913)
Notice after the "shadow" auth fails it says, "unknown user - trying the next
passdb", which is
checkpassword (which apparently succeeds), then it goes on to gssapi which also
succeeds. Is
there a way to only have it do checkpassword if all shadow and gssapi fail? My
mechanisms are:
auth_mechanisms = plain login gssapi
THX, --Mark
--Mark
-Original Message-
Date: Sun, 03 Dec 2017 22:28:53 +0200
Subject: Re: Howto authenticate smartPhone via Active Directory
From: Aki Tuomi
To: Mark Foley , dovecot@dovecot.org
with passdb ldap i guess.
---Aki Tuomi
Dovecot oy
Original message
From: Mark Foley
Date: 03/12/2017 21:18 (GMT+02:00)
To: dovecot@dovecot.org
Subject: Re: Howto authenticate smartPhone via Active Directory
Yes, you are right. This link:
https://www.redips.net/linux/android-email-postfix-auth/#section2
shows:
passdb pam {
}
used for authenticating Android. Problem #1 is that Slackware does not ship
with PAM and the
AD/DC Samba4 does not use it. It is used on Slackware for a domain member, but
I'm not sure I
should try configuring PAM on the AD/DC.
Is there some otherway I can get authentication using domain credentials
besides pam? the phone
can send user and password.
--Mark
-Original Message-
> Date: Sun, 03 Dec 2017 15:22:56 +0200
> Subject: Re: Howto authenticate smartPhone via Active Directory
> From: Aki Tuomi
> To: Mark Foley , dovecot@dovecot.org
>
> Actually you are authenticating gssapi clients from ad and everyone else from
> shadow. maybe you need to configure pam module?
> ---Aki TuomiDovecot oy
>
> Original message
> From: Mark Foley
> Date: 03/12/2017 06:03 (GMT+02:00)
> To: dovecot@dovecot.org
> Subject: Howto authenticate smartPhone via Active Directory
> I have a Samba4 Active Directory server. Dovecot authenticates AD Users with
> domain credentials
> using GSSAPI (Thunderbird client). I believe I have Dovecot set to attempt
> authentication via
> shadow first and. failing that, it does authenticate via GSSAPI.
>
> Smartphones connect to Dovecot via port 143 and SSL. They are not domain
> members so if the
> shadow authentication fails, no other methods are tried and no connection is
Re: Howto authenticate smartPhone via Active Directory
mj - thanks! That the first useful example I've received from any forum/list.
I'm getting ready
to try my config (have to do so after hours), but I have some probably
simple-minded questions:
Your example is not the complete dovecot-ldap.conf.ext file, right? Have you
just given me
differences in your config from the "original"? You've kept the hosts, base,
ldap_version,
scope, deref, debug_level, and auth_bind_userdn settings in your config, right?
Your dn is:
dn = cn=search_dovecit,cn=users,dc=company,dc=com
Mine (original) is:
dn = cn=user_for_bind,cn=Users,dc=dom
Can you tell me why you have "search_dovecit" versus "user_for_bind"? Is that
something I need
in order to make this work?
Is your "dc=company,dc=com" meta-syntax and you use your actual domain CNs
here, or is that
litterally what you have there?
My dnpass (original) is:
dnpass =
your example is:
dnpass = top_secret
Again, are the assigned values meta-syntax (meta-syntax in configs is not
obvious to me unless
it is bold, underlined, italicized and colored ... or uses brackets or some
other convention)?
If meta, what is actually supposed to go there?
With your "this user/passwd filter". Can you tell me why you have
"userAccountControl=514"? Is
that 514 bit documented somewhere? Your user_filer/pass_filter is *completely*
different from
my installed original.
You don't mention the user_attrs/pass_attrs settings. Is this because you use
the originals or
because you have commented them out? My current settings are:
user_attrs = quotaFieldAD=quota_rule=*:storage=%$MB
pass_attrs = userPassword=password
My auth_mechanisms are:
auth_mechanisms = plain login gssapi
Is this sufficient for ldap?
Thanks for your help --Mark
btw - I have been running Dovecot with AD for years, but for local Domain users
authenticating
via GSSAPI. Remote users (e.g. smartPhones) don't have that mechanism that I'm
aware of.
Currently they are authenticated via shadow, but I'd like to remove AD users
from /etc/passwd.
On Mon, 4 Dec 2017 09:04:57 +0100 mj wrote
>
> Hi Mark,
>
> Just to let you know that we are running dovecot with AD. (and I guess:
> *many* people are running that combination)
>
> It worked without issues, we are using in dovecot-ldap.conf.ext:
>
> > auth_bind = yes
>
> this user/passwd filter:
> > = (&(objectclass=person)(sAMAccountName=%n)(!(userAccountControl=514)))
>
> > dn = cn=search_dovecit,cn=users,dc=company,dc=com
> > dnpass = top_secret
>
> And not the 3268 port, but regular 389.
>
> Hope that helps.
>
> MJ
>
>
>
> On 12/04/2017 01:38 AM, Mark Foley wrote:
> > Unfortunately, I tried for weeks to figure out passdb ldap without success.
> > I guess I'm just
> > not knowledgeable enough about how to use ldap and Active Directory. The
> > dovecot wiki
> > https://wiki2.dovecot.org/AuthDatabase/LDAPm doesn't help me much. All it
> > says is:
> >
> > Active Directory
> >
> > When connecting to AD, you may need to use port 3268. Then again, not all
> > LDAP fields are
> > available in port 3268. Use whatever works.
> > http://technet.microsoft.com/en-us/library/cc978012.aspx
> >
> > I have not been able to find an example of someone using Dovecot and ldap
> > with AD.
> >
> > However, I have had some success with CheckPassword
> > (https://wiki2.dovecot.org/AuthDatabase/CheckPassword). Using a program I
> > wrote to do
> > ntlm_auth, I am able to authenticate the smartPhone user and pass the
> > required parameters back
> > to Dovecot. My auth-checkpasswd.conf.ext is the as-shipped standard except
> > pointing to my
> > checkpassword executable.
> >
> > passdb {
> > driver = checkpassword
> > args = /user/util/bin/checkpassword
> > }
> > userdb {
> > driver = prefetch
> > }
> >
> > The one issue I have with this at the moment is that dovecot runs
> > checkpassword for every user,
> > smartphone or otherwise:
> >
> > Dec 03 18:56:32 auth-worker(14903): Info:
> > shadow(charmaine,192.168.0.52,): unknown user - trying
> > the next passdb
> > Dec 03 18:56:32 auth: Debug:
> > checkpassword(charmaine,192.168.0.52,): execute:
> > /user/util/bin/checkpassword /usr/local/libexec/dovecot/checkpassword-reply
> > Dec 03 18:56:32 auth: Debug:
> > checkpassword(charmaine,192.168.0.52,): Received input:
> > Dec 03 18:56:32 auth: Debug:
> > checkpassword(charmaine,192.168.0.52,): exit_status=1
> > Dec 03 18:56:32 auth: Debug:
> > checkpassword(charmaine,192.168.0.52,): Crede
Can passdb be bypassed for non-plaintext authentication mechanisms
I am using Active directory authentication via gssapi for most users. In
dovecot.conf I have:
auth_mechanisms = plain login gssapi
auth_use_winbind = yes
I also have
passdb { driver = shadow }
userdb { driver = passwd }
for those few users who are NOT AD users.
Even though the AD users do not exist in /etc/passwd or /etc/shadow, Dovecot
ALWAYS first looks
them up in shadow, which ALWAYS fails.
The https://wiki2.dovecot.org/PasswordDatabase wiki says, "these databases
can't be used with
non-plaintext authentication mechanisms."
Is there a way to bypass checking passdb (and userdb?) for these mechanism?
--Mark
Re: Howto authenticate smartPhone via Active Directory
On Tue, 5 Dec 2017 16:42:15 +0100 mj wrote: > Hi, > > Not much time to reply now. > > On 12/05/2017 05:21 AM, Mark Foley wrote: > > mj - thanks! That the first useful example I've received from any > > forum/list. I'm getting ready > > to try my config (have to do so after hours), but I have some probably > > simple-minded questions: > > Well, that looks as if you are testing/trying out on your production > machine. Why not setup a seperate (virtual?) test server to play with..? > Use the same os version, with the same dovecot version. > Or clone your production machine, so you can test as much as you like, > without time pressure, at any given time. I've been playing with this ldap authentication for a couple of years off and on. Time isn't a problem. The issue with setting up a test environment is that I really need the domain workstations and external smartphone attempting to connect when I make a change so I can follow what's going on in the Dovecot log and maillog. It's rather simple to test a change, then put things back. I'll likely not go the test platform route for now, but thanks for the input. > > Your example is not the complete dovecot-ldap.conf.ext file, right? Have > > you just given me > > differences in your config from the "original"? You've kept the hosts, > > base, ldap_version, > > scope, deref, debug_level, and auth_bind_userdn settings in your config, > > right? > Not the complete file, no. I just provided the essentials. > [deleted] Ok, here's what I've come up with for dovecot-ldap.conf.ext hosts = mail.hprs.local base = dc=mail, dc=hprs, dc=local ldap_version = 3 scope = subtree deref = never debug_level = -1 auth_bind = yes auth_bind_userdn = %n@dom dn = cn=Administrator,cn=users,dc=hprs,dc=local dnpass = *** user_filter = (&(objectclass=person)(sAMAccountName=%n)(!(userAccountControl=514))) pass_filter = (&(objectclass=person)(sAMAccountName=%n)(!(userAccountControl=514))) I've enabled auth-ldap.conf.ext in 10-auth.conf. My doveconf is listed at bottom. Unfortuntately, this doesn't work. My remote devices are not even showing as trying to connect. For internal domain LAN users I get: Dec 06 01:08:10 imap-login: Info: Disconnected: Auth process broken (disconnected before auth was ready, waited 30 secs): user=<>, rip=192.168.0.52, lip=192.168.0.2, session=<3/ZyxaVfE8PAqAA0> I do see ldap listening on 389, imap[s] (Dovecot) listening on 143 and 993, these last two are opened externally through the firewall. > For the rest: my advise is that you *really* need to pay around with > this much more. Get yourself a test environment, and play and test. > > Plus: read some dovecot/ad howto's, and try things in your own environment. > > Quick google returns: > https://www.howtoforge.com/postfix-dovecot-authentication-against-active-directory-on-centos-5.x > I know my level of sophistication on this must sound like I've glibly posted a question hoping someone will do the work for me without my having to do any thinking myself, but believe me, I've been reading and experimenting with this for a very long time. I've got internal AD authentication working with GSSAPI and I've got a rather complex checkpassword program able to do authentication, so I don't think I'm a complete moron, although this project makes me feel that way. Now, I just want smartphones to authenticate with their owners' domain credentials and get them out of /etc/passwd. I believe I've read all the Dovecot wikis on ldap plus things from many other sites. I've been to that howtoforge site before. It mostly deals with setting up Postfix, which I'm not using. The dovecot bits make more sense in light of your feedback. I've tried that ldapsearch example: ldapsearch -x -h ad.example.com -D 'vmail' -W -b 'cn=users,dc=example,dc=com' with the domain user I specified in my dovecot-ldap.conf.ext with my host and dc info and I get the error ldap_bind: Strong(er) authentication required (8) additional info: BindSimple: Transport encryption required. I've seen confusing postings on this error having to do with port 636 and LDAPS -- no idea what they're talking about. My user is the Samba/domain administrator and has a pretty complex password. None of the sites I've visited on this error indicate it has anything to do with the actual password's complexity. Perhaps I'm just thick-skulled with all this. If you or anyone can see something obviously wrong with my conf, or have any suggestion at all on a baby step I can take to incm me forward, please let me know. Thanks, --Mark doveconf -n: # 2.2.3
Re: v2.3.0 release candidate released
On Mon, Dec 18, 2017 at 7:23 AM, Timo Sirainen wrote: > https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz > https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig > > It's finally time for v2.3 release branch! There are several new and > exciting features in it. I'm especially happy about the new logging and > statistics code, which will allow us to generate statistics for just about > everything. We didn't have time to implement everything we wanted for them > yet, and there especially aren't all that many logging events yet that can > be used for statistics. We'll implement those to v2.3.1, which might also > mean that some of the APIs might still change in v2.3.1 if that's required. > > We also have new lib-smtp server code, which was used to implement SMTP > submission server and do a partial rewrite for LMTP server. Please test > these before v2.3.0 to make sure we don't have any bad bugs left! > > BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1. > > Some of the larger changes: > > * Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3 > * Logging rewrite started: Logging is now based on hierarchical events. >This makes it possible to do various things, like: 1) giving >consistent log prefixes, 2) enabling debug logging with finer >granularity, 3) provide logs in more machine readable formats >(e.g. json). Everything isn't finished yet, especially a lot of the >old logging code still needs to be translated to the new way. > * Statistics rewrite started: Stats are now based on (log) events. >It's possible to gather statistics about any event that is logged. >See http://wiki2.dovecot.org/Statistics for details > * ssl_dh setting replaces the old generated ssl-parameters.dat > * IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error >instead of [UNKNOWNCTE] > * Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by >default due to potential security reasons (found by cPanel Security >Team). > > + Added support for SMTP submission proxy server, which includes >support for BURL and CHUNKING extension. > + LMTP rewrite. Supports now CHUNKING extension and mixing of >local/proxy recipients. > + auth: Support libsodium to add support for ARGON2I and ARGON2ID >password schemes. > + auth: Support BLF-CRYPT password scheme in all platforms > + auth: Added LUA scripting support for passdb/userdb. >See https://wiki2.dovecot.org/AuthDatabase/Lua > - Input streams are more reliable now when there are errors or when >the maximum buffer size is reached. Previously in some situations >this could have caused Dovecot to try to read already freed memory. > - Output streams weren't previously handling failures when writing a >trailer at the end of the stream. This mainly affected encrypt and >zlib compress ostreams, which could have silently written truncated >files if the last write happened to fail (which shouldn't normally >have ever happened). > - virtual plugin: Fixed panic when fetching mails from virtual >mailboxes with IMAP BINARY extension. > - Many other smaller fixes > > No issue compilng (and very very excited about this release, esp the Lua code, which is already super useful). I did have this one issue so far with the RC. I was previously using a git checkout of ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 with some of the Lua patches attached. That was working just fine (except for one thing I'll mention below). I rolled the RC and got this (and I was actually testing for the issue I had with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015): # doveadm -D acl set -u test1-sha...@test.com INBOX user=te...@test.com read list Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_lucene_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so Debug: Module loaded: /usr/lib/dovecot
Re: v2.3.0 release candidate released
On Mon, Dec 18, 2017 at 1:16 PM, Mark Moseley wrote: > On Mon, Dec 18, 2017 at 7:23 AM, Timo Sirainen wrote: > >> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz >> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig >> >> It's finally time for v2.3 release branch! There are several new and >> exciting features in it. I'm especially happy about the new logging and >> statistics code, which will allow us to generate statistics for just about >> everything. We didn't have time to implement everything we wanted for them >> yet, and there especially aren't all that many logging events yet that can >> be used for statistics. We'll implement those to v2.3.1, which might also >> mean that some of the APIs might still change in v2.3.1 if that's required. >> >> We also have new lib-smtp server code, which was used to implement SMTP >> submission server and do a partial rewrite for LMTP server. Please test >> these before v2.3.0 to make sure we don't have any bad bugs left! >> >> BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1. >> >> Some of the larger changes: >> >> * Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3 >> * Logging rewrite started: Logging is now based on hierarchical events. >>This makes it possible to do various things, like: 1) giving >>consistent log prefixes, 2) enabling debug logging with finer >>granularity, 3) provide logs in more machine readable formats >>(e.g. json). Everything isn't finished yet, especially a lot of the >>old logging code still needs to be translated to the new way. >> * Statistics rewrite started: Stats are now based on (log) events. >>It's possible to gather statistics about any event that is logged. >>See http://wiki2.dovecot.org/Statistics for details >> * ssl_dh setting replaces the old generated ssl-parameters.dat >> * IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error >>instead of [UNKNOWNCTE] >> * Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by >>default due to potential security reasons (found by cPanel Security >>Team). >> >> + Added support for SMTP submission proxy server, which includes >>support for BURL and CHUNKING extension. >> + LMTP rewrite. Supports now CHUNKING extension and mixing of >>local/proxy recipients. >> + auth: Support libsodium to add support for ARGON2I and ARGON2ID >>password schemes. >> + auth: Support BLF-CRYPT password scheme in all platforms >> + auth: Added LUA scripting support for passdb/userdb. >>See https://wiki2.dovecot.org/AuthDatabase/Lua >> - Input streams are more reliable now when there are errors or when >>the maximum buffer size is reached. Previously in some situations >>this could have caused Dovecot to try to read already freed memory. >> - Output streams weren't previously handling failures when writing a >>trailer at the end of the stream. This mainly affected encrypt and >>zlib compress ostreams, which could have silently written truncated >>files if the last write happened to fail (which shouldn't normally >>have ever happened). >> - virtual plugin: Fixed panic when fetching mails from virtual >>mailboxes with IMAP BINARY extension. >> - Many other smaller fixes >> >> > > No issue compilng (and very very excited about this release, esp the Lua > code, which is already super useful). > > I did have this one issue so far with the RC. I was previously using a git > checkout of ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 with some of the Lua > patches attached. That was working just fine (except for one thing I'll > mention below). I rolled the RC and got this (and I was actually testing > for the issue I had with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015): > > # doveadm -D acl set -u test1-sha...@test.com INBOX user=te...@test.com > read list > Debug: Loading modules from directory: /usr/lib/dovecot/modules > Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so > Debug: Module loaded: /usr/lib/dovecot/modules/ > lib02_lazy_expunge_plugin.so > Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so > Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so > Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so > Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so > Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_lucene_plugin.so > Debug: Loading modules from directory: /u
Re: v2.3.0 release candidate released
On Mon, Dec 18, 2017 at 2:32 PM, Timo Sirainen wrote: > On 18 Dec 2017, at 23.16, Mark Moseley wrote: > > > > doveadm(test1-sha...@test.com): Panic: file buffer.c: line 97 > > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) > .. > > /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so(+0x43fe) > > [0x6ba6997c33fe] -> > > Since the panic is coming from pigeonhole, did you recompile it also? And > what version of it? > > The previous version (that was running happily with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015) was 0.4.21. I had compiled 0.4.21 against ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015. RC1 + 0.5.0rc1 stopped backtracing on me and works ok (minus the 'read' thing I mentioned).
Re: Lua Auth
>
>
>
>
> 2) Is there an appropriate way to return data with spaces in it (or
> presumably other non-alphanum chars. My quota name had a space in it,
> which
> somehow got interpreted as 'yes' , i.e.:
>
> imap: Error: Failed to initialize quota: Invalid quota root quota: Unknown
> quota backend: yes
>
> I simply changed the space to an underscore as a workaround, but I'm
> curious if there's a better way. I tried various quoting without success.
> Didn't try escaping yet.
>
>
> 2) Instead of string, return a key value table. you can have spaces in
> values.
>
>
>
Does this work for auth_passdb_lookup too, or just auth_userdb_lookup? I've
been returning a table with auth_userdb_lookup just fine. But when I try
using it with passdb (and despite being very very sure that a 'password'
key exists in the table I'm returning from auth_passdb_lookup() -- I'm
logging it one line above the return), the passdb auth fails with this log
entry:
Dec 21 23:29:22 auth-worker(7779): Info:
lua(te...@test.com,10.20.103.32,):
No password returned (and no nopassword)
I guess it's not seeing the password key in the table I'm returning. If I
return a concat'd string ("password=... user=...") from
auth_passdb_lookup(), it works just fine.
I was also curious if there's a way to pass info between auth_userdb_lookup
and auth_passdb_lookup. I was trying to use a table with
auth_passdb_lookup() so I could take advantage of prefetch and thought that
if auth_passdb_lookup didn't take a table, I could stash data away and then
un-stash it in auth_userdb_lookup
Thanks!
> 3) response_from_template expands a key=value string into table by var
> expanding values.
>
>
> var_expand can be used to interpolation for any purposes. it returns a
> string. see https://wiki.dovecot.org/Variables for details on how to use
> it.
>
>
> Individual variable access is more efficient to do directly.
>
>
> ---
> Aki Tuomi
>
Re: Lua Auth
On Thu, Dec 21, 2017 at 9:51 PM, Aki Tuomi wrote:
>
> > On December 22, 2017 at 6:43 AM Mark Moseley
> wrote:
> >
> >
> > >
> > >
> > >
> > >
> > > 2) Is there an appropriate way to return data with spaces in it (or
> > > presumably other non-alphanum chars. My quota name had a space in it,
> > > which
> > > somehow got interpreted as 'yes' , i.e.:
> > >
> > > imap: Error: Failed to initialize quota: Invalid quota root quota:
> Unknown
> > > quota backend: yes
> > >
> > > I simply changed the space to an underscore as a workaround, but I'm
> > > curious if there's a better way. I tried various quoting without
> success.
> > > Didn't try escaping yet.
> > >
> > >
> > > 2) Instead of string, return a key value table. you can have spaces in
> > > values.
> > >
> > >
> > >
> > Does this work for auth_passdb_lookup too, or just auth_userdb_lookup?
> I've
> > been returning a table with auth_userdb_lookup just fine. But when I try
> > using it with passdb (and despite being very very sure that a 'password'
> > key exists in the table I'm returning from auth_passdb_lookup() -- I'm
> > logging it one line above the return), the passdb auth fails with this
> log
> > entry:
> >
> > Dec 21 23:29:22 auth-worker(7779): Info:
> > lua(te...@test.com,10.20.103.32,):
> > No password returned (and no nopassword)
> >
> > I guess it's not seeing the password key in the table I'm returning. If I
> > return a concat'd string ("password=... user=...") from
> > auth_passdb_lookup(), it works just fine.
> >
> > I was also curious if there's a way to pass info between
> auth_userdb_lookup
> > and auth_passdb_lookup. I was trying to use a table with
> > auth_passdb_lookup() so I could take advantage of prefetch and thought
> that
> > if auth_passdb_lookup didn't take a table, I could stash data away and
> then
> > un-stash it in auth_userdb_lookup
> >
> > Thanks!
> >
> >
>
> Yeah, this is a bug we have fixed =)
>
> https://github.com/dovecot/core/commit/c86575ac9776d0995355d03719c82e
> 7ceac802e6#diff-83374eeaee91d90e848390ba3c7b264a
>
>
I'm on rc1, so I appear to already have that git commit (as part of rc1).
# /usr/sbin/dovecot --version
2.3.0.rc1 (12aba5948)
For testing this, I tried replacing my passdb lookup with this:
function auth_passdb_lookup(req)
passdb_table = {}
passdb_table[ 'password' ] = 'test'
passdb_table[ 'user' ] = 'te...@test.com'
return dovecot.auth.PASSDB_RESULT_OK, passdb_table
end
and still get:
Dec 22 01:17:17 auth-worker(9711): Info:
lua(te...@test.com,10.20.103.32,):
No password returned (and no nopassword)
Replacing that return statement with this:
return dovecot.auth.PASSDB_RESULT_OK, 'password=test user=te...@test.com'
authenticates successfully.
Re: Lua Auth
On Fri, Dec 22, 2017 at 5:18 AM, wrote:
>
> > On December 22, 2017 at 8:20 AM Mark Moseley
> wrote:
> >
> >
> > On Thu, Dec 21, 2017 at 9:51 PM, Aki Tuomi wrote:
> >
> > >
> > > > On December 22, 2017 at 6:43 AM Mark Moseley
> > > wrote:
> > > >
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > 2) Is there an appropriate way to return data with spaces in it (or
> > > > > presumably other non-alphanum chars. My quota name had a space in
> it,
> > > > > which
> > > > > somehow got interpreted as 'yes' , i.e.:
> > > > >
> > > > > imap: Error: Failed to initialize quota: Invalid quota root quota:
> > > Unknown
> > > > > quota backend: yes
> > > > >
> > > > > I simply changed the space to an underscore as a workaround, but
> I'm
> > > > > curious if there's a better way. I tried various quoting without
> > > success.
> > > > > Didn't try escaping yet.
> > > > >
> > > > >
> > > > > 2) Instead of string, return a key value table. you can have
> spaces in
> > > > > values.
> > > > >
> > > > >
> > > > >
> > > > Does this work for auth_passdb_lookup too, or just
> auth_userdb_lookup?
> > > I've
> > > > been returning a table with auth_userdb_lookup just fine. But when I
> try
> > > > using it with passdb (and despite being very very sure that a
> 'password'
> > > > key exists in the table I'm returning from auth_passdb_lookup() --
> I'm
> > > > logging it one line above the return), the passdb auth fails with
> this
> > > log
> > > > entry:
> > > >
> > > > Dec 21 23:29:22 auth-worker(7779): Info:
> > > > lua(te...@test.com,10.20.103.32,):
> > > > No password returned (and no nopassword)
> > > >
> > > > I guess it's not seeing the password key in the table I'm returning.
> If I
> > > > return a concat'd string ("password=... user=...") from
> > > > auth_passdb_lookup(), it works just fine.
> > > >
> > > > I was also curious if there's a way to pass info between
> > > auth_userdb_lookup
> > > > and auth_passdb_lookup. I was trying to use a table with
> > > > auth_passdb_lookup() so I could take advantage of prefetch and
> thought
> > > that
> > > > if auth_passdb_lookup didn't take a table, I could stash data away
> and
> > > then
> > > > un-stash it in auth_userdb_lookup
> > > >
> > > > Thanks!
> > > >
> > > >
> > >
> > > Yeah, this is a bug we have fixed =)
> > >
> > > https://github.com/dovecot/core/commit/c86575ac9776d0995355d03719c82e
> > > 7ceac802e6#diff-83374eeaee91d90e848390ba3c7b264a
> > >
> > >
> >
> > I'm on rc1, so I appear to already have that git commit (as part of rc1).
> >
> > # /usr/sbin/dovecot --version
> > 2.3.0.rc1 (12aba5948)
> >
> > For testing this, I tried replacing my passdb lookup with this:
> >
> > function auth_passdb_lookup(req)
> > passdb_table = {}
> > passdb_table[ 'password' ] = 'test'
> > passdb_table[ 'user' ] = 'te...@test.com'
> >
> > return dovecot.auth.PASSDB_RESULT_OK, passdb_table
> > end
> >
> > and still get:
> >
> > Dec 22 01:17:17 auth-worker(9711): Info:
> > lua(te...@test.com,10.20.103.32,):
> > No password returned (and no nopassword)
> >
> > Replacing that return statement with this:
> >
> > return dovecot.auth.PASSDB_RESULT_OK, 'password=test user=te...@test.com
> '
> >
> > authenticates successfully.
>
> Fixed in https://github.com/dovecot/core/commit/
> e5fb6b3b7d4e79475b451823ea6c0a02955ba06b
>
>
>
Works like a charm now, thanks!
As a matter of 'best practices', in my current iteration of Lua auth, I
moved all my lookups to passdb (thus yesterday's emails to the list), so
that it could be used with prefetch. Belatedly realizing that LMTP doesn't
touch passdb, I rewrote the userdb lookup to call the same passdb lookup
(which only happens for non-passdb/prefetch things) and then it copies the
return table (but strips the 'userdb_' prefix). It's all working currently.
BUT, does that sound sane? Or is there some gotcha I'm heading towards
(yes, I realize the question is a bit vague -- just looking for very
general "No, don't do that").
I'm curious too if I can set vars in the passdb lookup and then access then
in userdb. Or is it random which auth-worker will handle the userdb lookup,
relative to which one handled the passdb lookup? I tried dropping things in
the req.userdb table in the passdb phase, but it was unset during the
userdb phase.
Re: Locks directory change
On Thu, Oct 26, 2017 at 7:30 AM, Aki Tuomi wrote: > > > On October 26, 2017 at 4:30 PM Federico Bartolucci > wrote: > > > > > > Hello, > > > > it's the first time for me writing to the list, I'm trying to change the > > location into which the Dovecot's locks are done reserving a special > > temporary directory on an other partition, then adding to the > > dovecont.conf the line: > > > > mail_location = maildir:~/Maildir:VOLATILEDIR=/tmp_lock/%2.256Nu/%u > > > > so that through the VOLATILEDIR directive the locks should be written in > > this path. > > We observe though that the locks for many users are still done in the > > real maildir (NFS mounted filesystem) as if in some situations this > > instruction is not effective. Anybody knows if are there other things to > > change or to do or what could be the reason? (for instance to login in a > > specific way or doing a particular operation). > > > > Regards, > > Federico > > Hi, VOLATILEDIR currently only affects vsize.lock and autoexpunge.lock. > > Aki > Are there plans to expand that in 2.3? Without knowing the ramifications, it'd be nice to have lastlogin use it, at least with director enabled.
Re: Dovecot 2.3.0 TLS
On Tue, Jan 23, 2018 at 10:05 AM, Aki Tuomi wrote:
>
> > On January 23, 2018 at 7:09 PM Arkadiusz Miśkiewicz
> wrote:
> >
> >
> > On Thursday 11 of January 2018, Aki Tuomi wrote:
> >
> > > Seems we might've made a unexpected change here when we revamped the
> ssl
> > > code.
> >
> > Revamped, interesting, can it support milions certs now on single
> machine? (so
> > are certs loaded by demand and not wasting memory)
> >
> > > Aki
> >
> >
> > --
> > Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )
>
> Unfortunately not. This time round it was about putting the ssl code
> mostly in one place, so that we use same code for all SSL connections.
>
>
Just to chime in, having some way of supporting SSL certs dynamically would
be tremendously useful. Like splitting out the retrieval of certs/key to a
socket, that would typically just be a built-in regular dovecot service
("go and get the certs that are configured in dovecot configs"), but could
also be a custom unix listener that could return certs/keys. Dovecot would
send in the local IP/port and/or SNI name (if there was one) to the socket
and then use whatever comes back. A perl/python/etc script doing the unix
listener could then grab the appropriate cert/key from wherever (and
dovecot would presumably have a time-based cache for certs/keys). This is
just wish-listing :)
Currently, I've got a million different domains on my dovecot boxes, so
allowing them all to use per-domain SSL is a bit challenging. I've been
searching for an SSL proxy that supports something like nginx/openresty's
"ssl_certificate_by_lua_file" (and can communicate the remote IP to dovecot
like haproxy does) to put in front of dovecot, to no avail. Having
something like that built directly into dovecot would be a dream -- or that
can at least farm that functionality out to a custom daemon).
AuthDatabase CheckPassword broken?
I had been using the CheckPassword authentication interface with dovecot 2.2.15, https://wiki2.dovecot.org/AuthDatabase/CheckPassword, and it was working. After upgrading to 2.2.33.2 CheckPassword no longer works. The referenced wiki page says, Checkpassword Interface Read NUL NUL from fd 3. I've checked the information read from fd 3 with 2.2.33.2 and I get followed by 3 nulls. I'm guessing the 2nd null is supposed to be the password. Why is this no longer working? How can I fix it? THX --Mark
Re: AuthDatabase CheckPassword broken?
On Thu, 1 Feb 2018 10:02:10 +0200 Aki Tuomi wrote:
>
> On 01.02.2018 08:00, Mark Foley wrote:
> > I had been using the CheckPassword authentication interface with dovecot
> > 2.2.15,
> > https://wiki2.dovecot.org/AuthDatabase/CheckPassword, and it was working.
> >
> > After upgrading to 2.2.33.2 CheckPassword no longer works. The referenced
> > wiki page says,
> >
> > Checkpassword Interface
> >
> > Read NUL NUL from fd 3.
> >
> > I've checked the information read from fd 3 with 2.2.33.2 and I get
> > followed by 3
> > nulls. I'm guessing the 2nd null is supposed to be the password.
> >
> > Why is this no longer working? How can I fix it?
> >
> > THX --Mark
> Our CI has test
>
> #!/usr/bin/env python
> # -*- coding: utf-8 -*-
> import os, sys
>
> DOVECOT_PW_FD = 3
>
> def checkPassword():
> with os.fdopen(DOVECOT_PW_FD, 'r') as s:
> data = s.read().split("\0")
> if data[0] != "testuser" or data[1] != "pass":
> return False
> os.environ["USER"] = data[0]
> os.environ["EXTRA"] = "userdb_uid=vmail userdb_gid=vmail"
> return True
>
> if __name__ == "__main__":
> if not checkPassword():
> sys.exit(1)
> os.execv(sys.argv[1], sys.argv[1:])
>
> And it seems to work.
>
> Aki
Thanks for the script. I'm testing this on a production system, so I'll have to
wait until
after business hours to test. Meanwhile, not being a python wizard, I have a
couple of
questions.
I have to run this script as my passdb { args } parameter, right?
On the line where it is checking for "testuser" and password "test", I assume
that if I want to
use a configured user I can just change these, right?
Likewise with "userdb_uid=vmail userdb_gid=vmail", what are these? UID/GID of
the user?
Is there a way in python to output the values in data[0] and data[1] to a file
so I can see
what's actually received? If after the 'split' line I added:
f = open("/tmp/checkpassword.log","a")
f.write("Name: " + data[0] + ", PW: " + data[1])
f.close()
Would that work?
--THX Mark
Re: AuthDatabase CheckPassword broken?
Script didn't run:
File "/root/tmp/checkpwtest.py", line 8
o?= with os.fdopen(DOVECOT_PW_FD, 'r') as s:
^
SyntaxError: invalid syntax
--Mark
-Original Message-
From: Mark Foley
Date: Thu, 01 Feb 2018 15:34:15 -0500
Organization: Ohio Highway Patrol Retirement System
To: dovecot@dovecot.org
Subject: Re: AuthDatabase CheckPassword broken?
On Thu, 1 Feb 2018 10:02:10 +0200 Aki Tuomi wrote:
>
> On 01.02.2018 08:00, Mark Foley wrote:
> > I had been using the CheckPassword authentication interface with dovecot
> > 2.2.15,
> > https://wiki2.dovecot.org/AuthDatabase/CheckPassword, and it was working.
> >
> > After upgrading to 2.2.33.2 CheckPassword no longer works. The referenced
> > wiki page says,
> >
> > Checkpassword Interface
> >
> > Read NUL NUL from fd 3.
> >
> > I've checked the information read from fd 3 with 2.2.33.2 and I get
> > followed by 3
> > nulls. I'm guessing the 2nd null is supposed to be the password.
> >
> > Why is this no longer working? How can I fix it?
> >
> > THX --Mark
> Our CI has test
>
> #!/usr/bin/env python
> # -*- coding: utf-8 -*-
> import os, sys
>
> DOVECOT_PW_FD = 3
>
> def checkPassword():
> with os.fdopen(DOVECOT_PW_FD, 'r') as s:
> data = s.read().split("\0")
> if data[0] != "testuser" or data[1] != "pass":
> return False
> os.environ["USER"] = data[0]
> os.environ["EXTRA"] = "userdb_uid=vmail userdb_gid=vmail"
> return True
>
> if __name__ == "__main__":
> if not checkPassword():
> sys.exit(1)
> os.execv(sys.argv[1], sys.argv[1:])
>
> And it seems to work.
>
> Aki
Thanks for the script. I'm testing this on a production system, so I'll have to
wait until
after business hours to test. Meanwhile, not being a python wizard, I have a
couple of
questions.
I have to run this script as my passdb { args } parameter, right?
On the line where it is checking for "testuser" and password "test", I assume
that if I want to
use a configured user I can just change these, right?
Likewise with "userdb_uid=vmail userdb_gid=vmail", what are these? UID/GID of
the user?
Is there a way in python to output the values in data[0] and data[1] to a file
so I can see
what's actually received? If after the 'split' line I added:
f = open("/tmp/checkpassword.log","a")
f.write("Name: " + data[0] + ", PW: " + data[1])
f.close()
Would that work?
--THX Mark
Re: limit pop login per user and per minute
On Thu, Mar 22, 2018 at 1:41 PM, Joseph Tam wrote: > On Thu, 22 Mar 2018, Markus Eckerl wrote: > > The problem is, that he misconfigured the servers of these customers. In >> detail: their servers are trying to fetch email every 2 - 5 seconds. For >> every email address. >> >> In the past I contacted the technician and told him about his mistake. >> He was not very helpful and simply told me that he is doing the same >> configuration since several years at all of his customer servers. >> Without problems. It is up to me to fix my problem myself. >> > > Seems to me you're bending over backwards to fix someone else's problem, > and what you really need is an "attitude adjustment" tool for obnoxious > clients who use your service like they're the only ones that matter. > > Apart from what others can suggest (I think dovecot allows delegation > of usage to a separate policyd service), you can perhaps use firewall > throttling e.g. > > https://making.pusher.com/per-ip-rate-limiting-with-iptables/ > > It can't do it per user, but perhaps it is better to set a global limit > and let your downstream client better manage and conserve a limited > resource. > > Might be a good use of the new authpolicy stuff. You could run a local weakforced with 1 minute windows and break auth for certain IPs that do more than one login per minute.
folders not visible on copied mail folders
We had a user quit recently. Three days ago I copied his entire Maildir folder to another user to that user's Maildir/.JoesEmail. I changed ownership and made the permission 'chmod -R og-rwx .', just like all the other files/directories of the new owner. This didn't work to show the new folder. Today, in his Thunderbird client, I subscribed to the 'JoesEmail' folder. I restarted dovecot and restarted Thunderbird. In Thunderbird, the 'JoesEmail' folder now shows, but it is empty and shows none of the subordinate mail folders. I ran 'doveadm index -u newowner JoesEmail' and 'doveadm force-resync -u newowner JoesEmail'. This didn't help. I did this once before with a previous user who quit and only changed ownership, no subscribing, no doveadm, and that worked. What am I doing wrong? THX --Mark
Re: folders not visible on copied mail folders
On Tue, 17 Jul 2018 08:06:24 +0200 Steffen Kaiser
wrote:
>
> On Mon, 16 Jul 2018, Mark Foley wrote:
>
> > We had a user quit recently. Three days ago I copied his entire Maildir
> > folder to another user
> > to that user's Maildir/.JoesEmail. I changed ownership and made the
> > permission 'chmod -R
> > og-rwx .', just like all the other files/directories of the new owner.
> > This didn't work to show
> > the new folder. Today, in his Thunderbird client, I subscribed to the
> > 'JoesEmail' folder. I
> > restarted dovecot and restarted Thunderbird.
> >
> > In Thunderbird, the 'JoesEmail' folder now shows, but it is empty and shows
> > none of the
> > subordinate mail folders. I ran 'doveadm index -u newowner JoesEmail' and
> > 'doveadm force-resync -u newowner JoesEmail'. This didn't help.
> >
> > I did this once before with a previous user who quit and only changed
> > ownership, no
> > subscribing, no doveadm, and that worked.
> >
> > What am I doing wrong?
>
> Your description might be interpreted one way or another, esp. "copied his
> entire Maildir folder ... to that user's Maildir/.JoesEmail".
>
> Also, it depends on how you have configured mail_location.
>
> If this means that you have:
> Maildir/.JoesEmail/{new,cur,tmp}
> Maildir/.JoesEmail/.mailbox/{new,cur,tmp}
> Maildir/.JoesEmail/.mailbox.submailbox/{new,cur,tmp}
> now, that will clash with the standard Maildir format:
> https://wiki2.dovecot.org/MailboxFormat/Maildir
>
> You would need to move the subfolders with a leading dot of .JoesEmail
> into:
> Maildir/.JoesEmail/{new,cur,tmp}
> Maildir/.JoesEmail.mailbox/{new,cur,tmp}
> Maildir/.JoesEmail.mailbox.submailbox/{new,cur,tmp}
>
> If you use :LAYOUT=fs to mail_location, .JoesEmail should spell JoesEmail
>
> Subscription is needed only, if the mail client "displays subscribed
> folders only" or does not "display all folders". The meaning of the
> setting varies from client to client.
>
> Another way would to keep the other account and share it via ACLs:
> https://wiki2.dovecot.org/SharedMailboxes/Shared
>
> Steffen Kaiser
Steffen, thanks for your reply. I did have the copied folders as shown in your
first example. I
changed that to what you show as the remedy. The target user's Maildir folder
now has:
drwx-- 5 mpress domusers 4096 2017-06-28 20:07 .Deleted\ Messages.Junk/
drwx-- 5 mpress domusers 4096 2018-07-16 23:22 .Delta\ Dental/
drwx-- 21 mpress domusers 4096 2018-07-17 16:48 .Dennis\ Email/
drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\ Email.Deleted\
Items/
drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\ Email.Deleted\
Items.Sent/
drwx-- 5 mpress domusers 4096 2018-07-17 17:02 .Drafts/
drwx-- 5 mpress domusers 4096 2018-07-17 16:35 .ESI/
Where '.Dennis Email' is the folder for the old user. I copied the old user's
'Maildir/.Deleted Items'
and 'Maildir/.Deleted Items/Sent' to the target user's 'Maildir/.Dennis
Email.Deleted Items'
and 'Maildir/.Deleted Items/Sent, respective. That how I understood what you
advised. There are
more such subfolders, but I thought I'd try this one first.
However, still only the "Dennis Email" folder shows in the mail client, empty,
no sub-folders
even though "Deleted Items.Sent/cur" has plenty of mail files (1522).
I did try running 'doveadm index -u mpress "Dennis Email"', again; and
restarting dovecot and
thunderbird again, but still nothing.
What else can I try?
THX --Mark
Re: folders not visible on copied mail folders
On Wed, 18 Jul 2018 07:23:06 +0200 Steffen Kaiser
wrote:
>
> On Tue, 17 Jul 2018, Mark Foley wrote:
> > On Tue, 17 Jul 2018 08:06:24 +0200 Steffen Kaiser
> > wrote:
> >>
> >> On Mon, 16 Jul 2018, Mark Foley wrote:
> >>
> >>> We had a user quit recently. Three days ago I copied his entire Maildir
> >>> folder to another user
> >>> to that user's Maildir/.JoesEmail. I changed ownership and made the
> >>> permission 'chmod -R
> >>> og-rwx .', just like all the other files/directories of the new owner.
> >>> This didn't work to show
> >>> the new folder. Today, in his Thunderbird client, I subscribed to the
> >>> 'JoesEmail' folder. I
> >>> restarted dovecot and restarted Thunderbird.
> >>>
> >>> In Thunderbird, the 'JoesEmail' folder now shows, but it is empty and
> >>> shows none of the
> >>> subordinate mail folders. I ran 'doveadm index -u newowner JoesEmail' and
> >>> 'doveadm force-resync -u newowner JoesEmail'. This didn't help.
> >>>
> >>> I did this once before with a previous user who quit and only changed
> >>> ownership, no
> >>> subscribing, no doveadm, and that worked.
> >>>
> >>> What am I doing wrong?
> >>
> >> Your description might be interpreted one way or another, esp. "copied his
> >> entire Maildir folder ... to that user's Maildir/.JoesEmail".
> >>
> >> Also, it depends on how you have configured mail_location.
> >>
> >> If this means that you have:
> >> Maildir/.JoesEmail/{new,cur,tmp}
> >> Maildir/.JoesEmail/.mailbox/{new,cur,tmp}
> >> Maildir/.JoesEmail/.mailbox.submailbox/{new,cur,tmp}
> >> now, that will clash with the standard Maildir format:
> >> https://wiki2.dovecot.org/MailboxFormat/Maildir
> >>
> >> You would need to move the subfolders with a leading dot of .JoesEmail
> >> into:
> >> Maildir/.JoesEmail/{new,cur,tmp}
> >> Maildir/.JoesEmail.mailbox/{new,cur,tmp}
> >> Maildir/.JoesEmail.mailbox.submailbox/{new,cur,tmp}
> >>
> >> If you use :LAYOUT=fs to mail_location, .JoesEmail should spell JoesEmail
> >>
> >> Subscription is needed only, if the mail client "displays subscribed
> >> folders only" or does not "display all folders". The meaning of the
> >> setting varies from client to client.
> >>
> >> Another way would to keep the other account and share it via ACLs:
> >> https://wiki2.dovecot.org/SharedMailboxes/Shared
> >>
> >> Steffen Kaiser
> >
> > Steffen, thanks for your reply. I did have the copied folders as shown in
> > your first example. I
> > changed that to what you show as the remedy. The target user's Maildir
> > folder now has:
> >
> > drwx-- 5 mpress domusers 4096 2017-06-28 20:07 .Deleted\
> > Messages.Junk/
> > drwx-- 5 mpress domusers 4096 2018-07-16 23:22 .Delta\ Dental/
> > drwx-- 21 mpress domusers 4096 2018-07-17 16:48 .Dennis\ Email/
> > drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\
> > Email.Deleted\ Items/
> > drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\
> > Email.Deleted\ Items.Sent/
> > drwx-- 5 mpress domusers 4096 2018-07-17 17:02 .Drafts/
> > drwx-- 5 mpress domusers 4096 2018-07-17 16:35 .ESI/
> >
> > Where '.Dennis Email' is the folder for the old user. I copied the old
> > user's 'Maildir/.Deleted Items'
> > and 'Maildir/.Deleted Items/Sent' to the target user's 'Maildir/.Dennis
> > Email.Deleted Items'
> > and 'Maildir/.Deleted Items/Sent, respective. That how I understood what
> > you advised. There are
> > more such subfolders, but I thought I'd try this one first.
>
> > However, still only the "Dennis Email" folder shows in the mail client,
> > empty, no sub-folders
> > even though "Deleted Items.Sent/cur" has plenty of mail files (1522).
> >
> > I did try running 'doveadm index -u mpress "Dennis Email"', again; and
> > restarting dovecot and
> > thunderbird again, but still nothing.
>
> First check if Dovecot thinks the folders are there:
>
> doveadm mailbox list -u "mpress" | grep Dennis
yes:
# doveadm mailbox list -u "mpress" | grep Dennis
Dennis Email
Dennis Email.Deleted Items
D
Re: folders not visible on copied mail folders
On Thu, 19 Jul 2018 08:11:40 +0200 Steffen Kaiser
wrote:
>
> On Thu, 19 Jul 2018, Mark Foley wrote:
> > On Wed, 18 Jul 2018 07:23:06 +0200 Steffen Kaiser
> > wrote:
> >>
> >> On Tue, 17 Jul 2018, Mark Foley wrote:
> >>> On Tue, 17 Jul 2018 08:06:24 +0200 Steffen Kaiser
> >>> wrote:
> >>>>
> >>>> On Mon, 16 Jul 2018, Mark Foley wrote:
> >>>>
> >>>>> We had a user quit recently. Three days ago I copied his entire
> >>>>> Maildir folder to another user
> >>>>> to that user's Maildir/.JoesEmail. I changed ownership and made the
> >>>>> permission 'chmod -R
> >>>>> og-rwx .', just like all the other files/directories of the new owner.
> >>>>> This didn't work to show
> >>>>> the new folder. Today, in his Thunderbird client, I subscribed to the
> >>>>> 'JoesEmail' folder. I
> >>>>> restarted dovecot and restarted Thunderbird.
> >>>>>
> >>>>> In Thunderbird, the 'JoesEmail' folder now shows, but it is empty and
> >>>>> shows none of the
> >>>>> subordinate mail folders. I ran 'doveadm index -u newowner JoesEmail'
> >>>>> and
> >>>>> 'doveadm force-resync -u newowner JoesEmail'. This didn't help.
> >>>>>
> >>>>> I did this once before with a previous user who quit and only changed
> >>>>> ownership, no
> >>>>> subscribing, no doveadm, and that worked.
> >>>>>
> >>>>> What am I doing wrong?
> >>>>
> >>>> Your description might be interpreted one way or another, esp. "copied
> >>>> his
> >>>> entire Maildir folder ... to that user's Maildir/.JoesEmail".
> >>>>
> >>>> Also, it depends on how you have configured mail_location.
> >>>>
> >>>> If this means that you have:
> >>>> Maildir/.JoesEmail/{new,cur,tmp}
> >>>> Maildir/.JoesEmail/.mailbox/{new,cur,tmp}
> >>>> Maildir/.JoesEmail/.mailbox.submailbox/{new,cur,tmp}
> >>>> now, that will clash with the standard Maildir format:
> >>>> https://wiki2.dovecot.org/MailboxFormat/Maildir
> >>>>
> >>>> You would need to move the subfolders with a leading dot of .JoesEmail
> >>>> into:
> >>>> Maildir/.JoesEmail/{new,cur,tmp}
> >>>> Maildir/.JoesEmail.mailbox/{new,cur,tmp}
> >>>> Maildir/.JoesEmail.mailbox.submailbox/{new,cur,tmp}
> >>>>
> >>>> If you use :LAYOUT=fs to mail_location, .JoesEmail should spell JoesEmail
> >>>>
> >>>> Subscription is needed only, if the mail client "displays subscribed
> >>>> folders only" or does not "display all folders". The meaning of the
> >>>> setting varies from client to client.
> >>>>
> >>>> Another way would to keep the other account and share it via ACLs:
> >>>> https://wiki2.dovecot.org/SharedMailboxes/Shared
> >>>>
> >>>> Steffen Kaiser
> >>>
> >>> Steffen, thanks for your reply. I did have the copied folders as shown in
> >>> your first example. I
> >>> changed that to what you show as the remedy. The target user's Maildir
> >>> folder now has:
> >>>
> >>> drwx-- 5 mpress domusers 4096 2017-06-28 20:07 .Deleted\
> >>> Messages.Junk/
> >>> drwx-- 5 mpress domusers 4096 2018-07-16 23:22 .Delta\ Dental/
> >>> drwx-- 21 mpress domusers 4096 2018-07-17 16:48 .Dennis\ Email/
> >>> drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\
> >>> Email.Deleted\ Items/
> >>> drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\
> >>> Email.Deleted\ Items.Sent/
> >>> drwx-- 5 mpress domusers 4096 2018-07-17 17:02 .Drafts/
> >>> drwx-- 5 mpress domusers 4096 2018-07-17 16:35 .ESI/
> >>>
> >>> Where '.Dennis Email' is the folder for the old user. I copied the old
> >>> user's 'Maildir/.Deleted Items'
> >>> and 'Maildir/.Deleted Items/Sent' to the target user's 'Maildir/.Dennis
> >>> Email.Deleted Items'
> >>> and 'Maildir/.Deleted Items/Sent,
Re: folders not visible on copied mail folders
Shortly after this post, I found a solution here:
http://forums.mozillazine.org/viewtopic.php?t=1097725
In order to see the .Dennis\ Email.Dennis\ Inbox sub-folder you have to
collapse and re-expand
the folder list in Thunderbird. It's that simple ... AND that annoyingly
obscure!
Thanks for your help! --Mark
-Original Message-
From: Mark Foley
Date: Thu, 19 Jul 2018 21:21:34 -0400
Organization: Ohio Highway Patrol Retirement System
To: dovecot@dovecot.org
Subject: Re: folders not visible on copied mail folders
On Thu, 19 Jul 2018 08:11:40 +0200 Steffen Kaiser
wrote:
>
> On Thu, 19 Jul 2018, Mark Foley wrote:
> > On Wed, 18 Jul 2018 07:23:06 +0200 Steffen Kaiser
> > wrote:
> >>
> >> On Tue, 17 Jul 2018, Mark Foley wrote:
> >>> On Tue, 17 Jul 2018 08:06:24 +0200 Steffen Kaiser
> >>> wrote:
> >>>>
> >>>> On Mon, 16 Jul 2018, Mark Foley wrote:
> >>>>
> >>>>> We had a user quit recently. Three days ago I copied his entire
> >>>>> Maildir folder to another user
> >>>>> to that user's Maildir/.JoesEmail. I changed ownership and made the
> >>>>> permission 'chmod -R
> >>>>> og-rwx .', just like all the other files/directories of the new owner.
> >>>>> This didn't work to show
> >>>>> the new folder. Today, in his Thunderbird client, I subscribed to the
> >>>>> 'JoesEmail' folder. I
> >>>>> restarted dovecot and restarted Thunderbird.
> >>>>>
> >>>>> In Thunderbird, the 'JoesEmail' folder now shows, but it is empty and
> >>>>> shows none of the
> >>>>> subordinate mail folders. I ran 'doveadm index -u newowner JoesEmail'
> >>>>> and
> >>>>> 'doveadm force-resync -u newowner JoesEmail'. This didn't help.
> >>>>>
> >>>>> I did this once before with a previous user who quit and only changed
> >>>>> ownership, no
> >>>>> subscribing, no doveadm, and that worked.
> >>>>>
> >>>>> What am I doing wrong?
> >>>>
> >>>> Your description might be interpreted one way or another, esp. "copied
> >>>> his
> >>>> entire Maildir folder ... to that user's Maildir/.JoesEmail".
> >>>>
> >>>> Also, it depends on how you have configured mail_location.
> >>>>
> >>>> If this means that you have:
> >>>> Maildir/.JoesEmail/{new,cur,tmp}
> >>>> Maildir/.JoesEmail/.mailbox/{new,cur,tmp}
> >>>> Maildir/.JoesEmail/.mailbox.submailbox/{new,cur,tmp}
> >>>> now, that will clash with the standard Maildir format:
> >>>> https://wiki2.dovecot.org/MailboxFormat/Maildir
> >>>>
> >>>> You would need to move the subfolders with a leading dot of .JoesEmail
> >>>> into:
> >>>> Maildir/.JoesEmail/{new,cur,tmp}
> >>>> Maildir/.JoesEmail.mailbox/{new,cur,tmp}
> >>>> Maildir/.JoesEmail.mailbox.submailbox/{new,cur,tmp}
> >>>>
> >>>> If you use :LAYOUT=fs to mail_location, .JoesEmail should spell JoesEmail
> >>>>
> >>>> Subscription is needed only, if the mail client "displays subscribed
> >>>> folders only" or does not "display all folders". The meaning of the
> >>>> setting varies from client to client.
> >>>>
> >>>> Another way would to keep the other account and share it via ACLs:
> >>>> https://wiki2.dovecot.org/SharedMailboxes/Shared
> >>>>
> >>>> Steffen Kaiser
> >>>
> >>> Steffen, thanks for your reply. I did have the copied folders as shown in
> >>> your first example. I
> >>> changed that to what you show as the remedy. The target user's Maildir
> >>> folder now has:
> >>>
> >>> drwx-- 5 mpress domusers 4096 2017-06-28 20:07 .Deleted\
> >>> Messages.Junk/
> >>> drwx-- 5 mpress domusers 4096 2018-07-16 23:22 .Delta\ Dental/
> >>> drwx-- 21 mpress domusers 4096 2018-07-17 16:48 .Dennis\ Email/
> >>> drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\
> >>> Email.Deleted\ Items/
> >>> drwx-- 5 mpress domusers 4096 2018-07-16 23:15 .Dennis\
> >>> Email.Deleted\ Items.Sent/
> >>&
Need to convert mbox to Maildir
I have a mbox file of emails. I want to convert this to Maildir giving me individual message files per email. I've looked at dsync, but as far as I can tell this wants a specific target user and it appears that it will "distribute" the converted messages into that user's INBOX. I don't want to put these mbox messages into any particular user's Maildir hierarchy, just export to file-per-message format to a destination directory of my choosing. Is this possible? THX --Mark
Re: Need to convert mbox to Maildir
On Tue, 2 Oct 2018 21:17:20 +0300 Sami Ketola wrote: > > > On 2 Oct 2018, at 21.05, Mark Foley wrote: > > > > I have a mbox file of emails. I want to convert this to Maildir giving me > > individual message > > files per email. I've looked at dsync, but as far as I can tell this wants > > a specific target > > user and it appears that it will "distribute" the converted messages into > > that user's INBOX. > > > > I don't want to put these mbox messages into any particular user's Maildir > > hierarchy, just > > export to file-per-message format to a destination directory of my > > choosing. > > > > Is this possible? > > > Yes. > > [root@ketola /]# mkdir /test > [root@ketola /]# chown vmail /test > [root@ketola /]# doveadm backup -u sami Maildir:/test/ > > done. > > Sami Excellent! Thank you. I'll give that a try. I also found: mb2md.pl downloadable from https://wiki.dovecot.org/Migration/MailFormat. --Mark
email not visible in users mail client
I have an odd issue. One user has an email in her Maildir/cur folder named: 1545229920.27374_0.mail:2,. She cannot see this message in her mail client (Thunderbird). All other emails have 'S' and 'W' components to the name, e.g. 1488471573.M167365P19808.mail,S=41356,W=42118:2,RS, but this one does not. Would that somehow make a difference in it being visible to the mail client? Why would this message have been saved without the 'S' and 'W' bits? In fact, there are two such message with this abbreviated file name, both from the same sender. Is there possibly something about the message that affects naming? Dovecot version 2.2.33.2 THX --Mark
IMAP preauth and stats-writer
I use IMAP preauth; I connect with Alpine over SSH which is very useful.
The last few upgrades this has become more difficult to to. Last time
(moving 2.2 -> 2.3, I think) I had to put in a workaround:
stats_writer_socket_path =
It prevented /usr/local/libexec/dovecot/imap attempting to
connect to a central stats service.
As of an upgrade today (2.3.2.1_1 -> 2.3.4_3 on FreeBSD) it looks like
that 'fix' stopped working, and I get:
imap(mark,)Error: net_connect_unix() failed: Permission denied
It goes to stderr, which breaks Alpine.
"()" is actually the filename. It seems that the empty string is no longer
an indication to disable it. Here it is with the default configuration:
imap(mark,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
Permission denied
I do also have a dovecot running as a system daemon, and, interestingly,
disable this and it's 'fixed'; no attempt to connect. However, disabling
the service is not an option (needed for smartphone)
The best I've come up with so far is when using preauth to hack it to send
stderr to /dev/null. And yes, probably Alpine is at fault for interpreting
stderr content (separate issue)
* Is there a way to cleanly disable reporting to the stats service?
Previously, running as preauth was all very clean.
* Just wanted to highlight that IMAP preauth is really useful. Even
though it might not be mainstream, it seems healthy to be able to easily
install dovecot as an unprivilidged user in a "unixy" way.
I'm on FreeBSD 11.2, with dovecot from ports. dovecot.conf below.
--
Mark
#
# Dovecot configuration
#
mail_location = maildir:~/Maildir
postmaster_address = postmaster
namespace {
inbox = yes
}
ssl = required
ssl_cert =
Re: I need some help with my Dovecot and Postfix configs - I'm unable to log in on my mail server
On 19/7/20 8:43 pm, Bernardo Reino wrote: You can also read: https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration and set-up the necessary XML file at your server so that Thunderbird can pickup the settings automatically. I've done this for one server, but don't have the details anymore in my head. The link above should explain that all though. This may help with auto configuration if PHP is available... https://raw.githubusercontent.com/netserva/sh/master/etc/_.well-known_autodiscover.php
Backup questions - errors
Hi, although i've search the archives, there are questions left regarding backup strategies and errors that occur. I'm using dovecot 2.3.11.3 with mdbox (mailbox size 50MB - 30GB). Every night, i ran the following command: doveadm -o mail_fsync=never -o plugin/quota= -o plugin/zlib_save=gz \ backup -u account mdbox:/nfs/storage/account/mdbox ### incremental backup Thanks to zlib compression, i save a lot of storage space, nonetheless i'd go with an incremental backup, but it seems to only merge changes, comparing to rsyncs' --link-dest option where you can put all "changes of the day" in an extra folder. I'd like to prevent loosing mail that gets deleted by accident/on purpose. How can i achieve this with dsync and w/o doing a full backup every day? ### backup errors After a few days w/o any errors, doveadm prints the following lines (only for the biggest, 30GB mailbox) two days in a row. As dovecot has self- healing capabilities, i think these errors/warnings can be ignored and should go away...? Error: Transaction log mdbox/dovecot.list.index.log: duplicate transaction log sequence (38) Error: Index mdbox/dovecot.list.index: Lost log for seq=38 offset=63312: Missing middle file seq=38 (between 38..4294967295, we have seqs 37,39): .log.2 contains file_seq=37 (initial_mapped=1, reason=Index mapped) Warning: fscking index file mdbox/dovecot.list.index Error: Fixed index file mdbox/dovecot.list.index: log_file_seq 38 -> 39 Error: Mailbox list index was marked as fsck'd mdbox/dovecot.list.index Error: Mailbox list index was marked as fsck'd mdbox/dovecot.list.index Error: Mailbox list index was marked as fsck'd mdbox/dovecot.list.index Thanks, -Mark -- Mark Patruck ( mark at wrapped.cx ) GPG key 0xF2865E51 / 187F F6D3 EE04 1DCE 1C74 F644 0D3C F66F F286 5E51 https://www.wrapped.cx
Re: Backup questions - errors
On 8/28/20 9:54 AM, Mark Patruck wrote:
Hi,
although i've search the archives, there are questions left regarding
backup strategies and errors that occur.
I'm using dovecot 2.3.11.3 with mdbox (mailbox size 50MB - 30GB).
Every night, i ran the following command:
doveadm -o mail_fsync=never -o plugin/quota= -o plugin/zlib_save=gz \
backup -u account mdbox:/nfs/storage/account/mdbox
### incremental backup
Thanks to zlib compression, i save a lot of storage space, nonetheless
i'd go with an incremental backup, but it seems to only merge changes,
comparing to rsyncs' --link-dest option where you can put all "changes
of the day" in an extra folder. I'd like to prevent loosing mail that
gets deleted by accident/on purpose. How can i achieve this with dsync
and w/o doing a full backup every day?
Any info/recommendation on this? I still haven't found
a solution yet.
### backup errors
After a few days w/o any errors, doveadm prints the following lines (only
for the biggest, 30GB mailbox) two days in a row. As dovecot has self-
healing capabilities, i think these errors/warnings can be ignored and
should go away...?
Error: Transaction log mdbox/dovecot.list.index.log: duplicate transaction
log sequence (38)
Error: Index mdbox/dovecot.list.index: Lost log for seq=38 offset=63312: Missing middle file
seq=38 (between 38..4294967295, we have seqs 37,39): .log.2 contains file_seq=37 (initial_mapped=1,
reason=Index mapped)
Warning: fscking index file mdbox/dovecot.list.index
Error: Fixed index file mdbox/dovecot.list.index: log_file_seq 38 -> 39
Error: Mailbox list index was marked as fsck'd mdbox/dovecot.list.index
Error: Mailbox list index was marked as fsck'd mdbox/dovecot.list.index
Error: Mailbox list index was marked as fsck'd mdbox/dovecot.list.index
As i already thought, these are gone. No new errors over the
last days.
Thanks,
-Mark
--
Mark Patruck ( mark at wrapped.cx )
GPG key 0xF2865E51 / 187F F6D3 EE04 1DCE 1C74 F644 0D3C F66F F286 5E51
https://www.wrapped.cx
Re: LMTP Authentication Error
On Sat, Oct 10, 2020 at 12:08 PM David Morsberger
wrote:
> I wish someone could help me. I’m trying to track auth in the lmtp code.
> Nice code base but I’m having trouble tracking the call stack for the error
>
> Sent from my iPhone
>
> > On Oct 9, 2020, at 08:00, David Morsberger wrote:
> >
> > Alexander,
> >
> > Do you see anything wrong in my config?
> >
> > David
> >
> > Sent from my iPhone
> >
> >> On Oct 7, 2020, at 18:19, David Morsberger
> wrote:
> >> On 2020-10-07 12:43, Alexander Dalloz wrote:
> > Am 07.10.2020 um 18:20 schrieb da...@mmpcrofton.com:
> > Any ideas on how to resolve the Userdb connect/lookup problem? My
> users are pinging me on Sieve support.
> > Thanks,
> > David
> >>> Provide a full output of "doveconf -n"?
> >>> Alexander
> >> Alexandar,
> >> Thanks and here you go.
> >> # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
> >> # Pigeonhole version 0.5.7.2 ()
> >> # OS: Linux 5.4.0-48-generic x86_64 Ubuntu 20.04.1 LTS
> >> # Hostname: mmp-mail.mmpcrofton.com
> >> base_dir = /var/run/dovecot/
> >> first_valid_uid = 150
> >> login_greeting = Dovecot ready.
> >> mail_gid = 150
> >> mail_location = mbox:~/mail:INBOX=/var/mail/%u
> >> mail_privileged_group = mail
> >> mail_uid = 150
> >> managesieve_notify_capability = mailto
> >> managesieve_sieve_capability = fileinto reject envelope
> encoded-character vacation subaddress comparator-i;ascii-numeric relational
> regex imap4flags copy include variables body enotify environment mailbox
> date index ihave duplicate mime foreverypart extracttext
> >> namespace inbox {
> >> inbox = yes
> >> location =
> >> mailbox Drafts {
> >> auto = subscribe
> >> special_use = \Drafts
> >> }
> >> mailbox Junk {
> >> auto = subscribe
> >> special_use = \Junk
> >> }
> >> mailbox Sent {
> >> auto = subscribe
> >> special_use = \Sent
> >> }
> >> mailbox "Sent Messages" {
> >> auto = no
> >> special_use = \Sent
> >> }
> >> mailbox Spam {
> >> auto = create
> >> special_use = \Junk
> >> }
> >> mailbox Trash {
> >> auto = subscribe
> >> special_use = \Trash
> >> }
> >> prefix =
> >> }
> >> passdb {
> >> args = /etc/dovecot/dovecot-sql.conf.ext
> >> driver = sql
> >> }
> >> plugin {
> >> sieve =
> file:/home/mail/rules/%u/;active=/home/mail/rules/%u/.dovecot.sieve
> >> sieve_dir = /home/mail/rules/%u
> >> }
> >> protocols = " imap lmtp sieve pop3 sieve"
> >> service auth {
> >> unix_listener /var/spool/postfix/private/auth {
> >> group = postfix
> >> mode = 0660
> >> user = postfix
> >> }
> >> }
> >> service lmtp {
> >> unix_listener /var/spool/postfix/private/dovecot-lmtp {
> >> group = postfix
> >> mode = 0600
> >> user = postfix
> >> }
> >> }
> >> ssl = required
> >> ssl_cert = >> ssl_client_ca_dir = /etc/ssl/certs
> >> ssl_dh = # hidden, use -P to show it
> >> ssl_key = # hidden, use -P to show it
> >> userdb {
> >> driver = prefetch
> >> }
> >> userdb {
> >> args = /etc/dovecot/dovecot-sql.conf.ext
> >> driver = sql
> >> }
> >> protocol lmtp {
> >> mail_plugins = " sieve"
> >> postmaster_address = da...@mmpcrofton.com
> >> }
> >> protocol imap {
> >> mail_max_userip_connections = 50
> >> }
>
Pretty sure you can set up multiple unix_listener's. What about creating
another one, inside the 'service auth' container? It'll need to have
unix_listener set to 'auth-userdb' (for dovecot's sake, which probably
means that you'll to leave it with default user/group/permissions) with a
'path' of /var/run/dovecot. And then rename the existing one to
auth-userdb-postfix (totally arbitrary), though note that that will change
the filename of the socket itself, so you'll need to change postfix to use
/var/spool/postfix/private/auth/auth-userdb-postfix (i.e. same last
component as the argument to 'unix_listener')
So you'd end up with something like:
service auth {
unix_listener auth-userdb {
path = /var/run/dovecot
mode = 0660 (or whatever the default is)
user = $dovecot_auth_user_dunno_what
group = $dovecot_auth_group_dunno_what
}
unix_listener auth-userdb-postfix {
path = /var/spool/postfix/private/auth
mode = 0660
user = postfix
group = postfix
}
}
And then postfix would have /var/spool/postfix/private/auth/auth-userdb-postfix
for its dovecot-related socket
Re: Sieve filter script EXECUTION FAILED
On Fri, Oct 30, 2020 at 11:34 AM @lbutlr wrote:
> On 30 Oct 2020, at 11:57, Aki Tuomi wrote:
> > But I think the sed here is missing 's' from start, so this does not
> actually do anything...
>
> Copy/paste/edit error. The s is there in the file.
>
> darkmode.sh:
> #!/bin/sh
> echo $1 | sed -e 's||* {color:white !important;
> background-color: black !important; } |'
>
> I am not sure about the $1. I think filter just pipes the message (or part
> of the message.
>
> I will see what happens without the echo I suppose.
>
> Nope, still the same.
>
> 32: starting `:contains' match with `i;ascii-casemap' comparator:
> 32: matching value ` lang="en">29-Oct-2020 ""
Feature Request: Redis support for username and TLS
I was wondering if there was any imminent support in 2.3.12+ for using a username to log into Redis, as well as support for using TLS to connect to Redis. And if not, I'd like to put in a feature request for those two things (AUTH with username/password, and TLS connections to Redis). Specifically, I was looking at using a username/password combo to log into Redis for the quota_clone plugin. I found the 'password' param in the source (not documented at https://wiki.dovecot.org/Dictionary). There's no 'username' param (the 'username' in the source seems to refer to the mailbox, for the purpose of building the key name). Redis 6 supports authenticating with a username and password, as well as the ability to listen on a TLS-enabled port. Both of these significantly improve security, combined with the new ACL system. Obviously, these Redis 6 features are brand new, so I'd be shocked if they were already supported. But it'd be awesome if those were added to Dovecot :) Currently, I've got a localhost Envoy proxy doing TCP proxying from localhost+non-TLS to my Redis TLS port, which is a kludge at best. There's a neat Envoy Redis proxy that *almost* does the trick but the Envoy Redis proxy unfortunately doesn't support MULTI/EXEC, which Dovecot quota_clone uses, or I'd be using that instead of a plain TCP proxy (since the Envoy Redis proxy can use a username/password+tls to connect to the upstream Redis).
LMDB support?
Hi all, is there any way I could use LMDB for userdb and passdb lookups?
Re: Why Last-login?
On Wed, Mar 3, 2021 at 11:16 AM @lbutlr wrote: > On 03 Mar 2021, at 05:33, Yassine Chaouche > wrote: > >> Am I missing some reason I would need/want to keep track of that > specific login time separately? > > > What about mbox files ? > > Is anyone foolish enough to use mbox in 2021? > > It's designed for dozens of kilobytes of mail. Perhaps hundreds of > kilobytes/ It is a horrible horrible format for hundreds of megabyte of > mail, it offers no advantages at all, and is fragile to corruption since it > stores everything in a single file. > > Specific to the 'why use last login' question, with millions of mailboxes, walking the filesystem is more than a little onerous (having done it many times over the years, and never remembering where I put the script from 'last time') and takes a good chunk of a day to run. We were doing file-based last-login for a while (yeah, still needs a fs walk, but at least is dead simple and requires no stat()'ing), till locking became an issue (nfs). We moved to redis a couple of months ago, and now determining things like "who hasn't logged into anything in 30 days" becomes a 1 minute run of a python script using redis SCAN. If you don't have a mountain of mailboxes and fs-walking isn't a problem, then there's def less need. Which means you don't have management repeatedly asking for 'active mailboxes' ;)
JMAP support?
On Fri Jan 25 2019 Aki wrote: > There is no JMAP support in any dovecot version yet. Two years on so what is the status of JMAP support in Dovecot now?
Re: JMAP support?
On 8/3/21 18:08, @lbutlr wrote: Two years on so what is the status of JMAP support in Dovecot now? I don't think anything has changed on this front. Last I saw, only Cyrus has partial support for JMAP mail, but not the other parts of JMAP. Such a pity. I just two weeks battling with Cyrus-imapd just to test out the Cypht JMAP client and frankly Cyrus is an ill-documented PITA, to put it bluntly. From charter-ietf-jmap-03 "Now that draft-ietf-jmap-mail is completed, the working group will produce specifications for related data types, beginning with calendars and contacts." So "JSON Mail Access Protocol" is done and jmap + websockets is on the way... https://www.rfc-editor.org/rfc/rfc8887.html so I guess, minus cyrus-imapd, it's down to these fledgling projects... https://github.com/search?q=jmap+server No doubt I will ask about Dovecots JMAP status in another 12 months.
Re: TLS connection closed unexpectedly
On Fri, Jan 7, 2022 at 1:34 PM John Fawcett wrote: > On 07/01/2022 21:03, Ken Wright wrote: > > On Fri, 2022-01-07 at 18:50 +0100, John Fawcett wrote: > >> it may or may not be related to the tls issue, but I think you will > >> want to investigate that message about the SQL query syntax error. > >> You are not going to be able to login if the query is giving errors. > >> Check the log doesn't reveal the cause. > > Know anything about SQL queries, John? Here's the user query in > > question: > > > > user_query = SELECT maildir, 2000 AS uid, 2000 AS gid FROM mailbox > > WHERE username = '%u' AND active='1' > > > > I copied this directly from the tutorial I've been following and this > > is the first time I've seen this error. > > > Hi Ken > > looks fine to me. However, mariadb is not accepting it. I suggest you > run with auth_debug = yes and check the logs. > > Does it help at all if you use backticks around the column names for uid and gid? I.e. from: user_query = SELECT maildir, 2000 AS uid, 2000 AS gid FROM mailbox WHERE username = '%u' AND active='1' to: user_query = SELECT maildir, 2000 AS `uid`, 2000 AS `gid` FROM mailbox WHERE username = '%u' AND active='1'
Re: IMAP preauth and stats-writer
Hi, many thanks for the quick replies and patch. So quick that I'm not able to respond in the same timeframe. Yes, I am working with FreeBSD pre-built packages; I still intend check out the ports from svn or do my own build, but have not had time yet. To answer specific question: On Sun, 6 Jan 2019, John Fawcett wrote: > One suggestion is to run with the default setting, but look at resolving > the permission problem for the default socket creation at > /var/run/dovecot/stats-writer rather than working round it. Potentially, though it's less logical that the daemon dovecot (which runs as a system user / priviledged) should not be accepting stats from an unpriviledged source. Yes, one could seek to get stats for all dovecot activity on the server. Assuming clients are trusted to feed valid stats and not cause some horrible DoS or similar. So I am heading for the opoosite where dovecot is really the self-contained 'imap' command without crossing any priviledge boundary; just a process consuing CPU and RAM resources like any other. To date I haven't used any of the dovecot stats (not dismissing them, just haven't had a cause to) Thanks again -- Mark
Re: "unknown user - trying the next userdb" Info in log
On Tue, Jan 29, 2019 at 9:58 PM James Brown via dovecot
wrote:
> On 30 Jan 2019, at 4:35 pm, Aki Tuomi wrote:
>
>
>
> On 30 January 2019 at 07:12 James Brown < jlbr...@bordo.com.au> wrote:
>
>
> >> My settings:
> ...
> >> userdb {
> >> driver = passwd
> >> }
> >> userdb {
> >> driver = prefetch
> >> }
> >> userdb {
> >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
> >> driver = sql
> >> }
>
> Well... there is that usetdb passwd which seems bit extraneous.
> ---
> Aki Tuomi
>
>
> I'd remove the
>
> userdb {
> driver = passwd
> }
>
> section
> ---
> Aki Tuomi
>
>
> Thanks Aki - the trick was finding where that setting was! Found it in
> auth-system.conf.ext.
>
> Commented it out and all works perfectly now.
>
> Thanks again Aki,
>
> James.
>
I'll throw in my 2 cents that it'd be great for a passdb/userdb block to
have a setting to suppress that message. I've actually changed my mind and
*not* used extra userdbs in the past (despite there being good reasons to
use them) entirely due to that log entry. We've got millions of mailboxes
and the noise in the logs would be unreal. And it would cause no end of
confusion for ops people looking at logs. I weighed the likelihood that I'd
end up being asked a million times why a login failed, when in reality it
hadn't, and decided it wasn't worth the headache.
Or alternatively only log that error when *all* of the passdbs/userdbs have
failed. Anything would be better than logging what *looks* like an error
but isn't. And, yes, i see 'info' in the log entry, but 'unknown user' is
far more eye-catching.
This is the part where someone points out that there already *is* a setting
for this and I stop talking :)
Doveadm service as non-root user
Running: Ubuntu xenial, dovecot 2.2.36 I've been working on moving our user base from maildir to mdbox and trying to come up with solutions for things like moving emails around. In the past, with maildir, our support guys could just mv the files around and done. For mdbox, I've been working on getting things set up to use doveadm. One weirdness I've seen is that in imports (i.e. doveadm import), mail gets copied correctly but the resulting files are left with root ownership (I don't have 'service doveadm' 'user' set, so I guess it defaults to root). It's typically new m.* files as well as the dovecot.list.index and dovecot.list.index.log files. Looking at strace, no chown is done on them, nor was there setuid. The import had no trouble finding the correct user in the db, so I know that it knows the correct UID (I can see it just fine in debug logs too). And it will happily import to existing m.* files with no permissions issues (but considering it's running as root, I wouldn't expect it to). I've seen this using 'import' via IMAPc as well as with both src and dest on the same server. I can see this behavior in both scenarios. We have a single shared UID for mail, so especially in that "src/dest on same server" case, it's not a matter of UID-mismatch. It's a director setup, so all doveadm commands are coming through the director. If I run the import directly on the backend (which obviously would be a bad idea in real life), the ownership of new m.* files seems to be correct (I can see it setuid'ing to the correct UID from userdb in strace). If I run the import on the director, I can get a new root-owned file every time it rolls over to the next m.* file. Two questions: * Is that a bug? Is this expected behavior? Seems like the expected thing would be to use the UID from userdb and either do a setuid (just like running 'doveadm import' locally did) or chown'ing any new files to the correct UID. I always always assume misconfiguration (vs bug, since it's almost never a bug) but I'm baffled on this one. * I see that it's possible to set a user for service doveadm and the wiki even suggests that it's a good idea in a single UID setup. If there are no mailboxes with any other UIDs, *will setting 'service doveadm' to the same UID possibly break anything*? I can't think of why it would, but I want to be duly diligent. Plus I'm a little leery about closing the door to ever having additional UIDs for mailboxes. Happy to provide 'doveconf -n' but wanted to check first, before spending 15 minutes gently obfuscating it :)
Re: Doveadm service as non-root user
On Fri, Feb 1, 2019 at 11:37 PM Aki Tuomi wrote: > > On 01 February 2019 at 23:16 Mark Moseley < moseleym...@gmail.com> wrote: > > > Running: Ubuntu xenial, dovecot 2.2.36 > > I've been working on moving our user base from maildir to mdbox and trying > to come up with solutions for things like moving emails around. In the > past, with maildir, our support guys could just mv the files around and > done. For mdbox, I've been working on getting things set up to use > doveadm. > > One weirdness I've seen is that in imports (i.e. doveadm import), mail > gets > copied correctly but the resulting files are left with root ownership (I > don't have 'service doveadm' 'user' set, so I guess it defaults to root). > It's typically new m.* files as well as the dovecot.list.index > and dovecot.list.index.log files. > > Looking at strace, no chown is done on them, nor was there setuid. The > import had no trouble finding the correct user in the db, so I know that > it > knows the correct UID (I can see it just fine in debug logs too). And it > will happily import to existing m.* files with no permissions issues (but > considering it's running as root, I wouldn't expect it to). > > I've seen this using 'import' via IMAPc as well as with both src and dest > on the same server. I can see this behavior in both scenarios. We have a > single shared UID for mail, so especially in that "src/dest on same > server" > case, it's not a matter of UID-mismatch. > > It's a director setup, so all doveadm commands are coming through the > director. If I run the import directly on the backend (which obviously > would be a bad idea in real life), the ownership of new m.* files seems to > be correct (I can see it setuid'ing to the correct UID from userdb in > strace). If I run the import on the director, I can get a new root-owned > file every time it rolls over to the next m.* file. > > Two questions: > > * Is that a bug? Is this expected behavior? Seems like the expected thing > would be to use the UID from userdb and either do a setuid (just like > running 'doveadm import' locally did) or chown'ing any new files to the > correct UID. I always always assume misconfiguration (vs bug, since it's > almost never a bug) but I'm baffled on this one. > > * I see that it's possible to set a user for service doveadm and the wiki > even suggests that it's a good idea in a single UID setup. If there are no > mailboxes with any other UIDs, *will setting 'service doveadm' to the same > UID possibly break anything*? I can't think of why it would, but I want to > be duly diligent. Plus I'm a little leery about closing the door to ever > having additional UIDs for mailboxes. > > Happy to provide 'doveconf -n' but wanted to check first, before spending > 15 minutes gently obfuscating it :) > > > Can you try > > doveadm import -U victim -u victim ... ? > --- > Aki Tuomi > Is that to test a generic 'import from sourceUser to dest user' (i.e. victim isn't literally the same in both -u and -U) or are you looking for a test where 'sourceUser' is the same email account as the destination? I just want to make sure I'm understanding right. The original tests (that result in the root-owned files) were all -U userA -u userB (i.e. different email accounts for src and dest), if you're asking about the former. If you're asking about the latter, I ran that and got the same result, a root-owned dovecot.list.index.log and dovecot.list.index and freshly created m.* files. The message count in the destination mailbox increases by the right number (no surprise since it's running as root), so the import itself is working. I should add that in both cases (different src/dest email account and same src/dest), the import works ok -- or at least increments the count in the index. It just leaves the email account in a broken state. Re-chown'ing it to the current permissions makes it happy again and the newly imported messages show up.
doveadm import with subfolder oddity
This has got to be something weird in my config. And the standard
disclaimer of '"happy to post doveconf -n, but wanted to see if this is
normal first" :)
Background: Ubuntu Xenial, running 2.2.36. Mailbox type is mdbox and I've
got a period separator in my inbox namespace:
namespace {
hidden = no
inbox = yes
list = yes
location =
mailbox Spam {
auto = no
autoexpunge = 1 weeks
special_use = \Junk
}
mailbox Trash {
auto = no
special_use = \Trash
}
prefix = INBOX.
separator = .
subscriptions = yes
type = private
}
If I do a import for a regular folder under INBOX, it works just fine:
doveadm import -u testbox2@testing.local -U testbox1@testing.local
mdbox:~/mdbox INBOX all mailbox Sent
... returns happily, message count gets incremented
If I try to do the same with a subfolder (and a subfolder that most
definitely exists on both source and destination side), I get an error:
doveadm import -u testbox2@testing.local -U testbox1@testing.local
mdbox:~/mdbox INBOX all mailbox Sub.Sub1
doveadm(testbox2@testing.local): Error: remote(10.1.17.98:4000): Mailbox
Sub.Sub1: Mailbox sync failed: Mailbox doesn't exist: Sub.Sub1
If I use / instead of . in my query, it works:
doveadm import -u testbox2@testing.local -U testbox1@testing.local
mdbox:~/mdbox INBOX all mailbox Sub/Sub1
... returns happily and message count gets incremented.
Since we're using '.' as our separator, that was a bit unexpected :)
Ironically, if I'm doing a IMAPc 'import', it works just fine with a query
of 'all mailbox Sub.Sub1'. It's only when importing from a local src and
local dest (i.e. source_location == mdbox:~/mdbox) that it fails. With
source_location set to 'imapc:', it works. I imagine that's due to using
straight IMAP on the source side.
Likely a misconfig on my part? Expected behavior?
I can see in the strace that the error is triggered when doveadm is looking
at the source mailbox. It looks for mdbox/mailboxes/Sub.Sub1/dbox-Mails
first, then falls back to mdbox/mailboxes/Sub/Sub1/dbox-Mails (which it
finds). Then a little bit later in the strace, it again looks
for mdbox/mailboxes/Sub.Sub1/dbox-Mails (which it doesn't find) but doesn't
try mdbox/mailboxes/Sub/Sub1/dbox-Mails this time, and then spits out
'Mailbox Sub.Sub1: Mailbox sync failed: Mailbox doesn't exist: Sub.Sub1'.
With a query of 'all mailbox Sub/Sub1', the stat() is
for mdbox/mailboxes/Sub/Sub1/dbox-Mails which it finds and uses happily.
Having to substitute the '.'s for '/'s in the 'mailbox' part of the query
isn't an awful workaround, but it very much feels like I'm doing something
wrong. This is a production setup, so everything else is otherwise working
fine. But I've only just begun working with 'doveadm import', so I might be
turning up some issues with my config.
Thanks! Sorry I'm so verbose :)
Unable to authenticate on Dovecot - auth-userdb issue?
Some general information: Mageia Linux 5.4.6-desktop-2.mga7 2.3.7.2 (3c910f64b) postfix + dovecot + mysql 192.168.1.105 (shuttle) the email server machine 192.168.1.103 (pvr) the mail client machine I am unable to authenticate to send email. I've looked at postfix but I can't get past dovecot's authentication. Here is what I'm seeing in logs: Jan 02 18:46:47 shuttle sshd[6660]: Connection closed by 192.168.1.100 port 48506 [preauth] Jan 02 18:47:05 shuttle postfix/smtpd[6352]: connect from pvr[192.168.1.103] Jan 02 18:47:16 shuttle postfix/smtpd[6352]: lost connection after CONNECT from pvr[192.168.1.103] Jan 02 18:47:16 shuttle postfix/smtpd[6352]: disconnect from pvr[192.168.1.103] commands=0/0 Jan 02 18:47:36 shuttle postfix/smtpd[6352]: connect from pvr[192.168.1.103] Jan 02 18:47:36 shuttle postfix/smtpd[6352]: 6345D4A4A97: client=pvr[192.168.1.103] Jan 02 18:47:37 shuttle postfix/cleanup[6500]: 6345D4A4A97: message-id=<> Jan 02 18:47:37 shuttle postfix/qmgr[1385]: 6345D4A4A97: from=, size=485, nrcpt=1 (queue active) Jan 02 18:47:37 shuttle postfix/smtpd[6352]: disconnect from pvr[192.168.1.103] helo=1 mail=1 rcpt=1 data=1 quit=1 commands=5 Jan 02 18:47:37 shuttle dovecot[6744]: lda(root@shuttle)<6744><>: Error: auth-master: userdb lookup(root@shuttle): connect(/run/dovecot/auth-userdb) failed: Permission denied (euid=8(mail) egid=12(mail) missing +r perm: /run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) Jan 02 18:47:37 shuttle dovecot[6744]: lda: Fatal: Internal error occurred. Refer to server log for more information. Jan 02 18:47:37 shuttle postfix/pipe[6743]: 6345D4A4A97: to=, relay=dovecot, delay=1.1, delays=1.1/0.01/0/0.06, dsn=4.3.0, status=deferred (temporary failure. Command output: lda(root@shuttle): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied ) ^C Note: this error references "/run/dovecot/auth-userdb". That isn't even supposed to be the location of that file. I have no idea why that location shows up. The correct location should be "/etc/dovecot/auth-userdb". The file does exist at that location. There is no "base_dir" configured in /etc/dovecot/dovecot.conf. When I do try an point the configuration at the correct base_dir, I get this when I try to restart dovecot: -- The unit dovecot.service has entered the 'failed' state with result 'exit-code'. Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(aggregator): unlink(/etc/dovecot/replication-notify-fifo) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(pop3): unlink(/etc/dovecot/login/pop3) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(old-stats): unlink(/etc/dovecot/old-stats) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(old-stats): unlink(/etc/dovecot/old-stats-mail) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(old-stats): unlink(/etc/dovecot/old-stats-user) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(log): unlink(/etc/dovecot/log-errors) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(lmtp): unlink(/etc/dovecot/lmtp) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(ipc): unlink(/etc/dovecot/ipc) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(ipc): unlink(/etc/dovecot/login/ipc-proxy) failed: Read-only file system Jan 02 18:51:50 shuttle dovecot[7226]: master: Error: service(indexer-worker): unlink(/etc/dovecot/indexer-worker) failed: Read-only file system And there are about 30 lines of "read-only file system" errors. I haven't been able to track down the cause of that. Once the line "base_dir = /etc/dovecot" is commented out in /etc/dovecot/dovecot.conf, I can start dovecot: # systemctl status dovecot ● dovecot.service - Dovecot IMAP/POP3 email server Loaded: loaded (/usr/lib/systemd/system/dovecot.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2020-01-02 18:54:15 MST; 5s ago Docs: man:dovecot(1) http://wiki2.dovecot.org/ Main PID: 7550 (dovecot) Memory: 3.8M CGroup: /system.slice/dovecot.service ├─7550 /usr/sbin/dovecot -F ├─7554 dovecot/anvil ├─7555 dovecot/log └─7556 dovecot/config Jan 02 18:54:15 shuttle systemd[1]: Started Dovecot IMAP/POP3 email server. Jan 02 18:54:15 shuttle dovecot[7550]: master: Dovecot v2.3.7.2 (3c910f64b) starting up for imap, pop3, lmtp Jan 02 18:54:15 shuttle dovecot[7550]: master: Error: t_readlink(/etc/dovecot/dovecot.conf) failed: readlink() failed: Invalid argument I have no idea what's up with the t_readlink error. Might be related to the errors above. I can't really find out much about it. Trying to send email shows no apparent errors:
RE: Unable to authenticate on Dovecot - auth-userdb issue?
uth-userdb {
group =
mode = 0666
user = $default_internal_user
}
user = dovecot
}
service imap-login {
inet_listener imap {
port = 143
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service pop3-login {
inet_listener pop3s {
port = 995
ssl = yes
}
}
service stats {
unix_listener stats-reader {
group = mail
mode = 0666
}
unix_listener stats-writer {
group = mail
mode = 0666
}
}
ssl = required
ssl_cert = mailto:ad+li...@uni-x.org>
Sent: Friday, January 10, 2020 11:34 AM
To: Mark ADAMS<mailto:mada...@msn.com>
Subject: Re: Unable to authenticate on Dovecot - auth-userdb issue?
Mark,
first of all: please take care to whom you reply. Do not communicate
directly with my list mail address. Please keep the discussion on the
dovecot list. Thanks.
Am 09.01.2020 um 18:29 schrieb Mark ADAMS:
> At this point, passdb does not support lookups according to the log. Is there
> something else I should be looking at?
>
> I’ve worked on this and seem to be making little progress. A sample
> transaction log looks like this:
>
>
> Jan 09 10:22:32 shuttle dovecot[26851]: master: Warning: SIGHUP received -
> reloading configuration
> Jan 09 10:23:04 shuttle postfix/smtpd[5448]: connect from pvr[192.168.1.103]
> Jan 09 10:23:04 shuttle dovecot[5432]: auth: Debug: Loading modules from
> directory: /usr/lib64/dovecot/modules/auth
> Jan 09 10:23:04 shuttle dovecot[5432]: auth: Debug: Module loaded:
> /usr/lib64/dovecot/modules/auth/lib20_auth_var_expand_crypt.so
> Jan 09 10:23:04 shuttle dovecot[5432]: auth: Debug: Read auth token secret
> from /run/dovecot/auth-token-secret.dat
> Jan 09 10:23:04 shuttle dovecot[5432]: auth: Debug: auth client connected
> (pid=0)
> Jan 09 10:23:20 shuttle postfix/smtpd[5448]: 0C6BF4A6302:
> client=pvr[192.168.1.103]
> Jan 09 10:23:30 shuttle postfix/cleanup[5459]: 0C6BF4A6302: message-id=<>
> Jan 09 10:23:30 shuttle postfix/qmgr[1385]: 0C6BF4A6302: from=,
> size=180, nrcpt=1 (queue active)
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Debug: master in: USER1
> root@shuttleservice=lda
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Debug: static(root): Performing
> userdb lookup
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Debug: pam(root): Performing
> passdb lookup
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Debug: pam(root): passdb doesn't
> support credential lookups
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Debug: pam(root): Finished
> passdb lookup
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Error: static(root): passdb
> doesn't support lookups, can't verify user's existence
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Debug: static(root): Finished
> userdb lookup
> Jan 09 10:23:30 shuttle dovecot[5432]: auth: Debug: userdb out: FAIL1
> Jan 09 10:23:30 shuttle dovecot[5466]: lda(root@shuttle)<5466><>: Error:
> auth-master: userdb lookup(root@shuttle): Auth USER lookup failed
> Jan 09 10:23:30 shuttle dovecot[5466]: lda: Fatal: Internal error occurred.
> Refer to server log for more information.
> Jan 09 10:23:30 shuttle postfix/pipe[5465]: 0C6BF4A6302: to=,
> relay=dovecot, delay=17, delays=17/0.01/0/0.06, dsn=4.3.0, status=deferred
> (tempora>
> Jan 09 10:23:31 shuttle sshd[5468]: Connection closed by 192.168.1.100 port
> 48324 [preauth]
> Jan 09 10:23:31 shuttle postfix/smtpd[5448]: disconnect from
> pvr[192.168.1.103] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5
>
>
> My current dovecot configuration looks like this:
>
> # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
> # OS: Linux 5.4.6-desktop-2.mga7 x86_64 Mageia 7
> # Hostname: shuttle
> auth_debug_passwords = yes
> auth_username_format = %Ln
> disable_plaintext_auth = no
> first_valid_uid = 0
> last_valid_uid = 10001
> mail_gid = 10001
> mail_location = mbox:~/mail:INBOX=/var/mail/%u
> mail_privileged_group = mail
> mail_uid = 10001
> namespace inbox {
>inbox = yes
>location =
>mailbox Drafts {
> special_use = \Drafts
>}
>mailbox Junk {
> special_use = \Junk
>}
>mailbox Sent {
> special_use = \Sent
>}
>mailbox "Sent Messages" {
> special_use = \Sent
>}
>mailbox Trash {
> special_use = \Trash
>}
>prefix =
> }
> passdb {
>args = %s
>driver = pam
> }
> plugin {
>sieve = file:~/sieve;active=~/.dovecot.sieve
> }
> service anvil {
>unix_listener anvil {
> group = mail
> mode = 0666
>}
> }
> service auth-worker {
>user = vmail
> }
> service auth {
>
imapc master login for shared mailboxes
Hi there, I'm trying to set up shared mailboxes such that any access to the shared namespace like INBOX/shared/ will take and use imapc to log in with * (ie the current user is the master account for imapc). I've set up like https://wiki2.dovecot.org/SharedMailboxes/ClusterSetup on centos8 latest dovecot build (a late 2.2 series version I believe. However I'm running in to a couple of issues: 1) setting imapc_master_user = %u has %u as rather than . I can't find any sort of variable substitution that will allow me to access the details of . From looking at the logs it seems to do a userdb lookup for to check it exists but then the substitutions appear to be using these rather than the current user's details 2) In a similar vein, I believe imapc_password = %w (ie trying to just get current user logging in with the password that was specified when they initially logged in, but as a master user) is also not substituting correctly and is just a blank string Is there any way around this so I don't have to worry about trying to do sharing based on the filesystem and the same process but rather just allow the main user to log in to any other account (in this case domain owner logging in to any accounts under their domain) ? Thanks Mark
Seive + Spamprobe terminates with signal 6
Ubuntu 20.04, Dovecot 2.3.7.2, SpamProbe v1.4d. For the past weeks my
sieve filters that call spamprobe have been crashing out for some users.
I have a dozen similar server setups and this is not happening on any
of the other servers and it was working just fine for a year up until
recently. This particular server is quite busy. There is plenty of disk
space and inode usage is about 2.5%. Permissions look the same as on
the other servers.
I'm hoping someone might have a suggestion as to the cause of this?
Apr 22 10:31:24 mail dovecot:
lmtp(x...@example.com)<33016>: Error: program
exec:/etc/dovecot/sieve/spamprobe (38010): Terminated abnormally with signal 6
Apr 22 10:31:24 mail dovecot:
lmtp(x...@example.com)<33016>: Error: sieve: global:
line 6: execute action: failed to execute to program `spamprobe': refer to server log for
more information. [2020-04-22 10:31:24]
~ cat /etc/dovecot/sieve/global.sieve
require ["vnd.dovecot.execute", "fileinto", "envelope", "variables",
"editheader"];
if header :contains "from" ["root@", "daemon@", "postmaster@"] { fileinto
"Trash";
} elsif header :contains "to" ["root@", "daemon@", "postmaster@"] { fileinto
"Trash"; }
if envelope :localpart :matches "to" "*" { set "lhs" "${1}"; }
if envelope :domain :matches "to" "*" { set "rhs" "${1}"; }
execute :pipe :output "SCORE" "spamprobe" ["-c", "-d",
"/home/u/${rhs}/home/${lhs}/.spamprobe", "receive"];
addheader :last "X-Spam" "${SCORE}";
if header :matches "X-Spam" "SPAM*" { fileinto "Junk"; }
plugin {
imapsieve_mailbox1_before = file:/etc/dovecot/sieve/retrain-as-spam.sieve
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_name = Junk
imapsieve_mailbox2_before = file:/etc/dovecot/sieve/retrain-as-good.sieve
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_name = *
listescape_char = mail_log_cached_only = yes
mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
mail_log_fields = uid box msgid size
sieve_before = file:/etc/dovecot/sieve/global.sieve
sieve_dir = ~/sieve
sieve_execute_bin_dir = /etc/dovecot/sieve
sieve_extensions = +notify +imapflags +vacation-seconds
sieve_global_dir = /etc/dovecot/sieve/
sieve_global_extensions = +vnd.dovecot.debug +editheader +vnd.dovecot.pipe
+vnd.dovecot.execute
sieve_max_redirects = 30
sieve_max_script_size = 1M
sieve_pipe_bin_dir = /etc/dovecot/sieve
sieve_plugins = sieve_imapsieve sieve_extprograms
sieve_quota_max_scripts = 0
sieve_quota_max_storage = 0
sieve_redirect_envelope_from = recipient
sieve_vacation_default_period = 60s
sieve_vacation_max_period = 0
sieve_vacation_min_period = 5s
sieve_vacation_send_from_recipient = yes
}
--
Mark Constable
0419 530 037
https://spiderweb.com.au
Re: problem with a public folder
On 25/4/20 10:35 am, David Mehler wrote:
#doveadm acl get -A "Public/TestFolder"
doveadm(usern...@example.com): Error: Can't open mailbox
Public/TestFolder: Mailbox doesn't exist: Public/TestFolder
Username ID Global Rights
# ls -ld public/.TestFolder
drwx-- 5 vmail vmail 512 Apr 15 23:20 .TestFolder/
Try renaming the above folder to TestFolder (remove the dot).
namespace {
location =
sdbox:/var/vmail/public/:CONTROL=~/mail/public:INDEX=~/mail/public:INDEXPVT=~/mail/public
prefix = Public/
separator = /
subscriptions = yes
type = public
}
--
Mark Constable
0419 530 037
https://spiderweb.com.au
--
Mark Constable
0419 530 037
https://spiderweb.com.au
Re: problem with a public folder
On 25/4/20 10:35 am, David Mehler wrote:
I'm trying to set up a public folder called TestFolder. I'm getting this error:
#doveadm acl get -A "Public/TestFolder"
doveadm(usern...@example.com): Error: Can't open mailbox
Public/TestFolder: Mailbox doesn't exist: Public/TestFolder
Username ID Global Rights
# ls -ld public/.TestFolder
drwx-- 5 vmail vmail 512 Apr 15 23:20 .TestFolder/
Try renaming the above folder to TestFolder (remove the dot).
namespace {
location =
sdbox:/var/vmail/public/:CONTROL=~/mail/public:INDEX=~/mail/public:INDEXPVT=~/mail/public
prefix = Public/
separator = /
subscriptions = yes
type = public
}
--
Mark Constable
0419 530 037
https://spiderweb.com.au
Re: Seive + Spamprobe terminates with signal 6
On 22/4/20 10:40 am, Mark Constable wrote: Ubuntu 20.04, Dovecot 2.3.7.2, SpamProbe v1.4d. For the past weeks my sieve filters that call spamprobe have been crashing out for some users. For Googles sake, solved. I reverted to the older Bionic version of the Ubuntu spamprobe package and that seems to have fixed my problem... ## Downgrade spamprobe to prevent crashing, add to 20.04 sources.list ## deb http://au.archive.ubuntu.com/ubuntu bionic universe apt-get update #apt-cache showpkg spamprobe apt-get install spamprobe=1.4d-14build1 echo "spamprobe hold" | dpkg --set-selections -- Mark Constable 0419 530 037 https://spiderweb.com.au
Re: SV: Marking all emails in "Trash" as opened, and also prohibiting email clients from creating new ma
On 10/5/20 3:33 am, Sebastian Nielsen wrote:
And then this in plugins.conf:
plugin {
sieve_plugins = sieve_imapsieve
imapsieve_mailbox1_name = Trash
imapsieve_mailbox1_before = file:/etc/dovecot/sieve/trash.sieve
}
Maybe adding this will help...
imapsieve_mailbox1_causes = COPY FLAG
I can no longer use TLS for Windows7 and Outlook
I currently use Ubuntu 20.04 with Dovecot 2.3.7.2 and OpenSSL 1.1.1f. A few months ago there was an update to all these systems and since then I've had to talk W7 and old Mac clients through disabling ports 993/995 with TLS enabled back to ports 143/110 without SSL or they could not pick up email. Thunderbird users (ie; me) were unaffected. Could anyone share a set of port 993/995 SSL settings known to work with Windows7 and Outlook16 using "dovecot -n|grep ^ssl_" please ? Mine is currently... ssl_ca =
Re: Send SPF SoftFails to Junk
On 14/6/20 8:07 am, Scott A. Wozny wrote: Messages from domains set with a softfail that come from bad sources are tagged in the message header with "Softfail (domain owner discourages use of this host)", but end up in the user's Inbox, anyway. I suppose I kind of figured DoceCot would see the SPF softfail in the message header and automagically route that message to Junk, but it didn't. I've looked around Google and the Dovecot documentation to see if there are any instructions on how to do this, but I can't find anything. You probably need to take advantage of sieve scripts so here is a super lightweight spam filtering system that might provide some useful clues.. https://markc.blog/server/using-spamprobe-via-sieve/
Re: Apple mail works but thunderbird unable to connect
make sure ports are set correctly in IPtables as it seems to be failing on TLS/SSL (open port 993) Mark On 6/22/2020 12:19 PM, Dshah wrote: Dear all, I am stuck on this issue for days, I am able to connect on my IMAP server using Apple mail client but thunderbird shows me this error: "improper command pipelining after EHLO" I have tried all the ports in manual config but it just is not working while Apple Mail is able to connect perfectly, here are my configuration details of "postconf -n" , "doveconf -n" and "master.cf <http://master.cf>" https://pastebin.com/M3XG9DXA Thanks in advance.
Re: Outlook vs Thunderbird
On 7/7/20 12:16 pm, The Doctor wrote: Got a client that usually uses Outlook I think 2010. This person tends to move their e-mails to certain folers. On Thunderbird, the move shows. Not on Outlook. Any explanation? Using IMAp, most folders should sync client and server. Just wondering if an old version of Outlook has passed its time. FWIW if they happen to be using Windows7 and dovecot has been updated recently then you could try disabling SSL/STARTTLS on port 143. -- Mark Constable 0419 530 037 https://spiderweb.com.au
Re: Outlook vs Thunderbird
On 7/7/20 3:50 pm, @lbutlr wrote: you could try disabling SSL/STARTTLS on port 143. What? I’ve never seen SSL/StARTTLS on port 143,a dn I doubt that would work? I thought you had a problem picking up IMAP mail. I see now you mean you move messages within Thunderbird and the Outlook 2010 app does not sync those changes. My mistake. FWIW I meant if the client is Windows7/old-Outlook then changing either 993/SSL or 143/STARTTLS to 143/NONE could help pick up the mail. We had to do this for a 100 or so clients a few months ago after upgrading to Ubuntu 20.04.
Re: Outlook vs Thunderbird (re disabling SSL)
On 8/7/20 2:04 am, Alexander Dalloz wrote: FWIW I meant if the client is Windows7/old-Outlook then changing either 993/SSL or 143/STARTTLS to 143/NONE could help pick up the mail. We had to do this for a 100 or so clients a few months ago after upgrading to Ubuntu 20.04. Curious, what's the rationale behind that move? Is it because that old beast of Outlook does not have the capabilities modern TLS/STARTTLS implementations require regarding TLS minimal version and ciphers? It involved Windows7 customers and older Apple device users. Recent versions of Thunderbird on Win7 still worked fine but even Outlook 2016 on Win7 could no longer pick up mail with SSL enabled. It happened after a Ubuntu server update to Dovecot and Openssl about 3 or 4 months ago. But plaintext auth for mail access, seriously? Tell me about it! We spent YEARS getting these same folks to change to secure settings (some of them have been with us for 20+ years) so it was heartbreaking to contact each one of them and talk them through disabling SSL. I spent a week trying every cypher combination I could find via Google for Dovecot but with the phone going off the hook from complaints by customers not being able to pick up their mail. We had to respond with some solution so, after a week, disabling SSL was very reluctantly the only option left. We lost ~40 customers to outlook.com because of this. Actually, there is a regedit "trick" for Win7 but that is beyond the ability of our customers to apply, and that doesn't help the older Apple device users. FWIW.
Re: Dovecot permission denied errors on NFS after upgrade to 2.2.17
On Mon, Jul 13, 2020 at 7:36 AM Claudio Corvino wrote: > Thanks Jochen, > > no mixups present at all, file assigned to UID 501. > > Since this problem started few hours after the Debian upgrade, I think > it is related to it. > > I don't know if something has changed on the NFS client side on Debian, > but I don't think so as aptlistchanges didn't notify me about it, nor if > Dovecot 2.2.17 treat NFS in other way. > > I'm stuck. > > On 13/07/20 16:07, Jochen Bern wrote: > > On 07/13/2020 03:45 PM, Claudio Corvino wrote: > >> in addition the "permission denied" error is > >> random, most of the time Dovecot works well. > > In *that* case, I'd say that UID/GID mapping problems can be ruled out. > > > >> How can I check the mappings NFS uses? > > You don't have any relevant options in the client's fstab entry, and > > I'll assume that there are none in the server's /etc/exports, either. > > That leaves only potential default mappings, which should be documented > > in the corresponding manpages. > > > > Also, since there's only *one* user/group involved, you can always > > "chown" a test file on one side and check with "ls -n" on the other to > > verify whether there are mixups. > > > > *Intermittent* failures of an NFS mount over a well-functioning LAN ... > > I'm thinking "file locking" now, but that's a *complicated* topic, to > > say the least ... > > > > https://en.wikipedia.org/wiki/File_locking#Problems > > > https://unix.stackexchange.com/questions/553645/how-to-install-nfslock-daemon > > > > Regards, > > This is just me throwing things out to look at, but did the client mount on the old server use NFS3 and the new upgraded client uses NFS4? Sometimes that can cause weirdness with id mapping.
Re: Outlook vs Thunderbird
On 16/7/20 5:54 am, Benny Pedersen wrote: FWIW I meant if the client is Windows7/old-Outlook then changing either 993/SSL or 143/STARTTLS to 143/NONE could help pick up the mail. windows 7 just need tls 1.0, why its need to disabled all, is as well beyong me, do not disable tls 1.0 in dovecot aslong one have windows 7 clients Would anyone with Windows7 clients be able to provide me with the EXACT set of ssl_* settings that should work with W7 please? I tried for a week with various combinations but nothing worked short of disabling SSL altogether. These are the remnants of some attempts... # 20200531 suggested by Aki Tuomi #ssl_min_protocol = TLSv1.0 #ssl_ciphers = ALL:!LOW:!SSLv2:!EXP:!aNULL # https://ssl-config.mozilla.org OLD # openssl dhparam -dsaparam 1024 > /etc/dovecot/dh.pem ssl_prefer_server_ciphers = yes #ssl_min_protocol = TLSv1 #ssl_cipher_list = ECDHE-ECDSA # https://ssl-config.mozilla.org MEDIUM # openssl dhparam -dsaparam 2048 > /etc/dovecot/dh.pem #ssl_prefer_server_ciphers = no #ssl_min_protocol = TLSv1.2 #ssl_cipher_list = ECDHE-ECDSA ~ dovecot --version 2.3.7.2 (3c910f64b) Apologies to the OP for hijacking this thread. -- Mark Constable 0419 530 037 https://spiderweb.com.au
Re: IMAP flags and dovecot-keywords not working as expected
sage file in
the IMAP folder
hierarchy and the category name (following the "~") indicates which Thunderbird
tag to map it
to. I'll not include the bash script to mass-append IMAP flags to these files
as that script
will need some revising based on my recent experimentation, but should be a
rather simple bash
exercise in any case.
Note that the Outlook messages are also the same MAPI files, only the client
used (Outlook
versus Thunderbird) are different. Outlook does not set IMAP flags to designate
categories.
Categories are apparently stored in the user's .pst file.
CUT--
Public Sub ListOutlookFolders()
Dim olApp As Outlook.Application
Dim olNamespace As Outlook.Namespace
Dim olFolder As Outlook.MAPIFolder
Set olApp = New Outlook.Application
Set olNamespace = olApp.GetNamespace("MAPI")
For Each olFolder In olNamespace.Folders
Debug.Print olFolder.Name; ":", olFolder.Description
ListFolders olFolder, 1
Next
Set olFolder = Nothing
Set olNamespace = Nothing
Set olApp = Nothing
End Sub
Sub ListFolders(myFolder As Outlook.MAPIFolder, Level As Integer)
Dim olFolder As Outlook.MAPIFolder
' go through each email
scanFolder myFolder
' Now we'll check for subfolders
For Each olFolder In myFolder.Folders
'Debug.Print ":"; String(Level * 2, "-"); olFolder.Name
'go through each email
scanFolder olFolder
If olFolder.Folders.Count > 0 Then
ListFolders olFolder, Level + 1
End If
Next
End Sub
Sub scanFolder(sFolder As Outlook.MAPIFolder)
Dim src As Folder
Dim oItem As Object
Dim propertyAccessor As Outlook.propertyAccessor
Set src = sFolder
Dim strHeader As String
For Each oItem In src.Items
If TypeOf oItem Is Outlook.MailItem And oItem.Categories <> "" Then
'Debug.Print "Cat: " + oItem.Categories
Set propertyAccessor = oItem.propertyAccessor
header =
propertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x007D001E";)
Dim headerLines() As String
headerLines() = Split(header, vbCrLf)
Dim thisHeader As Variant
For Each thisHeader In headerLines
If InStr(thisHeader, "Message-ID:") > 0 Then
Debug.Print thisHeader + "~" + oItem.Categories
Exit For
End If
Next
End If
Next
End Sub
--CUT-
Example of resulting output:
Message-ID: <201109011105.p81b5666028...@webserver.ohprs.org>~Red Category
Hopefully someone finds this useful.
THX --Mark
-Original Message-
> Subject: Re: IMAP flags and dovecot-keywords not working as expected
> To: dovecot@dovecot.org
> From: Peter Chiochetti
> Date: Sat, 30 Jul 2016 11:26:09 +0200
>
> Am 2016-07-30 um 08:00 schrieb Mark Foley:
> ?
> >
> > However, none of the tags show up correctly in Thunderbird. If I manually
> > set a message to
> > have a tag of 0, the corresponding IMAP file gets a flag of 'm', not 'a'
> > and the following is
> > added to the dovecot-keywords files:
> >
> > 12 $label1
> >
> > How can I fix this? Where is "$label1" text defined? Why did Thunderbird
> > not snag the text for
> > '0' from the dovecot-keywords file and give the IMAP file a tag of 'a'?
>
> Thunderbird flags are stored in the users prefs.js, eg:
> - user_pref("mailnews.tags.$label1.tag", "Important");
> - user_pref("mailnews.tags.$label1.color", "#FF");
>
> A kind of key->value assignment. The "$label[1-9]" keys are special,
> where the number magically corresponds to the keyboard shorcut to tag
> messages, 0 meaning clear all tags.
>
> There can be more than nine tags, but they wont have a shortcut then:
> - user_pref("mailnews.tags.ten.tag", "ten");
>
> 1) The server will only ever see the key. The user will only ever see
> the value.
>
> 2) If you rename a label in TB, then only the value will change and the
> server will still see the same key as before.
>
> 3) If you rename a key in dovecot, TB will not create a label for it and
> the affected messages will no longer appear tagged, if TB does not know
> about the key.
>
> 4) Dovecot adds to the keywords as it receives requests from clients:
> Very likely there is a limit of 26 (letters of the alphabet) per
> account; a-d=0-3 are already taken for internal use, so 22 remain.
>
>
> > My current theory is that the "Default" Thunderbird Tags corresponding to
> > IMAP flags are not
> > changeable and if new tags are create in Tbird, they get new flag letters.
> > That would, of
> > course, mean that if a user changes Thunderbird tag name, they would lose
> > all tag settings on
> > existing message. That doesn't seem right and I hope my theory is wrong.
>
> I think you are mostly wrong: as long as you only use TB to work and as
> long as you do not exceed the limit you should be save.
>
> Notice that tags are a scarce resource: any key you ever created counts
> toward the limit - reusing old tags requires you to text-edit both
> dovecot-keywords and TB prefs.js.
>
> --
> peter
>
Re: Implementing secondary quota w/ "Archive" namespace
On Sun, Nov 20, 2016 at 3:28 PM, Fred Turner wrote:
> Hey Everybody—
>
> Posted this to the list a couple of months ago, but didn’t get any
> responses. Is there a better place to ask this question about quota &
> namespace configuration? Seems like a lot of the discussion here is a
> little deeper/lower-level than my configuration question, like debugging
> and development…
>
> Thx,
> Fred
>
>
> > On Sep 20, 2016, at 02:28 PM, Fred Turner wrote:
> >
> > Hello folks—
> >
> > My first post, so please be gentle… :-)
> >
> > I have a client email server using SSDs for primary user mailboxes, but
> since the number of users keeps growing and they all seem to be very
> reluctant to delete anything, I’ve implemented an “Archive” namespace that
> stores its mailboxes on a larger HD RAID. The idea is that, as the users
> approach their quota, they move messages to the Archive mailboxes to
> alleviate space in their primary Inbox namespace. This secondary storage
> part is working well, but I’m having trouble w/ getting the quotas to work
> right. Here are the basics of the setup:
> >
> > Mac Pro Server 2012
> > Mac OS X Server 10.6.8
> > Dovecot 1.1.20apple0.5
> >
> > Here is how I’ve configured my namespaces (during testing):
> >
> > namespace private {
> > separator = /
> > prefix =
> > inbox = yes
> > }
> >
> > namespace private {
> > separator = /
> > prefix = testArchive/
> > location = maildir:/Shared Items/MailArchive/%u
> > subscriptions = yes
> > }
> >
> > My quota research has led me to try this:
> >
> > quota = maildir:User quota:ns=
> >
> > quota2 = maildir:ns=testArchive/
> > quota2_rule = *:storage=20G
> >
> > The first line is already in the default config, with the exception of
> the added “:ns=“ at the end. The 2nd line in the examples I saw had a
> middle component w/ the quota name, but when I tried that, like so:
> >
> > quota2 = maildir:Archive quota:ns=testArchive/
> >
> > my server fails and shows this in the logs:
> >
> >> Fatal: IMAP(*): Quota root test backend maildir: Unknown parameter:
> ns=testArchive/
> >
> >
> > Any idea why it doesn’t like that? Also, do I need to add a quota_rule
> for the primary quota? It does not have one normally in the Mac OS X Server
> config…
> >
> > Thus far in my testing, I’ve been able to get the 2 quotas to show up in
> Roundcube and Mac Mail.app. It’s a little messy…the first shows up as “User
> quota”, the 2nd as “ns=testArchive/“, presumably because I cannot leave the
> description field in there.
> >
> > Unfortunately, both quotas show the same amount of space in use. If I
> drop the primary quota to a mere 4MB for testing, and if I have 5.2MB of
> messages in a testArchive folder, the space used for “User quota” shows as
> 5.2MB (>100%), as does the “ns=testArchive/“ quota (which is 20GB). In
> actuality, the Inbox namespace is really only using a few KB— the 5.2MB is
> in the testArchive namespace. This means that I cannot move messages
> between either set of namespaces, and new messages are not delivered. So,
> the quota trouble here is negating the whole point of having the Archive
> namespace...
> >
> > Is there a way to get Dovecot to “see” the 2 quotas as unique/discrete?
> It seems like I’m close to accomplishing what I want, but just can’t quite
> get it to cooperate. And that “Unknown parameter” error is bewildering. Any
> ideas?
> >
> > Thx,
> > Fred
> >
> > P.S. I can add my Dovecot config to the thread upon request…didn’t want
> to make this initial message even longer.
>
I beat my head against basically the same wall a few years back (and
similarly felt like I was almost in reach but could never quite get it
working), so I'm highly interested in the same topic. But I'd love to hear
from someone smarter than me if this is even possible. I don't mind beating
my head against a wall if it's not for no reason.
Can anyone verify if this is even possible? Timo?
Re: Implementing secondary quota w/ "Archive" namespace
On Mon, Nov 21, 2016 at 6:20 PM, Fred Turner wrote:
> Yeah, I gradually figured out it wouldn't work yesterday when delving back
> into this and testing. No separate quotas per namespaces until 2.1 or
> something, I think?
>
> So, got any suggestions on getting it to work with v2.x? I found an old
> thread from 2013 by Andreas (I think?) and he didn't seem to quite be able
> to get it to work. Actually, though, I'd be happy to even be able to apply
> a quota to the primary Inbox namespace and none to the secondary "Archive"
> namespace, but my testing on a 10.10 Server wasn't having much success
> either.
>
> Thanks for the responses and input!
> Fred
>
> > On Nov 21, 2016, at 17:53, Timo Sirainen wrote:
> >
> >> On 20 Sep 2016, at 21.28, Fred Turner wrote:
> >>
> >> Mac Pro Server 2012
> >> Mac OS X Server 10.6.8
> >> Dovecot 1.1.20apple0.5
> >
> > That's an old one..
> >
> >> quota = maildir:User quota:ns=
> >>
> >> quota2 = maildir:ns=testArchive/
> >> quota2_rule = *:storage=20G
> >>
> >> The first line is already in the default config, with the exception of
> the added “:ns=“ at the end. The 2nd line in the examples I saw had a
> middle component w/ the quota name, but when I tried that, like so:
> >>
> >> quota2 = maildir:Archive quota:ns=testArchive/
> >>
> >> my server fails and shows this in the logs:
> >>
> >>> Fatal: IMAP(*): Quota root test backend maildir: Unknown parameter:
> ns=testArchive/
> >>
> >>
> >> Any idea why it doesn’t like that? Also, do I need to add a quota_rule
> for the primary quota? It does not have one normally in the Mac OS X Server
> config…
> >
> > You're trying to use Dovecot v2.x configuration in Dovecot v1.x. Sorry,
> won't work without upgrade.
>
So I've been playing with this and I mostly have things working. It's
2.2.26.0, btw. In all the below, both namespaces are working and I can
copy/move messages back and forth between them.
One thing that I've not figured out yet (though I'm sure I'm just missing
something scouring the docs):
If I move messages between namespaces, it appears to ignore the quotas I've
set on them. A *copy* will trigger the quota error. But a *move* just
happily piles on to the overquota namespace. Is that normal?
E.g., here's the maildirsize from the 'archive' namespace (with quotas set
absurdly low for testing) and I just moved some messages into it from INBOX:
2S,10C
32252 31
2809 1
and it'll just keep tacking on. As you can see it's over on bytes and # of
messages. But it will successfully block a copy. This behavior of ignoring
the quota for moves goes in both directions, from INBOX to 'archive' and
vice versa.
And note that the values above are what I set, so it *is* seeing the quota
just fine (and like I said, when I copy a message, it gets appropriately
blocked due to quota).
Is this the normal behavior for message moves?
Oh, and it's definitely a move:
A0004 UID MOVE 180 Archive.archive1..
* OK [COPYUID 1268932143 180 53] Moved UIDs...* 69 EXPUNGE..A0004 OK Move
completed (0.042 + 0.000 + 0.041 secs)...
BTW, since I spent a good deal of time before I figured this out, if you're
using SQL prefetch, the syntax for overrding the location in passdb
password_query becomes (with the example ns of 'archive'):
userdb_namespace/archive/location
instead of
namespace/archive/location
I couldn't for the life of me figure out why dovecot was
ignoring 'namespace/archive/location'. Writing this email helped me figure
it out, as usual :)
=
doveconf -n:
# 2.2.26 (54d6540): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 3.14.77 x86_64 Ubuntu 12.04.5 LTS
auth_cache_negative_ttl = 1 mins
auth_cache_size = 10 M
auth_cache_ttl = 10 mins
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login
base_dir = /var/run/dovecot/
debug_log_path = /var/log/dovecot/debug.log
default_client_limit = 3005
default_internal_user = doveauth
default_process_limit = 1500
deliver_log_format = M=%m, F=%f, S="%s" B="%p/%w" => %$
disable_plaintext_auth = no
first_valid_uid = 199
imap_capability = +UNSELECT
last_valid_uid = 201
listen = *
log_path = /var/log/dovecot/mail.log
mail_debug = yes
mail_location = maildir:~/Maildir
mail_nfs_storage = yes
mail_privileged_group = mail
mail_uid = 200
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date index ihave
duplicate mime foreverypart extracttext
namespace {
hidden = no
inbox = yes
list = yes
location =
prefix = INBOX.
separator = .
subscriptions = yes
type = private
}
namespace archive {
inbox = no
list = children
location = maildir:~/Archive
prefix = Archive.
separator = .
subscriptions = yes
type = private
}
passdb {
args = /etc/dovecot/include/sql.conf
driver = sql
}
plugin {
quota = mail
Re: Implementing secondary quota w/ "Archive" namespace
On Wed, Nov 23, 2016 at 6:05 PM, Timo Sirainen wrote:
> On 23 Nov 2016, at 0.49, Mark Moseley wrote:
> >
> > If I move messages between namespaces, it appears to ignore the quotas
> I've
> > set on them. A *copy* will trigger the quota error. But a *move* just
> > happily piles on to the overquota namespace. Is that normal?
>
> Probably needs a bit more thinking, but I guess the attached patch would
> help.
>
>
I appreciate the patch! Esp on a Weds night. I applied and rerolled
dovecot, but I can still move messages into the over-quota namespace.
I threw some i_debug's into quota_roots_equal() (and one right at the
top), but I don't ever see them in the debug logs. But both "ctx->moving"
and "src_box == NULL" are true, so it never calls quota_roots_equal anyway
in that patched 'if' clause in quota_check. I threw the following into
quota_check and it printed to the debug log for both if's:
if (ctx->moving ) i_debug("quota: quota_check: YES to ctx->moving"
);
if (src_box == NULL) i_debug("quota: quota_check: YES to src_box ==
NULL" );
Out of curiosity, in the Quota wiki page, it mentions that 'in theory there
could be e.g. "user quota" and "domain quota" roots'. That's also super
interesting to me. Does anyone have any experience with that? I.e. any
gotchas?
Re: Implementing secondary quota w/ "Archive" namespace
On Thu, Nov 24, 2016 at 10:52 AM, Timo Sirainen wrote: > On 24 Nov 2016, at 9.33, Mark Moseley wrote: > > > > On Wed, Nov 23, 2016 at 6:05 PM, Timo Sirainen wrote: > > > >> On 23 Nov 2016, at 0.49, Mark Moseley wrote: > >>> > >>> If I move messages between namespaces, it appears to ignore the quotas > >> I've > >>> set on them. A *copy* will trigger the quota error. But a *move* just > >>> happily piles on to the overquota namespace. Is that normal? > >> > >> Probably needs a bit more thinking, but I guess the attached patch would > >> help. > >> > >> > > I appreciate the patch! Esp on a Weds night. I applied and rerolled > > dovecot, but I can still move messages into the over-quota namespace. > > How about this updated patch? > > Nope, still lets me move messages into the over-quota namespace. Both these are true in quota_check: ctx->moving quota_move_requires_check > > Out of curiosity, in the Quota wiki page, it mentions that 'in theory > there > > could be e.g. "user quota" and "domain quota" roots'. That's also super > > interesting to me. Does anyone have any experience with that? I.e. any > > gotchas? > > > There's no automatic quota recalculation for domain quotas, because it > would have to somehow sum up all the users' quotas. Also I think that it > still does do the automatic quota recalculation if it gets into a situation > where it realizes that quotas are wrong, but it'll then just use the single > user's quota as the entire domain quota. So maybe it would work if you > externally sum up all the users' quotas and update it to the domain quota > in cronjob, e.g. once per hour. I guess it would be also nice if the > internal quota recalculation could be disabled and maybe execute an > external script to do it (similar to quota-warnings). > > > > > >
Re: Implementing secondary quota w/ "Archive" namespace
On Thu, Nov 24, 2016 at 9:10 PM, Mark Moseley wrote: > On Thu, Nov 24, 2016 at 10:52 AM, Timo Sirainen wrote: > >> On 24 Nov 2016, at 9.33, Mark Moseley wrote: >> > >> > On Wed, Nov 23, 2016 at 6:05 PM, Timo Sirainen wrote: >> > >> >> On 23 Nov 2016, at 0.49, Mark Moseley wrote: >> >>> >> >>> If I move messages between namespaces, it appears to ignore the quotas >> >> I've >> >>> set on them. A *copy* will trigger the quota error. But a *move* just >> >>> happily piles on to the overquota namespace. Is that normal? >> >> >> >> Probably needs a bit more thinking, but I guess the attached patch >> would >> >> help. >> >> >> >> >> > I appreciate the patch! Esp on a Weds night. I applied and rerolled >> > dovecot, but I can still move messages into the over-quota namespace. >> >> How about this updated patch? >> >> > Nope, still lets me move messages into the over-quota namespace. > > Both these are true in quota_check: > > ctx->moving > quota_move_requires_check > > > > >> > Out of curiosity, in the Quota wiki page, it mentions that 'in theory >> there >> > could be e.g. "user quota" and "domain quota" roots'. That's also super >> > interesting to me. Does anyone have any experience with that? I.e. any >> > gotchas? >> >> >> There's no automatic quota recalculation for domain quotas, because it >> would have to somehow sum up all the users' quotas. Also I think that it >> still does do the automatic quota recalculation if it gets into a situation >> where it realizes that quotas are wrong, but it'll then just use the single >> user's quota as the entire domain quota. So maybe it would work if you >> externally sum up all the users' quotas and update it to the domain quota >> in cronjob, e.g. once per hour. I guess it would be also nice if the >> internal quota recalculation could be disabled and maybe execute an >> external script to do it (similar to quota-warnings). >> >> Anything else I can try? I'm not sure how the logic in the quota system works, so I'm not sure what to suggest. What's the gist of the patch (i.e. what's it trying to do that it wasn't before)? If I can get a handle on that, I can start littering things with debug statements to try to track stuff down.
Re: Implementing secondary quota w/ "Archive" namespace
On Thu, Dec 1, 2016 at 4:37 AM, Timo Sirainen wrote: > On 1 Dec 2016, at 2.22, Mark Moseley wrote: > > > How about this updated patch? > > > Nope, still lets me move messages into the over-quota namespace. > > Both these are true in quota_check: > > ctx->moving > quota_move_requires_check > > .. > > Anything else I can try? I'm not sure how the logic in the quota system > works, so I'm not sure what to suggest. What's the gist of the patch (i.e. > what's it trying to do that it wasn't before)? > > If I can get a handle on that, I can start littering things with debug > statements to try to track stuff down. > > > I just messed up the if-check. This one is now committed and should work: > https://github.com/dovecot/core/commit/2ec4ab6f5a1172e86afc72c0f29f47 > 0d6fd2bd9a.diff > > that looks good. When I apply it, I get: quota-storage.c: In function ‘quota_save_finish’: quota-storage.c:337:15: error: ‘struct mail_save_context’ has no member named ‘copy_src_mail’ quota-storage.c:337:51: error: ‘struct mail_save_context’ has no member named ‘copy_src_mail’ make[4]: *** [quota-storage.lo] Error 1 But if I then also apply the previous patch you gave, though it fails in a number of sections: # patch -p1 < ~moseley/diff2 (Stripping trailing CRs from patch.) patching file src/lib-storage/mail-storage-private.h (Stripping trailing CRs from patch.) patching file src/lib-storage/mail-storage.c Hunk #1 succeeded at 2238 (offset -20 lines). Hunk #2 succeeded at 2255 (offset -20 lines). (Stripping trailing CRs from patch.) patching file src/plugins/quota/quota-storage.c Hunk #1 FAILED at 185. Hunk #2 FAILED at 242. Hunk #3 FAILED at 297. 3 out of 3 hunks FAILED -- saving rejects to file src/plugins/quota/quota-storage.c.rej BUT, it then compiles. I haven't tested it extensively, but with this latest patch, when I try to move mail to the over-quota Archive mailbox, it correctly fails! Awesome!
Domains on different IPs
I want to supply separate Letsencrypt certificates for each virtual domain and seeing that SNI does not work I need to allocate separate IPs. Could anyone give some pointers, or keywords to search for, on... a) how to make dovecot listen for different domains on different IPs? b) how to configure separate SSL certs for each of these IPs?
Re: dovecot-pigeonhole and 2.2.27
On 09/01/17 02:29, Aki Tuomi wrote: There were some non-compatible changes in 2.2.27 that makes older versions of pigeonhole incompatible, when external programs are used. We moved the program-client code from pigeonhole to dovecot core and made some changes to it to facilitate other uses for it. I'm testing with ubuntu zesty and it's still at dovecot 2.2.25 so where might I find the docs for the relevant changes in 2.2.27 that will no doubt affect me in another month or three? And would anyone have an example of how to use dovecot-pigeonhole to call spamprobe as an external program in the current 2.2.25? I've only been testing postfix/dovecot for a few weeks so I'm not familiar with most of the docs or ecosystem in general.
pigeonhole + spamprobe
Would anyone have an example of how to use dovecot-pigeonhole to call spamprobe as an external program in the current 2.2.25?
Apparent Maildir permission issue
I've just upgraded from Slackware 14.1 to 14.2. I've not done anything with dovecot -- it's the same version that was running before the upgrade. However, now I'm getting a permission error: /var/log/maillog: Jan 16 13:09:44 mail dovecot: imap(mark): Error: opendir(/home/HPRS/mark/Maildir) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +r perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail last message repeated 4 times Jan 16 13:09:44 mail dovecot: imap(mark): Error: stat(/home/HPRS/mark/Maildir/.Trash) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +x perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail dovecot: imap(mark): Error: stat(/home/HPRS/mark/Maildir/tmp) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +x perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail dovecot: imap(mark): Error: opendir(/home/HPRS/mark/Maildir) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +r perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:45 mail last message repeated 11 times Permission on that folder are: $ ls -ld /home/HPRS/mark/Maildir drwx-- 17 HPRS\mark domusers 4096 Dec 7 23:07 /home/HPRS/mark/Maildir/ Permissions are unchanged since before the backup. What do I do to fix this? THX --Mark
Re: Apparent Maildir permission issue
More info ... This is the only user having this permission problem. All other Thunderbird/dovecot users are getting mail file. They all have the same permissions set on their Maildir folder. --Mark -Original Message- From: Mark Foley Date: Mon, 16 Jan 2017 13:21:31 -0500 Organization: Ohio Highway Patrol Retirement System To: dovecot@dovecot.org Subject: Apparent Maildir permission issue I've just upgraded from Slackware 14.1 to 14.2. I've not done anything with dovecot -- it's the same version that was running before the upgrade. However, now I'm getting a permission error: /var/log/maillog: Jan 16 13:09:44 mail dovecot: imap(mark): Error: opendir(/home/HPRS/mark/Maildir) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +r perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail last message repeated 4 times Jan 16 13:09:44 mail dovecot: imap(mark): Error: stat(/home/HPRS/mark/Maildir/.Trash) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +x perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail dovecot: imap(mark): Error: stat(/home/HPRS/mark/Maildir/tmp) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +x perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail dovecot: imap(mark): Error: opendir(/home/HPRS/mark/Maildir) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +r perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:45 mail last message repeated 11 times Permission on that folder are: $ ls -ld /home/HPRS/mark/Maildir drwx-- 17 HPRS\mark domusers 4096 Dec 7 23:07 /home/HPRS/mark/Maildir/ Permissions are unchanged since before the backup. What do I do to fix this? THX --Mark
Re: Apparent Maildir permission issue
On Mon, 16 Jan 2017 17:51:48 -0500 Bill Shirley wrote: > > I've gotten errors like this when it was actually a selinux denial. If you're > running > selinux, check those logs too. > OK, this is getting serious -- mail not getting delivered. No, I am not running selinux. Here is the error I get in the maillog: Jan 24 16:42:49 mail dovecot: imap(mark): Error: stat(/home/HPRS/mark/Maildir/tmp) failed: Permission denied (euid=326(HPRS\mark) egid=100(users) missing +x perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Permission are: $ ls -l /home/HPRS/mark/Maildir/ total 200 drwx-- 2 HPRS\mark domusers 45056 Dec 19 08:13 cur/ -rw--- 1 HPRS\mark domusers 131 Jul 1 2016 dovecot-keywords -rw--- 1 HPRS\mark domusers 5249 Dec 7 23:06 dovecot-uidlist -rw--- 1 HPRS\mark domusers 8 Jul 7 2016 dovecot-uidvalidity -r--r--r-- 1 HPRS\mark domusers 0 Jan 16 2015 dovecot-uidvalidity.54b9def3 -rw--- 1 HPRS\mark domusers 4080 Nov 27 23:28 dovecot.index -rw--- 1 HPRS\mark domusers 88612 Dec 7 23:07 dovecot.index.cache -rw--- 1 HPRS\mark domusers 8748 Dec 7 23:07 dovecot.index.log -rw--- 1 HPRS\mark domusers 2016 Jul 7 2016 dovecot.mailbox.log drwx-- 2 HPRS\mark domusers 12288 Jan 13 12:10 new/ -rw--- 1 HPRS\mark domusers 137 Jul 7 2016 subscriptions drwx-- 2 HPRS\mark domusers 12288 Jan 13 12:10 tmp/ Permission on the Maildir folder for another user who is NOT having this problem: $ ls -l /home/HPRS/shay/Maildir/ total 88 drwx-- 2 HPRS\shay domusers 12288 Jan 24 15:50 cur/ -rw--- 1 HPRS\shay domusers41 Sep 13 11:59 dovecot-keywords -rw--- 1 HPRS\shay users 1442 Jan 24 15:48 dovecot-uidlist -rw--- 1 HPRS\shay domusers 8 Jan 18 15:13 dovecot-uidvalidity -r--r--r-- 1 HPRS\shay domusers 0 Jul 15 2016 dovecot-uidvalidity.5789a8ca -rw--- 1 HPRS\shay users 1408 Jan 20 08:18 dovecot.index -rw--- 1 HPRS\shay users12928 Jan 24 15:50 dovecot.index.cache -rw--- 1 HPRS\shay users20844 Jan 24 15:51 dovecot.index.log -rw--- 1 HPRS\shay domusers 2856 Jan 18 15:13 dovecot.mailbox.log drwx-- 2 HPRS\shay domusers 4096 Jan 24 15:48 new/ -rw--- 1 HPRS\shay users 2906 Jan 18 15:13 subscriptions drwx-- 2 HPRS\shay domusers 4096 Jan 24 15:48 tmp/ You can see that the tmp/ folders for both users are set exactly the same, yet user 'mark' is getting the permission error. mark's mail is not getting delivered; shay's mail is. Why? > On 1/16/2017 4:09 PM, Mark Foley wrote: > > More info ... > > > > This is the only user having this permission problem. All other > > Thunderbird/dovecot users are > > getting mail file. They all have the same permissions set on their Maildir > > folder. > > > > --Mark > > > > -Original Message- > > From: Mark Foley > > Date: Mon, 16 Jan 2017 13:21:31 -0500 > > Organization: Ohio Highway Patrol Retirement System > > To: dovecot@dovecot.org > > Subject: Apparent Maildir permission issue > > > > I've just upgraded from Slackware 14.1 to 14.2. I've not done anything with > > dovecot -- it's the > > same version that was running before the upgrade. However, now I'm getting > > a permission error: > > > > /var/log/maillog: > > > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > opendir(/home/HPRS/mark/Maildir) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +r perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:44 mail last message repeated 4 times > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > stat(/home/HPRS/mark/Maildir/.Trash) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +x perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > stat(/home/HPRS/mark/Maildir/tmp) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +x perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > opendir(/home/HPRS/mark/Maildir) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +r perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:45 mail last message repeated 11 times > > > > Permission on that folder are: > > > > $ ls -ld /home/HPRS/mark/Maildir > > drwx-- 17 HPRS\mark domusers 4096 Dec 7 23:07 /home/HPRS/mark/Maildir/ > > > > Permissions are unchanged since before the backup. > > > > What do I do to fix this? > > > > THX --Mark >
Re: Apparent Maildir permission issue
On Tue, 17 Jan 2017 12:25:27 +0200 Aki Tuomi wrote: > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > opendir(/home/HPRS/mark/Maildir) failed: Permission denied > (euid=326(HPRS\mark) egid=100(users) missing +r perm: > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > > Just wanted to point out that you have at different UID for the folder > than your EUID (gotten from userdb/passdb). > > Aki > Yes, very puzzling. I'm restoring some older dovecot logs now to see if that was true e.g. in 2016. Perhaps an upgrade of some other software caused a problem. On the other hand, the other user I mentioned in my Jan 24 17:15 message, shay, also shows this UID/EUID discrepancy, but that does not prevent her from getting mail and there is no permission denied error on her messages. More when I know more --Mark > On 16.01.2017 23:09, Mark Foley wrote: > > More info ... > > > > This is the only user having this permission problem. All other > > Thunderbird/dovecot users are > > getting mail file. They all have the same permissions set on their Maildir > > folder. > > > > --Mark > > > > -Original Message- > > From: Mark Foley > > Date: Mon, 16 Jan 2017 13:21:31 -0500 > > Organization: Ohio Highway Patrol Retirement System > > To: dovecot@dovecot.org > > Subject: Apparent Maildir permission issue > > > > I've just upgraded from Slackware 14.1 to 14.2. I've not done anything with > > dovecot -- it's the > > same version that was running before the upgrade. However, now I'm getting > > a permission error: > > > > /var/log/maillog: > > > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > opendir(/home/HPRS/mark/Maildir) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +r perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:44 mail last message repeated 4 times > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > stat(/home/HPRS/mark/Maildir/.Trash) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +x perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > stat(/home/HPRS/mark/Maildir/tmp) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +x perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:44 mail dovecot: imap(mark): Error: > > opendir(/home/HPRS/mark/Maildir) failed: Permission denied > > (euid=326(HPRS\mark) egid=100(users) missing +r perm: > > /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) > > Jan 16 13:09:45 mail last message repeated 11 times > > > > Permission on that folder are: > > > > $ ls -ld /home/HPRS/mark/Maildir > > drwx-- 17 HPRS\mark domusers 4096 Dec 7 23:07 /home/HPRS/mark/Maildir/ > > > > Permissions are unchanged since before the backup. > > > > What do I do to fix this? > > > > THX --Mark >
Re: Apparent Maildir permission issue
On Wed, 25 Jan 2017 08:01:00 +0100 (CET) Steffen Kaiser wrote: > 1) Why does both UIDs 326 and 10001 translate back to HPRS\mark ? > What HPRS\mark translates to? > > > Permission on that folder are: > > > > $ ls -ld /home/HPRS/mark/Maildir > > drwx-- 17 HPRS\mark domusers 4096 Dec 7 23:07 /home/HPRS/mark/Maildir/ > > 2) I guess this HPRS\mark is 10001 ? (And not 326) > > > Permissions are unchanged since before the backup. > > "backup"? You've restored the Maildir's from somewhere else? What was the > _numerical_ UID within the backup and what is it now? "backup" meaning I looked at the permissions on an older routine, backup. No, I did not restore anything. BUT ... I found the problem. I upgraded Samba4 10 days ago from version 4.2.12 to 4.4.8 and, in the course of researching this problem, I found that the A/D authentication was broken: with 4.2.12 on AD/DC: $ getent passwd mark HPRS\mark:*:10001:1:Mark Foley:/home/HPRS/mark:/bin/false With 4.4.8 on AD/DC: $ getent passwd mark HPRS\mark:*:326:100:Mark Foley:/home/HPRS/mark:/bin/bash The new version of Samba is giving me this bogus UID:GID. I've no idea why. I have posted messages on the Samba List asking for help on this. Email clients authenticate with Dovecot via Kerberos/GSSAPI and Dovecot was therefore trying to use 326:100 to access Maildir files/directories created with owner 10001:1. I've done a workaround by added the correct UID, GID for this user to /etc/passwd, although one is not suppose to have AD users in /etc/passwd. However, that is working for the time being. If anyone on this list has had this experience and knows what needs to be fixed, please let me know! Thanks -- Mark
Director+NFS Experiences
As someone who is about to begin the process of moving from maildir to mdbox on NFS (and therefore just about to start the 'director-ization' of everything) for ~6.5m mailboxes, I'm curious if anyone can share any experiences with it. The list is surprisingly quiet about this subject, and articles on google are mainly just about setting director up. I've yet to stumble across an article about someone's experiences with it. * How big of a director cluster do you use? I'm going to have millions of mailboxes behind 10 directors. I'm guessing that's plenty. It's actually split over two datacenters. In the larger, we've got about 200k connections currently, so in a perfectly-balanced world, each director would have 20k connections on it. I'm guessing that's child's play. Any good rule of thumb for ratio of 'backend servers::director servers'? In my larger DC, it's about 5::1. * Do you use the perl poolmon script or something else? The perl script was being weird for me, so I rewrote it in python but it basically does the exact same things. * Seen any issues with director? In testing, I managed to wedge things by having my poolmon script running on all the cluster boxes (I think). I've since rewritten it to run *only* on the lowest-numbered director. When it wedged, I had piles (read: hundreds per second) of log entries that said: Feb 12 06:25:03 director: Warning: director(10.1.20.5:9090/right): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.5:9090/right): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Because it was in testing, I didn't notice it and it was like this for several days till dovecot was restarted on all the director nodes. I'm not 100% on what happened, but my *guess* is that two boxes tried to update the status of the same backend server in rapid succession. * Assuming you're using NFS, do you still see non-trivial amounts of indexes getting corrupted? * Again, assuming NFS and assuming at least some corrupted indexes, what's your guess for success rate % for dovecot recovering them automatically? And how about success rate % for ones that dovecot wasn't able to do automatically but you had to use doveadm to repair it? Really what I'm trying to figure out is 1) how often sysops will need to manually recover indexes; and 2) how often admins *can't* manually recover indexes? * if you have unrecoverable indexes (and assuming you have snapshots on your NFS server), does grabbing the most recent indexes from the snapshots always work for recovery (obviously, up till the point that the snapshot was taken)? * Any gotchas you've seen anywhere in a director-fied stack? I realize that's a broad question :) * Does one of your director nodes going down cause any issues? E.g. issues with the left and right nodes syncing with each other? Or when the director node comes back up? * Does a backend node going down cause a storm of reconnects? In the time between deploying director and getting mailboxes converted to mdbox, reconnects for us will mean cold local-disk dovecot caches. But hopefully consistent hashing helps with that? * Do you have consistent hashing turned on? I can't think of any reason not to have it turned on, but who knows * Any other configuration knobs (including sysctl) that you needed to futz with, vs the default? I appreciate any feedback!
Re: Director+NFS Experiences
On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen wrote: > On 24 Feb 2017, at 0.08, Mark Moseley wrote: > > > > As someone who is about to begin the process of moving from maildir to > > mdbox on NFS (and therefore just about to start the 'director-ization' of > > everything) for ~6.5m mailboxes, I'm curious if anyone can share any > > experiences with it. The list is surprisingly quiet about this subject, > and > > articles on google are mainly just about setting director up. I've yet to > > stumble across an article about someone's experiences with it. > > > > * How big of a director cluster do you use? I'm going to have millions of > > mailboxes behind 10 directors. > > I wouldn't use more than 10. > > Cool > > I'm guessing that's plenty. It's actually split over two datacenters. > > Two datacenters in the same director ring? This is dangerous. if there's a > network connectivity problem between them, they split into two separate > rings and start redirecting users to different backends. > I was unclear. The two director rings are unrelated and won't ever need to talk to each other. I only mentioned the two rings to point out that all 6.5m mailboxes weren't behind one ring, but rather split between two > > > * Do you have consistent hashing turned on? I can't think of any reason > not > > to have it turned on, but who knows > > Definitely turn it on. The setting only exists because of backwards > compatibility and will be removed at some point. > > Out of curiosity (and possibly extremely naive), unless you've moved a mailbox via 'doveadm director', if someone is pointed to a box via consistent hashing, why would the directors need to share that mailbox mapping? Again, assuming they're not moved (I'm also assuming that the mailbox would always, by default, hash to the same value in the consistent hash), isn't their hashing all that's needed to get to the right backend? I.e. "I know what the mailbox hashes to, and I know what backend that hash points at, so I'm done", in which case, no need to communicate to the other directors. I could see that if you moved someone, it *would* need to communicate that mapping. Then the only maps traded by directors would be the consistent hash boundaries *plus* any "moved" mailboxes. Again, just curious.
Re: Director+NFS Experiences
On Thu, Feb 23, 2017 at 3:45 PM, Zhang Huangbin wrote: > > > On Feb 24, 2017, at 6:08 AM, Mark Moseley wrote: > > > > * Do you use the perl poolmon script or something else? The perl script > was > > being weird for me, so I rewrote it in python but it basically does the > > exact same things. > > Would you mind sharing it? :) > > > Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ > Time zone: GMT+8 (China/Beijing). > Available on Telegram: https://t.me/iredmail > > Attached. No claims are made on the quality of my code :) poolmon Description: Binary data
Re: Director+NFS Experiences
> > On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen wrote: > >> On 24 Feb 2017, at 0.08, Mark Moseley wrote: >> > >> > As someone who is about to begin the process of moving from maildir to >> > mdbox on NFS (and therefore just about to start the 'director-ization' >> of >> > everything) for ~6.5m mailboxes, I'm curious if anyone can share any >> > experiences with it. The list is surprisingly quiet about this subject, >> and >> > articles on google are mainly just about setting director up. I've yet >> to >> > stumble across an article about someone's experiences with it. >> > >> > * How big of a director cluster do you use? I'm going to have millions >> of >> > mailboxes behind 10 directors. >> >> I wouldn't use more than 10. >> >> > Cool > > > >> > I'm guessing that's plenty. It's actually split over two datacenters. >> >> Two datacenters in the same director ring? This is dangerous. if there's >> a network connectivity problem between them, they split into two separate >> rings and start redirecting users to different backends. >> > > I was unclear. The two director rings are unrelated and won't ever need to > talk to each other. I only mentioned the two rings to point out that all > 6.5m mailboxes weren't behind one ring, but rather split between two > > > >> >> > * Do you have consistent hashing turned on? I can't think of any reason >> not >> > to have it turned on, but who knows >> >> Definitely turn it on. The setting only exists because of backwards >> compatibility and will be removed at some point. >> >> > Out of curiosity (and possibly extremely naive), unless you've moved a > mailbox via 'doveadm director', if someone is pointed to a box via > consistent hashing, why would the directors need to share that mailbox > mapping? Again, assuming they're not moved (I'm also assuming that the > mailbox would always, by default, hash to the same value in the consistent > hash), isn't their hashing all that's needed to get to the right backend? > I.e. "I know what the mailbox hashes to, and I know what backend that hash > points at, so I'm done", in which case, no need to communicate to the other > directors. I could see that if you moved someone, it *would* need to > communicate that mapping. Then the only maps traded by directors would be > the consistent hash boundaries *plus* any "moved" mailboxes. Again, just > curious. > > Timo, Incidentally, on that error I posted: Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 any idea what would cause that? Is my guess that multiple directors tried to update the status simultaneously correct?
Re: Director+NFS Experiences
On Fri, Feb 24, 2017 at 11:41 AM, Francisco Wagner C. Freire < wgrcu...@gmail.com> wrote: > In our experience. A ring with more of 4 servers is bad, we have sync > problems everyone. Using 4 or less works perfect. > > Em 24 de fev de 2017 4:30 PM, "Mark Moseley" > escreveu: > >> > >> > On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen wrote: >> > >> >> On 24 Feb 2017, at 0.08, Mark Moseley wrote: >> >> > >> >> > As someone who is about to begin the process of moving from maildir >> to >> >> > mdbox on NFS (and therefore just about to start the >> 'director-ization' >> >> of >> >> > everything) for ~6.5m mailboxes, I'm curious if anyone can share any >> >> > experiences with it. The list is surprisingly quiet about this >> subject, >> >> and >> >> > articles on google are mainly just about setting director up. I've >> yet >> >> to >> >> > stumble across an article about someone's experiences with it. >> >> > >> >> > * How big of a director cluster do you use? I'm going to have >> millions >> >> of >> >> > mailboxes behind 10 directors. >> >> >> >> I wouldn't use more than 10. >> >> >> >> >> > Cool >> > Interesting. That's good feedback. One of the things I wondered about is whether it'd be better to deploy a 10-node ring or split it into 2x 5-node rings. Sounds like splitting it up might not be a bad idea. How often would you see those sync problems (and were they the same errors as I posted or something else)? And were you running poolmon from every node when you were seeing sync errors?
Re: Faster way to import Thunderbird pop emails into dovecot imap Maildirs?
On 03/03/17 07:11, Ian Evans wrote: Some time ago Thunderbird was using mbox-style format to store messages locally. It this is true today and for your installation, you can try the mb2md scripts: http://wiki2.dovecot.org/Migration/MailFormat Recent versions of Thunderbird can store local messages in Maildir format... https://wiki.mozilla.org/Thunderbird/Maildir https://mail.mozilla.org/pipermail/tb-enterprise/2015-June/001112.html
Re: letsencrypt
On 04/03/17 04:07, David Mehler wrote: With the web it was easy just let apache serve the token that letsencrypt needed and I got certificates. How do I do this with regards email? I know there have been some answers to this already but FWIW I use dehydrated directly from Github and this script sets it up as well as creates a pem version for mail hosts... https://raw.githubusercontent.com/markc/sh/master/bin/newssl Just change WPATH, VCONF and the nginx server snippet then reload apache instead of nginx. Then put a slightly modified version of this on a monthly cronjob... https://raw.githubusercontent.com/markc/sh/master/bin/allssl
"Connection queue full" error
Just a quickie: why is "Connection queue full" logged under Info, instead of something like error? Or at least have the word 'error' in it? Seems like a pretty error-ish thing to happen. Anything that causes the connection to fail from the server side should show up in a grep -i for error. I.e. I don't care about clients failing to match up SSL cipher suites; that's fine as Info (SSL errors ironically do have 'error' in them, though I assume that's coming from the ssl libs). But the server dropping connections due to running out of available daemons (and any other "server isn't working right" conditions) is definitely worthy of Error.
Using SpamProbe via only sieve scripts
FWIW this took me days to get right, and still needs tinkering, but it might make for a good starting point for anyone else needing something similar (ie; I don't use or need the overhead of spamassassin or rspamd). https://gist.github.com/markc/eeeb66ce30ea805af62631656cf86c4d Any comments or corrections on that page would be appreciated.
Users home path inside a sieve script
Perhaps I have not looked hard enough but is it possible to get at the current users auth userdb "home" (or $HOME) variable from inside a sieve script? Example?
Re: Users home path inside a sieve script
On 04/04/17 04:00, Stephan Bosch wrote:
Perhaps I have not looked hard enough but is it possible to get at
the current users auth userdb "home" (or $HOME) variable from
inside a sieve script? Example?
What do you want to do with it?
I was hoping to replace the 2 variables below with the actual home dir
path directly instead of building up "/home/u/${rhs}/home/${lhs}/"...
require ["vnd.dovecot.execute", "fileinto", "envelope", "variables",
"editheader"];
if envelope :localpart :matches "to" "*" { set "lhs" "${1}"; }
if envelope :domain :matches "to" "*" { set "rhs" "${1}"; }
execute :pipe :output "SCORE" "spamprobe" ["-c", "-d",
"/home/u/${rhs}/home/${lhs}/.spamprobe", "receive"];
addheader :last "X-Spam" "${SCORE}";
if header :matches "X-Spam" "SPAM*" { fileinto "Spam"; }
The above works but I am concerned it's fragile and might break when
aliases and CC/BCC are taken into consideration. The "home" directory
is obviously known to LMTP at this point so I was hoping it could be
somehow exposed inside a sieve script?
Host ... is being updated before previous update had finished
We just had a bunch of backend boxes go down due to a DDoS in our director cluster. When the DDoS died down, our director ring was a mess. Each box had thousands (and hundreds per second, which is a bit much) of log lines like the following: Apr 03 19:59:29 director: Warning: director(10.1.20.10:9090/left): Host 10.1.17.15 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.10:9090/left): Host 10.1.17.15 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.10:9090/left): Host 10.1.17.15 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.10:9090/left): Host 10.1.17.15 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.10:9090/left): Host 10.1.17.15 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.2:9090/right): Host 10.1.17.15 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.2:9090/right): Host 10.1.17.15 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.2:9090/right): Host 10.1.17.15 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.2:9090/right): Host 10.1.17.15 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=100 Apr 03 19:59:29 director: Warning: director(10.1.20.2:9090/right): Host 10.1.17.15 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=100 This was on every director box and the status of all of the directors in 'doveadm director ring status' was 'handshaking'. Here's a sample packet between directors: 19:51:23.552280 IP 10.1.20.10.56670 > 10.1.20.1.9090: Flags [P.], seq 4147:5128, ack 0, win 0, options [nop,nop,TS val 1373505883 ecr 1721203906], length 981 Q. [f.|.HOST 10.1.20.10 90901006732 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006733 10.1.17.15 100 U1491260800 HOST10.1.20.10 90901006734 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006735 10.1.17.15 100 U1491260800 HOST10.1.20.10 90901006736 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006737 10.1.17.15 100 U1491260800 HOST10.1.20.10 90901006738 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006739 10.1.17.15 100 U1491260800 HOST10.1.20.10 90901006740 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006741 10.1.17.15 100 U1491260800 HOST10.1.20.10 90901006742 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006743 10.1.17.15 100 U1491260800 HOST10.1.20.10 90901006744 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006745 10.1.17.15 100 U1491260800 HOST10.1.20.10 90901006746 10.1.17.15 100 D1491260800 HOST10.1.20.10 90901006747 10.1.17.15 100 U1491260800 SYNC10.1.20.10 90901011840 7 1491263483 3377546382 I'm guessing that D1491260800 is the user hash (with D for down), and the U version is for 'up'. I'm happy to provide the full tcpdump (and/or doveconf -a), though the tcpdump is basically all identical the one I pasted (same hash, same host). This seems pretty fragile. There should be some sort of tie break for that, instead of bringing the entire cluster to its knees. Or just drop the backend host completely. Or something, anything besides hosing things pretty badly. This is 2.2.27, on both the directors and backend. If the answer is upgrade to 2.2.28, then I'll upgrade immediately. I see commit a9ade104616bbb81c34cc6f8bfde5dab0571afac mentions the same error but the commit predates 2.2.27 by a month and a half. In the meantime, is there any doveadm command I could've done to fix this? I tried removing the host (doveadm director remove 10.1.17.15) but that didn't do anything. I didn't think to try to flush the mapping for that user till just now. I suspect that with the ring unsync'd, flushing the user wouldn't have helped. The only remedy was to kill dovecot on every box in the director cluster and then (with dovecot down on *all* of them) start dovecot back up. Restarting each director's dovecot (with other directors' dovecots still running) did nothing. Only by brining the entire cluster down did dovecot stop f
