Delete vs. Expunge in Public Namespace
All,
my IMAP client [1] marks messages as deleted, but doesn't expunge.
Dovecot (2.2.27-3+deb9u2) expunges them. Is this because the mailbox is
part of public namespace?
::1.38247-::1.00143: MALP6 STORE
48,49,50,51,52,53,54,55,56,57,58,59,60,61 +FLAGS (\Deleted)
::1.00143-::1.38247: * 48 FETCH (FLAGS (\Deleted \Seen))
* 49 FETCH (FLAGS (\Deleted \Seen))
* 50 FETCH (FLAGS (\Deleted \Seen))
* 51 FETCH (FLAGS (\Deleted \Seen))
* 52 FETCH (FLAGS (\Deleted \Seen))
* 53 FETCH (FLAGS (\Deleted \Seen))
* 54 FETCH (FLAGS (\Deleted \Seen))
* 55 FETCH (FLAGS (\Deleted \Seen))
* 56 FETCH (FLAGS (\Deleted \Seen))
* 57 FETCH (FLAGS (\Deleted \Seen))
* 58 FETCH (FLAGS (\Deleted \Seen))
* 59 FETCH (FLAGS (\Deleted \Seen))
* 60 FETCH (FLAGS (\Deleted \Seen))
* 61 FETCH (FLAGS (\Deleted \Seen))
MALP6 OK Store completed.
::1.38247-::1.00143: MALP7 SEARCH DELETED
Oct 2 10:18:03 hyptest dovecot: imap(chris): delete:
box=MailingListen/test125, uid=180,
msgid=<740a9ba3-d637-3fa0-1b87-ff8d79491bf0@ hyp.postbox.xyz>,
size=1509 Oct 2 10:18:03 hyptest dovecot: imap(chris): expunge:
box=MailingListen/test125, uid=167,
msgid=<740a9ba3-d637-3fa0-1b87-ff8d79491bf0 @hyp.postbox.xyz>, size=1509
namespace {
list = yes
location =
maildir:/var/vmail/public/:LAYOUT=fs:INDEXPVT=%h/.public-indexes prefix
= MailingListen/ separator = /
subscriptions = no
type = public
}
- Chris
[1]
https://raw.githubusercontent.com/quentinsf/IMAPdedup/master/imapdedup.py
Re: Delete vs. Expunge in Public Namespace
On Wed, 3 Oct 2018 11:04:05 +0300 Timo Sirainen wrote: > If it's sending CLOSE, that explains it. It's not just "some > servers", it's all IMAP servers. There's UNSELECT command to close > without expunging. Thank you! Sorry, didn't know that. It is sending close: ::1.38247-::1.00143: MALP9 CLOSE ::1.00143-::1.38247: MALP9 OK Close completed. ::1.38247-::1.00143: MALP10 LOGOUT ::1.00143-::1.38247: * BYE Logging out MALP10 OK Logout completed. - Chris
2.3.4 doesnt compile on FreeBSD 11.2 using clang
extract below, this has already been reported a while back but still no new patch, so this email is to serve as a reminder, if someone manually fixes it for the ports tree, I dont consider that a fix, ideally we need this fixed in the source code, as not everyone will install it from ports. Chris "clang40 -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-ssl-iostream -DPKG_RUNDIR=\""/var/run/dovecot"\" -DPKG_STATEDIR=\""/var/lib/dovecot"\" -DSYSCONFDIR=\""/etc/dovecot"\" -DBINDIR=\""/usr/bin"\" -I/usr/local/include -std=gnu99 -fdiagnostics-color -Wl,-rpath=/usr/local/llvm40/lib -O2 -pipe -march=native -fno-strict-aliasing -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/usr/local/include -MT test-event-stats.o -MD -MP -MF .deps/test-event-stats.Tpo -c -o test-event-stats.o test-event-stats.c clang-4.0: warning: -Wl,-rpath=/usr/local/llvm40/lib: 'linker' input unused [-Wunused-command-line-argument] test-event-stats.c:101:8: warning: implicit declaration of function 'kill' is invalid in C99 [-Wimplicit-function-declaration] (void)kill(stats_pid, SIGKILL); ^ test-event-stats.c:101:24: error: use of undeclared identifier 'SIGKILL' (void)kill(stats_pid, SIGKILL); ^ 1 warning and 1 error generated. gmake[2]: *** [Makefile:656: test-event-stats.o] Error 1 gmake[2]: Leaving directory '/usr/local/directadmin/custombuild/dovecot-2.3.4/src/lib-master' gmake[1]: *** [Makefile:565: install-recursive] Error 1 gmake[1]: Leaving directory '/usr/local/directadmin/custombuild/dovecot-2.3.4/src' gmake: *** [Makefile:683: install-recursive] Error 1"
news page not been updated
The url http://www.dovecot.org/doc/NEWS has not been updated for a while, I suggest either killing it or updating it again. Chris
Softlinks
All, I'm using Maildir. Is it possible to move all (or only some) maildirs with softlinks to another partition? - Chris
Re: Softlinks
Steffen Kaiser wrote: > On Wed, 28 Sep 2016, Chris wrote: >> I'm using Maildir. Is it possible to move all (or only some) maildirs >> with >> softlinks to another partition? > > I did this, but be prepared that you have left-overs when the user deletes > the mailbox (mail folder). Ok, thank you. I would just link top-level folders, e.g. /var/vmail/users/userA -> /data/vmail2/users/userA for some extra big mailboxes. - Chris
migrating servers
Hi I am trying to migrate from a Centos dovecot 1.0.7 install to a debian dovecot 2.1.7 install. I thought simply replicating the maildirs would be fine (using rsync) but when I fetch mail from the new server using IMAP thunderbird wants to download all the messages again as duplicates. I searched the wiki over to see if I could find a solution and while it appeared I could use doveadm backup that seems to only work pushing changes from the new server to the old (wrong direction) and that program does not exist on the earlier version of dovecot on the old server. If I can be very blunt the documentation on the wiki assumes far too much knowledge from the reader. It is often hard to decipher exactly what is meant by many of the obtuse instructions. But that is an aside can anyone give me pointers on how to do this migration for all my users so they don't need to sort through hundreds of duplicated emails? Chris
Re: migrating servers
I worked out what the problem was Because I had been planning this move for several months I was syncing home user directories including Maildir for several months unfortunately I forgot the --delete command to rsync so while users were deleting emails on the original server they were often being copied to the destination server before being deleted. Therefore there where many more emails in the folder than expected. many quite old. by resyncing with the --delete command I was able to bring the source and destination into proper sync and all the extra emails were cleaned up Of course the pop email accounts still had the duplicates I have had to advise the user to clean out there email folders and apologised for the inconvenience. Chris On 22/08/2015 5:42 PM, chris wrote: Hi I am trying to migrate from a Centos dovecot 1.0.7 install to a debian dovecot 2.1.7 install. I thought simply replicating the maildirs would be fine (using rsync) but when I fetch mail from the new server using IMAP thunderbird wants to download all the messages again as duplicates. I searched the wiki over to see if I could find a solution and while it appeared I could use doveadm backup that seems to only work pushing changes from the new server to the old (wrong direction) and that program does not exist on the earlier version of dovecot on the old server. If I can be very blunt the documentation on the wiki assumes far too much knowledge from the reader. It is often hard to decipher exactly what is meant by many of the obtuse instructions. But that is an aside can anyone give me pointers on how to do this migration for all my users so they don't need to sort through hundreds of duplicated emails? Chris
Re: ACL and LDAP
Hi Daniel, > in doveadm_set_mailbox_acls() and imap_set_mailbox_acls() the attributes > are read and used to set them on the folders. thank you. That's interesting. They're calling doveadm directly. I probably would have tried to use an IMAP-Client library. The former seems easier. - Chris
IMAP ACLs for groups
Dear All, is there any way to assign POSIX groups to Dovecot IMAP-ACLs? I've tried that today, but I could only open a public folder when my username was listed in the ACL. A group didn't work. How do you manage ACLs for intenseley used public folders with many users? What backend do you use for user management and ACLs? Is there any way to extend permissions? Are there any scripts available that set permissions or grant them when logging in? - Chris
Re: shared folders not working with . separator
Hugh Bragg wrote: > I've been trying this for weeks and never managed to get my mail client > to see shared folders. > Apparently imap_acl works fine as I can see these entries in the > dovecot-acl-list files and the correct entries persist in the client. > This is my first dovecot setup so I hope I've configured it right. > Everything else works beautifully, thanks. Are they shown with telnet? e.g. http://www.anta.net/misc/telnet-troubleshooting/imap.shtml
Script dovecot ACLs, Quota and doveadm
Dear All, I'd like to set a) Quota for mailboxes in private and public namespace b) ACLs for both. What's the easiest way to do this? Do I have to call doveadm for every mailbox (private and public)? Is it required to provide a username to doveadm? Can I use the same user for all if it is defined in global ACLs? For Cyrus there are perl modules which have the same functions as cyradm. Is there something like this for dovecot? Are there any modules available? Haven't found anything in CPAN. - Chris
Block public namespace mail when quota exceeded
Dear All, is it possible to temporarily reject mails, when quota of public namespace subfolder is exceeded? I'd check folder size with a cronjob. Dovecot is using a system user, so I guess blocking post and insert in ACL isn't a solution? I don't want to set the folder to read-only, that users can still delete mails. Thank you in advance. - Chris
Deliver to Public Mailbox
Dear All, I'd like dovecot-lda (deliver) to post a message to a public mailbox. It doesn't seem to work when I just use the -m parameter with the IMAP-Mailbox path, e.g. Public/info . What parameters are required? Do I have to change the headers with formail in any way? TIA! - Chris
Re: Problem setting owner
Luca Bertoncello wrote:
> I already added:
>
> mail_uid = 1005
> mail_gid = 8
>
> in 10-mail.conf and:
>
> user_attrs = \
> =user=exim, \
> =uid=1005, \
> =gid=8, \
> =quota_rule=*:bytes=%{ldap:quotaBytes}, \
> =home=/home/mailboxes/%{ldap:sAMAccountName}/Maildir, \
> =mail=maildir:/home/mailboxes/%{ldap:sAMAccountName}/Maildir
>
> in the dovecot-ldap.conf.ext, but it seems to ignore them...
Have you tried override_fields?
- Chris
speedup doveadm
All, I've to set ACLs in public namespace for more than 5000 folders. That takes some hours. I'm calling doveadm by a perl script. Is there any way to speedup doveadm? Is it possible to set ACLs for multiple folders or users in a single call? - Chris
Re: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x?
Am 18. Februar 2012 19:49 schrieb Ralf Hildebrandt : > * Ralf Hildebrandt : > >> # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* >> /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe >> /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe >> /home/h/e/hermel/Maildir/.Entw&APw-rfe >> /home/h/l/hlammert/Maildir/.Entw&APw-rfe >> ... >> /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe >> /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe >> /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe > > Some stats: > > # Name > === > 2 .Entw&-APw-rfe > 9 .Entw&,,0-rfe > 91 .Entw&APw-rfe > Use something like "Entwuerfe" and not "Entwürfe". -- Chris
Re: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x?
2012/2/18 Luigi Rosa : > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Ralf Hildebrandt said the following on 18/02/12 20:04: > >> I'm not sure which program insists on creating "Entwürfe" > > > Outlook? ;) Yes... M$ Outlook crap do this :) -- Chris
Re: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x?
2012/2/18 René Neumann : > Am 18.02.2012 19:59, schrieb Chris: >> Use something like "Entwuerfe" and not "Entwürfe". > > Come on ... software having problems with non-ASCII-stuff in 2012 is > just broken by design. Hence there should be no reason to use 'ue' > instead of ü anywhere. Dovecot has no problems with non-ASCII stuff. The german language is broken by design ... ß ä ö ü... :-) -- Chris
Re: [Dovecot] 2.1.1: subscriptions: Removing invalid entry
2012/2/28 Ralf Hildebrandt : > Admittedly, it's not a valid folder name and dovecot is right in > removing it: > > Feb 28 17:50:05 postamt dovecot: imap(kbaessle): Warning: Subscriptions file > /home/k/b/kbaessle/Maildir/subscriptions: Removing invalid entry: Entw�rfe > > ...but when looking at the subscription file AFTER the removal, that file > contained an empty line. I don't think that's correct. Is that fixed? -- Chris
Re: [Dovecot] sieve vacation redirect problem
2012/4/25 Robert Schetterer : >> That's totally broken.. You sure you recompiled Pigeonhole correctly against >> the same Dovecot version? >> > > its from latest > http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ > recompiled the debain way , never had bugged You are using packages compiled against debian sources under ubuntu??? -- Chris
Re: [Dovecot] sieve vacation redirect problem
2012/4/25 Robert Schetterer : > Am 25.04.2012 16:47, schrieb Chris: >> 2012/4/25 Robert Schetterer : >>>> That's totally broken.. You sure you recompiled Pigeonhole correctly >>>> against the same Dovecot version? >>>> >>> >>> its from latest >>> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >>> recompiled the debain way , never had bugged >> >> You are using packages compiled against debian sources under ubuntu??? >> >> -- >> Chris > > jep that works since years Thats really a very bad idea. Please recompile it under ubuntu 10.04. -- Chris
Re: [Dovecot] Remove leading and trailing spaces from folder names?
2012/7/19 Ralf Hildebrandt : > Right now we're migrating mailboxes from dovecot -> Exchange, and > Exchange cannot handle leading and trailing spaces in Folder names. You're migrating to M$ exchange??? :o -- Chris
Re: [Dovecot] Remove leading and trailing spaces from folder names?
2012/7/20 Wojciech Puchar : >> do you really think that he migrates to exchange >> because he have fun with it? > > so? is it my problem that he works in company managed by an idiot? > > Why dovecot users have to provide free support for microsoft software, while > microsoft are paid for their "products" thousands of dollars? > > Does it make sense? Do you think Ralf is migrating just for fun to M$ exchange? Ralf is certainly no friend of M$ software. I think he's freaked out when he heard about it. -- Chris
Re: [Dovecot] Authentication with lower case username ONLY
On Sep 15, 2010, at 10:07 AM, Jerry wrote: > > Out of morbid curiosity, why do you feel the need to convert the user > names to lower case? > Because %L in a config file is better than a user on the phone? If there are no upper-case letters in user names, there is no reason a login should fail because of an upper-case letter in a user name.
[Dovecot] Can't receive emails
Hello everyone, I installed postfix and dovecot (imap + pop) on a debian server. I can send emails via postfix and I could receive emails via dovecot before. Yesterday my server got shutdown and restarted and since then, I somehow have some problems. If I send some emails, I don't receive them in /var/vmail/%d/%n/Maildir But the login works fine (pop and imap). How can I figure out, why there are no mails comming? Thank you Bye, Chris dovecot --version 1.0.15 dovecot -n # 1.0.15: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.debug.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: Mailserver ready. first_valid_uid: 5000 last_valid_uid: 5000 mail_privileged_group: mail mail_location: maildir:/var/vmail/%d/%n/Maildir mail_debug: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: static args: uid=5000 gid=5000 home=/var/vmail/%d/%n allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail driver = mysql connect = host=localhost dbname=postfix user=MYUSER password=MYPASS default_pass_scheme = CRYPT password_query = SELECT username as user, password FROM mailbox WHERE username='%u';
Re: [Dovecot] Can't receive emails
Hello again, I searched every logfile I could find, but without any results :( How does it usually work? The mail is send by another server and on my server, does postfix receive the email or is it dovecots job? Who puts the mail into that mail directory? I guess it's postfix? Thank you in advance Bye, Chris Am 05.10.2010 11:59, schrieb Chris: Hello everyone, I installed postfix and dovecot (imap + pop) on a debian server. I can send emails via postfix and I could receive emails via dovecot before. Yesterday my server got shutdown and restarted and since then, I somehow have some problems. If I send some emails, I don't receive them in /var/vmail/%d/%n/Maildir But the login works fine (pop and imap). How can I figure out, why there are no mails comming? Thank you Bye, Chris dovecot --version 1.0.15 dovecot -n # 1.0.15: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.debug.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: Mailserver ready. first_valid_uid: 5000 last_valid_uid: 5000 mail_privileged_group: mail mail_location: maildir:/var/vmail/%d/%n/Maildir mail_debug: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: static args: uid=5000 gid=5000 home=/var/vmail/%d/%n allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail driver = mysql connect = host=localhost dbname=postfix user=MYUSER password=MYPASS default_pass_scheme = CRYPT password_query = SELECT username as user, password FROM mailbox WHERE username='%u';
Re: [Dovecot] Dovecot is not creating directories properly for new accounts
Dovecot is able to create the other directories after the main home folder is created, it might be having issues creating the parent directories On 09/03/2013 10:49 AM, Chris Lasater wrote: Hi, I have just set up a new dovecot server with Dovecot 2.2.5. When I create a new user and try to authenticate I get the following error Sep 03 14:33:38 imap(test2): Error: user test2: Initialization failed: Initializing mail storage from mail_location setting failed: mkdir(/home/user/apps/var/mail/imap/test2/mailboxes) failed: Permission denied (euid=503(imapd) egid=503(imapd) missing +w perm: /home/user/apps/var/mail, we're not in group 502(dovecot), dir owned by 502:502 mode=0775) Sep 03 14:33:38 imap(test2): Error: Invalid user settings. Refer to server log for more information. imapd is in the dovecot group and here are the permissions [user@MB1 logs]$ ll /home/user/apps/var/ total 4 drwxrwxr-x 4 dovecot dovecot 4096 Sep 3 14:35 mail [user@MB1 logs]$ id imapd uid=503(imapd) gid=503(imapd) groups=503(imapd),500(user),502(dovecot) if I su - imapd and then run mkdir -p /home/user/apps/var/mail/imap/test2/mailboxes everything starts working and the user can log in. Isn't Dovecot supposed to be able to create these?
[Dovecot] login_log_format_elements does not appear to be changing log format 2.2.5
Hi,
login_log_format_elements does not seem to change the login logs. I
have it set to the below setting and the word "home" does not even
appear. Is there something I have to do to for this? Also I use ldap
for authentication.
[user@MB1 logs]$ doveconf -n | grep -i log
debug_log_path = /home/user/apps/logs/dovecot_debug.log
info_log_path = /home/user/apps/logs/dovecot_info.log
log_path = /home/user/apps/logs/dovecot.log
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
home=%h
service imap-login {
chroot = login
executable = imap-login
type = login
user = $default_login_user
[user@MB1 dovecot]$ tail -1 ~/apps/logs/dovecot_info.log
Sep 04 18:19:55 imap-login: Info: Login: user=,
method=PLAIN, rip=10.100.5.6, lip=10.100.5.103, mpid=6571
[user@MB1 dovecot]$ dovecot --version
2.2.5
Re: [Dovecot] Doveadm with a 2nd Instance
I upgraded to 2.2.6 yesterday and was one 2.2.5 before. On 09/26/2013 07:06 PM, Daniel Parthey wrote: Hi, this bug should have been fixed by Timo in some 2.1.x release. The issue should be in the list archives too. Which version are you using? BTW: I'm still using doveadm -c /etc/dovecot-director/dovecot- director.conf director status to address the director instance. Regards Daniel
[Dovecot] userdb maildir permission denied in 2.1.7
Dear All,
I'm using version 2.1.7 which currently ships with debian stable.
I'd like to use
userdb {
driver = passwd-file
args = username_format=%n /etc/dovecot/users
default_fields = uid=vmail gid=vmail
override_fields =
skip = never
result_failure = continue
result_internalfail = continue
result_success = return-ok
}
from [1].
The error message is:
Jan 25 14:02:03 post1 dovecot: lda(chris): Error: user chris:
Initialization failed: Namespace '': stat(/home/vmail/chris/Maildir)
failed: Permission denied (euid=1000(chris) egid=1000(chris) missing +x
perm: /home/vmail/chris, dir owned by 1001:1001 mode=0700)
The skip and result_* commands aren't supported.
My users file:
chris:{MD5-CRYPT}$1$3uC3URg..:::/home/vmail/chris/:/bin/false::
Dovecot should access the Maildir with vmail:vmail (1001:1001) and not
chris:chris (1000:1000)!
Thank you in advance!
- Chris
[1] http://wiki2.dovecot.org/UserDatabase
Re: [Dovecot] userdb maildir permission denied in 2.1.7
Sorry, there's another issue with this configuration, not in the part I
quoted. Please ignore this thread. If there are further questions, I'll
ask again.
- Chris
On 01/25/2014 05:28 PM, Chris wrote:
> Dear All,
>
> I'm using version 2.1.7 which currently ships with debian stable.
>
> I'd like to use
>
> userdb {
> driver = passwd-file
> args = username_format=%n /etc/dovecot/users
>
> default_fields = uid=vmail gid=vmail
> override_fields =
>
> skip = never
>
> result_failure = continue
> result_internalfail = continue
> result_success = return-ok
> }
>
> from [1].
>
> The error message is:
>
> Jan 25 14:02:03 post1 dovecot: lda(chris): Error: user chris:
> Initialization failed: Namespace '': stat(/home/vmail/chris/Maildir)
> failed: Permission denied (euid=1000(chris) egid=1000(chris) missing +x
> perm: /home/vmail/chris, dir owned by 1001:1001 mode=0700)
>
> The skip and result_* commands aren't supported.
>
> My users file:
> chris:{MD5-CRYPT}$1$3uC3URg..:::/home/vmail/chris/:/bin/false::
>
> Dovecot should access the Maildir with vmail:vmail (1001:1001) and not
> chris:chris (1000:1000)!
>
> Thank you in advance!
>
> - Chris
>
>
> [1] http://wiki2.dovecot.org/UserDatabase
>
>
Manage Sieve Security?
Dear All, can I expose Dovecot's Manage Sieve port to the internet or should I restrict access (e.g. by IP address or VPN)? -- Christian
Auth SEGV on sparc64, alignment problem?
Apologies first for using two addresses, but I can’t currently read my email
at distal.com. :-)
I was previously running dovecot2-2.2.29.1_2 on FreeBSD 11 on sparc64.
Trying to debug a problem I was having with one of my clients, I upgraded to
dovecot-2.2.33.2_4 on that same server. However, I cannot connect now, log
shows:
Feb 20 16:55:00 westeros dovecot: master: Dovecot v2.2.33.2 (d6601f4ec)
starting up for imap, pop3, lmtp
Feb 20 16:55:31 westeros dovecot: auth: Fatal: master: service(auth): child
25395 killed with signal 11 (core dumped)
Feb 20 16:55:31 westeros dovecot: master: Error: service(auth): command startup
failed, throttling for 2 secs
Feb 20 16:55:31 westeros dovecot: imap-login: Disconnected: Auth process broken
(disconnected before auth was ready, waited 0 secs): user=<>, rip=2001::xxx,
lip=2001:470:e24c:200::ae25, TLS handshaking, session=
Feb 20 16:55:33 westeros dovecot: auth: Fatal: master: service(auth): child
25398 killed with signal 11 (core dumped)
Feb 20 16:55:33 westeros dovecot: master: Error: service(auth): command startup
failed, throttling for 4 secs
Feb 20 16:55:33 westeros dovecot: imap-login: Disconnected: Auth process broken
(disconnected before auth was ready, waited 2 secs): user=<>, rip=2001::xxx,
lip=2001:470:e24c:200::ae25, session=
Feb 20 16:55:37 westeros dovecot: master: Error: service(auth): command startup
failed, throttling for 8 secs
Feb 20 16:55:37 westeros dovecot: auth: Fatal: master: service(auth): child
25400 killed with signal 11 (core dumped)
Loading the core file, as described https://www.dovecot.org/bugreport.html ,
shows the error in libc somewhere:
(gdb) bt full
#0 __unaligned_load (
p=0x617070656e640e6d , size=4)
at /usr/src/release-11.1.0/lib/libc/sparc64/sys/__sparc_utrap_align.c:45
val = 0
i = 0
#1 0x109f9f6c in __unaligned_fixup (uf=0x7fdee40)
at /usr/src/release-11.1.0/lib/libc/sparc64/sys/__sparc_utrap_align.c:78
addr =
val =
insn = 3254807616
sig =
#2 0x109f9d50 in __sparc_utrap (uf=0x7fdee40)
at /usr/src/release-11.1.0/lib/libc/sparc64/sys/__sparc_utrap.c:100
sig = 272013984
#3 0x1094a10c in __sparc_utrap_gen () from /lib/libc.so.7
No symbol table info available.
#4 0x1094a10c in __sparc_utrap_gen () from /lib/libc.so.7
No symbol table info available.
Previous frame identical to this frame (corrupt stack?)
(gdb)
As this is a sparc64, with 8-byte alignment requirements, I’m guessing that’s
the issue. Many a piece of software has failed to respect that and crashed.
But, I’m not sure.
Does anyone have any suggestions? I’ve built it locally (via ports), so if
there are compiler options I can/should try, I certainly can try.
Thanks…
- Chris
Re: Auth SEGV on sparc64, alignment problem?
Sadly, that doesn’t help either. Over the past day, I’ve built and installed
a different branch of the OS (stable/11, instead of release/11.1), to see if a
new compiler/libc might change things. Sadly, it does not.
In the same situation now, auth fails immediately with signal 11. Running
gdb on auth (from build dir, compiled -g -O2) shows something similar.
- Chris
# gdb work/dovecot-2.2.33.2/src/auth/.libs/auth
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "sparc64-marcel-freebsd"...
(gdb) list
372 /* ask auth master to disconnect us */
373 auth_worker_client_send_shutdown();
374 }
375 }
376
377 int main(int argc, char *argv[])
378 {
379 int c;
380
381 master_service = master_service_init("auth", 0, &argc, &argv,
"w");
(gdb) run
Starting program:
/usr/ports/mail/dovecot/work/dovecot-2.2.33.2/src/auth/.libs/auth
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb) bt
#0 0x4022a380 in ?? ()
#1 0x0008 in ?? ()
Previous frame identical to this frame (corrupt stack?)
(gdb)
> On Feb 21, 2018, at 02:01, Aki Tuomi wrote:
>
> Your core dump looks a bit broken. Since it seems to die instantly, can
> you try gdb /path/to/auth and just run it?
>
> Aki
Re: Auth SEGV on sparc64, alignment problem?
Fancy, while not fun. :-) But thanks, that does work. Doing that, n’ing
over calls to strcmp, it failed:
passdbs_init () at passdb.c:313
313 passdb_register_module(&passdb_ldap);
(gdb)
passdb_register_module (iface=0x280120) at passdb.c:33
33 old_iface = passdb_interface_find(iface->name);
(gdb)
passdb_interface_find (name=0x16fe60 "ldap") at passdb.c:20
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
26 return NULL;
(gdb)
27 }
(gdb)
passdb_register_module (iface=0x280120) at passdb.c:34
34 if (old_iface != NULL && old_iface->verify_plain == NULL) {
(gdb)
37 } else if (old_iface != NULL) {
(gdb)
41 array_append(&passdb_interfaces, &iface, 1);
(gdb)
42 }
(gdb)
passdbs_init () at passdb.c:314
314 passdb_register_module(&passdb_sql);
(gdb)
315 passdb_register_module(&passdb_sia);
(gdb)
316 passdb_register_module(&passdb_static);
(gdb)
317 passdb_register_module(&passdb_oauth2);
(gdb)
318 }
(gdb)
main_preinit () at main.c:186
186 userdbs_init();
(gdb)
188 password_schemes_init();
(gdb)
190 services = read_global_settings();
(gdb)
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb)
Cannot find bounds of current function
(gdb)
Next step I’ll stop before that and be more careful about n’ing things, but.
Just passing on context while I have it.
Thanks. More later.
- Chris
> On Feb 22, 2018, at 02:25, Aki Tuomi wrote:
>
> Hi!
>
> Unfortunately we do not have a Sparc64 with any OS at hand. Maybe you could
>
> break main
> r
> s
>
> until it breaks?
>
> Aki
>
Re: Auth SEGV on sparc64, alignment problem?
Okay. Got to the next bit pretty quickly.:
Breakpoint 4, auth_settings_read (service=0x0, pool=0x4104b020,
output_r=0x7fdf6d0) at auth-settings.c:522
522 input.module = "auth";
(gdb) n
523 input.service = service;
(gdb) n
524 if (master_service_settings_read(master_service, &input,
(gdb) s
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb)
So, why did it not step into master_service_settings_read ? Trying again:
523 input.service = service;
(gdb) s
524 if (master_service_settings_read(master_service, &input,
(gdb) list
519
520 i_zero(&input);
521 input.roots = set_roots;
522 input.module = "auth";
523 input.service = service;
524 if (master_service_settings_read(master_service, &input,
525 output_r, &error) < 0)
526 i_fatal("Error reading configuration: %s", error);
527
528 pool_ref(pool);
(gdb) p input
$1 = {roots = 0x27fbd8, config_path = 0x0, preserve_environment = false,
preserve_user = false, preserve_home = false, never_exec = false,
use_sysexits = false, parse_full_config = false, module = 0x16ad70 "auth",
service = 0x0, username = 0x0, local_ip = {family = 0, u = {ip6 = {
__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0,
0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {
s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {
__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0,
0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}},
local_name = 0x0}
(gdb) p &input
$2 = (struct master_service_settings_input *) 0x7fdf5a8
(gdb) p output_r
$3 = (struct master_service_settings_output *) 0x7fdf6d0
(gdb) p &error
$4 = (const char **) 0x7fdf598
(gdb) p error
$6 = 0x10dbd0 "@\005?\204\001"
(gdb) p master_service
$5 = (struct master_service *) 0x4103
(gdb) s
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb)
Any ideas here? I’m not sure where to look next…
- Chris
> On Feb 22, 2018, at 10:10, Chris Ross wrote:
>
> Fancy, while not fun. :-) But thanks, that does work. Doing that, n’ing
> over calls to strcmp, it failed:
>
> passdbs_init () at passdb.c:313
> 313 passdb_register_module(&passdb_ldap);
> (gdb)
> passdb_register_module (iface=0x280120) at passdb.c:33
> 33old_iface = passdb_interface_find(iface->name);
> (gdb)
> passdb_interface_find (name=0x16fe60 "ldap") at passdb.c:20
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (st
Re: Auth SEGV on sparc64, alignment problem?
(long gdb output, you’ve been warned)
Okay. So, the libdovecot shared library in /usr/local was stripped.
Replaced that, and got farther. gdb walk below.
It looks to me like it gets deep into the OS’s vfork/execv where it catches a
trap/crashes. Is this a problem I can catch, or something wrong with running
in gdb? I notice this is a SIGTRAP, where the binary when run out of gdb gets
a SIGSEGV, and that’s what a loaded core shows.
Thanks for any assistance.
- Chris
Breakpoint 3, master_service_exec_config (service=0x4103,
input=0x7fdf5a8) at master-service-settings.c:125
125 const char **conf_argv, *binary_path = service->argv[0];
(gdb) n
128 (void)t_binary_abspath(&binary_path);
(gdb) n
130 if (!service->keep_environment && !input->preserve_environment)
{
(gdb)
131 if (input->preserve_home)
(gdb)
133 if (input->preserve_user)
(gdb)
135 if ((service->flags & MASTER_SERVICE_FLAG_STANDALONE)
!= 0)
(gdb)
136
master_service_import_environment("LOG_STDERR_TIMESTAMP");
(gdb)
140 if (getenv(DOVECOT_PRESERVE_ENVS_ENV) == NULL)
(gdb)
146 if (input->use_sysexits)
(gdb)
150 i = 0;
(gdb)
151 argv_max_count = 11 + (service->argc + 1) + 1;
(gdb)
152 conf_argv = t_new(const char *, argv_max_count);
(gdb)
153 conf_argv[i++] = DOVECOT_CONFIG_BIN_PATH;
(gdb)
154 if (input->service != NULL) {
(gdb)
158 conf_argv[i++] = "-c";
(gdb)
159 conf_argv[i++] = service->config_path;
(gdb)
160 if (input->module != NULL) {
(gdb)
161 conf_argv[i++] = "-m";
(gdb)
162 conf_argv[i++] = input->module;
(gdb)
163 if (service->want_ssl_settings) {
(gdb)
168 if (input->parse_full_config)
(gdb)
171 conf_argv[i++] = "-e";
(gdb)
172 conf_argv[i++] = binary_path;
(gdb)
173 memcpy(conf_argv+i, service->argv + 1,
(gdb)
175 i += service->argc;
(gdb)
177 i_assert(i < argv_max_count);
(gdb)
178 execv_const(conf_argv[0], conf_argv);
(gdb) p conf_argv
$3 = (const char **) 0x41016e48
(gdb) p conf_argv[0]
$4 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) p *conf_argv
$5 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) s
execv_const (path=0x4064f6d8 "/usr/local/bin/doveconf", argv=0x41016e48)
at execv-const.c:23
23 (void)execv(path, argv_drop_const(argv));
(gdb) p parth
No symbol "parth" in current context.
(gdb) p path
$6 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) s
argv_drop_const (argv=0x41016e48) at execv-const.c:13
13 for (count = 0; argv[count] != NULL; count++) ;
(gdb) p argv
$7 = (const char * const *) 0x41016e48
(gdb) p argv[0]
$8 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) p argv[1]
$9 = 0x4064f708 "-c"
(gdb) p argv[2]
$10 = 0x4104 "/usr/local/etc/dovecot/dovecot.conf"
(gdb) p argv[3]
$11 = 0x4064f710 "-m"
(gdb) p argv[4]
$12 = 0x16ad70 "auth"
(gdb) p argv[5]
$13 = 0x4064f728 "-e"
(gdb) p argv[6]
$14 = 0x7fdfd18
"/usr/ports/mail/dovecot/work/stage/usr/local/libexec/dovecot/auth"
(gdb) p argv[7]
$15 = 0x0
(gdb) n
15 ret = t_new(char *, count + 1);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
18 return ret;
(gdb)
19 }
(gdb)
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb) b argv_drop_const
Breakpoint 4 at 0x405d50b8: file execv-const.c, line 13.
(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Re: Auth SEGV on sparc64, alignment problem?
> On Feb 22, 2018, at 15:21, Josef 'Jeff' Sipek wrote: > >> Loading the core file, as described >> https://www.dovecot.org/bugreport.html , shows the error in libc >> somewhere: > > I read the your other mails in this thread; can you run things as before and > do a 'bt full' on the core file with the debug-symbol-enabled libdovecot? > gdb seems to be catching the SIGTRAPs, which is making things a bit confusing. > >> (gdb) bt full >> #0 __unaligned_load ( >>p=0x617070656e640e6d , size=4) No difference there. I changed the install process to not strip things, and manually copied in all of the libs in /usr/local/lib/dovecot again with unstripped (I think libtool stripped them, I just rejiggered makefiles and install-sh). Loading a core from a SEGV shows: Loaded symbols for /libexec/ld-elf.so.1 #0 __unaligned_load ( p=0x706172736572690a , size=4) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap_align.c:45 45 val = (val << 8) | p[i]; (gdb) bt full #0 __unaligned_load ( p=0x706172736572690a , size=4) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap_align.c:45 val = 0 i = 0 #1 0x40adb7cc in __unaligned_fixup (uf=0x7fdf110) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap_align.c:78 addr = val = insn = 3254806592 sig = #2 0x40adb5b0 in __sparc_utrap (uf=0x7fdf110) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap.c:100 sig = 16 #3 0x40a2c1cc in __sparc_utrap_gen () from /lib/libc.so.7 No symbol table info available. #4 0x40a2c1cc in __sparc_utrap_gen () from /lib/libc.so.7 No symbol table info available. Previous frame identical to this frame (corrupt stack?) (gdb) (Which as you note below, that address is actually “parseri\n”) > This address looks like ASCII - "append\x0em", so my theory at the moment > is: > > (1) something clobbers a pointer > (2) the CPU attempts to execute a load from the address > (3) a utrap is generated to handle unaligned load > (4) the utrap code attempts to emulate the unaligned load > (5) the CPU fails to access the address since it is bogus, and a SIGSEGV is >generated > > Now, I'm have no idea why it'd first try to work around the alignment > requirement before doing a quick sanity check and generating SIGSEGV to > begin with, but that's my theory based on the info available so far. > Hopefully, a stack trace from a core file will help. Unfortunately it seems not to have. But, good catch on the pointer value there being ASCII data. Let me know if you have any other ideas. - Chris
Config problem: Service #0 is missing name
Okay. So following up on the problems I was discussing weeks ago, I have hand-patched dovecot to work around problems that I believe may be in libc and not dovecot. Trying to get the properly built and installed dovecot-2.2.35, however, is giving me an error I haven’t seen before. I presume that something got munged while I was hand-configuring and installing/uninstalling/reinstalling the port in recent weeks. Running on a FreeBSD 11-stable machine, when I run “service dovecot start”, it tells me: % sudo service dovecot start Starting dovecot. Fatal: Error reading configuration: Invalid settings: Service #0 is missing name /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot % Surprisingly, googling for “Service #0 is missing name” didn’t yield a suggestion. My config is the same as it had been previously, I believe. Does anyone have an idea what might be wrong here? Thanks. - Chris
Config problem: Service #0 is missing name
Okay. So following up on the problems I was discussing weeks ago, I have hand-patched dovecot to work around problems that I believe may be in libc and not dovecot. Trying to get the properly built and installed dovecot-2.2.35, however, is giving me an error I haven’t seen before. I presume that something got munged while I was hand-configuring and installing/uninstalling/reinstalling the port in recent weeks. Running on a FreeBSD 11-stable machine, when I run “service dovecot start”, it tells me: % sudo service dovecot start Starting dovecot. Fatal: Error reading configuration: Invalid settings: Service #0 is missing name /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot % Surprisingly, googling for “Service #0 is missing name” didn’t yield a suggestion. My config is the same as it had been previously, I believe. Does anyone have an idea what might be wrong here? Thanks. - Chris
Re: Config problem: Service #0 is missing name
> On Mar 25, 2018, at 10:37, Aki Tuomi wrote:
>
> Without looking at your config I'd say you have a nameless service section,
> like
>
> service {
>
> }
>
> Name goes after keyword service.
Yeah, I don’t seem to. This was the same config that was working before
things stopped for other reasons last month. I think I may’ve overlooked
something in my hand-coded replacement for libc functionality. I’m testing
that now.
[time goes by]
Yup. My error. Sorry for the noise on the list, it’s running now.
- Chris
Inbox quota usage doubled when mailbox_list_index enabled, under some circumstances
Hello,
I searched through the list archives for anything that appeared to be
similar to this but I didn't find any good matches. I apologize if this
has been brought up before.
Beginning with Dovecot 2.2.34, reported quota usage of a user's inbox
can be doubled when the following criteria are met:
1) quota plugin is enabled
2) mailbox_list_index=yes
3) A sub-folder of the inbox exists whose name also matches the prefix
(for example, "INBOX.Test"), even if that folder has no contents.
(Apparently, some mail clients such as K-9 Mail on Android can create
these folders automatically)
Downgrading to 2.2.33.2 allows this configuration to work normally. I
looked through the changes in git between 2.2.33.2 and 2.2.34 and they
appeared extensive, so unfortunately I wasn't able to pinpoint a
specific change that caused this.
Reproduced on a CentOS 7 server with Dovecot 2.2.34 built from git
sources according to
https://wiki2.dovecot.org/CompilingSource#Compiling_Dovecot_From_Git
using the following minimal configuration:
==
root@centos7 [~]# dovecot -n
# 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-862.9.1.el7.x86_64 x86_64 CentOS Linux release
7.5.1804 (Core)
# Hostname: centos7.local
mail_location = maildir:~/Maildir
mail_plugins = quota
mailbox_list_index = yes
namespace inbox {
inbox = yes
location =
prefix = INBOX.
separator = .
}
passdb {
driver = pam
}
plugin {
quota = maildir:Mailbox
}
userdb {
driver = passwd
}
==
With the system user "myuser" and a minimal Maildir setup containing 1
email in the inbox, and one sub-folder named "INBOX.Test":
==
root@centos7 [~]# ls -lAR /home/myuser/Maildir
/home/myuser/Maildir:
total 4
drwxr-xr-x. 2 myuser myuser 77 Jul 25 09:36 cur
drwxr-xr-x. 5 myuser myuser 56 Jul 25 15:19 .INBOX.Test
-rw-r--r--. 1 myuser myuser 14 Jul 25 15:19 maildirsize
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 new
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 tmp
/home/myuser/Maildir/cur:
total 12416
-rw-r--r--. 1 myuser myuser 12712627 Jul 25 09:07
1532529376.M543965P58007.centos7.local,S=12712627,W=12877782:2,S
/home/myuser/Maildir/.INBOX.Test:
total 0
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 cur
-rw-r--r--. 1 myuser myuser 0 Jul 25 09:36 maildirfolder
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 new
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 tmp
/home/myuser/Maildir/.INBOX.Test/cur:
total 0
/home/myuser/Maildir/.INBOX.Test/new:
total 0
/home/myuser/Maildir/.INBOX.Test/tmp:
total 0
/home/myuser/Maildir/new:
total 0
/home/myuser/Maildir/tmp:
total 0
==
Given this configuration, when "mailbox_list_index" is enabled, after
quota recalculation, quota reports that there are 2 messages and that
double the amount of storage is used:
==
root@centos7 [~]# doveadm quota recalc -u myuser; doveadm quota get -u
myuser
Quota name Type Value
Limit %
Mailbox STORAGE 24830
- 0
Mailbox MESSAGE 2
- 0
==
When "mailbox_list_index" is disabled and quota is recalculated, it will
report the message and storage counts correctly:
==
root@centos7 [~]# doveadm -o 'mailbox_list_index=no' quota recalc -u
myuser; doveadm quota get -u myuser
Quota name Type Value
Limit %
Mailbox STORAGE 12415
- 0
Mailbox MESSAGE 1
- 0
==
I am hoping that someone with much more familiarity with the code path
involved here could take a look at this issue and possibly submit a fix
for it -- that would be greatly appreciated. Please let me know if you
need any more details regarding the replication steps, though I hope
that I didn't miss anything.
Thank you!
--
Chris Dillon
Technical Analyst III
Migrations Specialist
cPanel, Inc.
smime.p7s
Description: S/MIME Cryptographic Signature
Re: Inbox quota usage doubled when mailbox_list_index enabled, under some circumstances
I failed to mention previously that the behavior also exists in latest 2.2.36 built from git sources, and I believe also exists in 2.3.2. Though, I wasn't able to successfully build release-2.3.2 or master-2.3 branches from git in the minimal test environment due to an odd compilation error that I have not yet attempted to resolve, but I used an internal binary build of 2.3.2 which has a few (though probably not affecting this particular issue) patches applied to it. -- Chris Dillon Technical Analyst III Migrations Specialist cPanel, Inc. Register Now for cPanel Conference 2018 Oct 1 - 3, 2018, Houston, Texas https://conference.cpanel.com smime.p7s Description: S/MIME Cryptographic Signature
Re: Inbox quota usage doubled when mailbox_list_index enabled, under some circumstances
I’ve had the opportunity to test the same configuration with a fresh build of
the git master branch (2.4.devel) and the issue also occurs there. I see that
"mailbox_list_index = yes" is now enabled by default. It can still be disabled
via "mailbox_list_index = no" which allows the quota to be calculated correctly.
==
root@ubuntu1804:~# dovecot -n
# 2.4.devel (44282aeeb): /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 4.15.0-30-generic x86_64 Ubuntu 18.04.1 LTS
# Hostname: ubuntu1804
mail_location = maildir:~/Maildir
mail_plugins = quota
namespace inbox {
inbox = yes
location =
prefix = INBOX.
separator = .
}
passdb {
driver = pam
}
plugin {
quota = maildir:Mailbox
}
userdb {
driver = passwd
}
==
(To summarize from my previous message -- other than "mailbox_list_index =
yes", second most important part of replication is that there is at least one
email in the real inbox and at least one sub-folder named "INBOX" in maildir
format)
root@ubuntu1804:~# ls -ld
/home/myuser/Maildir/cur/1532529376.M543965P58007.centos7.local\,S\=12712627\,W\=12877782\:2\,S
/home/myuser/Maildir/.INBOX.Test/
-rw-rw-r-- 1 myuser myuser 12712627 Aug 14 18:28
'/home/myuser/Maildir/cur/1532529376.M543965P58007.centos7.local,S=12712627,W=12877782:2,S'
drwxrwxr-x 5 myuser myuser 87 Aug 14 18:56
/home/myuser/Maildir/.INBOX.Test/
=
(In the following example usage is doubled, there is only one email)
root@ubuntu1804:~# doveadm quota recalc -u myuser; doveadm quota get -u myuser
Quota name TypeValue Limit
%
MailboxSTORAGE 24830 -
0
MailboxMESSAGE 2 -
0
==
(In the following example it works correctly with mailbox_list_index disabled)
root@ubuntu1804:~# doveadm -o 'mailbox_list_index=no' quota recalc -u myuser;
doveadm quota get -u myuser
Quota name TypeValue Limit
%
MailboxSTORAGE 12415 -
0
MailboxMESSAGE 1 -
0
==
Best Regards
HTTP DoveAdm API - Possible bug?
Hi all,
It seems that setting userMask in the doveadm http api's "user" command
to anything involving wildcards ?s or *s causes the API to fail.
When using the API with a userMask set to the fully qualified email
address, I get the expected fields back.
From a python shell, with an asterisk causes the following behaviour
(note that doveadm is a python module that wrappers the dovecot API into
Python's requests module).
>>> doveadm.do_query("http://[:::::]:24280";,
"API_KEY_HERE", [ ["user", { "userMask": ["*"] }, "c01"] ])
'[["doveadmResponse",,"c01"]]'
If I run the same command with a valid mailbox in the userMask Field, I
get the response I expect:
>>> doveadm.do_query("http://[[:::::]:24280";,
"API_KEY_HERE", [ ["user", { "userMask": ["t...@test.mydomain.com"] },
"c01"] ])
'[["doveadmResponse",{"t...@test.mydomain.com":{"uid":"x","gid":"y","home":"/mail/test.mydomain.com/users/test","mail":"maildir:/mail/test.mydomain.com/users/test/Maildir/:INDEX=/indexes/test.mydomain.com/test","nice":"10"}},"c01"]]'
Chatting with people in #dovecot revealed similar behaviour elsewhere so
it doesn't appear to be just me. I've also selectively edited the above
to avoid leaking data.
We're running on the following
# 2.2.34 (874deae): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.22 (22940fb7)
# OS: Linux 4.9.0-7-amd64 x86_64 Debian 9.5 ext4
If you need a full copy of the config, please let me know and I'll talk
to my client, however, I don't think this is config related.
Thanks in advance.
Regards,
Chris Malton
--
Delta V Technologies Limited
0 402 402www.deltav-tech.co.uk
Office: 17 Elm Close, Southampton, SO16 7DT
Company No. 11006104 Registered in England and Wales
Re: HTTP DoveAdm API - Possible bug?
Thanks Aki,
We'll see if we can backport this internally to the version we're
currently running as a temporary fix until we can do a proper upgrade.
Regards,
Chris
On 19/09/18 07:57, Aki Tuomi wrote:
Yeah. This is a bug, there is a fix pending.
From bb200128c83610d213b2ff2e59f1e0440ecbd2c9 Mon Sep 17 00:00:00 2001
From: Aki Tuomi
Date: Tue, 18 Sep 2018 20:37:38 +0300
Subject: [PATCH] doveadm: Fix doveadm user output when called from server
Was forgotten in a13b1245bee0b6524b4aeb3c8fd9e34af648b746
---
src/doveadm/doveadm-auth-server.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/doveadm/doveadm-auth-server.c
b/src/doveadm/doveadm-auth-server.c
index 44278e467e..d9e4e01c74 100644
--- a/src/doveadm/doveadm-auth-server.c
+++ b/src/doveadm/doveadm-auth-server.c
@@ -170,6 +170,10 @@ cmd_user_list(struct auth_master_connection *conn,
const char *username, *user_mask = "*";
unsigned int i;
+ doveadm_print_init(DOVEADM_PRINT_TYPE_FORMATTED);
+ doveadm_print_formatted_set_format("%{username}\n");
+ doveadm_print_header_simple("username");
+
if (users[0] != NULL && users[1] == NULL)
user_mask = users[0];
@@ -180,7 +184,7 @@ cmd_user_list(struct auth_master_connection *conn,
break;
}
if (users[i] != NULL)
- printf("%s\n", username);
+ doveadm_print(username);
}
if (auth_master_user_list_deinit(&ctx) < 0)
i_fatal("user listing failed");
--
Delta V Technologies Limited
0 402 402www.deltav-tech.co.uk
Office: 17 Elm Close, Southampton, SO16 7DT
Company No. 11006104 Registered in England and Wales
Re: Ubuntu 18.04 (Bionic) packages now available
W dniu 23/11/2018 o 12:44, Aki Tuomi pisze: Please find instructions on how to use them at https://repo.dovecot.org/ Thank you. I was always interested why those packages cannot be in upstream, but people maintain their own repositories for them. Is it too slow-moving to get a pkg into upstream or it is in upstream already, but not up to date? Other reasons? Just curiosity, as I'll be in similar position soon, trying to maintain own package. Best regards, Chris Narkiewicz
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1
Hi Aki,
Yes, the same connect error: imap(meuser,)Error:
net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
The mail location in "doveconf -n" (belo) does not reflect my preferences.
In Alpine, if I acknowledge the error I get, which tells me to add
novalidate-cert in my pinerc file, it prompts me for my POSIX username and
password. After that, I get my /var/spool/mail inbox, which is not what I
want.
doveconf -n
# 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10
# Hostname: myhost-t450s
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
protocols = " imap"
ssl_cert = You are still getting the connect error?
>
> Can you share your doveconf -n?
>
> Aki
>
> > On 31 December 2018 at 16:42 cpblpublic+dove...@gmail.com wrote:
> >
> >
> > Hi Aki,
> >
> > Thanks. I put that in the .conf file I mentioned, and it made no
> > difference.
> >
> > Chris
> >
> > > Hi!
> > >
> > > In your case it's probably easiest to change
> > >
> > > service stats {
> > > unix_listener stats-writer {
> > > mode = 0666
> > > }
> > > }
> > >
> > > Aki
> > >
> > > > On 31 December 2018 at 16:05 cpblpublic+dove...@gmail.com wrote:
> > > >
> > > >
> > > >
> > > > I use a localhost dovecot imap server in order to access maildir
> > > > folders from Alpine (see 2009 instructions for my set up here
> > > > https://cpbl.wordpress.com/2009/11/07/alpine-offlineimap-and-gmail-under-ubuntu/
> > > > )
> > > >
> > > > My configuration has worked nicely since 2009, but failed after a
> > > > recent upgrade of dovecot (during basic upgrade from Ubuntu 18.04 to
> > > > 18.10).
> > > >
> > > > The version of Alpine did not change. Uninstalling dovecot-core and
> > > > dovecot-imapd, and reinstalling the older versions, fully fixes the
> > > > problem.
> > > >
> > > > I would like to know how to get my configuration working with the new
> > > > version of dovecot.
> > > >
> > > > Here's what the problem looks like:
> > > >
> > > > I have a file auto-dovecot.conf with only the following two lines.
> > > >
> > > > verbose_proctitle = yes
> > > > mail_location =
> > > > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > > >
> > > > Other /etc/ files are unchanged from installation defaults.
> > > >
> > > > Alpine calls imapd when it starts, with the following command:
> > > >
> > > > /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-dovecot.conf
> > > >
> > > >
> > > > When I run that command, I get:
> > > >
> > > > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer)
> > > > failed: Permission denied
> > > > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> > > > SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
> > > > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
> > > > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
> > > > WITHIN
> > > > CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY]
> > > > Logged in as meuser
> > > >
> > > >
> > > >
> > > >
> > > > so I am guessing that the first line is the issue, since when the older
> > > > version I get instead:
> > > >
> > > > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
> > > > ENABLE
> > > > IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
> > > > THREAD=ORDEREDSUBJECT
> > > > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
> > > > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
> > > > WITHIN
> > > > CONTEXT=SEARCH LIST-STATUS BINARY MOVE] Logged in as meuser
> > > >
> > > >
> > > > Can anyone help me? I cannot find documentation for using a localhost
> > > > server for Alpine, except my own from 2009.
> > > >
> > > > Many thanks!
> > > >
> > > > Chris
> > >
>
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1 (fwd)
Hi Aki,
Oops; I did not think carefully enough about your instructions. Here is the
output from
$ dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
# 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
# Hostname: cpbl-t450s
doveconf: Warning: please set ssl_dh= /etc/dovecot/dh.pem
mail_location =
maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
service stats {
unix_listener stats-writer {
mode = 0666
}
}
ssl_dh = # hidden, use -P to show it
verbose_proctitle = yes
I then followed the instructions in the Warning lines above and now I get:
dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
# 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
# Hostname: cpbl-t450s
mail_location =
maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
service stats {
unix_listener stats-writer {
mode = 0666
}
}
ssl_dh = # hidden, use -P to show it
verbose_proctitle = yes
However, this does not help my primary symptom:
$ /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
Permission denied
* PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND
URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE
UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY]
Logged in as meuser
Thanks!
Chris
On Mon, Dec 31, 2018 at 10:27 AM Aki Tuomi wrote:
Did you remove the service section before taking doveconf -n? 'cos I
can't see it here..
Also, did you restart dovecot after config change?
Aki
> On 31 December 2018 at 16:54 Chris BL
wrote:
>
>
> Hi Aki,
>
> Yes, the same connect error: imap(meuser,)Error:
net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
>
> The mail location in "doveconf -n" (belo) does not reflect my
preferences.
>
> In Alpine, if I acknowledge the error I get, which tells me to add
> novalidate-cert in my pinerc file, it prompts me for my POSIX username
and
> password. After that, I get my /var/spool/mail inbox, which is not
what I
> want.
>
>
> doveconf -n
> # 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.2 ()
> # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10
> # Hostname: myhost-t450s
> mail_location = mbox:~/mail:INBOX=/var/mail/%u
> mail_privileged_group = mail
> namespace inbox {
> inbox = yes
> location =
> mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special_use = \Junk
> }
> mailbox Sent {
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> special_use = \Sent
> }
> mailbox Trash {
> special_use = \Trash
> }
> prefix =
> }
> passdb {
> driver = pam
> }
> protocols = " imap"
> ssl_cert = ssl_client_ca_dir = /etc/ssl/certs
> ssl_dh = # hidden, use -P to show it
> ssl_key = # hidden, use -P to show it
> userdb {
> driver = passwd
> }
>
>
>
> On Mon, 31 Dec 2018, Aki Tuomi wrote:
>
> > You are still getting the connect error?
> >
> > Can you share your doveconf -n?
> >
> > Aki
> >
> > > On 31 December 2018 at 16:42 cpblpublic+dove...@gmail.com wrote:
> > >
> > >
> > > Hi Aki,
> > >
> > > Thanks. I put that in the .conf file I mentioned, and it made no
> > > difference.
> > >
> > > Chris
> > >
> > > > Hi!
> > > >
> > > > In your case it's probably easiest to change
> > > >
> > > > service stats {
> > > > unix_listener stats-writer {
> > > > mode = 0666
> > > > }
> > > > }
> > > >
> > > > Aki
> > > >
> > > > > On 31 De
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1 (fwd)
Hi Aki,
I am not 100% sure. I do not really know what my OS does. "ps -A" says
dovecot is running, but if I say "sudo service dovecot stop", my email
client works fine still (that is, assuming the old version of dovecot is
installed). ie the on-demand instances of dovecot-imapd that Alpine calls
are, I assume, started and stopped by Alpine. (?)
Is my OS using dovecot for other stuff, like managing system mail (e.g.
root's reports of cron jobs, which still go to /var/mail, but I never read
them)? I am not certain. But I believed that any other use of dovecot
would be using a different .conf file, so I was hoping I just had to know
how to make my custom one (-c flag) work.
Thanks,
Chris
On Mon, 31 Dec 2018, Aki Tuomi wrote:
> Am I assuming correctly that you are not, in fact, running anything but imap
> process from Dovecot?
>
> Aki
>
> > On 31 December 2018 at 18:35 Chris BL wrote:
> >
> >
> >
> > Hi Aki,
> >
> > Oops; I did not think carefully enough about your instructions. Here is the
> > output from
> >
> > $ dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> >
> > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # Pigeonhole version 0.5.2 ()
> > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > # Hostname: cpbl-t450s
> > doveconf: Warning: please set ssl_dh= > doveconf: Warning: You can generate it with: dd
> > if=/var/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam
> > -inform der > /etc/dovecot/dh.pem
> > mail_location =
> > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > service stats {
> > unix_listener stats-writer {
> > mode = 0666
> > }
> > }
> > ssl_dh = # hidden, use -P to show it
> > verbose_proctitle = yes
> >
> >
> > I then followed the instructions in the Warning lines above and now I get:
> >
> > dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # Pigeonhole version 0.5.2 ()
> > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > # Hostname: cpbl-t450s
> > mail_location =
> > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > service stats {
> > unix_listener stats-writer {
> > mode = 0666
> > }
> > }
> > ssl_dh = # hidden, use -P to show it
> > verbose_proctitle = yes
> >
> >
> > However, this does not help my primary symptom:
> >
> >
> > $ /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
> > Permission denied
> > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
> > SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
> > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE
> > UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
> > WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+
> > NOTIFY] Logged in as meuser
> >
> >
> > Thanks!
> > Chris
> >
> >
> >
> > On Mon, Dec 31, 2018 at 10:27 AM Aki Tuomi
> > wrote:
> > Did you remove the service section before taking doveconf -n? 'cos I
> > can't see it here..
> >
> > Also, did you restart dovecot after config change?
> >
> > Aki
> >
> > > On 31 December 2018 at 16:54 Chris BL
> > wrote:
> > >
> > >
> > > Hi Aki,
> > >
> > > Yes, the same connect error: imap(meuser,)Error:
> > net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
> > >
> > > The mail location in "doveconf -n" (belo) does not reflect my
> > preferences.
> > >
> > > In Alpine, if I acknowledge the error I get, which tells me to add
> > > novalidate-cert in my pinerc file, it prompts me for my POSIX
> > username and
> > > password. After that, I get my /var/spool/mail inbox, which is not
> > what I
> > > want.
> > >
> > >
> > > doveconf -n
> > > # 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
> > > # Pigeonhole version 0.5.2 ()
> > > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10
> &g
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1 (fwd)
Hi Aki,
Bingo! Putting that line in dovecot.conf rather than my custom file, and
then using my old custom file, unchanged, from Alpine works perfectly.
My only remaining question, then, would be ... what was this (change)
about / or is there any documentation that I should/could have read to
know what to do without asking a lead developer?
Thank you very much.
I hope you had a great start to the New Year!
Happy 2019,
Chris
On Mon, 31 Dec 2018, Aki Tuomi wrote:
> If your client directly executes imap binary, it is not mandatory for dovecot
> to be running, but you need to put the service section into
> /etc/dovecot/dovecot.conf and restart the dovecot service.
>
> Putting it to your local config file does not affect the global service.
>
> Aki
>
> > On 31 December 2018 at 22:16 Chris BL wrote:
> >
> >
> >
> > Hi Aki,
> >
> > I am not 100% sure. I do not really know what my OS does. "ps -A" says
> > dovecot is running, but if I say "sudo service dovecot stop", my email
> > client works fine still (that is, assuming the old version of dovecot is
> > installed). ie the on-demand instances of dovecot-imapd that Alpine calls
> > are, I assume, started and stopped by Alpine. (?)
> >
> > Is my OS using dovecot for other stuff, like managing system mail (e.g.
> > root's reports of cron jobs, which still go to /var/mail, but I never read
> > them)? I am not certain. But I believed that any other use of dovecot
> > would be using a different .conf file, so I was hoping I just had to know
> > how to make my custom one (-c flag) work.
> >
> > Thanks,
> > Chris
> >
> >
> > On Mon, 31 Dec 2018, Aki Tuomi wrote:
> >
> > > Am I assuming correctly that you are not, in fact, running anything but
> > > imap process from Dovecot?
> > >
> > > Aki
> > >
> > > > On 31 December 2018 at 18:35 Chris BL
> > > > wrote:
> > > >
> > > >
> > > >
> > > > Hi Aki,
> > > >
> > > > Oops; I did not think carefully enough about your instructions. Here is
> > > > the output from
> > > >
> > > > $ dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > >
> > > > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > # Pigeonhole version 0.5.2 ()
> > > > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > > > # Hostname: cpbl-t450s
> > > > doveconf: Warning: please set ssl_dh= > > > doveconf: Warning: You can generate it with: dd
> > > > if=/var/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam
> > > > -inform der > /etc/dovecot/dh.pem
> > > > mail_location =
> > > > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > > > service stats {
> > > > unix_listener stats-writer {
> > > > mode = 0666
> > > > }
> > > > }
> > > > ssl_dh = # hidden, use -P to show it
> > > > verbose_proctitle = yes
> > > >
> > > >
> > > > I then followed the instructions in the Warning lines above and now I
> > > > get:
> > > >
> > > > dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > # Pigeonhole version 0.5.2 ()
> > > > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > > > # Hostname: cpbl-t450s
> > > > mail_location =
> > > > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > > > service stats {
> > > > unix_listener stats-writer {
> > > > mode = 0666
> > > > }
> > > > }
> > > > ssl_dh = # hidden, use -P to show it
> > > > verbose_proctitle = yes
> > > >
> > > >
> > > > However, this does not help my primary symptom:
> > > >
> > > >
> > > > $ /usr/lib/dovecot/imap -c
> > > > /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer)
> > > > failed: Permission denied
> > > > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> > > > SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=OR
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1
Hello Stephan,
I'm interested to see this get fixed (and ideally one day for it to be
really easy for people to set up Alpine so that it competes fairly with
heavier GUIs!), but I do not know that I am qualified to help (beyond
light testing). Is there someone from Alpine on this thread? The second
(askubuntu.com) link you give is my own question (and answer, based on Aki
Tuomi's earlier responses).
Thanks and best wishes,
Chris
On Sun, 6 Jan 2019, Stephan Bosch wrote:
>
> Op 31/12/2018 om 16:53 schreef Chris Barrington-Leigh:
> > I then followed the instructions in the Warning lines above and now I get:
> >
> > dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # Pigeonhole version 0.5.2 ()
> > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > # Hostname: cpbl-t450s
> > mail_location =
> > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > service stats {
> > unix_listener stats-writer {
> > mode = 0666
> > }
> > }
> > ssl_dh = # hidden, use -P to show it
> > verbose_proctitle = yes
> >
> >
> > However, this does not help my primary symptom:
> >
> >
> > $ /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
> > Permission denied
> > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
> > SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND
> > URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED
> > I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH
> > LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY] Logged in as meuser
> >
>
> I notice that you use an explicit configuration file for your imap invocation.
> Which likely means that this is something different from the config used the
> main dovecot service running on your system. That service block needs to be in
> the configuration of the dovecot instance running the stats service (which is
> likely just the main /etc/dovecot/dovecot.conf). Given the location of your
> own config file, I'm assuming you cannot manage the main dovecot instance?
>
> BTW, there is a related thread with a different solution to this issue:
>
> https://www.dovecot.org/pipermail/dovecot/2019-January/114151.html
>
> We're still working on getting that bug fixed, so that will not help you
> immediately, unless you can patch dovecot.
>
> BTW, I find it a bit weird that Alpine would fail on this. It is not a fatal
> error, so the imap service will just work. The error is sent to /dev/stderr,
> so I'd normally expect a service like Alpine to be able to ignore output sent
> there.
>
> The question below is about the same issue and their config suggests that
> stderr is sent to a log file:
>
> https://askubuntu.com/questions/1104056/how-to-use-local-dovecot-imap-with-alpine-18-10-changes
>
> So, I really don't understand what the core of the problem is. Why does Alpine
> fail on this at all?
>
> Regards,
>
> Stephan.
>
>
>
>
Dovecot 2.3 no longer accepts ssl_key_password
Hit a little problem when I upgraded a system from FreeBSD 10.3 to 11.2. I did not receive any errors in the upgrade. The system is running 4 jails and everything seems to work except in Dovecot dovecot-2.3.4_5 where when using the exact same configuration which worked in 10.3 with the same password protected certificate key. (doveconf -n -P shows the correct password.) ssl_ca =
Test on sending only
I had to move off of a server to this one too fast. Having problems If this goes through, if someone could reply to ch...@bennettconstruction.us instead of on-list. Thanks, Chris Bennett
Thanks, got replys
I've got a mix of some receive only, some send only. I tried to slam together two different configs really fast. This helped. I'll look things over and post some configs after a bit. Thanks again! Chris Bennett
Problems, virtual and alias, Dovecot, Neomutt, OpenSMTPD on OpenBSD
I'm having some problems. I'm using virtual users with real domains.
That is working, but the folders are not showing up properly in neomutt,
for both virtual users and system users.
When I use K9 on Android, the folders show up correctly.
Sending is working, but I don't have auth setup right on OpenSMTPD, so
I'm sending locally, not remotely (including from K9, which can't login
to send). That's really probably not relevant here, but I thought I'd
mention it. I'm a bit embarrassed to admit that after having to move
everything off of one server super rapidly to another already working
to save money, I realized that I had made some wrong assumptions.
Oops but +1 for me. So now I know what to fix there and why.
If anyone has a good .neomuttrc or hints, I would appreciate that help.
But I can save that problem for the neomutt mailing list.
Here are copies of my doveconf -n, dovecot/passwd, vaddr, vdomains and
vusers. Obscured of course. Plus the DNS IP's of this server and the one
other server, which isn't running dovecot yet. Everything is using
maildir on both servers. DNSSEC on all but bennettconstruction.us and
mail.bennettconstruction.us
Is there anything wrong, missing or screwed up here?
I do intend to use encrypted passwds once I get submission to OpenSMTPD
working and add the right ports for that.
So far, I really haven't found the three different dovecot logs to be
particularly helpful, but I can send excerpts of those if helpful.
Thanks. I appreciate any help. This is my first go at IMAP and someone
sent me some config info a while ago, which really helped a ton.
I really like this software, I used it previously a few years ago for
POP3. I do find the documentation missing some sample configs with some
explanations for why and how. At some point I would like to add some
more complicated things, but I don't understand what some config options
really do. I also understand that documentation vs. code work are always
in conflict. 28 hour days would really help! :-}
doveconf -n:
# 2.3.10.1 (a3d0e1171): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.10 (bf8ef1c2)
# OS: OpenBSD 6.7 amd64 ffs
# Hostname: cowboyup.xyz
auth_debug = yes
auth_debug_passwords = yes
auth_socket_path = /var/dovecot/auth-userdb
auth_verbose = yes
auth_verbose_passwords = sha1
debug_log_path = /var/log/dovecot-debug.log
first_valid_uid = 1000
hostname = consulting-diy-construction.com
imap_id_log = *
imap_id_send = *
info_log_path = /var/log/dovecot-info.log
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
lmtp_client_workarounds = whitespace-before-path mailbox-for-path
log_path = /var/log/dovecot.log
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_debug = yes
mail_location = maildir:/home/vmail/%n@%d/Maildir
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext
mbox_write_locks = fcntl
mmap_disable = yes
namespace inbox {
inbox = yes
list = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
subscriptions = yes
}
passdb {
driver = bsdauth
}
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
plugin {
mail_log_fields = uid box msgid size
sieve = file:~/sieve;active=~/.dovecot.sieve
}
postmaster_address = postmas...@consulting-diy-construction.com
protocols = imap lmtp sieve
service auth-worker {
user = root
}
service auth {
unix_listener auth-userdb {
group = vmail
mode = 0666
user = vmail
}
}
service dict {
unix_listener dict {
group = vmail
mode = 0600
user = vmail
}
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
service_count = 1
}
service imap {
process_limit = 1024
}
service lmtp {
unix_listener lmtp {
mode = 0666
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
inet_listener sieve_deprecated {
port = 2000
}
}
ssl_dh = # hidden, use -P to show it
ssl_prefer_server_ciphers = yes
userdb {
driver = passwd
}
userdb {
args = uid=vmail gid=vmail home=/home/vmail/%n@%d
driver = static
}
verbose_ssl = yes
protocol lmtp {
mail_plugins =
}
protocol imap {
mail_max_userip_connections = 10
mail_plugins =
}
protocol lda {
mail_plugins =
}
local 172.107.198.230 {
ssl_cert =
Re: Providers running dovecot?
On Sat, Oct 10, 2020 at 04:31:07PM +0200, Olivier Cailloux wrote: > Le vendredi 09 octobre 2020 à 11:22 +0200, Piotr Auksztulewicz a > écrit : > > On Fri, Oct 09, 2020 at 10:15:10AM +0200, Olivier Cailloux wrote: > > > The real, “final” question I am interested in is, but which might be > > > slightly off-topic on this list (the reason I asked the other > > > question), is to find providers that satisfy these two conditions: > > > a) offer free e-mail accounts > > > b) implement correctly the IMAP SEARCH feature of RFC 3501. > > > > IMO this is the right question to ask, even here. > > You are probably right, in retrospect, I should have started with that > question. > > > > > > That is because my client uses the IMAP SEARCH feature, and it is > > > usually implemented incorrectly (e.g. in GMail or MS Exchange). > > > > Probably it would be more informative to describe which features you > > need that are implemented "incorrectly". > > Well, support of the IMAP SEARCH command is the specific feature I > need. See https://tools.ietf.org/html/rfc3501#section-6.4.4. > > > > > Reality check: RFCs are not government-enforced standards. There are many > > sensible RFCs that never got implemented widely, or nearly at all, are > > implemented partially, or there are widely deployed not-fully-compliant > > software systems. If your client software requires a feature that's not > > widely available, you're just limiting your audience. > > Sad but true, I believe you are completely right. I will perhaps have > to abandon my hope of finding compliant providers. > > > > > You may try to find a different way to achieve your goal using the > > features that are widely implemented. Real, successful software packages > > very often contain options to do some quirks in order to stay interoperable > > with existing noncompliant implementations. > > This is precisely the problem in my case: it is impossible to work > nicely around the lack of IMAP SEARCH feature on the side of my > software, which is client-side, because that support must be provided > server side. In a nutshell, the SEARCH command lets a client ask a > server: “give me all e-mails whose subject and date match such and such > criteria”. As a client, if the server does not implement IMAP SEARCH, I > simply can’t know which e-mails match such and such criteria, short of > downloading all e-mail headers and filtering them, which is orders of > magnitude slower if my user has many e-mails in her box. > > Admittedly, I can work around this more or less nicely, e.g. by > downloading all headers once, storing them on the device of my user, > and searching this local database, instead of re-downloading all > headers every time my software runs. (This is how Thunderbird, and, I > suppose, most MUAs out there, work.) But this creates other > inconvenience for the user: this database takes space, takes time and > bandwidth to build, has to be re-built when the user changes device, > there is a security issue with having these e-mail headers stored > locally; not talking about the fact that it will make my software much > more complex for a single feature that really should, conceptually, be > implemented server side. Hence my willingness to actively try to find > compliant providers before giving up. > > An alternative is to try to understand what exactly bugs in the > implementation of IMAP SEARCH of each of the main providers out there > (GMail; MS Exchange; and so on) and work around this on a case-by-case > basis. I suppose this has been investigated already by some developers; > if anybody knows where I could ask about this, I’d be very happy to ask > there, as I guess this discussion is becoming completely OT for this > list. > > But the general lack of support for remote search in well known > softwares such as Thunderbird (that seem to systematically perform > searches client-side, on the local database) makes me rather > pessimistic about the possibility of working around those bugs; after > all, if the server sometimes (or often) replies incorrectly, as my > tests indicate, there may be nothing the client can do to guess what > the right answer is. > -- > Olivier > > Some projects just can't get around the lack of compliance from vendors, whether it's from incompetence or their not needing/desiring to do it. Several years ago, I wanted to move lpd forward. After weeks of looking at the hardware with non-compliance, I finally just had to drop the project because it was truly a vast and hopeless situation. It was an excellent, but frustrating lesson. I wish you luck, but you might be in the same spot I found myself. Chris Bennett
Move older messages to Old folder after a bunch of changes (sql + other stuff)
I'm starting to get things better done now. Moved to all virtual users over IMAP. Using a SQL database for users/passwords. Setting up folders working properly with neomutt. Right now, that leaves me with just new mail in the INBOX, and other folders. I have a bunch of old mail in each users Maildirs in their real home folders. Right now, I'd like to move all the old mail into a folder (doesn't really matter what it's called). Right now is actually a perfect time to do this by good luck in timing. How should I go about creating a new folder for those messages and then adding the old mail from system user home Maildir folders to the virtual users folders I'm using? Thanks for any help, Chris Bennett
Re: Recommended Protocols?
On Tue, Nov 10, 2020 at 09:07:37AM -0600, Raymond Herrera wrote: > > I have arrived to a preliminary conclusion. The error that I am getting is > this: > > dovecot: imap-login: Disconnected: TLS: SSL_read() failed: SSL > routines:ssl3_read_bytes:sslv3 alert bad certificate: SSL alert number 42 > SSLv3 is no longer considered safe. You shouldn't use it. If the mail client program can't use something newer (I only allow TLSv2 and TLSv3), I would question whether it's a good choice. Because runaway spam is the kiss of death with blacklisting and IP reputation. Isn't sslv3 disallowed in the default config??? If it's only for reading and not sending mail, then just manually set it up as sslv3 allowed. Hopefully this is helpful. Debug logs can be helpful \o/, or just confusing :-[ but they are long to read through. Good luck! Chris Bennett
Re: Dovecot and mutt
On Mon, Jan 25, 2021 at 08:52:14AM -0700, @lbutlr wrote: > I have tried, but failed,. To write a sieve and script to strip HTML parts of > messages and if the message is only HTML to pipe it through w3m and add the > html portion as an emo attachment (in case it has links that need clicking, > like on some 'confirm you exist' emails. I truly hate those. Most often they now require Javascript, too. I use ssh and neomutt. I'm going to write a macro to take the html attachment and put it in a website directory. I've been doing it the long hard manual way. I hate forced javascript. No excuse but sloppiness to have that on a confirm you exist page. In any case, this is just nice to vent a little steam out. I don't think we can do much except chastise users of mailing lists. Sounds like a good macro to send a polite form letter reply to evildoers. > Honestly, I do not main HTML per se, it is when the HTML specifies font size, > colors, background colors, and other garbage like that that I despise it. A > well formed HTML message is is fine, but those are very rare. +1 Thanks for the great software and long hard work to find the most miniscule hidden bugs! Chris Bennett
Re: invalid vsize-hdr
nd trailing whitespace "
>>
>> # Most (but not all) settings can be overridden by different protocols
>> and/or
>> # source/destination IPs by placing the settings inside sections, for
>> example:
>> # protocol imap { }, local 127.0.0.1 { }, remote 10.0.0.0/8 { }
>>
>> # Default values are shown for each setting, it's not required to uncomment
>> # those. These are exceptions to this though: No sections (e.g.
>> namespace {})
>> # or plugin settings are added by default, they're listed only as examples.
>> # Paths are also just examples with the real defaults being based on
>> configure
>> # options. The paths listed here are for configure --prefix=/usr
>> # --sysconfdir=/etc --localstatedir=/var
>>
>> # Protocols we want to be serving.
>> #protocols = imap pop3 lmtp
>> protocols = imap pop3
>> # A comma separated list of IPs or hosts where to listen in for connections.
>> # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces.
>> # If you want to specify non-default ports or anything more complex,
>> # edit conf.d/master.conf.
>> #listen = *, ::
>> listen = *, ::
>>
>> # Base directory where to store runtime data.
>> #base_dir = /var/run/dovecot/
>>
>> # Name of this instance. In multi-instance setup doveadm and other commands
>> # can use -i to select which instance is used (an
>> alternative
>> # to -c ). The instance name is also added to Dovecot processes
>> # in ps output.
>> #instance_name = dovecot
>>
>> # Greeting message for clients.
>> #login_greeting = Dovecot ready.
>>
>> # Space separated list of trusted network ranges. Connections from these
>> # IPs are allowed to override their IP addresses and ports (for logging and
>> # for authentication checks). disable_plaintext_auth is also ignored for
>> # these networks. Typically you'd specify your IMAP proxy servers here.
>> #login_trusted_networks =
>> login_trusted_networks = 10.5.1.0/24
>> # Space separated list of login access check sockets (e.g. tcpwrap)
>> #login_access_sockets =
>>
>> # With proxy_maybe=yes if proxy destination matches any of these IPs,
>> don't do
>> # proxying. This isn't necessary normally, but may be useful if the
>> destination
>> # IP is e.g. a load balancer's IP.
>> #auth_proxy_self =
>>
>> # Show more verbose process titles (in ps). Currently shows user name and
>> # IP address. Useful for seeing who are actually using the IMAP processes
>> # (eg. shared mailboxes or if same uid is used for multiple accounts).
>> #verbose_proctitle = no
>>
>> # Should all processes be killed when Dovecot master process shuts down.
>> # Setting this to "no" means that Dovecot can be upgraded without
>> # forcing existing client connections to close (although that could also be
>> # a problem if the upgrade is e.g. because of a security fix).
>> #shutdown_clients = yes
>>
>> # If non-zero, run mail commands via this many connections to doveadm
>> server,
>> # instead of running them directly in the same process.
>> #doveadm_worker_count = 0
>> # UNIX socket or host:port used for connecting to doveadm server
>> #doveadm_socket_path = doveadm-server
>>
>> # Space separated list of environment variables that are preserved on
>> Dovecot
>> # startup and passed down to all of its child processes. You can also give
>> # key=value pairs to always set specific settings.
>> #import_environment = TZ
>>
>> ##
>> ## Dictionary server settings
>> ##
>>
>> # Dictionary can be used to store key=value lists. This is used by several
>> # plugins. The dictionary can be accessed either directly or though a
>> # dictionary server. The following dict block maps dictionary names to URIs
>> # when the server is used. These can then be referenced using URIs in format
>> # "proxy::".
>>
>> dict {
>> #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
>> #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext
>> }
>>
>> # Most of the actual configuration gets included below. The filenames are
>> # first sorted by their ASCII value and parsed in that order. The
>> 00-prefixes
>> # in filenames are intended to make it easier to understand the ordering.
>> !include conf.d/*.conf
>>
>> # A config file can also tried to be included without giving an error if
>> # it's not found:
>> !include_try local.conf
>> ---
>>
>> I need assistance. I appreciate the help.
>>
>> Chris
>>
>>
>>
>> --
>> Christopher Wensink
>> IS Administrator
>> Five Star Plastics, Inc
>> 1339 Continental Drive
>> Eau Claire, WI 54701
>> Office: 715-831-1682
>> Mobile: 715-563-3112
>> Fax: 715-831-6075
>> cwens...@five-star-plastics.com
>> www.five-star-plastics.com
>>
>>
>
> For what it's worth... I know less than nothing, but a quick search
> turned up an apparent issue with cpanel which sounds similar:
>
> https://forums.cpanel.net/threads/dovecot-errors.626131/
>
> John Tulp
>
>
Re: Password Schemes
On Sat, Sep 11, 2021 at 08:07:31PM -0500, John Schmerold wrote:
> My /etc/dovecot/conf.d/auth-passwdfile.conf.ext is configured to use MD5
>
>passdb {
> driver = passwd-file
> args = scheme=MD5 username_format=%n /etc/exim4/domains/%d/passwd
>}
>
>userdb {
> driver = passwd-file
> args = username_format=%n /etc/exim4/domains/%d/passwd
>}
>
> /home/account/conf/mail/domain.com/passwd has a mixture of MD5 &
> SHA512-CRYPT:
>
>
> scanner:{MD5}$1$M5QuU7QI$AE7Nnorb8KC5KMvyYfVcr0:account:mail::/home/account:0:userdb_quota_rule=*:storage=0M
>
> test:{SHA512-CRYPT}$6$towo0IVjzBgZ0htU$uTFbyJ3aPunrhsEEC2alHz6SEuPyBdL3JYDWc6Z0ZtA2cMFjFVJNqAwn04OKQfsu99DNcDGu21zkvdYbsPmgJ0:account:mail::/home/account:0:userdb_quota_rule=*:storage=0M
>
> Everything is working fine, is this by design? In other words does the {MD5}
> vs {SHA512-CRYPT} in passwd over-rule auth-passwdfile.conf.ext ?
>
If you can, I would get rid of MD5. It's no longer secure. Sending out
mountains of spam if a password gets cracked, could be problematic. :-{
I'm getting ready to drop using MD5 on secure cookies for that very
reason. Website software, not dovecot.
Hopefully that's helpful. I dropped one of my bare metal servers because
the company couldn't keep other spammers off of the IP block I was in.
They refused to do anything to clean up their blacklist, which included
me unfortunately.
Chris Bennett
Re: Strategies for protecting IMAP (e.g. MFA)
On Mon, Nov 15, 2021 at 02:14:31PM -0600, Tyler Montney wrote: > I've just recently started using mailing lists, so replying is still a bit > awkward to me. (Probably be easier if we'd use forums.) Forums are a mixed bag. Some love them, some hate them. I previously ran a forum for LedgerSMB. It had some usage, but people mostly preferred the mailing lists. I was also running an add-on by the forum author that allowed posting incoming emails to boards. I never setup an outgoing to the mailing list. I wanted to. No time. However, reading and searching emails from a mailing list in a forum is *nice*. Really nice. You can see the entire thread at once. If any mailing list I'm part of wants to do a read-only posting of email only messages like that, I would put up a forum for that only, for free. I would only make something like that public if the mailing list owners wanted it done. - As far as 2FA over a phone, I hate it. When traveling, my phone only works in USA, Canada and Mexico. If I were in Guatemala, I'd have no access to my DNS server company. Ugh! -- Chris Bennett
Re: silly quesiton
On Tue, Jan 25, 2022 at 03:50:12AM -0900, justina colmena ~biz wrote: > Maybe a future programming project idea: I want a system that will store all > mail messages and user account info in, say, a postgresql transactional > database, a little more manageable and reliable than ad hoc databasing with > those flat files all over the place cluttering up the system. > I am in progress moving towards something like that. As of right now, perl, dovecot for IMAP, neomutt and OpenSMTPD. Right now, .neomuttrc files *only* exist during the usage of neomutt. They have random names, cannot be written to and are immediately erased after neomutt starts (not quits). That is a very small window of threat. I would very much like to put all of the messages into PostgreSQL also instead of file folders under the user vmail. This is just a side project. As I have been advised, there is no need to even write a configuration file at all, but there are some issues with dbh that I need to solve with a different database module. If someone can read files that never exist, well... At some point you have to at least consider trusting something. That or just turn it all off and get another career. -- Chris Bennett
Re: silly quesiton [ot]
On Sun, Jan 30, 2022 at 09:46:53PM -0500, dove...@ptld.com wrote: > Storing mail in a db... at the end of the day isn't it still just a file (.db > file) on the drive? > Aren't you just adding bloat and complexity vs just storing the mail directly > (maildir format) to a file on the drive? > > What do you think you are saving? Security? > If someone can read files on your server, they can equally read a maildir or > a .db file. > K.I.S.S. I gain modularity for a system. The database is the foundation. I am working with: 1. Dovecot 2. Neomutt 3. OpenSMTPD Now, if I decide to drop or addon some new program, I can just adjust and/or add some new tables. Write a new stored procedure. Drop in a new Perl module or subroutine. 1. Dovecot 2. Neomutt 3. OpenSMTPD 4. Xyz 5. Abc 6. SuperDuperMail-ThingyPlus So what I am working for is a system that is united. Add a new user and email, CLI program, bang. All done. Change a password with a web interface. Click. All done. I'm in no rush. This is a fun side project. I have already done this type of work successfully for other kinds of projects, so it's different, but not really outside of my past experience. Secure today is wide open tomorrow. File, memory, etc. all get broken eventually. I'm much more worried about my own mistakes than that of others. :-* -- Chris Bennett
Re: silly quesiton [ot]
On Mon, Jan 31, 2022 at 06:23:28AM +, Sam Kuper wrote:
> On Sun, Jan 30, 2022 at 07:49:56PM -0900, justina colmena ~biz wrote:
> > On January 30, 2022 6:30:44 PM AKST, Sam Kuper wrote:
> >> On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote:
> >>> On January 30, 2022 5:46:53 PM AKST, dove...@ptld.com wrote:
> >>>> Storing mail in a db... at the end of the day isn't it still just a
> >>>> file (.db file) on the drive?
> >>>>
> >>>> Aren't you just adding bloat and complexity vs just storing the
> >>>> mail directly (maildir format) to a file on the drive? [...]
> >>>
> >>> You'll get better indexing and fast full text search by storing your
> >>> emails in a database rather than a flat file, hopefully after
> >>> decoding any attachments. Especially for spam scoring, analysis, and
> >>> classification. Much better performance deleting or moving specific
> >>> messages, too.
> >>
> >> Do you have evidence to back up these claims, specifically re: mail
> >> servers?
> >>
> >> Like-for-like benchmarks, for instance?
> >
> > Just ideas.
>
> OK, no then.
>
>
> > Removing or deleting a single message from near the beginning of a
> > large flat file takes an inordinate amount of time because the
> > remainder of the flat file has to be rewritten all the way from the
> > point of the deleted message to the end of the file and then
> > truncated.
>
> You might want to look up what Maildir is before making bold but
> apparently unfounded claims about it.
>
> Maildir is not a "large flat file". It is a set of conventions that
> amount to a database specification, in the traditional sense of the word
> "database": a system for storing data. (Not a relational database.)
>
Many people haven't ever had to deal with the old "database" style of
files instead of tables and columns.
Maildir does show it's age with the little complexities it has.
> DJB developed Maildir to gain performance and reliability improvements
> over mbox files. Unlike Maildirs, mbox files *are* "large flat files".
Corrupt your mbox file and bad things happen!
I also like being able to throw in some older backed up email when I
find I need a few more to fill out that important thread from 3 years
ago with Maildir.
Maildir does not have the relational database problem of needing to keep
up with updates to the database software.
And nothing works very well when you suddenly discover that the company
you are renting servers from decides to close up and turn everything
off. While you are in another country with internet cafes only and don't
even have a laptop with you! Happened to me once. 8-{
--
Chris Bennett
Re: dovecot mailing list (this mailing list), DKIM, SPF and DMARC
On Sat, Feb 12, 2022 at 12:58:03PM +0100, Sebastian Nielsen wrote: > Thats a TLD ban. Meaning *.ru is banned. > > same applies for my domain for example, I ban *.xyz, *.date and a few others. > I don't understand at all why banning tld is reasonable. I'm not rich. I buy .rocks and .xyz .rocks really works well with the domain name. .xyz is short, memorable and easy to type. I can't afford to buy domain names that cost $200 a year to purchase. .com .net .info , etc. have run out of the names I wish to use. I have never ever sent a single spam email, but you would block my emails? Bluntly said, but without malice, that attitude favors the rich over the poor. I refuse to trust the BIG guys. My dad uses yahoo and gets emails yanked away while he is reading it. Also, I can't find a server company that has IP blocks that are clean enough. I truly wish I could. There are many other methods to block spam. IMHO, blocking by tld is a bit harsh. But you have the right to do whatever method you wish. I will only point out my thoughts. SPAM sucks! :-) -- Chris Bennett
Re: email location - files or sql
On Wed, Feb 23, 2022 at 08:48:11PM -0500, Robert Moskowitz wrote: > > > I liked Maildir at the time and still do. Only a 1,000+ emails a day. My > wife keeps a lot (10K messages) on the server, I keep all of my various > boxes on the server small. So do my other users. > > > Personally, I would not use a relational database as a mail store unless > > specifically required by the mail system of your choice. Email is not > > organised in a way that benefits from a RDB. > > I was there for the beginning of RDB. Almost had NOMAD shoved down my > throat (UNIVAC) and did work with RIM (BCS) then R:Base. Was gamma release > site of DB2, where we worked out how to do a UNION which was not supported > in the original design. I have seen email systems that stuff the messages > into RDB and really wonder if the hammer really fits. > What is the opinion on repairs to problems that occur? Easier to fix in a DB? That does give a simple set of fixed points and dates to make repairs simple. I backup into a tar.gz and that preserves a timepoint, but how in the heck could a fix a problem that occurs? That seems like a nightmare problem. Does anyone have any method to do that? I'd love to know that if it exists. I love Maildir, but the backups are very large. I don't have a huge set of emails, but downloading a copy to home instead of my other server is a big task. Since I use a single ISP and once had a company shut down all servers, which makes me nervous. -- Hmm, Chris Bennett
Re: Problems with userdb and mail deliveribility
I realize that I dumpimg a lot of info out there. Good news is that I´m online but not in production. I can shoot for system users, but I really want to get the virtual users working. This is a bit of a project. Thanks for anyone who wants to help. I was in a hotel room when I started and now on Amtrak train with connections fading in and out. Someone shared some of their configs, but I still confused. I may just scrap what Ie done and start fresh. Would if be helpful if I followed up with each file inline I was kirring then 40kb limit. Thanks, Chris Bennett
Re: Problems with userdb and mail deliveribility
On Sun, May 17, 2020 at 01:00:53PM -0600, @lbutlr wrote: > On 17 May 2020, at 11:08, Chris Bennett > wrote: > > > > I realize that I dumpimg a lot of info out there. > > But not what we need. > > doveconf -n > postfix -n (or equivalent) > > What MTA you are using (postfix, etc) > > Errors in mail.log showing the failures. > > > > I can shoot for system users, but I really want to get the virtual users > > working. This is a bit of a project. > > user=vmail delay=2h50m40s result=TempFail stat=Error (temporary failure: > "mail.lmtp: LMTP server error: 550 5.1.1 > User doesn't exist: > > Probably your LMTP lookup is malformed, probably in your MTA. > > lmtp(rodeo)<45419>: Error: > mkdir(/var/vmail//rodeo/Maildir) failed: Permission denied (euid=1000(rodeo) > egid=1000(rodeo) missing +w perm: /var, dir owned by 0:0 mode=0755) > > You have permission issues. vmail (and all files and folders in vmail) should > be owned by the vmail user, not by root. > Thanks. Sorry for such a delay in responding, but I was out in the boonies for awhile. I am using OpenSMTPD. The problems for this were indeed as you suggested and with the passwd files. This is all working now, but I have some other problems which I will work on too. I will continue to work on those and ask another thread if I cannot get that worked out. Thanks again. Chris Bennett
Panic: file auth-request.c
erdb {
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
verbose_ssl = yes
protocol lmtp {
mail_fsync = optimized
}
protocol lda {
mail_fsync = optimized
}
protocol imap {
mail_max_userip_connections = 50
}
protocol pop3 {
mail_max_userip_connections = 20
}
--
Chris Wik
Anu Internet Services
www.anu.net | www.cwik.ch
Re: Panic: file auth-request.c
From: Aki Tuomi To: Dovecot Mailing List , Chris Wik Sent: 18/09/2016 8:06 AM Subject: Re: Panic: file auth-request.c > Sep 17 19:34:57 mail dovecot: auth: Panic: file auth-request.c: line 1049 > (auth_request_lookup_credentials): assertion failed: > (request->credentials_scheme == scheme) > Sep 17 19:34:57 mail dovecot: auth: Error: Raw backtrace: > /usr/local/lib/dovecot/libdovecot.so.0(+0x89470) [0x7fa9cb8af470] -> > /usr/local/lib/dovecot/libdovecot.so.0(+0x8954e) [0x7fa9cb8af54e] -> > /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fa9cb851f75] -> > dovecot/auth() [0x4165bc] -> dovecot/auth() [0x4221fb] -> dovecot/auth() > [0x41620b] -> dovecot/auth(auth_request_lookup_credentials_callback+0x58) > [0x4162f8] -> dovecot/auth(passdb_handle_credentials+0x6a) [0x4254ba] -> > dovecot/auth() [0x425b62] -> dovecot/auth() [0x41c1f8] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fa9cb8c207c] > -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) > [0x7fa9cb8c3377] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7fa9cb8c2105] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7fa9cb8c22b8] -> > /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7fa9cb857f33] -> dovecot/auth(main+0x2eb ) [0x40ccdb] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fa9c9dc2b15] -> dovecot/auth() [0x40cf15] Hi! This has been fixed with https://github.com/dovecot/core/commit/6c969ac21a43cc10ee1f1a91a4f39e4864c886cb Aki Tuomi Dovecot oy Great, good to hear! In my local source of 2.2.5, the deleted lines are lines 1048-1049. In the patch the lines are 1068-1069. I think maybe we'll wait for 2.2.6 and not try to patch it ourselves, we aren't using the new features in 2.2.5 yet and 2.2.4 has been stable for us... Chris -- Chris Wik Anu Internet Services www.anu.net | www.cwik.ch
Re: Panic: file auth-request.c
From: Tanstaafl To: Sent: 19/09/2016 5:44 PM Subject: Re: Panic: file auth-request.c On 9/17/2016 2:15 PM, Chris Wik wrote: > So we upgraded to a new CentOS 7 server with SSD RAID, fast CPUs and > tons of RAM. No more load problems. We compiled the latest dovecot > from source (as the version from CentOS yum repo is already quite > old, figure we might as well run the latest version since we were > upgrading anyway). Then on 9/18/2016 6:50 AM, Chris Wik wrote: > In my local source of 2.2.5, ??? Latest dovecot version is 2.2.25 - or was that (hopefully) a typo? http://www.dovecot.org/download.html Yes, typo, sorry. I have 2.2.25 sources and the line numbers don't match the diff. We'll wait for 2.2.26, unless someone from Dovecot would like us to test the patch? In which case I'll try removing the 2 lines and recompiling and see if it works. Chris
Transitioning away from mail_location = maildir:~
Hi,
I have been using this setup for years:
mail_home = /var/mail/vhosts/%d/%n
mail_location = maildir:~
I have since learned that mail_home and mail_location should be
different. I plan to use this:
mail_home = /var/mail/vhosts/%d/%n
mail_location = maildir:~/mail
I would like the transition to be transparent for my email clients. I
don't want mail/directories/sieve_scripts to disappear.
1. Should I manually create the maildir:~/mail directories?
2. Should I move files from /var/mail/vhosts/%d/%n into maildir:~/mail?
Which files? (In other words: which files are "home directory" files
and which files are "mail files"?)
Thank you,
Chris
Example home directory:
drwx-- .
drwxr-xr-x ..
drwx-- .Archives
drwx-- .Archives.2016
drwx-- .Archives.2017
drwx-- .Drafts
drwx-- .Junk
drwx-- .Notes
drwx-- .Sent
drwx-- .Sent
drwx-- .Trash
drwx-- .work
drwx-- .home
drwx-- .todo
lrwx-- .dovecot.sieve
-rw--- .dovecot.sieve.log
-rw--- .dovecot.sieve.log.0
-rw--- .dovecot.svbin
drwx-- cur
-rw--- dovecot-keywords
-rw--- dovecot-uidlist
-rw--- dovecot-uidvalidity
-r--r--r-- dovecot-uidvalidity.55411048
-rw--- dovecot.index
-rw--- dovecot.index.cache
-rw--- dovecot.index.log
-rw--- dovecot.mailbox.log
drwx-- new
drwx-- sieve
-rw--- subscriptions
drwx-- tmp
$ dovecot -n
# 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: FreeBSD 10.3-RELEASE-p11 amd64
lmtp_save_to_detail_mailbox = yes
mail_gid = 1002
mail_home = /var/mail/vhosts/%d/%n
mail_location = maildir:~
mail_privileged_group = vpostfix
mail_uid = 1002
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users
driver = passwd-file
}
plugin {
recipient_delimiter = -
sieve = file:~/sieve;active=~/.dovecot.sieve
}
protocols = imap pop3 lmtp sieve
recipient_delimiter = -
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = postfix
mode = 0600
user = postfix
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0666
user = postfix
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
ssl_cert = xxx
ssl_key = xxx
userdb {
args = username_format=%u /usr/local/etc/dovecot/users
driver = passwd-file
}
protocol lmtp {
mail_plugins = " sieve"
postmaster_address = xxx
}
protocol lda {
mail_plugins = " sieve"
}
running alternate dovecot instances on the same server
I'm posting to the list, but not on the list. I presume that means a reply-all to get to me as well as the list? We have two servers (dovecot --version: 2.2.22 (fe789d2)) that handle email for two different departments. We are transitioning mail service to the University central IT. They need to move accounts in an automated fashion and therefore need a master password to our dovecot servers. However, we are running with LDAP authentication, and I understand that a master password is not possible in that configuration. Would it be possible to run an alternate dovecot process that would use local account authentication, have a master password, and use an alternate port for connecting? Ideally it would only read accounts without changing anything, and would not interfere with the operation of the other dovecot process. I'm hoping that I could copy the configuration files, make these changes, and then launch it manually without any startup scripts in /etc/inetd.conf. Oh, by the way, we are running Ubuntu 16.04 LTS and have contracts with Ubuntu Advantage for ongoing patch support. The dovecot version is from the distribution, installed with aptitude. -- ------- Chris Hoogendyk - O__ Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~ - University of Massachusetts, Amherst --- Erdös 4
Re: running alternate dovecot instances on the same server
Aki,
Thankyou for your advice. I finally got around to this. I'm retired, working part time, and have
more to do than fits the time. Anyway . . .
I did the configuration in /etc/dovecot/local.conf, which is included in the
/etc/dovecot/dovecot.conf. However, the dovecot.conf includes the /etc/dovecot/conf.d/* before the
local.conf. I believe that means that the entries in the conf.d come first and have precedence. I
was getting authorization failures. The section of my local.conf is as follows:
#
# master passwd added 4/30/2022 based on email from Aki Tuomi on Dovecot
support list 3/21/2022,
# modified based on example from
https://doc.dovecot.org/configuration_manual/authentication/master_users/
# also added userdb at bottom of this and removed lines from
conf.d/auth-system.conf.ext to
resolve precedence.
# CGH
#
auth_master_user_separator = *
passdb {
driver = passwd-file
args = /etc/dovecot/passwd.masterusers
master = yes
result_success = continue
}
passdb {
driver = pam
args = session=yes %Ls
}
userdb {
driver = passwd
}
I had found entries in the /etc/dovecot/conf.d/auth-system.conf.ext that set drivers to pam for
passdb and passwd for userdb. I commented those two lines out since I had them covered in my
local.conf. That failed with the control process exiting with an error code. I quickly uncommented
those two lines in auth-system.conf.ext and it started just fine (I have a lot of users dependent on
this, although it is Saturday afternoon and a bit slow). Dovecot starts up alright with the above
local.conf, but master user doesn't seem to work.
I'm testing with the following (master username and passwrd replaced):
chrisho@marlin:/etc/dovecot$ telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
STARTTLS AUTH=PLAIN]
Dovecot ready.
1 login chrisho*masteruser masterpassword
1 NO [AUTHORIZATIONFAILED] Authorization failed
2 exit
Connection closed by foreign host.
What I'm seeing in the logs is:
Apr 30 19:32:29 marlin auth[20859]: pam_unix(dovecot:auth): authentication
failure; logname=
uid=0 euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:29 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
Apr 30 19:32:31 marlin auth[20859]: pam_unix(imap:auth): authentication
failure; logname= uid=0
euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:31 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
The output of doveconf -n is as follows:
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-223-generic x86_64 Ubuntu 16.04.7 LTS
auth_master_user_separator = *
default_process_limit = 200
first_valid_gid = 98
first_valid_uid = 1000
login_access_sockets = tcpwrap
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_max_userip_connections = 8
mail_privileged_group = mail
mbox_write_locks = dotlock fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
passdb {
args = /etc/dovecot/passwd.masterusers
driver = passwd-file
master = yes
result_success = continue
}
passdb {
args = session=yes %Ls
driver = pam
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_reuse_xuidl = yes
pop3_uidl_format = %08Xv%08Xu
protocols = " imap pop3"
service imap-login {
inet_listener imap {
address = localhost
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service tcpwrap {
unix_listener login/tcpwrap {
group = $default_login_user
mode = 0600
user = $default_login_user
}
}
ssl = required
ssl_ca = And, yes, Ubuntu 16.04 is EOL; however, we do have an Ubuntu Advantage account for this server. So
we do get security patches and the hope is that we can do a release upgrade this summer.
Any further guidance would be much appreciated. If any further information is
needed, I can provide it.
On 3/21/22 1:57 AM, Aki Tuomi wrote:
On 20/03/2022 22:36 Chris
Re: running alternate dovecot instances on the same server
is Saturday afternoon and a bit slow). Dovecot starts up alright with the
above local.conf, but master user doesn't seem to work.
I'm testing with the following (master username and passwrd replaced):
chrisho@marlin:/etc/dovecot$ telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
STARTTLS AUTH=PLAIN]
Dovecot ready.
1 login chrisho*masteruser masterpassword
1 NO [AUTHORIZATIONFAILED] Authorization failed
2 exit
Connection closed by foreign host.
What I'm seeing in the logs is:
Apr 30 19:32:29 marlin auth[20859]: pam_unix(dovecot:auth): authentication
failure; logname=
uid=0 euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:29 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
Apr 30 19:32:31 marlin auth[20859]: pam_unix(imap:auth): authentication
failure; logname= uid=0
euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:31 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
The output of doveconf -n is as follows:
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-223-generic x86_64 Ubuntu 16.04.7 LTS
auth_master_user_separator = *
default_process_limit = 200
first_valid_gid = 98
first_valid_uid = 1000
login_access_sockets = tcpwrap
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_max_userip_connections = 8
mail_privileged_group = mail
mbox_write_locks = dotlock fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
passdb {
args = /etc/dovecot/passwd.masterusers
driver = passwd-file
master = yes
result_success = continue
}
passdb {
args = session=yes %Ls
driver = pam
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_reuse_xuidl = yes
pop3_uidl_format = %08Xv%08Xu
protocols = " imap pop3"
service imap-login {
inet_listener imap {
address = localhost
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service tcpwrap {
unix_listener login/tcpwrap {
group = $default_login_user
mode = 0600
user = $default_login_user
}
}
ssl = required
ssl_ca = And, yes, Ubuntu 16.04 is EOL; however, we do have an Ubuntu Advantage account for this server. So
we do get security patches and the hope is that we can do a release upgrade this summer.
Any further guidance would be much appreciated. If any further information is needed, I can
provide it.
On 3/21/22 1:57 AM, Aki Tuomi wrote:
On 20/03/2022 22:36 Chris Hoogendyk wrote:
I'm posting to the list, but not on the list. I presume that means a reply-all to get to me as
well
as the list?
We have two servers (dovecot --version: 2.2.22 (fe789d2)) that handle email
for two different
departments.
We are transitioning mail service to the University central IT. They need to
move accounts in an
automated fashion and therefore need a master password to our dovecot servers.
However, we are
running with LDAP authentication, and I understand that a master password is
not possible in that
configuration.
Hi!
It is totally possible to use LDAP with master password, using configuration
like this:
# this must be first
passdb {
driver = static
args = password=masterpass
}
# current passdb config
# you probably already have this
userdb {
driver = ldap
args = /path/to/ldap/userdb
}
If this does not work, please send your `doveconf -n` as well.
Aki
--
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
Re: Configuring master password when using pam/openldap [was: "running alternate dovecot instances on the same server"]
I've changed the subject line on this email to more closely address the current
issue.
I'm hoping that with the most recent release out the door for a while there might be time to answer
this.
I believe I have the configuration done correctly to use a master password, but for some reason
Dovecot seems to be going to ldap rather than just transferring to the requested user account. Since
I don't really know what Dovecot is doing, it's hard to know what to try or whether it will actually
work. In my mind it seems like Dovecot is could do something similar to a `sudo su user`, bypassing
the usual authentication since it was given the master password.
Details below and already on the dovecot mailing list.
I've got a limited time to get this running, because central IT wants to transfer these accounts
from our department server to the central IT servers this month. We have hundreds of accounts, so it
doesn't scale to try to copy over individual accounts using individual's passwords.
Thank you,
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
On 5/1/22 4:59 PM, Chris Hoogendyk wrote:
Progress???
I realized that the error might indicate I had an empty conf paragraph or whatever you call it
when dovecot failed to start. That is I commented out the driver = pam, but left the enclosing
passdb { } with no contents. That was the error. So I commented out everything in
/etc/dovecot/conf.d/auth-system.conf.ext. That worked and doveconf -n gave me the right precedence
for masteruser and regular logins.
Then my login from telnet still failed. So I had two terminal windows, one with `sudo journalctl
-u dovecot.service -f` to follow the journal entries form dovecot, and another to do my telnet
localhost 143. Now, when I tried to login with the masteruser, I got an error in the journal that
said:
May 01 16:15:50 marlin dovecot[31944]: auth-worker(32577): Error:
pam(chrisho,127.0.0.1):
pam_authenticate() failed: Authentication failure (/etc/pam.d/imap missing?)
hmm. No idea what to put there. /etc/pam.d/ contains a bunch of entries, including dovecot. Then
there are some entries that start with common and include common-auth, common-session,
common-password, etc. The dovecot entry simply includes those. So, I tried just setting up a
symlink from imap pointing at the dovecot file. Then I tried logging in again via telnet. Now I
simply get:
May 01 16:26:40 marlin dovecot[31944]: imap-login: Aborted login (auth
failed, 2 attempts in 112
secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured,
session=<2BspEfndKMN/AAAB>
The entry of that line in the journal seemed to be slightly delayed. It showed up when I entered
`2 logout`.
At this point, my dovecot -n output is as follows:
chrisho@marlin:/etc/dovecot$ sudo doveconf -n
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-223-generic x86_64 Ubuntu 16.04.7 LTS
auth_master_user_separator = *
default_process_limit = 200
first_valid_gid = 98
first_valid_uid = 1000
login_access_sockets = tcpwrap
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_max_userip_connections = 8
mail_privileged_group = mail
mbox_write_locks = dotlock fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/passwd.masterusers
driver = passwd-file
master = yes
result_success = continue
}
passdb {
args = session=yes %Ls
driver = pam
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_reuse_xuidl = yes
pop3_uidl_format = %08Xv%08Xu
protocols = " imap pop3"
service imap-login {
inet_listener imap {
address = localhost
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service tcpwrap {
unix_listener login/tcpwrap {
group = $default_login_user
mode = 0600
user = $default_login_user
}
}
ssl = required
ssl_ca =
Aki,
Thankyou for your advice. I finally got around to this. I'm retired, working part time, and have
more to do than fits the time. Anyway . . .
I did the configuration in /etc/dovec
Re: Configuring master password when using pam/openldap [was: "running alternate dovecot instances on the same server"]
Hoping that someone can help with this. It's a critical project that requires getting the master
password to work.
I've managed to get the logging at maximum level and what I'm seeing is the
following. Testing with:
chrisho@marlin:~$ telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
STARTTLS AUTH=PLAIN]
Dovecot ready.
a login chrisho+
a NO [AUTHORIZATIONFAILED] Authorization failed
* BYE Disconnected for inactivity.
Connection closed by foreign host.
chrisho@marlin:~$
And what I see in the log files is:
Jun 17 12:16:10 marlin dovecot: auth: Debug: client in:
AUTH#0111#011PLAIN#011service=imap#011secured#011session=#011lip=127.0.0.1#011rip=127.0.0.1#011lport=143#011rport=60150#011resp
(previous base64 data may contain sensitive data)
Jun 17 12:16:10 marlin dovecot: auth: Debug:
passwd-file(,127.0.0.1,master,):
Master user lookup for login: chrisho
Jun 17 12:16:10 marlin dovecot: auth: Debug:
passwd-file(,127.0.0.1,master,):
lookup: user= file=/etc/dovecot/passwd.masterusers
Jun 17 12:16:10 marlin dovecot: auth:
passwd-file(,127.0.0.1,master,): Master
user logging in as chrisho
Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug:
pam(chrisho,127.0.0.1): lookup
service=imap
Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug:
pam(chrisho,127.0.0.1): #1/1 style=1
msg=Password:
Jun 17 12:16:12 marlin dovecot: auth-worker(9763): pam(chrisho,127.0.0.1):
pam_authenticate()
failed: Authentication failure (password mismatch?) (given password:
)
Jun 17 12:18:53 marlin dovecot: imap-login: Disconnected: Inactivity (auth
failed, 1 attempts in
163 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1,
secured,
I've replaced sensitive content with , , , and . BUT,
note that in the next to the last line, where dovecot is attempting to authenticate the user via pam
using the master password!! Why?? 1st, it shouldn't be trying to authenticate, it already has the
master user authenticated. 2nd, why would it try to use what it should know is the master password
to authenticate the regular user?
Irrespective of those two questions, which are given primarily as what I would see as significant
clues, what do I need to do to get it to work? Is this an issue with my dovecot configuration? An
issue with my pam configuration? Or a bug in dovecot? Note also that I am using ldap as a user
database. One of the criteria set up in pam is that the user must also exist in the local
/etc/passwd, but the authentication goes to ldap.
Any help very much appreciated.
My doveconf output is already included in this thread, so I won't repeat it. The only thing I've
changed is the debug levels. If there is anything else that would be useful to know, please let me
know.
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
On 6/5/22 2:40 PM, Chris Hoogendyk wrote:
I've changed the subject line on this email to more closely address the current
issue.
I'm hoping that with the most recent release out the door for a while there might be time to
answer this.
I believe I have the configuration done correctly to use a master password, but for some reason
Dovecot seems to be going to ldap rather than just transferring to the requested user account.
Since I don't really know what Dovecot is doing, it's hard to know what to try or whether it will
actually work. In my mind it seems like Dovecot is could do something similar to a `sudo su user`,
bypassing the usual authentication since it was given the master password.
Details below and already on the dovecot mailing list.
I've got a limited time to get this running, because central IT wants to transfer these accounts
from our department server to the central IT servers this month. We have hundreds of accounts, so
it doesn't scale to try to copy over individual accounts using individual's passwords.
Thank you,
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
On 5/1/22 4:59 PM, Chris Hoogendyk wrote:
Progress???
I realized that the error might indicate I had an empty conf paragraph or whatever you call it
when dovecot failed to start. That is I commented out the driver = pam, but left the enclosing
passdb { } with no contents. That was the error. So I commented out everything in
/etc/dovecot/conf.d/auth-system.conf.ext. That worked and doveconf -n
Re: Configuring master password when using pam/openldap [was: "running alternate dovecot instances on the same server"]
So this is resolved. Turns out that getting rid of the "result_success = continue" fixed it. Which is odd, because I thought from the documentation that that was required. Anyway. Fixed. On 6/20/22 2:53 PM, Chris Hoogendyk wrote: Hoping that someone can help with this. It's a critical project that requires getting the master password to work. I've managed to get the logging at maximum level and what I'm seeing is the following. Testing with: chrisho@marlin:~$ telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. a login chrisho+ a NO [AUTHORIZATIONFAILED] Authorization failed * BYE Disconnected for inactivity. Connection closed by foreign host. chrisho@marlin:~$ And what I see in the log files is: Jun 17 12:16:10 marlin dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011session=#011lip=127.0.0.1#011rip=127.0.0.1#011lport=143#011rport=60150#011resp (previous base64 data may contain sensitive data) Jun 17 12:16:10 marlin dovecot: auth: Debug: passwd-file(,127.0.0.1,master,): Master user lookup for login: chrisho Jun 17 12:16:10 marlin dovecot: auth: Debug: passwd-file(,127.0.0.1,master,): lookup: user= file=/etc/dovecot/passwd.masterusers Jun 17 12:16:10 marlin dovecot: auth: passwd-file(,127.0.0.1,master,): Master user logging in as chrisho Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug: pam(chrisho,127.0.0.1): lookup service=imap Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug: pam(chrisho,127.0.0.1): #1/1 style=1 msg=Password: Jun 17 12:16:12 marlin dovecot: auth-worker(9763): pam(chrisho,127.0.0.1): pam_authenticate() failed: Authentication failure (password mismatch?) (given password: ) Jun 17 12:18:53 marlin dovecot: imap-login: Disconnected: Inactivity (auth failed, 1 attempts in 163 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, I've replaced sensitive content with , , , and . BUT, note that in the next to the last line, where dovecot is attempting to authenticate the user via pam using the master password!! Why?? 1st, it shouldn't be trying to authenticate, it already has the master user authenticated. 2nd, why would it try to use what it should know is the master password to authenticate the regular user? Irrespective of those two questions, which are given primarily as what I would see as significant clues, what do I need to do to get it to work? Is this an issue with my dovecot configuration? An issue with my pam configuration? Or a bug in dovecot? Note also that I am using ldap as a user database. One of the criteria set up in pam is that the user must also exist in the local /etc/passwd, but the authentication goes to ldap. Any help very much appreciated. My doveconf output is already included in this thread, so I won't repeat it. The only thing I've changed is the debug levels. If there is anything else that would be useful to know, please let me know. --- Chris Hoogendyk - O__ Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~ - University of Massachusetts, Amherst --- Erdös 4 On 6/5/22 2:40 PM, Chris Hoogendyk wrote: I've changed the subject line on this email to more closely address the current issue. I'm hoping that with the most recent release out the door for a while there might be time to answer this. I believe I have the configuration done correctly to use a master password, but for some reason Dovecot seems to be going to ldap rather than just transferring to the requested user account. Since I don't really know what Dovecot is doing, it's hard to know what to try or whether it will actually work. In my mind it seems like Dovecot is could do something similar to a `sudo su user`, bypassing the usual authentication since it was given the master password. Details below and already on the dovecot mailing list. I've got a limited time to get this running, because central IT wants to transfer these accounts from our department server to the central IT servers this month. We have hundreds of accounts, so it doesn't scale to try to copy over individual accounts using individual's passwords. Thank you, --- Chris Hoogendyk - O__ Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~ - University of Massachusetts, Amherst --- Erdös 4 On 5/1/22 4:59 PM, Chris Hoogendyk wrote: Progress??? I realized that the error might indicate I had an empty conf paragraph or whatever you call it when d
Re: ot: how to t/s TBird problems ?
Over the last several months we have seen what seems like large delays in email delivery as well, we get emails at 11AM that are time stamped at 9:10. I thought it was a networking issue, but I can’t be sure. I wish I knew more about coding, to look under the hood to examine things further. Sent from my iPhone > On Oct 23, 2022, at 7:17 AM, Voytek Eymont wrote: > > > >> On Sat, October 22, 2022 11:29 am, Joseph Tam wrote: >> >> I haven't seen anyone else replying, but there doesn't seem anything >> anomalous with the output. The session commands-repliesd is is more or >> less what I expect, although to make sense of this, you'll have to splice >> the input and output files together using timestamps to see the sequential >> flow of data. > ... >> Typically, if some resource limit is hit, one side or the other will >> create a log or notification. Your INBOX is large, but not outrageous. You >> can test it directly by creating smaller subsets of the INBOX messages and >> see if the problem goes away. > > Joseph, > > thank you very much for the follow up! > you won't believe it, literally minutes before your email I got this email > from the 'problem user' (below) > > thank you to all who responded! > > - I guess if TB debug log was enabled (as was suggested)- maybe the issue > would become apparent from TB debug log ? > > - I guess i should encourage POP users to switch to IMAP anyhow ? > > got this from problem user: > --- > Mozilla Thunderbird released an update which I just installed. > > Problem solved. > > I guess Tbird had a problem that the new release addressed. > > I'm sorry for the inconvenience. > > I'm mystified why my issue was only with one account. Perhaps it was > something to do with the size of the database. > > --- > yesterday it was > --- > I'm still experiencing a 40 second delay to retrieve emails for > xxx > > I have changed the pop port to 110 for the server but that did not > work at all. > > I have reinstalled my email client TBird but no change, anyway all the > other accounts on TBird are working ok but they are MAPI not POP. > > > Voytek >
doveadm sending invalid AUTHENTICATE to uw-imap
I'm migrating a legacy uw-imap system to Dovecot, on a Rocky (RHEL) 8 server running Dovecot 2.3.16-3 from their repos. I am using a master user to import all users for an imaps connection from the old server to the new. On a trial run however, it worked for about half the users. Half are giving an error of the form: dsync(user): Error: imapc(host:993): Command '1 AUTHENTICATE PLAIN ' failed with BAD: 1 Missing or invalid argument to AUTHENTICATE I can't seem to get the IMAP command for the users that did work. However, on the face of it, that is an invalid AUTHENTICATE command. If I take that string and brake it up into (what I've googled is) the proper form of multi-command form of 1 AUTHENTICATE PLAIN + then the login succeeds. I have not been able to find anyone else with this problem in my search. Is this a known issue, is there a way to force the multi-line AUTHENTICATE, something else I'm missing ? Any help is appreciate on this! -Chris -- --- ==== Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
Re: doveadm sending invalid AUTHENTICATE to uw-imap
On Wed, 8 Feb 2023, Aki Tuomi wrote: > Can you try setting imapc_sasl_mechanisms to login, maybe it works better? And Stephan Bosch wrote: > Can you make a protocol log (tcp dump of commands sent by client and > replies sent by server) for one of these sessions? e.g. using ngrep if > connections aren't secured. I was using imaps initially. Switching to imap over port 143 to do the tcpdump had the side effect of switching to LOGIN authentication, evidently uw-imap is sending different capability strings. It still doesn't work though. Both from the error and the dump I can tell "doveadm" is sending the user's id only without the "*masteruser" and the master user password. Plain connection banner: * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS STARTTLS] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 16:45:22 -0500 (EST) SSL Banner on 993: * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS AUTH=PLAIN AUTH=LOGIN] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 16:53:36 -0500 (EST) > > On 08/02/2023 06:24 EET Chris Candreva wrote: > > > > > > I'm migrating a legacy uw-imap system to Dovecot, on a Rocky (RHEL) 8 > > server running Dovecot 2.3.16-3 from their repos. I am using a master user > > to import all users for an imaps connection from the old server to the > > new. On a trial run however, it worked for about half the users. Half are > > giving an error of the form: > > > > dsync(user): Error: imapc(host:993): > > Command '1 AUTHENTICATE PLAIN ' failed with > > BAD: > > 1 Missing or invalid argument to AUTHENTICATE > > > > I can't seem to get the IMAP command for the users that did work. However, > > on the face of it, that is an invalid AUTHENTICATE command. If I take that > > string and brake it up into (what I've googled is) the proper form of > > multi-command form of > > > > 1 AUTHENTICATE PLAIN > > + > > > > > > then the login succeeds. I have not been able to find anyone else with > > this problem in my search. Is this a known issue, is there a way to force > > the multi-line AUTHENTICATE, something else I'm missing ? Any help is > > appreciate on this! > > > > -Chris > > > > > > > > -- > > --- > > ==== > > Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris > -- --- Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
Re: doveadm sending invalid AUTHENTICATE to uw-imap
For anyone searching in the future: The eventual cause turns out to be, as far as I can tell, something screwy in uw-imap or the base64 decode function it uses on this old Solaris server. It seems to be something odd with the number of characters in the hash. tl;dr I'm going to do my migration with 2 different master users with different length passwords, which will cover all cases and allow me to migrate all the users. Problem solved for this case. Now the long version for the archive: To diagnose the problem I eventually wrote a short perl program using IO::Socket::SSL that prints the UW-Imap banner and the '+' second prompt, logs what it receives back, and exits. This let me see that in all cases, doveadm sends the single line AUTHENTICATE command, so that wasn't the problem. It had never occurred to me to reverse the hash and see what was being sent. I had been testing from telnet / "openssl s_client" using a string "\0user@masteruser\0masterpass" . Doveadm is sending "user\0masteruser\0masterpass" . Different input, different results. Somewhere in my testing, I forgot to add the -n to echo -e "\0user@masteruser\-masterpass" | base64 (so the input to base64 had a trailing newline) and to my surprise, a user that didn't work before worked ! That mistake is how I figured out the extra character in the password was somehow making a difference. So -- since this is a one time migration, and it's repeatable, I've come up with the users that work with the padding, and the ones that work without, and will run the import each way for each list, turn off the legacy server and be done with this ! Thanks for the pointers, and if anyone else runs into this bizare situation hopefully they find this ! Of course if anyone knows why this happens I'd love to hear it. -Chris On Wed, 8 Feb 2023, Chris Candreva wrote: > > On Wed, 8 Feb 2023, Aki Tuomi wrote: > > > Can you try setting imapc_sasl_mechanisms to login, maybe it works better? > > And Stephan Bosch wrote: > > > Can you make a protocol log (tcp dump of commands sent by client and > > replies sent by server) for one of these sessions? e.g. using ngrep if > > connections aren't secured. > > > I was using imaps initially. Switching to imap over port 143 to do the > tcpdump had the side effect of switching to LOGIN authentication, > evidently uw-imap is sending different capability strings. It still > doesn't work though. Both from the error and the dump I can tell "doveadm" > is sending the user's id only without the "*masteruser" and the > master user password. > > Plain connection banner: > * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS > STARTTLS] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 16:45:22 > -0500 (EST) > > SSL Banner on 993: > * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS > AUTH=PLAIN AUTH=LOGIN] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 > 16:53:36 -0500 (EST) > > > > > > On 08/02/2023 06:24 EET Chris Candreva wrote: > > > > > > > > > I'm migrating a legacy uw-imap system to Dovecot, on a Rocky (RHEL) 8 > > > server running Dovecot 2.3.16-3 from their repos. I am using a master > > > user > > > to import all users for an imaps connection from the old server to the > > > new. On a trial run however, it worked for about half the users. Half are > > > giving an error of the form: > > > > > > dsync(user): Error: imapc(host:993): > > > Command '1 AUTHENTICATE PLAIN ' failed > > > with BAD: > > > 1 Missing or invalid argument to AUTHENTICATE > > > > > > I can't seem to get the IMAP command for the users that did work. > > > However, > > > on the face of it, that is an invalid AUTHENTICATE command. If I take > > > that > > > string and brake it up into (what I've googled is) the proper form of > > > multi-command form of > > > > > > 1 AUTHENTICATE PLAIN > > > + > > > > > > > > > then the login succeeds. I have not been able to find anyone else with > > > this problem in my search. Is this a known issue, is there a way to force > > > the multi-line AUTHENTICATE, something else I'm missing ? Any help is > > > appreciate on this! > > > > > > -Chris > > > > > > > > > > > > -- > > > --- > > > > > > Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris > > > > -- --- Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
Re: Cannot reach documentation site
On Sun, 26 Feb 2023, Ken Bass wrote: > doc.dovecot.org does not have a DNS entry. It worked a few days ago, but not > now. [chris@newpop ~]$ host doc.dovecot.org doc.dovecot.org is an alias for talvi.dovecot.org. talvi.dovecot.org has address 94.237.105.223 talvi.dovecot.org has IPv6 address 2a04:3545:1000:720:acc1:5bff:fe5e:459 -- --- ==== Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
Incorrect saved dates in mailboxes
Hello: We have a server (Debian 11) using Dovecot 2.3 which has been in place for about a year. We are using mbox format for our mailboxes, and do not use MySQL, just plain user directories under /home. Recently, I noticed that our expunge script is not working, and I don't think it ever has on this server. On further inspection, it looks like the saved date for emails in the folders we want to expunge is set to a recent date (yesterday) for almost all messages. For example, when I run the command "doveadm -f tab fetch -u username date.saved mailbox Trash", the date for almost all of the messages is yesterday at the exact same time. I tried this on other users and they have this exact same timestamp shown for most of their messages, too. I tried checking for "date.received" and that shows correct, it is "date.saved" that is not working. Any idea of how or why this would not be showing the correct date? I've used this setup in the past on other servers with dovecot and it has worked great, no idea why we would be seeing this issue now. Thanks in advance. Chris ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Incorrect saved dates in mailboxes
On 2023-05-17 18:28, Joseph Tam wrote: Chris Szilagyi writes: Recently, I noticed that our expunge script is not working, and I don't think it ever has on this server. On further inspection, it looks like the saved date for emails in the folders we want to expunge is set to a recent date (yesterday) for almost all messages. For example, when I run the command "doveadm -f tab fetch -u username date.saved mailbox Trash", the date for almost all of the messages is yesterday at the exact same time. I tried this on other users and they have this exact same timestamp shown for most of their messages, too. I tried checking for "date.received" and that shows correct, it is "date.saved" that is not working. Any idea of how or why this would not be showing the correct date? I've used this setup in the past on other servers with dovecot and it has worked great, no idea why we would be seeing this issue now. I recall having the same problem. I think the "date.saved" is not instantiated in the cache until you query for it. So the value you were shown is when you last dumped its value if it wasn't previously set -- your run of same values coincided when you ran "doveadm fetch". My expunge script just uses date.received instead -- it seems to work. Joseph Tam OK, that does make sense so far. Today I have checked, and the correct dates seem to be showing up now after the initial date (ever since that initial run of the "dovecot expunge" command was run on all mailboxes). So in our case I have the expunge set to delete older than 30 days. I assume if I wait 30 days from now, it will start working? Or did your issue come back repeatedly? I too thought about using date.received, but in our case date.saved is more ideal. Thank you for your reply. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
IMAP account can't save any email with attachment
Hi, I'm getting a curious problem where if I write a draft without an attachment and click save. It'll work without any issue at all. But if I do the same, then attach a file to the email, it'll sit there for a couple of minutes before timing out (I'm using thunderbird), it'll eventually give you a message saying "Your draft message was not copied to your drafts folder (Drafts) due to network or file access errors." I've got all of dovecots verbose logging turned on. I'm using dovecot as a submission server through to the postfix server to do the actual sending. All the logging is turned on there too. But I can't figure out what the problem is. Is there anything I can look for in the logs that will help me out? chris ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: IMAP account can't save any email with attachment
Here is the info from dovecot -n
dovecot.mail-server and postfix.mail-server are valid dns entries for
themselves. It's running on a kubernetes cluster so those hostnames are
provided by the namespace and pod name, they work too, you can ping them
and it works for everything except emails with attachments, for some reason
# 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.4 ()
# OS: Linux 4.9.0-9-amd64 x86_64 Debian 10.13 ext4
# Hostname: dovecot.mail-server.svc.cluster.local
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = yes
disable_plaintext_auth = no
first_valid_gid = 8
first_valid_uid = 8
haproxy_timeout = 5 secs
haproxy_trusted_networks = 10.0.0.0/8
hostname = s3.mydomain.com
log_path = /dev/stderr
mail_access_groups = mail
mail_debug = yes
mail_gid = mail
mail_home = /mail/%d/%n
mail_location = maildir:/mail/%d/%n
mail_plugins = " zlib"
mail_privileged_group = mail
mail_uid = mail
maildir_stat_dirs = yes
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
postmaster_address = i...@mydomain.com
protocols = " imap lmtp pop3 submission"
service auth-worker {
unix_listener auth-worker {
group = mail
mode = 0660
user = $default_internal_user
}
user = mail
}
service auth {
user = $default_internal_user
}
service dict {
unix_listener dict {
group = mail
mode = 0660
}
}
service imap-login {
inet_listener imap {
haproxy = yes
port = 143
}
inet_listener imaps {
haproxy = yes
port = 993
ssl = yes
}
}
service lmtp {
inet_listener lmtp {
haproxy = no
port = 24
}
}
service pop3-login {
inet_listener pop3 {
haproxy = yes
port = 110
}
inet_listener pop3s {
haproxy = yes
port = 995
ssl = yes
}
}
service submission-login {
inet_listener submission {
haproxy = yes
port = 587
}
}
ssl_cert =
wrote:
>
> > Op 20 jul. 2023 om 14:26 heeft Chris Thomas
> het volgende geschreven:
> >
> >
> > Hi,
> >
> > I'm getting a curious problem where if I write a draft without an
> attachment and click save. It'll work without any issue at all.
> >
> > But if I do the same, then attach a file to the email, it'll sit there
> for a couple of minutes before timing out (I'm using thunderbird), it'll
> eventually give you a message saying
> >
> > "Your draft message was not copied to your drafts folder (Drafts) due to
> network or file access errors."
> >
> > I've got all of dovecots verbose logging turned on.
>
> Cool! So … where is it?
>
> > I'm using dovecot as a submission server through to the postfix server
> to do the actual sending. All the logging is turned on there too. But I
> can't figure out what the problem is.
> >
> > Is there anything I can look for in the logs that will help me out?
> >
> > chris
> > ___
> > dovecot mailing list -- dovecot@dovecot.org
> > To unsubscribe send an email to dovecot-le...@dovecot.org
>
>
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
Re: IMAP account can't save any email with attachment
omain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Added userdb setting: plugin/quota_rule=*:bytes=0 > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Effective uid=8, gid=8, home=/mail/mydomain.com/myuser > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, > list=yes, subscriptions=yes location=maildir:/mail/mydomain.com/myuser > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > maildir++: root=/mail/mydomain.com/myuser, index=, indexpvt=, control=, > inbox=/mail/mydomain.com/myuser, alt= > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Mailbox Drafts: Mailbox opened because: SELECT In the thunderbird client, I wait for ages before a popup appears saying "Your draft message was not copied to your drafts folder (Drafts) due to network or file access errors. You can retry or save the draft locally to Local Folders" I've tried searching around for information on what the problem could be, but I've not found anything that would explain this problem. Have any ideas? Chris On Thu, Jul 20, 2023 at 3:20 PM William Edwards wrote: > > > Op 20 jul. 2023 om 14:26 heeft Chris Thomas > het volgende geschreven: > > > > > > Hi, > > > > I'm getting a curious problem where if I write a draft without an > attachment and click save. It'll work without any issue at all. > > > > But if I do the same, then attach a file to the email, it'll sit there > for a couple of minutes before timing out (I'm using thunderbird), it'll > eventually give you a message saying > > > > "Your draft message was not copied to your drafts folder (Drafts) due to > network or file access errors." > > > > I've got all of dovecots verbose logging turned on. > > Cool! So … where is it? > > > I'm using dovecot as a submission server through to the postfix server > to do the actual sending. All the logging is turned on there too. But I > can't figure out what the problem is. > > > > Is there anything I can look for in the logs that will help me out? > > > > chris > > ___ > > dovecot mailing list -- dovecot@dovecot.org > > To unsubscribe send an email to dovecot-le...@dovecot.org > > ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Using / in folder names
Maybe this is over simplifying things but can you just open the pst file in outlook and then rename the folder, then import? Sent from my iPhone > On Aug 3, 2023, at 9:03 PM, Scott wrote: > > Is this possible at all ? > > I am trying to import a PST file and it has a slash in the folder name. > Dovecot errors out with : > > APPEND "Caixa de Entrada_VIDA/S1" (\Seen) > NO [CANNOT] Invalid mailbox name: Name must not have '/' characters > > My namespace separator is the dot and I configured the listescape plugin to > escape / but it still doesn't work. > > The listescape documentation says: > > The / character is disallowed on POSIX systems. > > This plugin allows you to use all of these characters, as long as the > virtual separator (i.e. what is set by the separator setting and used as > such by the IMAP protocol) is changed to something else, which means that > the plugin does not make it possible to use the virtual separator in folder > names. > > > So it seems that as long as I use the dot, it should work ? Except it > doesn't... > > How can I do this import ? > > Thank you! > > ___ > dovecot mailing list -- dovecot@dovecot.org > To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
RE: The future of SIS
On Mon, 16 Oct 2023, Marc wrote: > Is this feature really useful? I can imagine if you are twitter or ig and > everyone is posting the same video this could be usefull. Are there any stats > on this available, so you know what to expect implementing deduplication. In an office where people insist on mailing documents to everyone, and using email as a document storage system, yes, it is very useful. -- --- ==== Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Different realm for different listeners?
I see this has been asked a few times over the years (but not in several years), and the response was along the lines of "maybe someday", so I figured I'd see if someday was here yet... I have a need to have a different default realm for different listeners. Basically, I've a bunch of different domains, and a few of them insist their users not be required to authenticate with "user@domain" and just use "user", and I'm trying to make this work without separate daemons and/or VMs. I'll be using Dovecot 2.2 (most likely 2.2.10 as provided by RHEL/CentOS 7). Thanks. If there's not a way to do this directly in Dovecot, does anybody have any suggestion that doesn't require several different VMs? -- Chris Adams
Re: Different realm for different listeners?
Once upon a time, Tom Hendrikx said: > You could drop the default realm completely, and create a second > passdb lookup which uses only the username part to lookup credentials. > This means that, as long as you have no conflicts as usernames > 'j...@foo.com' vs 'j...@bar.com', you'd be able to support username > 'john'. The domains I need to handle are separate telephone company ISPs, so they all want common things like "sales", so that won't work. All my users are in MySQL. Reading the docs, I see that "%l" expands to the local IP address "almost everywhere". If that's available in the userdb SQL expansion, I could write a (somewhat convoluted) SQL statement that adds a domain based on the local IP. Does that sound like it'll work? Not quite as "elegant" as just setting auth_default_realm in separate listeners, but should work I guess. -- Chris Adams
Re: Different realm for different listeners?
Once upon a time, Chris Adams said:
> All my users are in MySQL. Reading the docs, I see that "%l" expands to
> the local IP address "almost everywhere". If that's available in the
> userdb SQL expansion, I could write a (somewhat convoluted) SQL
> statement that adds a domain based on the local IP. Does that sound
> like it'll work? Not quite as "elegant" as just setting
> auth_default_realm in separate listeners, but should work I guess.
Just a follow-up for the archives: this worked. This is my
MySQL password_query:
password_query = \
SELECT username AS user, password, \
concat('/srv/mail/',maildir) AS userdb_home, 1000 AS userdb_uid, \
1000 AS userdb_gid \
FROM mailbox WHERE active = 1 AND username = case \
when '%d' <> '' then '%u' \
when '%l' = '10.0.9.73' then '%u...@domain1.com' \
when '%l' = '10.0.9.74' then '%u...@domain2.net' \
else '%u' end
--
Chris Adams
Re: postfix sasl -> haproxy -> dovecot auth
Once upon a time, Edgaras Lukoševičius said: > What I need is to make smtp authentication balanced and keep everything in > backend (private network) If you have more than one Postfix server, each one must talk to its own private Dovecot server for auth. The Dovecot auth protocol includes a client (Postfix) assigned ID, and Postfix uses the process ID. If you have multiple Postfix servers talking to one Dovecot server, you'll get ID conflicts and dropped auths. I ended up putting a local instance of Dovecot on each Postfix server, with no protcols configured except for auth. Not quite as HA, but I have my monitoring system doing SMTP AUTH (never have had a problem with the setup); you could probably have HAProxy do it as well (IIRC it can do some basic expect-style send/receive). -- Chris Adams
Re: v2.2.17 release candidate released
On May 9, 2015, at 19:13 , Jerry wrote: >> 2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed (see >> <554773d2.9060...@dovecot.fi> from Teemu Huovila).[...]. But, maybe it's >> worth >> solving this issue for those non-linux systems without bash installed per >> default before releasing dovecot v2.2.17, finally. >> >> Besides that: Thanks for all of you involved making dovecot available! > > For what its worth, I run FreeBSD 10.1 and I have Bash. Personally, I could > not work without it. It is not a big deal to install it on FreeBSD. My > question is does it have to be the default shell in dovecot v2.2.17 or just > installed? I run FreeBSD 9, 10, and 10.1. And I don't have bash installed except on the one system I use to build lots of ports. Because, there are other ports that also presume the availability of bash, and therefore require it. I'd much rather be able to build dovecot without bash, because I don't have bash installed on my FreeBSD 10.1 mail server, where I run dovecot. I'd also be willing to help by porting any of the scripts that require bash to something else that is available in the base installation of FreeBSD as well as the variety of Linux distributions. I agree that it's not hard to install the bash port. But if you don't plan to use it for anything else, it's better not to have to. - Chris
Problem with sieve not triggering randomly?
I'm running CentOS 7.1 with Dovecot 2.2.10 (CentOS packaged version). I
have Postfix feeding messages to Dovecot for local delivery via LMTP,
and I have Amavisd for spam filtering. For messages above a certain
threshold, Amavisd adds the "X-Spam-Flag: YES" header, and I file that
into the Spam folder using a default (system-wide) sieve script (there
aren't any per-user sieve scripts set up):
require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
fileinto "Spam";
}
I can see it work on my test messages and from lmtp log entries.
However, it doesn't seem to always work. I had a user complain about a
spam increase, and I looked in their mailbox. I see a message with the
"X-Spam-Flag: YES" header, but it is in the INBOX (and the corresponding
lmtp log entry shows that's where it went).
Why would the sieve script work on some messages and not others? I
don't see any errors in the log. I see a bunch of messages from the
same source to different users, and they were all filed into INBOX, not
Spam.
Looking at the message in question, it is a pretty bogus message. There
are multiple From: and Subject: headers. Would that cause sieve to
"punt" on the message (but not log anything about it)?
--
Chris Adams
Re: Problem with sieve not triggering randomly?
Once upon a time, Stephan Bosch said: > You can check the handling of a particular message yourself using the > sieve-test tool (there is a man page for it). By specifying the `-t - > -Tlevel=matching` options, you'll get detailed information on why a > particular decision is made. Hmm, that's weird. sieve-test says it would store the message into the Spam folder (as expected), but it was definitely delivered to INBOX. > Without insight in your configuration (output from `dovecot -n`) and an > example of a message that is mishandled, I cannot tell you more. Here you go. I changed the local part of the user's email address in the spam message, and the doveadm password in the doveconf output, but that's all. http://www.cmadams.net/misc/dove/doveconf-n.txt http://www.cmadams.net/misc/dove/spam-message.txt I have three Dovecot VMs behind a pair of Dovecot director VMs; this is from the actual host that handled the delivery (all three backend hosts are showing the same issue). Users are in MySQL and mail is on NFS. There are some things configured, like managesieve, that aren't currently used. This is a new setup, migrated from Courier; I made the switch a couple of weeks ago (not my first time using Dovecot, just finally got a chance to migrate this setup). Thanks. -- Chris Adams
