Re: Unexpected accounts showing up in replication status (doveadm replicator status '*')
Just an update that I think I know what has been going on. I believe I've been tripped up by two things. 1) Not always clearly reading the actual component being referred to when "userid", "user" and "username" are used in official online documentation or other web sources. 2) Setting up dovecot to accept delivery / auth on %n and dropping the domain. For #1 it is descriptions such as the following from the online documentation which tripped me up %u user full username (e.g. user@domain) %n username user part in user@domain, same as %u if there's no domain i.e. %n referring both to "username" and "user" where for test...@example.com %n is testing and %u is test...@example.com Another one which could be clearer is the dovecot-sql.conf.ext file gives a example database with username and domain fields and the same file has an example password_query with the userid field. For #2 failure to get to grips with #1 and also various examples online which aren't fully context explained meant I had an inconsistent set of queries. After getting to grips with that I changed my mysql auth to be consistent and always return username and domain e.g. [Note in my database username contains "testing" and domain "example.com") password_query = SELECT username, domain [etc] user_query = [does not return username or domain] iterate_query = SELECT username,domain FROM users i.e. note they all now return two separate fields with "testing" and "example.com" which dovecot is merging as per the documentation. With that set up both dovadm user '*' and doveadm -A [etc] return and use test...@example.com The last problem I had was that in my mail delivery (from an alias file) I was using testing: "dovecot-lda -d testing" And that seems to trigger replication to trigger on testing as well as test...@example.com being triggered when the user made changes to their account. So having changed the mysql as per above and the delivery to testing: "dovecot-lda -d test...@example.com" and having removed all entries without @domain from replication with doveadm replicator remove [entries without domains] doveadm repliciator status now returns the correct number of accounts and doveadm repliciator status '*' does not have "duplicate" entries. Fingers crossed that is it and there are no other call which will cause replication on "testing" as separate from "test...@example.com" On 04/01/19 09:11, Daniel Schütze wrote: I have a replication set up with two dovecot 2.3.4 servers. Messages appear to be replicating fine between them, however there is an oddity when I run doveadm replicator status '*' and I'm worried work is being carried out needlessly. Both machines have identical configs and have a mysql backend for username/passwords. Both machines point to the same mysql server and same tables. When I run doveadm user '*' On either machine I am presented with the same correct list of users. The users are presented in the form: username and not username@domain. When I run doveadm replicator status '*' On either machine I receive a list of usernames which includes not only username but also username@domain. Both of these "accounts" have fast sync, full sync and success sync entries and these are not the same times between username and username@domain (hence I'm worried unnecessary work is being carried out). My ee dovecot-sql.conf.ext Has these entires re usernames user_query = SELECT home, uid, gid FROM users WHERE username = '%n' password_query = SELECT username AS user, domain, password, home AS userdb_home, uid AS userdb_uid, gid AS userdb_gid FROM users WHERE username = '%u' [Having copied these in I do see the different usage of %u and %n] iterate_query = SELECT username AS user,domain FROM users The users table is as follows +--+--+--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +--+--+--+-+-+---+ | username | varchar(128) | NO | | NULL | | | domain | varchar(128) | NO | | NULL | | | password | varchar(64) | NO | | NULL | | | home | varchar(255) | NO | | NULL | | | uid | int(11) | NO | | NULL | | | gid | int(11) | NO | | NULL | | | active | char(1) | NO | | Y | | | mail_replica | varchar(255) | NO | | NULL | | +--+--+--+-+-+---+ I assume that the duplication of username with and without domains isn't as intended, any suggestions where I am going wrong? I did previously have the servers running with the iterate query commented out and when that was the case doveadm user '*' returned the correct number of accounts but in the format username@domain and the replic
Dovecot can't access mailbox that exists (nested)
Hello,
I have this strange situation on nested mailboxes (directories)
# dovecot --version
2.2.36 (1f10bfa63)
# doveconf -n
# 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.24 (124e06aa)
# OS: Linux 3.10.0-693.21.1.el7.x86_64 x86_64 CentOS Linux release
7.5.1804 (Core)
mail_access_groups = vmail
mail_location = maildir:%h
mail_plugins = quota stats quota_clone
mailbox_list_index = yes
managesieve_sieve_capability = fileinto envelope encoded-character
subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body environment mailbox date index ihave duplicate
mime foreverypart extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix = INBOX.
separator = .
type = private
}
protocols = imap lmtp pop3 sieve
service imap-login {
inet_listener imap-dsync {
port = 1143
}
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
process_limit = 300
process_min_avail = 1
}
service lmtp {
inet_listener lmtp {
port = 24
}
process_min_avail = 5
}
service managesieve-login {
inet_listener sieve {
port = 4190
ssl = yes
}
process_min_avail = 5
}
service pop3-login {
inet_listener pop3-dsync {
port = 1110
}
inet_listener pop3 {
port = 110
}
inet_listener pop3s {
port = 995
ssl = yes
}
process_limit = 300
process_min_avail = 1
}
service stats {
fifo_listener stats-mail {
group = vmail
mode = 0600
user = vmail
}
fifo_listener stats-user {
group = vmail
mode = 0600
user = vmail
}
}
protocol lmtp {
mail_plugins = quota stats quota_clone sieve
}
protocol imap {
imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
mail_max_userip_connections = 50
mail_plugins = quota stats quota_clone imap_quota imap_stats
}
protocol sieve {
mail_max_userip_connections = 10
}
protocol pop3 {
mail_plugins = quota stats quota_clone
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_enable_last = yes
}
# ls -1a (Mailbox structure)
cur
dovecot.index
dovecot.index.cache
dovecot.index.log
dovecot.list.index
dovecot.list.index.log
dovecot.mailbox.log
dovecot-uidlist
dovecot-uidvalidity
.Drafts
.Junk
maildirfolder
maildirsize
new
.Received 1.1522486 - Subdirname.Received
.Received 1.1522486 - Subdirname.Sent
.Sent
sieve
subscriptions
tmp
.Trash
.Trash.Sent (2)
# doveadm mailbox list -u n...@domain.com
INBOX
INBOX.Trash
INBOX.Trash.Sent (2)
INBOX.Sent
INBOX.Received 1
INBOX.Received 1.1522486 - Subdirname
INBOX.Received 1.1522486 - Subdirname.Sent
INBOX.Received 1.1522486 - Subdirname.Received
INBOX.Junk
INBOX.Drafts
# doveadm mailbox status -u n...@domain.com -t all INBOX.*
doveadm(n...@domain.com): Error: Mailbox INBOX.Received 1: Failed to
lookup mailbox status: Mailbox doesn't exist: INBOX.Received 1
doveadm(n...@domain.com): Error: Mailbox INBOX.Received 1.1522486 -
Subdirname: Failed to lookup mailbox status: Mailbox doesn't exist:
INBOX.Received 1.1522486 - Subdirname
messages=16 recent=0 unseen=1 vsize=3432379
The problem is that it's not possible to move this mailbox between
servers using dovecot dsync/imapc. But I can see the messages inside
these mailboxes using external IMAP/POP clients.
Re: Dovecot can't access mailbox that exists (nested)
On 21 Jan 2019, at 07:38, Edgaras Lukoševičius wrote: > > # doveadm mailbox status -u n...@domain.com -t all INBOX.* You’re doing better than I, whenever I try to use * I get # doveadm mailbox status -u krem...@kreme.com -t all * messages=0 recent=0 unseen=0 vsize=0 # doveadm mailbox list -u krem...@kreme.com Archive Drafts Junk dovecot macports Sent Trash bind freebsd postfix bbedit vim_use mailman applescript httpd spamassassin rsync users INBOX -- I intend to live forever -- so far, so good!
Re: Dovecot can't access mailbox that exists (nested)
On 21 January 2019 at 17:16 "@lbutlr" < krem...@kreme.com> wrote: On 21 Jan 2019, at 07:38, Edgaras Lukoševičius < edgaras.lukosevic...@gmail.com> wrote: # doveadm mailbox status -u n...@domain.com -t all INBOX.* You’re doing better than I, whenever I try to use * I get # doveadm mailbox status -u krem...@kreme.com -t all * messages=0 recent=0 unseen=0 vsize=0 # doveadm mailbox list -u krem...@kreme.com Archive Drafts Junk dovecot macports Sent Trash bind freebsd postfix bbedit vim_use mailman applescript httpd spamassassin rsync users INBOX -- I intend to live forever -- so far, so good! Have you tried "*"? --- Aki Tuomi
FOSDEM
Hello all, Several Open-Xchange/Dovecot folks will be attending FOSDEM in Brussels on 2-3 February 2019. For those of you planning to attend: we would love to meet and chat with members of the Dovecot community! Come find us wandering around the talks, or look for us in the evening. Rumor is Brussels has some good beer... Also, we'd love to invite everyone to a talk we are giving on the Chat Over IMAP initiative we have been working on. It takes place Sunday at 1105 in the Real-Time Communications room: https://fosdem.org/2019/schedule/event/chat_over_imap/ Hope to see some of you there! michael
How to connect to a remote server and execute eg a search
Is there a page that explains how to connect to a remote server? What is the syntax of socket_path? doveadm search -S XXX -u testuser mailbox INBOX ALL
Re: How to connect to a remote server and execute eg a search
> On 21 Jan 2019, at 21.32, Marc Roos wrote: > > > Is there a page that explains how to connect to a remote server? What is > the syntax of socket_path? > > > doveadm search -S XXX -u testuser mailbox INBOX ALL -S : Sami
RE: How to connect to a remote server and execute eg a search
Thanks Sami, And I should allow access on the remote server via login_access_sockets? And the authenticated user via this connection is then testuser I presume? > > > Is there a page that explains how to connect to a remote server? What > is the syntax of socket_path? > > > doveadm search -S XXX -u testuser mailbox INBOX ALL -S : Sami
Moving messages between servers with different configurations
I wanted to move messages from a mbox mailbox on server A to mdbox on server B. I thought I could do this by connecting to the remote server with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox INBOX/test" but I guess this will only allow and move messages internally on server B? Should I use dsync, or is there another way to move the messages?
Changing the imaps port #
Dovecot 2.3.4, FreeBSD 11.2
Due to comcast buisness ISP intercepting imaps I need to have my clients
connect to non-standard port (). Previously I had been using stunnel
to receive the imaps connection and forward it to the imap port over
127.0.0.1. But I would like to retire stunnel and have my imap clients
connect remotely.
I have configured the imap-login service -
service imap-login {
inet_listener imap {
address = 127.0.0.1, ::1
port = 143
}
inet_listener imaps {
port =
}
process_min_avail = 3
service_count = 0
vsz_limit = 1 G
}
But I am not seeing any listeners on port -
root@ns(1001)# netstat -an | grep
Using sockstat on FreeBSD:
root@ns(1002)# sockstat | grep dovecot | grep tcp
root dovecot 34800 15 tcp4 *:4190
*:*
root dovecot 34800 37 tcp4 127.0.0.1:143
*:*
root dovecot 34800 38 tcp6 ::1:143
*:*
I have mail_debug = yes, but I don't see any failures. What is the best
way to debug why I am not seeing this port number?
Thanks,
Patrick
Re: Changing the imaps port #
On 21 Jan 2019, at 20:17, Patrick Mahan wrote:
> Due to comcast buisness ISP intercepting imaps
At you sure about that? I've been using comcast business for 7 years and the do
not block 143, 993 587 or 25. they do block 110, but that's fine, I stopped
supporting POP around 2001.
Other than 110, they block DHCP, NETBIOS, SNMP, and ports 445, 520, and 1080.
They will block port 25 on a individual basis, but I've no idea what their
criteria is for that.
> I need to have my clients connect to non-standard port (). Previously I
> had been using stunnel to receive the imaps connection and forward it to the
> imap port over 127.0.0.1. But I would like to retire stunnel and have my
> imap clients connect remotely.
An stunnel or a reverse proxy is the best way to do this, honestly.
As for why your config isn't working, my only guess is maybe you need to
specify ssl?
inet_listener imaps {
port = 999
ssl = yes
}
?
--
If you write the word "monkey" a million times, do you start to think you're
Shakespeare? -- Steven Wright
Re: Changing the imaps port #
Yes, I am pretty sure about that. I originally was connected via AT&T DSL
but wanted the fast access of cable modem. I need permanent IPs which
required me to contract with Comcast buisness. Once I switched over, I was
no longer able to access my imap server, which was as I mentioned, stunnel
listening on the imaps port and forwarding to dovecot listening on the imap
port.
I was getting connection refused on my laptop (thunderbird) email client
when I was not at home. I validated that it was not because it was
reaching my email server. So who ever was rejecting it, I assumed it was
somewhere inside the comcast network. Once I switch to a non-standard
port, I was able to connect again.
Re needing to say ssl = yes, I thought that was implied for imaps?
I can go back to stunnel, just thought it was an unnecessary layer.
Thanks,
Patrick
On Mon, Jan 21, 2019 at 8:46 PM @lbutlr wrote:
> On 21 Jan 2019, at 20:17, Patrick Mahan wrote:
> > Due to comcast buisness ISP intercepting imaps
>
> At you sure about that? I've been using comcast business for 7 years and
> the do not block 143, 993 587 or 25. they do block 110, but that's fine, I
> stopped supporting POP around 2001.
>
> Other than 110, they block DHCP, NETBIOS, SNMP, and ports 445, 520, and
> 1080. They will block port 25 on a individual basis, but I've no idea what
> their criteria is for that.
>
> > I need to have my clients connect to non-standard port ().
> Previously I had been using stunnel to receive the imaps connection and
> forward it to the imap port over 127.0.0.1. But I would like to retire
> stunnel and have my imap clients connect remotely.
>
> An stunnel or a reverse proxy is the best way to do this, honestly.
>
> As for why your config isn't working, my only guess is maybe you need to
> specify ssl?
>
> inet_listener imaps {
> port = 999
> ssl = yes
>}
>
> ?
>
>
> --
> If you write the word "monkey" a million times, do you start to think
> you're
> Shakespeare? -- Steven Wright
>
>
Re: How to connect to a remote server and execute eg a search
> On 21 Jan 2019, at 22.00, Marc Roos wrote:
>
>
>
> Thanks Sami, And I should allow access on the remote server via
> login_access_sockets? And the authenticated user via this connection is
> then testuser I presume?
>
No. It's the doveadm service that it's contacting:
service doveadm {
service_count = 1000
inet_listener {
port = 2425
}
}
and passwords need to match on both servers:
doveadm_password = secret
Sami
Re: Changing the imaps port #
On 21 Jan 2019, at 21:58, Patrick Mahan wrote: > Re needing to say ssl = yes, I thought that was implied for imaps? I would think so, but all the examples I found in a quick search explicitly set it when changing the port. Also, unlike regular comcast, I have found the tech support on their business to be pretty decent. I’d call them and find out why 993 is blocked. -- Badges? We ain't got no badges. We don't need no badges. I don't have to show you any stinking badges.
Re: Changing the imaps port #
On Tue, 22 Jan 2019 at 06:17, Patrick Mahan wrote:
> Dovecot 2.3.4, FreeBSD 11.2
>
> Due to comcast buisness ISP intercepting imaps I need to have my clients
> connect to non-standard port (). Previously I had been using stunnel
> to receive the imaps connection and forward it to the imap port over
> 127.0.0.1. But I would like to retire stunnel and have my imap clients
> connect remotely.
>
> I have configured the imap-login service -
>
> service imap-login {
> inet_listener imap {
> address = 127.0.0.1, ::1
> port = 143
> }
> inet_listener imaps {
> port =
> }
> process_min_avail = 3
> service_count = 0
> vsz_limit = 1 G
> }
>
> But I am not seeing any listeners on port -
>
> root@ns(1001)# netstat -an | grep
>
> Using sockstat on FreeBSD:
>
> root@ns(1002)# sockstat | grep dovecot | grep tcp
> root dovecot 34800 15 tcp4 *:4190
>*:*
> root dovecot 34800 37 tcp4 127.0.0.1:143
> *:*
> root dovecot 34800 38 tcp6 ::1:143
> *:*
>
> I have mail_debug = yes, but I don't see any failures. What is the best
> way to debug why I am not seeing this port number?
>
> Thanks,
>
> Patrick
>
>
I would leave dovecot config untouched and use a firewall (PF) to translate
port to whatever dovecot listens to.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", grep ^[^#] :-)
