Duplicate users with multiple userdb iteration

[Slavko]

Hi,

recently i decided to use quota_over_flag with passwd_file userdb. The
main idea is to prevent dovecot (or vmail) user to have write access to
file with passwords.

To do it, i setup separate userdb as this (SQL userdb is SQLite based,
thus the same problem with write):

passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
userdb {
  args = username_format=%u /etc/dovecot/over_quota
  default_fields = quota_over_flag=1
  driver = passwd-file
  result_internalfail = continue-fail
  result_success = continue-fail
}
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}

For testing i add one user into over_quota file:

sla...@slavino.eu:::

With this setting the doveadm user with user in over_quota seems to
work:

doveadm user sla...@slavino.eu
field   value
uid 2
gid 2
home/srv/vmail/slavino.eu/slavko
mailmaildir:~/Maildir
quota_over_flag 1
quota_rule  *:storage=0M
quota2_rule *:storage=0M

But the user iteration shows duplicate users:

doveadm user '*'
sla...@slavino.eu
sla...@slavino.eu
...

As we discussed on IRC, it is how iteration works, as it emit users as
soon as it meet it, but this doesn't correspond with "multiple userdb
are merged" in docs (for iteration).

I am surprised by this mainly because the particular userdb has set
result_success to "continue-fail" and thus have to set user not found as
with any user, which is not in this db. But it seems, that iteration
ignores this result.

I wrote this as we discussed via IRC, i am not subscribed, thus please
CC me.

regards

-- 
Slavko
http://slavino.sk


pgpDv9UrbARt0.pgp
Description: Digitálny podpis OpenPGP

Dovecot 2.3.13 virtual mailboxes subscription problem

[Slavko]

Hi,

by discussion on IRC channel is post details here.


I am using virtual plugin in dovecot 2.3.13 and i have porblems with
subsription to these virtual mailboxes. I can setup autosubscribe to
them and it works as expected. I can SUBSCRIBE to them, dovecot returns
OK and they are added in subscriptions file, but then they are not
shown in LSUB output. I can UNSUBSCRIBE from subscribed virtual
mailbox (dovecot returns OK), but it is not removed from subscriptions
file and remains there...

I bother with this for long time. I tried to setup mailboxes without
UTF-8 chars, but that doesn't matter nad currently problematic mailbox
has not any nonASCII char (Virtual.Dnes). I tried to define mailboxes
in parent namespace too, doesn't help, nor defining only Virtual in
parent namespace (with and without separator at end). Some evidence
follows.

The (my) subscriptions file (the Virtual.Dnes is at end):

-
V   2


Virtual Dnes
-

IMAP LIST (only Virtual namespace), as shown by dovecot's imap command:

-
* PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
  SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
  MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
  LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
  WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY
  PREVIEW=FUZZY STATUS=SIZE SAVEDATE LITERAL+ NOTIFY METADATA
  SPECIAL-USE QUOTA ACL RIGHTS=texk] Logged in as sla...@slavino.sk

* LIST (\Noselect \HasChildren) "." Virtual
* LIST (\HasNoChildren) "." Virtual.Dnes
* LIST (\HasNoChildren) "." Virtual.D&APQ-le&AX4-it&AOk-
* LIST (\HasNoChildren) "." Virtual.Pr&AO0-lohy
* LIST (\Noselect \HasChildren) "." Verejn&AOk-
* LIST (\HasNoChildren \UnMarked) "." Verejn&AOk-.Novinky
. OK List completed (0.004 + 0.000 + 0.003 secs).
-

One can see that there are three virtual mailboxes, the Virtual.Dnes,
the Virtual.Dôležité (encoded) and the Virtual.Prílohy (encoded too),
including Virtual namespace.

IMAP LSUB output (again only Virtual namespace):

-
* PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
  SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
  MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
  LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
  WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY
  PREVIEW=FUZZY STATUS=SIZE SAVEDATE LITERAL+ NOTIFY METADATA
  SPECIAL-USE QUOTA ACL RIGHTS=texk] Logged in as sla...@slavino.sk
  
* LSUB () "." Virtual.D&APQ-le&AX4-it&AOk-
* LSUB () "." Virtual.Pr&AO0-lohy
a OK Lsub completed (0.001 + 0.000 secs).
-

As one can see, there are only auto-subscribed Virtual mailboxes, the
Virtual.Dnes is missing, despite that it is in subscriptions file.

The doveconf -n output (can be wrapped):

-
# 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 4.19.0-20-amd64 x86_64 Debian 10.12 
# Hostname: servac.skk
auth_default_realm = slavino.sk
auth_mechanisms = plain login
auth_policy_check_after_auth = no
auth_policy_hash_nonce = # hidden, use -P to show it
auth_policy_server_timeout_msecs = 5500
auth_policy_server_url = http://127.0.0.1:9090/
auth_verbose = yes
first_valid_uid = 130
imap_idle_notify_interval = 15 mins
imap_logout_format = in=%i out=%o deleted=%{deleted}
expunged=%{expunged} trashed=%{trashed} hdr_count=%{fetch_hdr_count}
hdr_bytes=%{fetch_hdr_bytes} body_count=%{fetch_body_count}
body_bytes=%{fetch_body_bytes} autoexp=%{autoexpunged} last_valid_uid =
130 lda_original_recipient_header = Envelope-to
lmtp_add_received_header = no lmtp_hdr_delivery_address = none
lmtp_rcpt_check_quota = yes lmtp_save_to_detail_mailbox = yes
login_trusted_networks = 127.0.0.1 ::1
mail_attachment_detection_options = add-flags no-flags-on-fetch
exclude-inlined content-type=!application/*-signature
content-type=!application/pgp-keys mail_attribute_dict =
file:%h/dovecot-attributes mail_home = /srv/vmail/%Ln mail_location =
maildir:~/Maildir mail_plugins = " acl fts fts_xapian notify quota
virtual quota_clone" mail_privileged_group = mail
mail_server_admin = mailto:postmas...@slavino.sk
mail_vsize_bg_after_count = 100
mailbox_list_index_very_dirty_syncs = yes
maildir_very_dirty_syncs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext vnd

imapsieve removeflag ignored

[Slavko]

Hi,

i have working SPAM reporting implementend by imapsieve in
debian's 2.3.13 dovecot.

Recently i decide to add personal spam filter by bogofilter and
use logic to learn/unlearn SPAM/HAM based on message flags.
Here is example of moving message from Junk folder (the
opossite logic is similar):

if hasflag "$Junk" {
    execute :pipe "bogofilter" ["-Sn"];
removeflag "$Junk";
debug_log "Learned HAM (and unlearned)"; }
else {
execute :pipe "bogofilter" ["-n"];
debug_log "Learned HAM (only)";
}
# mark as learned
addflag "$NotJunk";

   report ...

When i move message, the debug lines in log shows, that
logic works and flag is added as expected, but the other
flag is not removed. Thus after moving mail from Junk, which
has $Junk flag, it has both flags, the $Junk and $NotJunk.

I tried to move removeflag action outside of condition and
add it just before addflag, but the result is the same.

For now i do not know if here is something wrong in my script,
or i did some other mistake. On IRC cmouse tell, that removeflag
was fixed in some early version (2.3.5 or so), but it doesn't work
here.

Please, how i can get it working?

regards

PS: please CC me, i am not subscribed

-- 
Slavko
https://www.slavino.sk/