Hello
I have some performance problem with
doveadm quota get -u user@domain
after migrating from 2.3.21 to 2.4.1 and enabling fts-flatcurve plugin for mail
account with 1330 folders and subfolders.
Quota on 2.3.21 was stored in maildirsize.
After upgrading dovecot to 2.4.2 quota driver is set to count
When I run doveadm quota get -u user@domain command it takes about 15 minutes.
Mail account quota is used about 6.7GB and about 10k messeges.
Doveadm process use 100% CPU when it try to count quota values.
When I run command doveadm with "strace" to trace callbacks it looks like:
14:51:00.577061 newfstatat(AT_FDCWD, "/maildovecot/*******/Maildir/Foldery
przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714",
{st_mode=S_IFDIR|0700, st_size=4096, ...}, 0) = 0
14:51:00.577295 openat(AT_FDCWD, "/maildovecot/*******/Maildir/Foldery
przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714",
O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 13
14:51:00.577680 rmdir("/maildovecot/*******/Maildir/Foldery
przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1
ENOTEMPTY (Directory not empty)
14:51:00.577765 rmdir("/maildovecot/*******/Maildir/Foldery
przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1
ENOTEMPTY (Directory not empty)
14:51:00.577847 rmdir("/maildovecot/*******/Maildir/Foldery
przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1
ENOTEMPTY (Directory not empty)
14:51:00.577929 rmdir("/maildovecot/*******Maildir/Foldery
przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1
ENOTEMPTY (Directory not empty)
or
14:33:44.529146 newfstatat(AT_FDCWD, "/maildovecot//******//Maildir/Foldery
przechowywania/programy lojalnosciowe/vitay/fts-flatcurve/cur", 0x7fff0d9cf270,
0) = -1 ENOENT (No such file or directory)
and it loops 2374 times
grep fts-flatcurve strace_quota | grep rossmann | wc -l
2374
My configuration:
quota "User quota" {
driver = count
quota_storage_grace = 10M
quota_mail_size = 50M
quota_message_count = 1500000
quota_warning warn-80-storage {
quota_storage_percentage = 80
quota_warning_resource = storage
quota_warning_threshold = over
execute quota-warning {
args = 80 size %{user} %{userdb:quota_storage_size}
}
}
quota_warning warn-90-storage {
quota_storage_percentage = 90
quota_warning_resource = storage
quota_warning_threshold = over
execute quota-warning {
args = 90 size %{user} %{userdb:quota_storage_size}
}
}
quota_warning warn-90-message {
quota_message_percentage = 95
quota_warning_resource = message
quota_warning_threshold = over
execute quota-warning {
args = 90 inodes
}
}
}
service quota-warning {
executable = script /dovecot/bin/quota.sh
user = mail
unix_listener quota-warning {
user = mail
}
}
cat conf.d/90-fts.conf
language_tokenizers = generic email-address
language_tokenizer_generic_algorithm = simple
language_filters = lowercase snowball stopwords
fts_autoindex = yes
fts_autoindex_max_recent_msgs = 100
fts_message_max_size = 20M
fts_driver = flatcurve
fts_decoder_driver = script
fts_decoder_script_socket_path = decode2text
service decode2text {
executable = script /dovecot/bin/attachment2text.sh
unix_listener decode2text {
mode = 0666
user = dovecot
group = dovecot
}
}
fts_header_excludes {
Authentication-Results = yes
Received = yes
DKIM-* = yes
X-* = yes
Comments = yes
Date = yes
Message-ID = yes
Received = yes
}
mailbox TRASH {
fts_autoindex = no
}
mailbox SPAM {
fts_autoindex = no
}
mailbox confirmed-spam {
fts_autoindex = no
}
mailbox confirmed-ham {
fts_autoindex = no
}
language en {
default = yes
filters = lowercase snowball english-possessive stopwords
}
fts flatcurve {
commit_limit = 500
min_term_size = 2
optimize_limit = 10
rotate_count = 5000
rotate_time = 5s
substring_search = no
}
My questions are:
Why "doveadm quota get" tries to open "fts-flatcurve" directories and tries to
find "cur" or "ndex.714" directory?
Is there any options to exclude all "fts-flatcurve" directories from quota ?
Do you observe performance hit for mail accounts with many folders and
subfolders after enabling fts-flatcurve?
_______________________________________________
dovecot mailing list -- [email protected]
To unsubscribe send an email to [email protected]
