I recently got FTS search with Clucene working on FreeBSD 10.1 (I had some
trouble with 9.3, postponed solving it, then restarted with 10.1 by manually
installing libtextcat, a listed dependency on
http://wiki2.dovecot.org/Plugins/FTS/Lucene, but not part of the FreeBSD
makefile: maybe that helped, maybe it was irrelevant).
However, I now get some anomalous results. I'm putting them together as they
may be related. Or, maybe not.
1) Searching from Thunderbird returns 35x as many results for the same term as
searching from the command line. I'm not sure if the indexes are created or
enumerable.
2) There seems to be a glitch in the maildir naming structure that results in a
double slash (//) which doesn't seem to impact IMAP serving or Sieve, but maybe
is causing the problem above with Clucene. Any hints?
If I run a search from Thunderbird and "run search on server" for a somewhat
infrequently used term like "titanium," I get results in more or less every
archive folder one by one over about 20 minutes of searching and get about 516
results. It seems to run a little faster without the "run search on server"
checkbox selected, but Thunderbird on Windows is abandonware and the only
useful datapoint is 516 results. Maybe it is time for Claws.
Anyway, if I run:
# doveadm search -u u...@domain.com mailbox '*' body titanium
doveadm(u...@domain.com): Error:
stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory
doveadm(u...@domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal
error occurred. Refer to server log for more information. [2015-03-11 09:11:16]
bcadc922ba26b953a25701002812f29a 812478
bcadc922ba26b953a25701002812f29a 9338037
bcadc922ba26b953a25701002812f29a 9342918
bcadc922ba26b953a25701002812f29a 9343205
bcadc922ba26b953a25701002812f29a 9346096
bcadc922ba26b953a25701002812f29a 9350977
bcadc922ba26b953a25701002812f29a 9351379
bcadc922ba26b953a25701002812f29a 9352318
bcadc922ba26b953a25701002812f29a 9352441
bcadc922ba26b953a25701002812f29a 9352501
3d5d982e3a89b853c12c01002812f29a 1901940
3d5d982e3a89b853c12c01002812f29a 1908073
3d5d982e3a89b853c12c01002812f29a 1941453
3d5d982e3a89b853c12c01002812f29a 2036962
3d5d982e3a89b853c12c01002812f29a 2038360
I get 15 results.
# doveadm fetch -u u...@domain.com "mailbox" mailbox-guid
3d5d982e3a89b853c12c01002812f29a uid 1908073
mailbox: INBOX
All are from the inbox.
This takes roughly 30 seconds (much faster, that's good, though still not super
fast) and in the logs I see promising messages like:
Mar 11 09:27:52 host dovecot: indexer-worker(u...@domain.com): Indexed 3708
messages in Commerce
but also slightly less promising messages like:
Mar 11 09:27:52 host dovecot: indexer-worker(u...@domain.com): Warning: Maildir
/mail/domain.com/user//.Bulkmail: Expunged message reappeared, giving a new UID
(old uid=9335155, file=1426089637.M350915P91305.host,S=23845,W=24282:2,)
If I execute
# doveadm index -A '*'
I get
doveadm(u...@domain.com): Error:
stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory
doveadm(u...@domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal
error occurred. Refer to server log for more information. [2015-03-11 09:51:26]
(there's nothing in the server log) But note in both previous results a double
slash (//) in the directory structure. I don't think that's right.
So:
Does # doveadm search search subfolders? The internets say either '*' or INBOX
should search the hierarchy and the search command does set off indexing
operations on every subfolder as indicated by the logs, but the results are
only from the top level folder.... perhaps that is related to...
Why do some errors report a double slash (//) in directory names? I don't seem
to be having any other symptoms (if, indeed, this is one) of incompatibilities.
My sieve filters work just fine and are accessible from the managesieve plugin
in Thunderbird.
My config file is below:
# 2.2.15: /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.6 (3e924b1b6c5c+)
# OS: FreeBSD 10.1-RELEASE amd64
auth_default_realm = domain.com
auth_mechanisms = plain login
auth_realms = domain.com domain2.org
disable_plaintext_auth = no
first_valid_gid = 5000
first_valid_uid = 5000
hostname = domain.com
imap_idle_notify_interval = 29 mins
last_valid_gid = 5000
last_valid_uid = 5000
lda_mailbox_autocreate = yes
listen = *
mail_gid = 5000
mail_location = maildir:~:CONTROL=/var/no-quota/%u
mail_plugins = quota mail_log notify fts fts_lucene
mail_temp_dir = /var/tmp
mail_uid = 5000
maildir_broken_filename_sizes = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave duplicate
spamtest spamtestplus editheader
mbox_lazy_writes = no
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
expire = Trash
fts = lucene
fts_autoindex = yes
fts_lucene = whitespace_chars=@. mime_parts
mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
mail_log_fields = uid box msgid size
quota = maildir:User quota
quota_exceeded_message = Storage quota for this account has been exceeded,
please try again later.
quota_rule = *:storage=50G
quota_rule2 = Trash:storage=+30%%
quota_rule3 = Sent:storage=+30%%
quota_warning = storage=90%% quota-warning 90 %u
quota_warning2 = storage=75%% quota-warning 75 %u
sieve = ~/.dovecot.sieve
sieve_before = /usr/local/etc/dovecot/sieve/
sieve_dir = ~/sieve
sieve_extensions = +spamtest +spamtestplus +relational
+comparator-i;ascii-numeric +editheader
sieve_global_dir = /usr/local/etc/dovecot/sieve
}
postmaster_address = u...@domain.com
protocols = imap lmtp sieve sieve
sendmail_path = /usr/local/sbin/sendmail
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
service_count = 1
vsz_limit = 128 M
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = vmail
}
ssl = required
ssl_cert = </usr/local/etc/ca/host-cert.pem
ssl_key = </usr/local/etc/ca/host-unencrypted-key.pem
ssl_prefer_server_ciphers = yes
userdb {
driver = prefetch
}
userdb {
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
protocol lda {
mail_plugins = quota mail_log notify fts fts_lucene sieve
}
protocol imap {
mail_max_userip_connections = 20
mail_plugins = quota mail_log notify fts fts_lucene imap_quota imap_zlib
}
protocol lmtp {
mail_plugins = quota mail_log notify fts fts_lucene sieve
}
(END)
