Okay I'm progressing here. If I copy an email within my MUA to another
folder and then make another copy back to my Inbox, the new copied email
is correctly and immediately indexed ! If I delete an email, the search
result is also correct and immediate.
Is it possible that my problem come from Dovecot LDA ?
In 10-mail.conf I have:
# cat dovecot/conf.d/10-mail.conf| grep -v -e "^$" -e "^#" -e "^ #"
mail_location = maildir:/var/vmail/%Ln
namespace {
type = private
location = maildir:/var/vmail/%Ln
inbox = yes
mailbox Trash {
auto = subscribe
special_use = \Trash
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Sent {
auto = subscribe # autocreate, autosubscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
}
mail_uid=vmail
mail_gid=vmail
auth_socket_path = /var/run/dovecot/auth-userdb
mail_plugins = $mail_plugins quota stats fts fts_solr
And in 15-lda.conf:
# cat dovecot/conf.d/15-lda.conf| grep -v -e "^$" -e "^#" -e "^ #"
postmaster_address = postmaster@mydomain
hostname = mydomain
recipient_delimiter = +
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
protocol lda {
mail_plugins = $mail_plugins sieve virtual
}
And the lda portion of "doveconf -n" become:
protocol lda {
mail_plugins = " quota stats fts fts_solr sieve virtual"
}
Here is my postfix relevant conf:
################## MAIN.CF ##############################
# postconf -n | grep dovecot
dovecot_destination_recipient_limit = 1
smtpd_sasl_type = dovecot
virtual_transport = dovecot
#########################################################
################## MASTER.CF ##############################
dovecot unix - n n - - pipe
flags=DRhu user=vmail:vmail
argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -a ${recipient}
-d ${user}@${nexthop}
#########################################################
On 28/05/2018 13:30, kfx wrote:
On 28/05/2018 13:23, kfx wrote:
On 28/05/2018 13:04, Timo Sirainen wrote:
On 28 May 2018, at 13.28, kfx <kada...@gmail.com
<mailto:kada...@gmail.com>> wrote:
Especially what is in the "fts" header vs. "next uid" header? Does
the UID in "fts" header keep changing every time you save a new
mail? I suppose it will.
Diff between 2 emails:
next uid = 30104 | next uid = 30105
last_indexed_uid = 30103 | last_indexed_uid = 30104
So Dovecot thinks it has indexed everything.
You could also monitor (e.g. tcpdump/wireshark) the network traffic
between Dovecot <-> Solr what happens when a new mail arrives. I
suspect Dovecot sends it to Solr, which for whatever reason just
ignores the update.
################### TCPDUMP ####################################
POST /solr/dovecot/update HTTP/1.1
Host: localhost:8983
Date: Mon, 28 May 2018 10:18:05 GMT
Transfer-Encoding: chunked
Connection: Keep-Alive
Content-Type: text/xml
<add><doc><field name="uid">37581</field><field
name="box">e0c58a3093235153110800003ea484a8</field><field
name="user">username</field><field
name="id">37581/e0c58a3093235153110800003ea484a8/username</field><field
name="body">Search Pattern: Kai8oovi
..
</field></doc></add>
And Dovecot sends the mail.
######### SOLR'S RESPONSE ###############
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
</lst>
</response>
And Solr receives it. Your tcpdump doesn't show <commit
softCommit="true" waitSearcher="true"/> being sent though. Do you see
it being sent anywhere?
Yes:
################### TCPDUMP ###################################
POST /solr/dovecot/update HTTP/1.1
Host: localhost:8983
Date: Mon, 28 May 2018 10:18:05 GMT
Expect: 100-continue
Content-Length: 47
Connection: Keep-Alive
Content-Type: text/xml
<commit softCommit="true" waitSearcher="true"/>
################### /TCPDUMP ###################################
Does it make the mails visible if you run it yourself? Or if you run
hard commit? :
curl http://<host>:8983/solr/update?commit=true
# curl http://127.0.0.1:8983/solr/update?commit=true
<html>
[0/0]
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/update. Reason:
<pre> Not Found</pre></p>
</body>
</html>
# curl http://127.0.0.1:8983/solr/dovecot/update?commit=true
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
</lst>
</response>
# doveadm search -u username mailbox INBOX body Kai8oovi
==> No result ('Kai8oovi' is the search pattern, it should returns 4
results)
In the web interface of solr at http://127.0.0.1:8983/solr/#/~cores/dovecot
I can see:
lastModified: less than a minute ago
version:1428772
numDocs:6353615
maxDoc:6356213
deletedDocs:2598
So it IS indexing :(
Just below I see the "optimized:" parameter followed by an icon which
seems saying that is NOT "optimized". Don't know if it's relevant.
This is driving me crazy :(
