Re: Solr/Tika patches for dovecot 2.3.21

On 03/01/2024 10:16, Aki Tuomi via dovecot wrote: On 09/12/2023 15:02 EET John Fawcett wrote: Hi I've just made updated versions of 3 patches against the latest release 2.3.21 in case they are useful to someone or might get considered for official inclusion. John _

Re: Solr/Tika patches for dovecot 2.3.21

> On 09/12/2023 15:02 EET John Fawcett wrote: > > > Hi > > I've just made updated versions of 3 patches against the latest release > 2.3.21 in case they are useful to someone or might get considered for > official inclusion. > > John > > *dovecot-2.3.21-tika-http-auth.patch* > > Allows

Re: solr-jetty package in Debian 11 bullseye

On 5/25/22 13:30, John Gateley wrote: I am in the process of upgrading the OS on my mailserver, currently Debian 9, to Debian 11. The solr-jetty package does not appear to exist in Debian 11. The Solr package that was included in Debian and derivatives was REALLY ancient -- Solr 3.6.  Looks

Re: Solr FTS - message deletes not working as expected

> Op 18 nov. 2021 om 17:15 heeft Shawn Heisey het > volgende geschreven: > > On 11/3/21 11:45 PM, Shawn Heisey wrote: >> Manual expunges of existing messages also are not sending a delete request >> to Solr. I waited several minutes for that too. > > Update on this, since you're all on th

Re: Solr FTS - message deletes not working as expected

On 11/3/21 11:45 PM, Shawn Heisey wrote: Manual expunges of existing messages also are not sending a delete request to Solr.  I waited several minutes for that too. Update on this, since you're all on the edge of your seat waiting. :) I did a Send test with TypeApp, a mail client for Android.

Re: Solr FTS - message deletes not working as expected

On 11/3/21 12:38 PM, Michael Slusarz wrote: Have you tried another client? I tried evolution on Linux. I can't work out how to do an expunge in evolution.  I'm not sure that it CAN do it.  Whenever I delete a message, even with shift-delete, it is moved to the Trash folder (with delete and

Re: Solr FTS - message deletes not working as expected

On 11/3/2021 11:10 PM, Shawn Heisey wrote: Then I downloaded the source archive from the main site, extracted it, and the configure script was included in that.  After compiling, I found the .so file and moved it into place, and I will be testing it. The new library is significantly larger tha

Re: Solr FTS - message deletes not working as expected

On 11/3/2021 10:12 PM, Shawn Heisey wrote: Maybe I can do a custom compile of the source code and replace the /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so file with what the compile produces.  I'm going to try that, and see if it explodes. :) Bit of a problem trying to do this. I pulled

Re: Solr FTS - message deletes not working as expected

On 11/3/21 1:09 PM, Michael Slusarz wrote: For Solr, there's a code path in the FTS expunge code that will silently toss expunge requests: if (ctx->last_indexed_uid == 0 || uid > ctx->last_indexed_uid + 100) { /* don't waste time asking Solr to expunge a m

Re: Solr FTS - message deletes not working as expected

> On 11/03/2021 12:56 PM Shawn Heisey wrote: > > > Thunderbird does NOT necessarily process expunges immediately. Depends on > > what else it is doing in the background. So you can't click delete in the > > UI and not immediately see anything on the backend and definitively > > correlate the

Re: Solr FTS - message deletes not working as expected

On 11/3/21 12:38 PM, Michael Slusarz wrote: Have you tried another client? I only have two clients configured:  Thunderbird 78.13.0 on Linux and Thunderbird 91.2.1 on Windows.  Behavior is the same on both. I will see if I can get another client configured.  Windows Mail is included on Wind

Re: Solr FTS - message deletes not working as expected

Have you tried another client? Thunderbird does NOT necessarily process expunges immediately. Depends on what else it is doing in the background. So you can't click delete in the UI and not immediately see anything on the backend and definitively correlate the two. Another option is to ensure

Re: Solr FTS - message deletes not working as expected

On 10/28/21 8:00 AM, Shawn Heisey wrote: Also, when I send a message with Thunderbird, which deletes the message in Drafts and adds one to Sent, I am not seeing a delete request in the Solr log.  I do see the add. So this isn't limited to just the shift-delete workflow. I have confirmed thi

Re: Solr FTS - message deletes not working as expected

On 10/26/21 12:18 PM, Shawn Heisey wrote: But if I use shift-delete in Thunderbird, which deletes the message immediately without going through Trash, things are different. Also, when I send a message with Thunderbird, which deletes the message in Drafts and adds one to Sent, I am not seeing

Re: Solr FTS - when does indexing happen?

Since most people will want fts_autoindex, the wiki page should include it in its example configuration that goes into 90-plugin.conf. Possibly better ... maybe it should default to "yes". It's probably a safe bet the developers, who are experts on these systems, probably have good reason n

Re: Solr FTS - when does indexing happen?

On 9/4/2021 4:52 PM, Shawn Heisey wrote: I see something talking about autoindex, but it does not have an example so that I can see where it needs to go.  I cannot work it out from what is there. With a little googling, I was able to figure out where it needs to go. And now it acts like I was

Re: Solr FTS - when does indexing happen?

On 9/4/2021 4:06 PM, Steve Dondley wrote: As I recall, indexing an email is triggered immediately when an email is received if you have you dovecot settings set properly to trigger the indexing. The dovecot documentation for FTS, it spells it out. See https://doc.dovecot.org/configuration_man

Re: Solr FTS - when does indexing happen?

On 2021-09-03 12:43 PM, Shawn Heisey wrote: I have Solr FTS on my dovecot install.  I followed the instructions on the dovecot wiki. How long a delay should I expect to see between new mail being delivered with the dovecot LDA and an indexing request sent to Solr?  Because I get a LOT of email f

Re: Solr and FTS - assertion failure [proposed patch for upper bound on rows in solr search]

On 30/12/2020 16:04, Antonino Esposito wrote: > Hi, > > in the latest weeks i'm working on the Solr integration and > immediately i've faced the assertion failure errors, on 2.0.19, 2.2.9 > and 2.3.11.3 servers in our network. > Reading the thread on debian ML, I realize this issue is related to >

Re: Solr and FTS - assertion failure

On 30/12/2020 16:04, Antonino Esposito wrote: > Hi, > > in the latest weeks i'm working on the Solr integration and > immediately i've faced the assertion failure errors, on 2.0.19, 2.2.9 > and 2.3.11.3 servers in our network. > Reading the thread on debian ML, I realize this issue is related to >

Re: solr and dovecot 2.2.36

Hi Maciej, version 6.6.x works fine, but probably also 7.7.x with schema from Dovecot 2.3. Ciao Il 18/08/20 14:00, Maciej Milaszewski ha scritto: Hi I have dovecot-2.2.36.4 (director) + 5 nodes dovecot (dovecot-2.2.36.4) What version of Solr do you recommend ? -- Alessio Cecchi Postmaster

Re: solr and dovecot 2.2.36

Hi I tested ver solr-8.6.0 but not found schema for 2.2.x with version 6.6.x works fine On 18.08.2020 14:59, Alessio Cecchi wrote: > > Hi Maciej, > > version 6.6.x works fine, but probably also 7.7.x with schema from > Dovecot 2.3. > > Ciao > > Il 18/08/20 14:00, Maciej Milaszewski ha scritto: >>

Re: solr and dovecot 2.2.36

* Maciej Milaszewski, 18.08.20 14:00 > I have dovecot-2.2.36.4 (director) + 5 nodes dovecot (dovecot-2.2.36.4) > What version of Solr do you recommend ? Don't know about 2.2.36.4, but for 2.3.11.3 both solr-7.7.3 and solr-8.6.0 appear to work fine. I'm only running a small setup with a handful of

Re: solr and dovecot-2.2.36.4

On 7/17/2020 3:23 AM, Maciej Milaszewski wrote: I try schema.xml and solrconfig.xml from working solr-6.6.5 (dovecot) "dovecot: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Error initializing QueryElevationComponent" That looks like a Solr error. This list is not

Re: solr fts and removing accounts

Citát Shawn Heisey : On 1/19/2020 3:31 AM, Wojciech Puchar wrote: i use solr fts indexing. It worls very well but it have one database per system, not per user. Lets suppose i delete one or more e-mail users in system. How to remove them in solr database to reclaim space? I cannot say w

Re: solr fts and removing accounts

On 1/19/2020 3:31 AM, Wojciech Puchar wrote: i use solr fts indexing. It worls very well but it have one database per system, not per user. Lets suppose i delete one or more e-mail users in system. How to remove them in solr database to reclaim space? I cannot say whether there is anything t

Re: Solr commit and optimize: which user?

On 15/12/2019 17:13, John Gateley wrote: > Hi, > > I have Solr FTR working with dovecot, and need to do the commit and > optimize recommended here: > https://wiki.dovecot.org/Plugins/FTS/Solr > > Should this run as root, or as each mail user. The documentation above > implies, but doesn't state, >

Re: Solr for Dovecot on Debian 9, package has correct binary?

>If I want to use solr, do I have to build dovecot myself? I'd prefer to >use >the debian package (Debian 9 for now). No. Install solr support with apt-get install dovecot-solr -- Christian Kivalo

Re: Solr, Dovecot & macOS / iOS

On 13 Aug 19, at 05:58 , James Brown wrote: > > b) does Mail.app and other mail clients on Macs or iOS devices perform > searches on their local copy of mail or does it just send a search request to > the server? Mail.app uses spotlight on the local data, so if your users are all Mac then sol

Re: Solr, Dovecot & macOS / iOS

> Le 13 août 2019 à 14:53, Sami Ketola a écrit : > > > >> On 13 Aug 2019, at 15.37, Jean-Daniel via dovecot > > wrote: >> >> >> >>> Le 13 août 2019 à 14:16, Sami Ketola via dovecot >> > a écrit : >>> >>> >>> On 13 Aug 2019, at

Re: Solr, Dovecot & macOS / iOS

> On 13 Aug 2019, at 15.37, Jean-Daniel via dovecot wrote: > > > >> Le 13 août 2019 à 14:16, Sami Ketola via dovecot a >> écrit : >> >> >> >>> On 13 Aug 2019, at 14.58, James Brown via dovecot >>> wrote: >>> >>> I’m thinking of getting Solr working with my Dovecot server. Server is ne

Re: Solr, Dovecot & macOS / iOS

> Le 13 août 2019 à 14:16, Sami Ketola via dovecot a > écrit : > > > >> On 13 Aug 2019, at 14.58, James Brown via dovecot >> wrote: >> >> I’m thinking of getting Solr working with my Dovecot server. Server is new >> 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thou

Re: Solr, Dovecot & macOS / iOS

> On 13 Aug 2019, at 14.58, James Brown via dovecot wrote: > > I’m thinking of getting Solr working with my Dovecot server. Server is new > 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands > of messages. > > But I’m not sure if: > > a) it will make enough of a d

Re: solr

On 7/10/2019 2:49 AM, Maciej Milaszewski IQ PL via dovecot wrote: On the other hand solr replication is quite complicated process and rollback or master-slave switch in this case is non-trivial task, that may have result in whole dataset inconsistency. Do you have any experience in such cases ?

Re: solr vs fts

On 2019-07-04 13:35, Felix Zielcke via dovecot wrote: Am Donnerstag, den 04.07.2019, 12:27 +0300 schrieb Aki Tuomi via dovecot: On 4.7.2019 12.22, Maciej Milaszewski IQ PL via dovecot wrote: > Hi > So you're advised to use a solr or something else? > Using any FTS is advisable, currently suit

Re: solr vs fts

Hi, Is Clucene no longer prefered/developed indexer? Thanks. Dave. On 7/4/19, Felix Zielcke via dovecot wrote: > Am Donnerstag, den 04.07.2019, 12:27 +0300 schrieb Aki Tuomi via > dovecot: >> On 4.7.2019 12.22, Maciej Milaszewski IQ PL via dovecot wrote: >> > Hi >> > So you're advised to use a

Re: solr vs fts

Am Donnerstag, den 04.07.2019, 12:27 +0300 schrieb Aki Tuomi via dovecot: > On 4.7.2019 12.22, Maciej Milaszewski IQ PL via dovecot wrote: > > Hi > > So you're advised to use a solr or something else? > > > > Using any FTS is advisable, currently suitable ones would be SOLR or > Xapian (see https

Re: solr vs fts

On 4.7.2019 12.22, Maciej Milaszewski IQ PL via dovecot wrote: >>> A few clients have 25K and more e-mail >>> >>> I thinking about use solr like: >>>  fts = solr >>>  fts_solr = debug url=http://IP:8983/solr/ (solr in external machine) >>> >>> Does it make sense ? use dovecot_indexes and fts ? >>

Re: solr vs fts

>> A few clients have 25K and more e-mail >> >> I thinking about use solr like: >>  fts = solr >>  fts_solr = debug url=http://IP:8983/solr/ (solr in external machine) >> >> Does it make sense ? use dovecot_indexes and fts ? >> What is the difference in performance? >> > Hi! > > Dovecot indexes a

Re: solr vs fts

On 4.7.2019 12.14, Maciej Milaszewski IQ PL via dovecot wrote: > Hi > I have a question about tunning dovecot-2.2.36.x > > Mail was stared in storage via nfs in MAILDIR via > /home/us/usern...@domain.ltd/MAILDIR/ > I use additionally local dovecot_indexes via SSD disk > (/var/dovecot_indexes%h) >

Re: SOLR/Index?

On 15/04/2019 11:38, Larry Rosenman via dovecot wrote: > ⌂63% [l...@thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX > ⌂67% [l...@thebighonker.lerctr.org:~] 1 $ mail -s "test fts1970" > l...@lerctr.org > test fts1970 > > test fts1970 > . > EOT > [l...@thebighonker.lerct

Re: SOLR/Index?

19 at 4:05 AM John Fawcett via dovecot < dovecot@dovecot.org> wrote: > On 15/04/2019 10:59, Larry Rosenman via dovecot wrote: > > I'll run a full test when I'm back in front of areal computer vs. My > phone.(in a few hours) > > Get Outlook for Android <https://

Re: SOLR/Index?

> *From:* dovecot on behalf of John > Fawcett via dovecot > *Sent:* Monday, April 15, 2019 3:57:08 AM > *To:* Dovecot Mailing List > *Subject:* Re: SOLR/Index? >   > On 15/04/2019 10:31, Larry Rosenman via dovecot wrote: >> It always shows the au

Re: SOLR/Index?

cot Mailing List Subject: Re: SOLR/Index? On 15/04/2019 10:31, Larry Rosenman via dovecot wrote: It always shows the autoindex. And yes built from sources. I'm the FreeBSD port maintainer for mail/docecot. This has been happening for several releases. Get Outlook for Android&

Re: SOLR/Index?

t;https://aka.ms/ghei36> > > > *From:* dovecot on behalf of John > Fawcett via dovecot > *Sent:* Monday, April 15, 2019 2:06:55 AM > *To:* dovecot@dovecot.org > *Subject:* Re: SOLR/Index? >   > On 15/04/2019 08:09, Larry R

Re: SOLR/Index?

ia dovecot Sent: Monday, April 15, 2019 2:06:55 AM To: dovecot@dovecot.org Subject: Re: SOLR/Index? On 15/04/2019 08:09, Larry Rosenman via dovecot wrote: Note the hits after the fts rescan/index. Get Outlook for Android<https://aka.ms/ghei36> From: Aki Tuo

Re: SOLR/Index?

April 15, 2019 12:55:07 AM > *To:* Larry Rosenman; John Fawcett > *Cc:* Dovecot Mailing List > *Subject:* Re: SOLR/Index? >   > > > On 15.4.2019 3.33, Larry Rosenman via dovecot wrote: >> ⌂72% [l...@thebighonker.lerctr.org:~] $ doveadm search mailbox &

Re: SOLR/Index?

Note the hits after the fts rescan/index. Get Outlook for Android<https://aka.ms/ghei36> From: Aki Tuomi Sent: Monday, April 15, 2019 12:55:07 AM To: Larry Rosenman; John Fawcett Cc: Dovecot Mailing List Subject: Re: SOLR/Index? On 15.4.2019 3.33,

Re: SOLR/Index?

On 15.4.2019 3.33, Larry Rosenman via dovecot wrote: > ⌂72% [l...@thebighonker.lerctr.org:~] $ doveadm search mailbox > lists/freebsd/ports-commiters  body 'sysutils' > [l...@thebighonker.lerctr.org:~] $ doveadm fts rescan > [l...@thebighonker.lerctr.org:~] $ doveadm index -q > lists/freebsd/ports

Re: SOLR/Index?

⌂72% [l...@thebighonker.lerctr.org:~] $ doveadm search mailbox lists/freebsd/ports-commiters body 'sysutils' [l...@thebighonker.lerctr.org:~] $ doveadm fts rescan [l...@thebighonker.lerctr.org:~] $ doveadm index -q lists/freebsd/ports-commiters ⌂64% [l...@thebighonker.lerctr.org:~] $ tail -f /var/

Re: SOLR/Index?

On 15/04/2019 01:39, Larry Rosenman via dovecot wrote: > > full solr.log at: > https://www.lerctr.org/~ler/solr.log > > The search DOES make it to SOLR: > ⌂77% [l...@thebighonker.lerctr.org:~] 130 $ grep sysutils > /var/log/solr/solr.log > 2019-04-14 18:31:34.749 INFO  (qtp349420578-7538) [   x:dov

Re: SOLR/Index?

full solr.log at: https://www.lerctr.org/~ler/solr.log The search DOES make it to SOLR: ⌂77% [l...@thebighonker.lerctr.org:~] 130 $ grep sysutils /var/log/solr/solr.log 2019-04-14 18:31:34.749 INFO (qtp349420578-7538) [ x:dovecot] o.a.s.c.S.Request [dovecot] webapp=/solr path=/select params={q

Re: SOLR/Index?

On 15/04/2019 01:15, Larry Rosenman via dovecot wrote: > Given all the discussion on FTS/Solr, etc, I have a question: > > I have autoindex set, and searching in roundcube most of the time does > NOT find things, > HOWEVER if I do: > doveadm fts rescan > doveadm index > > I can find things in the m

Re: [PATCH] Re: Solr connection timeout hardwired to 60s

On 4/14/2019 7:59 AM, John Fawcett via dovecot wrote: From dovecot point of view I can see the following as potentially useful features: 1) a configurable batch size would enable to tune the number of emails per request and help stay under the 60 seconds hard coded http request timeout. A config

Re: Solr connection timeout hardwired to 60s

On 14/04/2019 17:55, John Fawcett via dovecot wrote: > The solr server is a small test virtual machine with 0.2 (shared) vCPU > and 0.6MB of memory and non SSD storage. It can index around 2000 emails > per minute when there is no other activity. Average email size is about > 45Kb. I'm not indexing

Re: Solr connection timeout hardwired to 60s

On 14/04/2019 17:16, Peter Mogensen via dovecot wrote: > sorry... I got distracted half way and forgot to put a meaningfull > subject so the archive could figure out the thread. - resending. > > On 4/14/19 4:04 PM, dovecot-requ...@dovecot.org wrote: > >>> Solr ships with autoCommit set to 15 second

Re: [PATCH] Re: Solr connection timeout hardwired to 60s

On 14/04/2019 16:04, Aki Tuomi via dovecot wrote: > >> On 14 April 2019 16:59 John Fawcett via dovecot < dovecot@dovecot.org >> > wrote: >> >> >> On 13/04/2019 17:16, Shawn Heisey via dovecot wrote: >>> On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: If this v

Re: Solr connection timeout hardwired to 60s

sorry... I got distracted half way and forgot to put a meaningfull subject so the archive could figure out the thread. - resending. On 4/14/19 4:04 PM, dovecot-requ...@dovecot.org wrote: >> Solr ships with autoCommit set to 15 seconds and openSearcher set to >> false on the autoCommit.? The aut

Re: [PATCH] Re: Solr connection timeout hardwired to 60s

On 14 April 2019 16:59 John Fawcett via dovecot < dovecot@dovecot.org> wrote: On 13/04/2019 17:16, Shawn Heisey via dovecot wrote: On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: If th

[PATCH] Re: Solr connection timeout hardwired to 60s

On 13/04/2019 17:16, Shawn Heisey via dovecot wrote: > On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: >> If this value was made configurable people could set it to what they >> want. However the underlying problem is likely on solr configuration. > > The Jetty that is included in Solr has it

Re: Solr connection timeout hardwired to 60s

On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: If this value was made configurable people could set it to what they want. However the underlying problem is likely on solr configuration. The Jetty that is included in Solr has its idle timeout set to 50 seconds. But in practice, I have n

Re: Solr connection timeout hardwired to 60s

On 12/04/2019 12:09, Peter Mogensen via dovecot wrote: > Looking further at tcpdumps of the Dovecot->Solr traffic and Solr > metrics it doesn't seem like there's anything suspicious apart from the > TCP windows running full and Dovecot backing of ... until it times out > and close the connection. >

Re: Solr connection timeout hardwired to 60s

Looking further at tcpdumps of the Dovecot->Solr traffic and Solr metrics it doesn't seem like there's anything suspicious apart from the TCP windows running full and Dovecot backing of ... until it times out and close the connection. >From my understanding of how Dovecot operates towards Solr i

Re: Solr connection timeout hardwired to 60s

On 4/4/19 6:57 PM, Peter Mogensen wrote: > > > On 4/4/19 6:47 PM, dovecot-requ...@dovecot.org wrote: >> For a typical Solr index, 60 seconds is an eternity. Most people aim >> for query times of 100 milliseconds or less, and they often achieve >> that goal. > > I'm pretty sure I get these wh

Re: Solr connection timeout hardwired to 60s

On 4/4/2019 6:42 PM, M. Balridge via dovecot wrote: What is a general rule of thumb for RAM and SSD disk requirements as a fraction of indexed document hive size to keep query performance at 200ms or less? How do people deal with the JAVA GC world-stoppages, other than simply doubling or tripling

Re: Solr connection timeout hardwired to 60s

> I'm a denizen of the solr-u...@lucene.apache.org mailing list. > [...] > Here's a wiki page that I wrote about that topic. This wiki is going > away next month, but for now you can still access it: > > https://wiki.apache.org/solr/SolrPerformanceProblems That's a great resource, Shawn. I am

Re: Solr connection timeout hardwired to 60s

On 4/4/19 6:47 PM, dovecot-requ...@dovecot.org wrote: > For a typical Solr index, 60 seconds is an eternity. Most people aim > for query times of 100 milliseconds or less, and they often achieve > that goal. I'm pretty sure I get these while indexing, not querying. Apr 04 16:44:50 host doveco

Re: Solr connection timeout hardwired to 60s

On 4/4/2019 2:21 AM, Peter Mogensen via dovecot wrote: What's the recommended way to handling timeouts on large mailboxes given the hardwired request timeout of 60s in solr-connection.c: http_set.request_timeout_msecs = 60*1000; I'm a denizen of the solr-u...@lucene.apache.org mailing list

Re: Solr connection timeout hardwired to 60s

Hi Shawn Am 04.04.19 um 16:12 schrieb Shawn Heisey via dovecot: > On 4/4/2019 2:21 AM, Peter Mogensen via dovecot wrote: > Here's a wiki page that I wrote about that topic.  This wiki is going > away next month, but for now you can still access it: > > https://wiki.apache.org/solr/SolrPerformanc

Re: Solr connection timeout hardwired to 60s

On 4/4/2019 2:21 AM, Peter Mogensen via dovecot wrote: What's the recommended way to handling timeouts on large mailboxes given the hardwired request timeout of 60s in solr-connection.c: http_set.request_timeout_msecs = 60*1000; I'm a denizen of the solr-u...@lucene.apache.org mailing list

Re: Solr - complete setup (update)

On 2019-01-30 07:33, Stephan Bosch wrote: (forgot to CC mailing list) Op 26/01/2019 om 20:07 schreef Joan Moreau via dovecot: *- Bugs so far* -> Line 620 of fts_solr dovecot plugin : the size oof header is improperly calculated ("huge header" warning for a simple email, which kilss the ind

Re: Solr - complete setup (update)

(forgot to CC mailing list) Op 26/01/2019 om 20:07 schreef Joan Moreau via dovecot: *- Bugs so far* -> Line 620 of fts_solr dovecot plugin : the size oof header is improperly calculated ("huge header" warning for a simple email, which kilss the index of that considered email, so basically M

Re: Solr - complete setup (update)

*- Installation:* -> Create a clean install using the default, (at least in the Archlinux package), and do a "sudo -u solr solr create -c dovecot ". The config files are then in /opt/solr/server/solr/dovecot/conf and datafiles in /opt/solr/server/solr/dovecot/data On my system (Debian) these

Re: Solr - complete setup (update)

Op 26/01/2019 om 15:24 schreef Hendrik Boom: On Sat, Jan 26, 2019 at 01:44:16PM +0100, Stephan Bosch wrote: Hi Joan, Op 14/01/2019 om 07:44 schreef Joan Moreau via dovecot: Hi Stephan, What's up with that ? Thank you so much On 2019-01-05 02:04, Stephan Bosch wrote: Debian does somethin

Re: Solr - complete setup (update)

On Sat, Jan 26, 2019 at 01:44:16PM +0100, Stephan Bosch wrote: > Hi Joan, > > Op 14/01/2019 om 07:44 schreef Joan Moreau via dovecot: > > > > Hi Stephan, > > > > What's up with that ? > > > > Thank you so much > > > > On 2019-01-05 02:04, Stephan Bosch wrote: > > > > > Hi, > > > > > > Op 04/

Re: Solr - complete setup (update)

Hi Joan, Op 14/01/2019 om 07:44 schreef Joan Moreau via dovecot: Hi Stephan, What's up with that ? Thank you so much On 2019-01-05 02:04, Stephan Bosch wrote: Hi, Op 04/01/2019 om 05:36 schreef Joan Moreau via dovecot: Hi This is the summary of my work with SOLR-Dovecot, in my *quest t

Re: Solr -> Xapian ?

st UID. (Supporting out-of-order indexing would be rather difficult to keep track of.) Q2 : WHen Indexing an email, the data is not passed by "build_key". Why so ? What is the link with "build_more" ? The idea is that it calls something like: - build_key(type=hdr, hdr_name=From) -

Re: Solr - complete setup (update)

Yes, the " -property update.autoCreateFields -value false " seems interesting However, we smash the created schema just after On 2019-01-14 23:25, Stephan Bosch wrote: Op 14/01/2019 om 07:44 schreef Joan Moreau via dovecot: Hi Stephan, What's up with that ? Thank you so much Working o

Re: Solr - complete setup (update)

Op 14/01/2019 om 07:44 schreef Joan Moreau via dovecot: Hi Stephan, What's up with that ? Thank you so much Working on it, somewhat anyway. BTW, did you see this ? : """ $ sudo -u solr /opt/solr/bin/solr create -c dovecot WARNING: Using _default configset with data driven schema functi

Re: Solr - complete setup (update)

Hi Stephan, What's up with that ? Thank you so much On 2019-01-05 02:04, Stephan Bosch wrote: Hi, Op 04/01/2019 om 05:36 schreef Joan Moreau via dovecot: Hi This is the summary of my work with SOLR-Dovecot, in my *quest to reproduce the previoulsy excellent work of fts_squat* @Aki :

Re: Solr -> Xapian ?

On 13 Jan 2019, at 10.45, Joan Moreau via dovecot wrote: > > Now, I can see in the logs that several times, the dovecot calls the > fts_backend_xapian_update_set_mailbox with box == NULL. WHy so ? > fts-api.h says: /* Switch to updating the specified mailbox. box may also be set to NULL to

Re: Solr -> Xapian ?

because fts_squat is set to be deleted Xapian and similar libraries offers a very easy interface for FTS (and basically, I have done it already) On 2019-01-07 18:31, Michael Slusarz wrote: Maybe a dumb question (I admit I haven't followed this thread very closely)... But why are you writi

Re: Solr -> Xapian ?

I found the solution o this using SEQ_RANGE_ARRAY_ADD(&RESULT->DEFINITE_UIDS, UID); Now, I can see in the logs that several times, the dovecot calls the fts_backend_xapian_update_set_mailbox with box == NULL. WHy so ? THank you On 2019-01-12 21:40, Joan Moreau via dovecot wrote: I somehow

Re: Solr -> Xapian ?

I somehow fixed the folder issue. (seems some unix rights after too many tests) Getting back on the "fts_results" structure: I am trying: I_ARRAY_INIT(&(RESULT->DEFINITE_UIDS),R->SIZE); I_ARRAY_INIT(&(RESULT->MAYBE_UIDS),0); uint32_t uid; for(i=0;isize;i++) { try { uid=atol(back

Re: Solr -> Xapian ?

additionally, my logic is that the backend stores one databalse per mailox in /xapian-indexes (in the "root" dir of the user), the name od the database is the GUID of the mailbox For INBOX, that works perfectly, and database is properly createdm and backed starts indexing all emails For othe

Re: Solr -> Xapian ?

THank you Now, for the results I see the member of fts_result is : ARRAY_TYPE(seq_range) definite_uids; I have the UID as a aray of uint32_t * How to put my UIDs into this "definite_uids" ? Obviously this is not a simple array/pointer. How to say someting similar to result->definite_uids

Re: Solr -> Xapian ?

On 11 Jan 2019, at 21.23, Joan Moreau via dovecot wrote: > > The below patch resolves the compilation error > > $ diff -p compat.h compat.h.joan > *** compat.h 2019-01-11 20:21:00.726625427 +0100 > --- compat.h.joan 2019-01-11 20:14:41.729109919 +0100 > *** struct iovec; > *** 202,2

Re: Solr -> Xapian ?

El 04/01/19 a las 03:20, Joan Moreau via dovecot escribió: > What about consedering linking Dovecot with Xapian librairies instead of > going to nightmare Solr ? > https://xapian.org/features given that notmuch already does a good job at indexing email (although only supports maildirs afaik), woul

Re: Solr -> Xapian ?

first indexes all the missing mails. So the latest UID is supposed to be the greatest UID. (Supporting out-of-order indexing would be rather difficult to keep track of.) Q2 : WHen Indexing an email, the data is not passed by "build_key". Why so ? What is the link with "build_

Re: Solr -> Xapian ?

The idea is that it calls something like: - build_key(type=hdr, hdr_name=From) - build_more(" t...@iki.fi") - build_key(type=hdr, hdr_name=Subject) - build_more("Re: Solr -> Xapian ?") - build_key(type=body_part) - build_more("message body piece") - build_more(&q

Re: Solr -> Xapian ?

so ? What is the link with "build_more" ? The idea is that it calls something like: - build_key(type=hdr, hdr_name=From) - build_more(" t...@iki.fi") - build_key(type=hdr, hdr_name=Subject)

Re: Solr -> Xapian ?

e rather difficult to keep track of.) Q2 : WHen Indexing an email, the data is not passed by "build_key". Why so ? What is the link with "build_more" ? The idea is that it calls something like: - build_key(type=hdr, hdr_name=From) - build_more("t...@iki.fi") -

Re: Solr -> Xapian ?

idea is that it calls something like: - build_key(type=hdr, hdr_name=From) - build_more("t...@iki.fi") - build_key(type=hdr, hdr_name=Subject) - build_more("Re: Solr -> Xapian ?") - build_key(type=body_part) - build_more("message body piece") - build_more("messa

Re: Solr -> Xapian ?

2 : WHen Indexing an email, the data is not passed by "build_key". Why so ? What is the link with "build_more" ? The idea is that it calls something like: - build_key(type=hdr, hdr_name=From) - build_more("t...@iki.fi") - build_key(type=hdr, hdr_name=Subjec

Re: Solr -> Xapian ?

name=From) - build_more("t...@iki.fi") - build_key(type=hdr, hdr_name=Subject) - build_more("Re: Solr -> Xapian ?") - build_key(type=body_part) - build_more("message body piece") - build_more("message body piece2") ... > Q3 : Searching/Lookup : TH

Re: Solr -> Xapian ?

Maybe a dumb question (I admit I haven't followed this thread very closely)... But why are you writing a new FTS driver? If squat allegedly does everything you need it to do, why don't you just take that plugin and fix it up to do what you need? That seems way easier than trying to create a FT

Re: Solr -> Xapian ?

Hi ANyone to answer specifically ? Q1 : get_last_uid -> Is this the last UID indexed (which may be not the greatest value), or the gratest value (which may not be the latest) (the code of existing plugins is unclear about this, Solr looks for the greatest for insance) Q2 : WHen Indexing an

Re: Solr -> Xapian ?

and finally , for fts_backend__lookup_multi, why is that backend dependent ? Would- nt the below function below be the same for any backend ? Waiting fro your feedback on all those questions Thank you JM - static int fts_backend_xapian_lookup_multi(struct fts_backe

Re: Solr -> Xapian ?

for fts_backend_xxx_lookup, where is specidifed in which field (to, cc, subject, body, from, all) to lookup ? On 2019-01-06 16:03, Joan Moreau wrote: For "rescan " and "optimize", wouldn't it be the dovecot core who indicate which are to be dismissed (expunged), or re-ask for indexing a particul

Re: Solr -> Xapian ?

For "rescan " and "optimize", wouldn't it be the dovecot core who indicate which are to be dismissed (expunged), or re-ask for indexing a particular (or all) uid ? WHy would the backend be aware of the transactions on the mailbox ??? There is alredy "fts_backend_xxx_update_expunge", so I beleiv

  1   2   >