Re: dovecot full text search

2019-12-15 Thread Joan Moreau
Hi The first run of indexing on a large existing mailbox is indeed slow, and I would run "doveadm index -A -q \*" before putting the system in production. Besides the Ram disk, what kind of solution would you suggest ? On 2019-12-10 19:28, Wojciech Puchar via dovecot wrote: Where do write

Re: dovecot full text search

2019-12-10 Thread Wojciech Puchar via dovecot
Where do write ops take place? to the xapian index subdirectory Maybe mount that path to a RAM disk rather than looking for anorher solution. not a solution for a problem but workaround Am 10.12.2019 um 15:50 schrieb Wojciech Puchar via dovecot : what FTP module should i use instead

Re: dovecot full text search

2019-12-10 Thread Admin via dovecot
Where do write ops take place? Maybe mount that path to a RAM disk rather than looking for anorher solution. > > Am 10.12.2019 um 15:50 schrieb Wojciech Puchar via dovecot > : > > what FTP module should i use instead of squat that is probably no longer > supported or no longer at all? > > i

dovecot full text search

2019-12-10 Thread Wojciech Puchar via dovecot
what FTP module should i use instead of squat that is probably no longer supported or no longer at all? i want to upgrade my dovecot installation. it currently uses squat but i found it often crashes on FTS on large mailboxes. i found "xapian" addon for dovecot but while it works excellent AF

Re: Dovecot Full Text Search: HTTP 500 : Unknown fieldType 'text_general' specified on field text. [SERIOUS]

2015-03-05 Thread Muzaffer Tolga Ozses
Make that *text* instead of *text_general* On 5 March 2015 at 12:14, Kevin Laurie wrote: > Hi Muzzafer, > I get the error as specified below when i try to added it in as a field:- > I dont think text_general is a valid field? > > HTTP ERROR 500 > > Problem accessing /solr/. Reason: > > {msg=

Re: Dovecot Full Text Search: HTTP 500 : Unknown fieldType 'text_general' specified on field text. [SERIOUS]

2015-03-05 Thread Kevin Laurie
Anyone here can enlighten me on this? On Thu, Mar 5, 2015 at 5:14 PM, Kevin Laurie wrote: > Hi Muzzafer, > I get the error as specified below when i try to added it in as a field:- > I dont think text_general is a valid field? > > HTTP ERROR 500 > > Problem accessing /solr/. Reason: > > {msg=

Dovecot Full Text Search: HTTP 500 : Unknown fieldType 'text_general' specified on field text. [SERIOUS]

2015-03-05 Thread Kevin Laurie
Hi Muzzafer, I get the error as specified below when i try to added it in as a field:- I dont think text_general is a valid field? HTTP ERROR 500 Problem accessing /solr/. Reason: {msg=SolrCore 'collection1' is not available due to init failure: Could not load conf for core collection1: Unkn

Re: Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS]

2015-03-05 Thread Kevin Laurie
Below is my schema.xml

Re: Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS]

2015-03-05 Thread Leon Kyneur
In your schema.XML check you have defined: On 05/03/2015 7:11 PM, "Kevin Laurie" wrote: > Hello, > My dovecot constantly runs into this error. > I want to fix this one last time, I am tired of troubleshooting so > please someone give me a lasting and proper solution for this error. I > think i

Re: Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS]

2015-03-05 Thread Muzaffer Tolga Ozses
Paste your xml file here. On 5 March 2015 at 10:11, Kevin Laurie wrote: > Hello, > My dovecot constantly runs into this error. > I want to fix this one last time, I am tired of troubleshooting so > please someone give me a lasting and proper solution for this error. I > think its a problem with

Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS]

2015-03-05 Thread Kevin Laurie
Hello, My dovecot constantly runs into this error. I want to fix this one last time, I am tired of troubleshooting so please someone give me a lasting and proper solution for this error. I think its a problem with the dovecot-solr module. Please tell me how do I find the root of this problem with

Re: [Dovecot] Full text search improvements

2013-12-05 Thread Timo Sirainen
On 5.12.2013, at 10.40, Steffen Kaiser wrote: >> 9. Attachments can be translated to indexable UTF-8 text already with >> fts_decoder setting by doing it via a conversion script. This could also >> support Apache Tika server directly. > > This means some kind of MIME type based (or file type g

Re: [Dovecot] Full text search improvements

2013-12-05 Thread Steffen Kaiser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 30 Nov 2013, Timo Sirainen wrote: 7. Don't index non-text data? For example if there is large block of base64 data or something else that definitely doesn't look like text, it's pretty useless to index it. Then again, we do want to index al

Re: [Dovecot] Full text search improvements

2013-12-04 Thread Michael M Slusarz
Quoting Timo Sirainen : 1. Support for multiple languages. Use textcat while indexing to guess the language of the indexed data. FWIW, you could probably use the Content-Language header (if it exists) to at least give a hint. No guarantee it is correct, but it's a better starting place t

Re: [Dovecot] Full text search improvements

2013-12-04 Thread Metro Domain Admin
Substring match is important to us, so we'd love to see Squat reinstated with speed improvements. It seems like Solr can handle substrings as well ([Edge]NGramFilterFactory), but for small deployments, having the engine built right in is a plus.

Re: [Dovecot] Full text search improvements

2013-12-02 Thread Timo Sirainen
On 3.12.2013, at 0.09, Mike Abbott wrote: >> Do you think [moving IMAP IDLE connections to a separate imap-idle process] >> would work for you also? > > Probably. It always depends on the details. Forking a new imap process > every time there's a little input to read or output to send might

Re: [Dovecot] Full text search improvements

2013-12-02 Thread Mike Abbott
> Do you think [moving IMAP IDLE connections to a separate imap-idle process] > would work for you also? Probably. It always depends on the details. Forking a new imap process every time there's a little input to read or output to send might perform poorly under load. Having a pool of ready

Re: [Dovecot] Full text search improvements

2013-12-02 Thread Gedalya
On 12/02/2013 02:41 PM, Timo Sirainen wrote: Currently I’m thinking that most of the reasons for client_limit>1 can be avoided just by moving IMAP IDLE connections to a separate imap-idle process where they wait until they have more work to do. Do you think that would work for you also? I was

Re: [Dovecot] Full text search improvements

2013-12-02 Thread Timo Sirainen
On 2.12.2013, at 20.50, Mike Abbott wrote: >> how [FTS indexing] could be improved for everyone in future > > For sites which set client_limit > 1 it would help performance not to stall > for INDEXER_WAIT_MSECS when polling the indexer for input. Currently dovecot > unwinds back out to the ma

Re: [Dovecot] Full text search improvements

2013-12-02 Thread Mike Abbott
> how [FTS indexing] could be improved for everyone in future For sites which set client_limit > 1 it would help performance not to stall for INDEXER_WAIT_MSECS when polling the indexer for input. Currently dovecot unwinds back out to the main command loop repeatedly to allow other clients to

[Dovecot] Full text search improvements

2013-11-30 Thread Timo Sirainen
FTS indexing is something I hear quite often nowadays. I’ve added some hacks to make it work better for some installations, but it’s about time to think about the whole design and how it could be improved for everyone in future. Here are some of my initial thoughts. Currently Dovecot supports 3

Re: [Dovecot] Full-text search

2013-02-22 Thread Timo Sirainen
On 18.2.2013, at 18.10, Valery V. Sedletski wrote: > I discovered that the full-text search (fts) plugin can work without > SQUAT/LUCENE/SOLR backend. I.e., Dovecot creates separate indexes for > header search in files dovecot.index and dovecot.index.cache. Even, the > search by headers is fast e

[Dovecot] Full-text search

2013-02-18 Thread Valery V. Sedletski
Hi all I discovered that the full-text search (fts) plugin can work without SQUAT/LUCENE/SOLR backend. I.e., Dovecot creates separate indexes for header search in files dovecot.index and dovecot.index.cache. Even, the search by headers is fast enough, and can search for phrases. Also, it seems that

Re: [Dovecot] Full text search in attachments

2012-08-19 Thread Timo Sirainen
On 19.8.2012, at 21.57, Mailing wrote: > Am 19.08.2012 17:26, schrieb Timo Sirainen: >> On 11.8.2012, at 5.28, Mailing wrote: >> I updated the wiki with: >> >> * See the decode2text.sh script included in Dovecot for how to use this. > > I want to send the complete attachments (unparsed) to solr

Re: [Dovecot] Full text search in attachments

2012-08-19 Thread Mailing
Hi Timo, Am 19.08.2012 17:26, schrieb Timo Sirainen: On 11.8.2012, at 5.28, Mailing wrote: I updated the wiki with: * See the decode2text.sh script included in Dovecot for how to use this. I want to send the complete attachments (unparsed) to solr server and let solr do the parsing work. I

Re: [Dovecot] Full text search in attachments

2012-08-19 Thread Timo Sirainen
On 11.8.2012, at 5.28, Mailing wrote: > is it possible to use the Solr full text search plugin for indexing mail > attachments? I found a very old patch and some hints regarding a fts_decoder > script that I don't understand. > > Making Solr indexing PDF or Office files shouldn't be that diffic

[Dovecot] Full text search in attachments

2012-08-10 Thread Mailing
Hello, is it possible to use the Solr full text search plugin for indexing mail attachments? I found a very old patch and some hints regarding a fts_decoder script that I don't understand. Making Solr indexing PDF or Office files shouldn't be that difficult, but how can I enable the plugin t