Here it is: Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Effective uid=8, gid=8, home=/var/spool/mail/iwascoding/paul Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota root: name=User quota backend=dict args=:file:/var/spool/mail/iwascoding/paul/dovecot-quota Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=60000 Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota grace: root=User quota bytes=214748364 (10%) Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: dict quota: user=p...@iwascoding.com, uri=file:/var/spool/mail/iwascoding/paul/dovecot-quota, noenforcing=0 Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: fs: root=/var/spool/mail/iwascoding/paul/mdbox, index=, indexpvt=, control=, inbox=, alt= Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: FTS Xapian: Partial=2, Full=20 DB_PATH=/var/spool/mail/iwascoding/paul/mdbox/xapian-indexes Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: Mailbox opened because: indexing Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: FTS Xapian : Mailbox sent : Last UID=0 Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: FTS Xapian : Mailbox sent : Last UID=0 Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Namespace : Using permissions from /var/spool/mail/iwascoding/paul/mdbox: mode=0700 gid=default Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 1: Opened mail because: fts indexing Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Opening RW /var/spool/mail/iwascoding/paul/mdbox/xapian-indexes/db_9ddfe10d8a8a8a568c120000654d370e Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 2: Opened mail because: fts indexing Jan 14 09:26:08 mail dovecot: indexer-worker(p...@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 3: Opened mail because: fts indexing
Thank you! > On 14. Jan 2019, at 10:11, Joan Moreau via dovecot <dovecot@dovecot.org> > wrote: > > Can you send the log part that includes the "init" of the plugins (something > similar as below) ? > > WHich version of Xapian are you on ? > > Jan 14 09:10:04 gjserver dovecot[31082]: > indexer-worker(ad...@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: > FTS Xapian: Partial=2, Full=20 > DB_PATH=/data/mail/grosjo.net/admin/xapian-indexes > Jan 14 09:10:04 gjserver dovecot[31082]: > indexer-worker(ad...@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: > FTS Xapian : Mailbox Mail : Last UID=815055 > Jan 14 09:10:04 gjserver dovecot[31082]: > indexer-worker(ad...@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: > FTS Xapian : Mailbox Mail : Last UID=815055 > Jan 14 09:10:04 gjserver dovecot[31082]: > indexer-worker(ad...@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: > Opening RW > /data/mail/grosjo.net/admin/xapian-indexes/db_5c935034609bc14c0e550000d6a3092d > > > > > > > On 2019-01-14 10:08, Paul Hecker via dovecot wrote: > >> Hi, >> >> I installed and tested your version, but the indexer process crashes >> reproducible with the following command after about 2000 messages were >> indexed: >> >> doveadm index -u p...@iwascoding.com <mailto:p...@iwascoding.com> -q \* >> >> Jan 14 09:26:15 mail dovecot: indexer-worker(p...@iwascoding.com >> <mailto:p...@iwascoding.com>)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox >> sent: UID 2038: Opened mail because: fts indexing >> Jan 14 09:26:15 mail dovecot: indexer-worker: Error: terminate called after >> throwing an instance of 'std::bad_alloc' >> Jan 14 09:26:15 mail dovecot: indexer-worker: Error: what(): >> std::bad_alloc >> Jan 14 09:26:15 mail dovecot: indexer: Error: Indexer worker disconnected, >> discarding 48 requests for p...@iwascoding.com <mailto:p...@iwascoding.com> >> Jan 14 09:26:15 mail dovecot: indexer-worker(p...@iwascoding.com >> <mailto:p...@iwascoding.com>)<16777><IKpfOqBHPFyJQQAADYqDFA>: Fatal: master: >> service(indexer-worker): child 16777 killed with signal 6 (core dumps >> disabled - https://dovecot.org/bugreport.html#coredumps >> <https://dovecot.org/bugreport.html#coredumps>) >> >> I tried to delete the message, but this does not help (crashes e.g. after >> message 2029 or 2044). Other folders with fewer messages were successfully >> indexed before. >> >> Sorry, could not convince dovecot to create core dumps (read the docs, >> changed /proc/sys/kernel/core_pattern, added LimitCORE=unlimited/infinity, >> even created /etc/systemd/system/dovecot.service.d/coredump.conf to no >> avail). Custom Dovecot 2.3.4 on Debian Stretch. >> >> Thanks, >> Paul >> >> >>> On 14. Jan 2019, at 07:42, Joan Moreau via dovecot <dovecot@dovecot.org >>> <mailto:dovecot@dovecot.org>> wrote: >>> >>> Thank you Stephan. >>> >>> The version here shall be up and running : >>> https://github.com/grosjo/fts-xapian <https://github.com/grosjo/fts-xapian> >>> >>> >>> >>> >>> >>> >>> On 2019-01-14 00:07, Stephan Bosch wrote: >>> >>>> >>>> >>>> Op 13/01/2019 om 21:25 schreef Joan Moreau via dovecot: >>>>> >>>>> >>>>> I tried to combined it, the "autoreconf" errors are solved >>>>> >>>>> Now, when I type "make install", the lib is not pushed into dovecot >>>>> folder, but somewhere in /usr/local/... >>>>> >>>>> How to adjust this to have it arriving in the proper folder ? >>>>> >>>> >>>> Depends on your system. It mostly a matter of setting a proper --prefix >>>> directory for configure, but other paths are configurable as well. I >>>> usually check what the official distribution package for Dovecot is doing >>>> and use that as a basis. >>>> >>>> For Debian I use the following configure command: >>>> >>>> ./configure --with-ldap=plugin --with-ssl=openssl --with-sql=plugin >>>> --with-lua=plugin --with-pgsql --with-mysql --with-sqlite \ >>>> --with-gssapi=plugin --with-solr --with-ioloop=best >>>> --enable-maintainer-mode \ >>>> --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib >>>> --localstatedir=/var --mandir=/usr/share/man \ >>>> --infodir=/usr/share/info >>>> --with-moduledir=/usr/lib/dovecot/modules --disable-rpath --disable-static >>>> >>>> Regards, >>>> >>>> Stephan >>>> >>>>> >>>>> On 2019-01-13 21:01, Tuomi, Aki wrote: >>>>> >>>>>> You copied your Makefile.am there. Stephan made you a working version, >>>>>> can you try that? >>>>>> (sorry for dup) >>>>>> Aki >>>>>> -------- Original message -------- >>>>>> From: Joan Moreau <j...@grosjo.net <mailto:j...@grosjo.net>> >>>>>> Date: 13/01/2019 21:39 (GMT+02:00) >>>>>> To: Stephan Bosch <step...@rename-it.nl <mailto:step...@rename-it.nl>> >>>>>> Cc: Aki Tuomi <aki.tu...@open-xchange.com >>>>>> <mailto:aki.tu...@open-xchange.com>> >>>>>> Subject: Re: [FTS Xapian] Beta release >>>>>> >>>>>> I used the skeleton from Aki : https://github.com/grosjo/fts-xapian >>>>>> <https://github.com/grosjo/fts-xapian> >>>>>> >>>>>> However, when I try to act as a visitor, I reach teh follwoing error: >>>>>> >>>>>> # autoreconf -vi >>>>>> autoreconf: Entering directory `.' >>>>>> autoreconf: configure.ac: not using Gettext >>>>>> autoreconf: running: aclocal -I m4 >>>>>> autoreconf: configure.ac: tracing >>>>>> autoreconf: running: libtoolize --copy >>>>>> libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, '.'. >>>>>> libtoolize: copying file './ltmain.sh' >>>>>> libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. >>>>>> libtoolize: copying file 'm4/libtool.m4' >>>>>> libtoolize: copying file 'm4/ltoptions.m4' >>>>>> libtoolize: copying file 'm4/ltsugar.m4' >>>>>> libtoolize: copying file 'm4/ltversion.m4' >>>>>> libtoolize: copying file 'm4/lt~obsolete.m4' >>>>>> autoreconf: running: /usr/bin/autoconf >>>>>> autoreconf: running: /usr/bin/autoheader >>>>>> autoreconf: running: automake --add-missing --copy --no-force >>>>>> configure.ac:9: installing './compile' >>>>>> configure.ac:11: installing './config.guess' >>>>>> configure.ac:11: installing './config.sub' >>>>>> configure.ac:7: installing './install-sh' >>>>>> configure.ac:7: installing './missing' >>>>>> src/Makefile.am: installing './depcomp' >>>>>> /usr/share/automake-1.16/am/depend2.am: error: am__fastdepCXX does not >>>>>> appear in AM_CONDITIONAL >>>>>> /usr/share/automake-1.16/am/depend2.am: The usual way to define >>>>>> 'am__fastdepCXX' is to add 'AC_PROG_CXX' >>>>>> /usr/share/automake-1.16/am/depend2.am: to 'configure.ac' and run >>>>>> 'aclocal' and 'autoconf' again >>>>>> src/Makefile.am: error: C++ source seen but 'CXX' is undefined >>>>>> src/Makefile.am: The usual way to define 'CXX' is to add 'AC_PROG_CXX' >>>>>> src/Makefile.am: to 'configure.ac' and run 'autoconf' again. >>>>>> src/Makefile.am:11: warning: variable 'NOPLUGIN_LDFLAGS' is defined but >>>>>> no program or >>>>>> src/Makefile.am:11: library has 'NOPLUGIN' as canonical name (possible >>>>>> typo) >>>>>> autoreconf: automake failed with exit status: 1 >>>>>> >>>>>> >>>>>> >>>>>> On 2019-01-13 20:24, Stephan Bosch wrote: >>>>>> >>>>>> Oh, right, a distribution tarball doesn't include some of the >>>>>> necessary files for your repository like autogen.sh and >>>>>> .gitignore. The attached tarball includes all those and is ready >>>>>> for `git init`. The previous tarball was made with `make >>>>>> distcheck` from this one. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Stephan. >>>>>> >>>>>> Op 13/01/2019 om 20:14 schreef Stephan Bosch: >>>>>> >>>>>> Hi Joan, >>>>>> >>>>>> Op 13/01/2019 om 19:03 schreef Aki Tuomi: >>>>>> >>>>>> Yes, from compiling point of view it is done. >>>>>> >>>>>> Unfortunately what is not done is all the other work >>>>>> involved, such as fixing all the inevitable bugs it has >>>>>> and maintaining it. We do not want, at this moment, take >>>>>> up maintaining and developing yet another FTS plugin as >>>>>> we have plenty of things to do already. >>>>>> >>>>>> I invite you to setup your own repository and provide >>>>>> this plugin from there, being the maintainer of this >>>>>> plugin. We can add a link to your plugin on our FTS page >>>>>> so people can also find it. >>>>>> >>>>>> There are other plugins like this, e.g. >>>>>> https://github.com/st3fan/dovecot-xaps-plugin >>>>>> <https://github.com/st3fan/dovecot-xaps-plugin> >>>>>> >>>>>> >>>>>> I turned the code you provided into a separate plugin >>>>>> package. The distribution tarball is attached. >>>>>> >>>>>> Notable changes: >>>>>> >>>>>> - Added example copyright headers and COPYING and AUTHORS >>>>>> files. You should modify those to your preference. >>>>>> - Added README and INSTALL files (in markdown using Pandoc). >>>>>> Those need to be amended with details. >>>>>> - Amended the plugin code to display a debug message with the >>>>>> plugin name and version upon plugin load. >>>>>> >>>>>> I advise you to turn this into a git repository and continue >>>>>> from there. >>>>>> >>>>>> I do not recommend releasing this plugin with the >>>>>> -fpermissive flag and the resulting warning as it is now. But >>>>>> I'm assuming this is still a work in progress, so that is OK. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Stephan. >>>>>> >>>>>> >>>>>> On 13 January 2019 at 19:52 Joan Moreau <j...@grosjo.net >>>>>> <mailto:j...@grosjo.net> >>>>>> <mailto:j...@grosjo.net <mailto:j...@grosjo.net>>> wrote: >>>>>> >>>>>> >>>>>> The only point here of this fts-xapian is to get rid >>>>>> of solr (because it >>>>>> is just a nightmare to setup) and squat (because it is >>>>>> considere >>>>>> obsolete). >>>>>> >>>>>> I already sent the changed in configure.ac, makefile.am, >>>>>> etc.. in order >>>>>> to include it in the dovecot, and it compiles properly >>>>>> >>>>>> The only remaining point is to push it in hte git (yes, >>>>>> everything is >>>>>> already done) >>>>>> >>>>>> On 2019-01-13 18:45, Aki Tuomi wrote: >>>>>> >>>>>> On 13 January 2019 at 17:05 Joan Moreau via >>>>>> dovecot <dovecot@dovecot.org >>>>>> <mailto:dovecot@dovecot.org> >>>>>> <mailto:dovecot@dovecot.org >>>>>> <mailto:dovecot@dovecot.org>>> wrote: >>>>>> >>>>>> Hi >>>>>> >>>>>> Please find attached the beta release of FTS >>>>>> Xapian, with the objective >>>>>> to replace fts_squat that is being deprecated. >>>>>> >>>>>> Configuration is exactly the same as for >>>>>> fts_squat: >>>>>> >>>>>> plugin { >>>>>> >>>>>> plugin = fts fts_xapian (...) >>>>>> fts = xapian >>>>>> fts_autoindex = yes >>>>>> fts_enforced = yes >>>>>> fts_xapian = partial=2 full=20 >>>>>> >>>>>> This is installed on my production server >>>>>> (>120Gb of mailboxes), and I >>>>>> will observe it during the coming days. >>>>>> >>>>>> I will definitely appreciate that this is added >>>>>> in the core git of >>>>>> docevot, in order to have a versionning of >>>>>> it, to remove squat and let >>>>>> basic users able to avoid Solr alternative as >>>>>> much as possible. >>>>>> >>>>>> Thanks >>>>>> >>>>>> JM >>>>>> >>>>>> Hi! >>>>>> >>>>>> I still recommend you setup a, say, github >>>>>> repository for your plugin. We are not able to >>>>>> currently include your work in dovecot core as it >>>>>> is more work than just pushing the code into the >>>>>> repo. Maybe it can be included in the future. >>>>>> >>>>>> If you want, I can help you in setting up the >>>>>> required configuration scripts and such to make >>>>>> it possible to compile it as plugin. >>>>>> >>>>>> Then anyone can download it and install it for >>>>>> their dovecot, even if dovecot itself has been >>>>>> installed from packages, and also makes it >>>>>> possible for package maintainers to consider >>>>>> including it in distributions. >>>>>> >>>>>> Aki >>>>>> >>>> >>>> >> >> >> Paul Hecker >> p...@iwascoding.com <mailto:p...@iwascoding.com> >> http://www.iwascoding.com <http://www.iwascoding.com/> >> >> iwascoding GmbH >> Weserstr 18, 10247 Berlin >> HRB 97269 B Amtsgericht Berlin-Charlottenburg >> Geschäftsführer: Paul Hecker, Ilja Iwas >> >> >> >> >> >> >> >> Paul Hecker p...@iwascoding.com <mailto:p...@iwascoding.com> http://www.iwascoding.com <http://www.iwascoding.com/> iwascoding GmbH Weserstr 18, 10247 Berlin HRB 97269 B Amtsgericht Berlin-Charlottenburg Geschäftsführer: Paul Hecker, Ilja Iwas
smime.p7s
Description: S/MIME cryptographic signature
