This is not a bug in virtual plugin, but in some email which contains invalid unicode sequence somehow. Can you send me a core file? This should not have occured ofc but would be nice to know how it ended up here.
Aki > On January 3, 2018 at 5:35 PM Jakobus Schürz <wertsto...@nurfuerspam.de> > wrote: > > > Hi there! > > I compiled dovecot 2.3 from git. Because there is already a bug in > virtual-plugin, and i hoped, it get fixed... but it doesn't. So this > is the error-message from the log > > Jän 03 16:27:08 aldebaran dovecot[26460]: > indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>: > Panic: file unichar.c: line 160 (uni_ucs4_to_utf8_c): assertion > failed: (uni_is_valid_ucs4(chr)) > Jän 03 16:27:08 aldebaran dovecot[26460]: > indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>: > Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021) > [0x7f8299f7a021] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed) > [0x7f8299f7a0ed] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f8299eec481] -> > /usr/local/lib/dovecot/libdovecot.so.0(uni_ucs4_to_utf8_c+0xa0) > [0x7f8299fb1500] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xa75e0) > [0x7f8299f5b5e0] -> > /usr/local/lib/dovecot/libdovecot.so.0(mail_html2text_more+0xc5) > [0x7f8299f5b775] -> > /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xcfcc) [0x7f82990aefcc] > -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_parser_more+0x27) > [0x7f82990aeca7] -> > /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x5e9) > [0x7f82990acc39] -> > /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x1122d) [0x7f82990b322d] > -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a) > [0x7f82958e316a] -> > /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d] > -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a) > [0x7f82958e316a] -> > /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d] > -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) > [0x7f829a2641be] -> dovecot/indexer-worker [jakob > Synoptic/AKTUELL](+0x2533) [0x562227882533] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) > [0x7f8299f91bf9] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) > [0x7f8299f93499] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52) > [0x7f8299f91d02] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f8299f91f18] -> > /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f8299f0f1e3] -> dovecot/indexer-worker [jakob > Synoptic/AKTUELL](main+0xe7) [0x562227881f47] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) > [0x7f8299b352b1] -> dovecot/indexer-worker [jakob > Synoptic/AKTUELL](_start+0x2a) [0x56222788201a] > Jän 03 16:27:08 aldebaran dovecot[26460]: indexer: Error: Indexer > worker disconnected, discarding 1 requests for jakob > Jän 03 16:27:08 aldebaran dovecot[26460]: > imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: indexer failed to index > mailbox Synoptic/AKTUELL > Jän 03 16:27:08 aldebaran dovecot[26460]: > indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>: > Fatal: master: service(indexer-worker): child 26476 killed with signal > 6 (core dumps disabled) > Jän 03 16:27:09 aldebaran dovecot[26460]: > indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>: > Error: lucene index > /var/lib/dovecot/db/indexes/Maildir/jakob/lucene-indexes: > IndexWriter() failed (#1): Lock obtain timed out > Jän 03 16:27:10 aldebaran dovecot[26460]: > indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>: > Error: Mailbox Synoptic/AKTUELL: Transaction commit failed: BUG: > Unknown internal error (attempted to index 1488 messages (UIDs > 15214..16775)) > Jän 03 16:27:10 aldebaran dovecot[26460]: > imap(jakob)<26480><lP5r2OBhKJvAqAAO>: Error: indexer failed to index > mailbox Synoptic/AKTUELL > Jän 03 16:27:11 aldebaran dovecot[26460]: > imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Panic: file mail-index.c: line > 793 (mail_index_close): assertion failed: (index->open_count > 0) > Jän 03 16:27:11 aldebaran dovecot[26460]: > imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: Raw backtrace: > /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021) [0x7fb0fbd3a021] -> > /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed) [0x7fb0fbd3a0ed] -> > /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb0fbcac481] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf39a8) > [0x7fb0fc0d99a8] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd92d1) > [0x7fb0fc0bf2d1] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd9363) > [0x7fb0fc0bf363] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_mailbox_free+0x9) > > [0x7fb0fc0b0f39] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd05a4) > [0x7fb0fc0b65a4] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x19) > [0x7fb0fc030c99] -> > /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xabdc) > [0x7fb0f729bbdc] -> > /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9) > [0x7fb0f729bcb9] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) > [0x7fb0fc030c0a] -> > /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xac60) > [0x7fb0f729bc60] -> > /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9) > [0x7fb0f729bcb9] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) > [0x7fb0fc030c0a] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13) > [0x7fb0fc030c93] -> dovecot/imap [jakob 192.168.0.14 > LOGOUT](imap_client_close_mailbox+0x5b) [0x56514d31305b] -> > dovecot/imap [jakob 192.168.0.14 LOGOUT](cmd_logout+0x35) > [0x56514d309485] -> dovecot/imap [jakob 192.168.0.14 > LOGOUT](command_exec+0x64) [0x56514d312ca4] -> dovecot/imap [jakob > 192.168.0.14 LOGOUT](+0x1b0b2) [0x56514d3110b2] -> dovecot/imap [jakob > 192.168.0.14 LOGOUT](+0x1b13c) [0x56514d31113c] -> dovecot/imap [jakob > 192.168.0.14 LOGOUT](client_handle_input+0x1a5) [0x56514d311565] -> > dovecot/imap [jakob 192.168.0.14 LOGOUT](client_input+0x82) > [0x56514d311be2] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) > [0x7fb0fbd51bf9] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) > [0x7fb0fbd53499] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52) > [0x7fb0fbd51d02] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7fb0fbd51f18] -> > /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7fb0fbccf1e3] > Jän 03 16:27:11 aldebaran dovecot[26460]: > imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Fatal: master: service(imap): > child 26472 killed with signal 6 (core dumps disabled) > > > I get this errors when i open or search in a virtual folder. i have > enabled fts_lucene. > > My config is: > > # dovecot -n > # 2.3.devel (393bfbea8): /usr/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.0.rc1 (d68c23a1) > # OS: Linux 4.13.0-trunk-amd64 x86_64 Debian 9.3 btrfs > auth_mechanisms = plain login cram-md5 digest-md5 > auth_socket_path = /usr/local/var/run/dovecot/auth-userdb > first_valid_uid = 1000 > imap_capability = +XDOVECOT > imap_client_workarounds = tb-extra-mailbox-sep > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > listen = *, ::1 > login_trusted_networks = 127.0.0.1/8 192.168.0.0/24 192.168.1.0/24 > 172.17.0.0/24 172.18.0.0/24 > mail_gid = vmail > mail_home = /var/mail/%u > mail_location = > maildir:/var/mail/%u/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:INDEXPVT=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u > mail_plugins = zlib quota acl listescape mail_log notify virtual fts > fts_lucene > mail_privileged_group = vmail > mail_server_admin = mailto:ja...@xundeenergie.at > mail_shared_explicit_inbox = yes > mail_uid = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate mime foreverypart > extracttext > namespace { > list = children > location = > maildir:/var/mail/public/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/public/%u:INDEXPVT=/var/lib/dovecot/db/indexes/public/%u > prefix = Roseggergasse/ > separator = / > subscriptions = no > type = public > } > namespace Geteilt { > hidden = no > inbox = no > list = children > location = > maildir:/var/mail/%%u/Maildir:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/shared/%u/%%u:INDEX=/var/lib/dovecot/db/indexes/shared/%u/%%u > prefix = Geteilt/%%n/ > separator = / > subscriptions = no > type = shared > } > namespace Mailarchiv { > hidden = no > inbox = no > list = children > location = > maildir:/var/mail/mailarchiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/mailarchiv/%u:INDEXPVT=/var/lib/dovecot/db/indexes/mailarchiv/%u > mailbox incoming { > auto = create > } > mailbox outgoing { > auto = create > } > prefix = Mailarchiv/ > separator = / > subscriptions = no > type = private > } > namespace Real { > hidden = yes > list = no > location = virtual:/var/mail/real:INDEX=/var/lib/dovecot/db/indexes/real/%u > prefix = Real/ > separator = / > subscriptions = no > } > namespace Synoptic { > hidden = no > list = children > location = > virtual:/var/mail/virtual:INDEX=/var/lib/dovecot/db/indexes/virtual/%u > mailbox INBOX/Archives { > auto = no > special_use = \Archive > } > mailbox INBOX/Drafts { > auto = no > special_use = \Drafts > } > mailbox INBOX/Entwürfe { > auto = no > special_use = \Drafts > } > mailbox INBOX/Junk { > auto = no > special_use = \Junk > } > mailbox INBOX/Sent { > auto = no > special_use = \Sent > } > mailbox INBOX/Spam { > auto = no > special_use = \Junk > } > prefix = Synoptic/ > separator = / > subscriptions = no > } > namespace inbox { > hidden = no > inbox = yes > location = > mailbox Archiv { > auto = no > special_use = \Archive > } > mailbox Archive { > auto = no > special_use = \Archive > } > mailbox Archives { > auto = no > special_use = \Archive > } > mailbox "Deleted Messages" { > auto = no > autoexpunge = 30 days > special_use = \Trash > } > mailbox Drafts { > auto = no > special_use = \Drafts > } > mailbox Entwürfe { > auto = no > special_use = \Drafts > } > mailbox "Gelöschte Elemente" { > auto = no > autoexpunge = 30 days > special_use = \Trash > } > mailbox "Gelöschte Objekte" { > auto = no > autoexpunge = 30 days > special_use = \Trash > } > mailbox Gesendet { > auto = no > special_use = \Sent > } > mailbox "Gesendete Elemente" { > auto = no > special_use = \Sent > } > mailbox "Gesendete Objekte" { > auto = no > special_use = \Sent > } > mailbox Important { > auto = no > } > mailbox Junk { > auto = subscribe > autoexpunge = 30 days > special_use = \Junk > } > mailbox Mistkübel { > auto = no > autoexpunge = 30 days > special_use = \Trash > } > mailbox Papierkorb { > auto = no > autoexpunge = 30 days > special_use = \Trash > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > auto = no > special_use = \Sent > } > mailbox Spam { > auto = no > autoexpunge = 30 days > special_use = \Junk > } > mailbox Synoptic/Alle { > auto = no > comment = All my messages > special_use = \All > } > mailbox Trash { > auto = no > autoexpunge = 30 days > special_use = \Trash > } > mailbox Wichtig { > auto = create > } > prefix = INBOX/ > separator = / > subscriptions = no > type = private > } > namespace subscriptions { > hidden = yes > list = no > location = > prefix = > subscriptions = yes > } > passdb { > args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users > driver = passwd-file > } > plugin { > acl = vfile:/etc/dovecot/dovecot-acl:cache_secs=300 > acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db > fts = lucene > fts_autoindex = yes > fts_autoindex_max_recent_msgs = 80 > fts_index_timeout = 90s > fts_lucene = whitespace_chars=@. normalize no_snowball > mail_home = /var/mail/%u > setting_name = sieve, managedsieve > sieve = file:/var/mail/%u/sieve/;active=/var/mail/%u/sieve/%u.sieve > } > postmaster_address = postmaster@localhost > protocols = imap pop3 lmtp imap lmtp sieve pop3 sieve > service anvil { > unix_listener anvil-auth-penalty { > mode = 00 > } > } > service auth { > unix_listener auth-client { > group = vmail > mode = 0666 > user = Debian-exim > } > unix_listener auth-userdb { > group = vmail > mode = 0666 > user = vmail > } > } > service imap-login { > inet_listener imap { > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 1 > service_count = 0 > } > service imap { > executable = imap postlogin > process_limit = 1024 > vsz_limit = 400 M > } > service lmtp { > executable = lmtp -L > user = vmail > vsz_limit = 400 M > } > service postlogin { > executable = script-login -d rawlog > } > ssl = required > ssl_cert = </usr/local/etc/dovecot/dovecot.crt > ssl_dh = # hidden, use -P to show it > ssl_key = # hidden, use -P to show it > userdb { > args = username_format=%u /usr/local/etc/dovecot/users > default_fields = home=/var/mail/%u > driver = passwd-file > } > verbose_proctitle = yes > protocol lmtp { > auth_username_format = %n > mail_plugins = zlib quota acl listescape mail_log notify virtual > fts fts_lucene quota sieve acl > } > protocol lda { > mail_plugins = zlib quota acl listescape mail_log notify virtual > fts fts_lucene quota sieve acl > } > protocol imap { > mail_max_userip_connections = 10 > mail_plugins = zlib quota acl listescape mail_log notify virtual > fts fts_lucene imap_quota imap_acl > } > > > The virtual Folder-definitions are: > > # find /var/mail/virtual/ /var/mail/real/ > /var/mail/virtual/ > /var/mail/virtual/GINBOX > /var/mail/virtual/GINBOX/dovecot-virtual > /var/mail/virtual/Neue > /var/mail/virtual/Neue/dovecot-virtual > /var/mail/virtual/Markierte > /var/mail/virtual/Markierte/dovecot-virtual > /var/mail/virtual/Ungelesene > /var/mail/virtual/Ungelesene/dovecot-virtual > /var/mail/virtual/Sent > /var/mail/virtual/Sent/dovecot-virtual > /var/mail/virtual/Drafts > /var/mail/virtual/Drafts/dovecot-virtual > /var/mail/virtual/Trash > /var/mail/virtual/Trash/dovecot-virtual > /var/mail/virtual/AKTUELL > /var/mail/virtual/AKTUELL/dovecot-virtual > /var/mail/virtual/Archives > /var/mail/virtual/Archives/dovecot-virtual > /var/mail/virtual/dovecot-acl-list > /var/mail/virtual/WICHTIG > /var/mail/virtual/WICHTIG/dovecot-virtual > /var/mail/real/ > /var/mail/real/Alle > /var/mail/real/Alle/dovecot-virtual > /var/mail/real/dovecot-acl-list > /var/mail/real/einjahr > /var/mail/real/einjahr/dovecot-virtual > /var/mail/real/zweijahr > /var/mail/real/zweijahr/dovecot-virtual > /var/mail/real/dreijahr > /var/mail/real/dreijahr/dovecot-virtual > > # cat /var/mail/virtual/AKTUELL/dovecot-virtual > Real/Alle > inthread refs younger 2592000 > > # cat /var/mail/real/Alle/dovecot-virtual > !INBOX > INBOX/* > -INBOX/Trash > -INBOX/Trash/* > -INBOX/Archives > -INBOX/Archives/* > -INBOX/Drafts > -INBOX/Drafts/* > -INBOX/Junk > -INBOX/Junk/* > -INBOX/SYSTEM > -INBOX/SYSTEM/* > all > > The others are similar. > > Greez > > Jakob
