Originally posted and debugged here:
https://github.com/slusarz/dovecot-fts-flatcurve/issues/66
I have to following panic with fts flatcurve enabled:
```
Panic: file index-search-result.c: line 132 (index_search_result_update_flags):
assertion failed: (result->search_args->args == &search_arg)
Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x74e9890a2846] ->
/usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x74e9890a2962] ->
/usr/lib/dovecot/libdovecot.so.0(+0x10d81b) [0x74e9890af81b] ->
/usr/lib/dovecot/libdovecot.so.0(+0x10d8b7) [0x74e9890af8b7] ->
/usr/lib/dovecot/libdovecot.so.0(+0x5e2e2) [0x74e9890002e2] ->
/usr/lib/dovecot/libdovecot-storage.so.0(+0x518a4) [0x74e9891d38a4] ->
/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x274e)
[0x74e988bf8a9e] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x5c)
[0x74e9891eff9c] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x39) [0x74e9891f0039] ->
dovecot/imap(cmd_select_full+0x1df) [0x62354bd8326f] ->
dovecot/imap(command_exec+0xa4) [0x62354bd8b444] ->
dovecot/imap(+0x22392) [0x62354bd89392] ->
dovecot/imap(+0x22444) [0x62354bd89444] ->
dovecot/imap(client_handle_input+0x1bd) [0x62354bd8989d] ->
dovecot/imap(client_input+0x74) [0x62354bd89e64] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x74e9890c603d] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13a)
[0x74e9890c77aa] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x74e9890c60e4] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x74e9890c62a0] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x74e989036ad7] ->
dovecot/imap(main+0x570) [0x62354bd7ad20] ->
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x74e988c29d90] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x74e988c29e40] ->
dovecot/imap(_start+0x25) [0x62354bd7ade5]
```
It happens when accessing virtual folder with filters like this:
```
something-*/excep-dev*
header x-abc-exception "1"
```
The error happens on subsequent access of the folder:
- Removing dovecot.index{,.log} files allows to access the virtual folder again
- dovecot.index gets recreated on access and the folder can be accessed
multiple times
- Marking an e-mail read triggers the error, though! Now each next access
raises the error
- Removing dovecot.index{,.log} restores access again and the e-mails are still
marked as read (so this information is not lost), until the next e-mail is
marked as read: then the folder cannot be accessed again
Slusarz debugged the backtrace and wrote a detailed description at
https://github.com/slusarz/dovecot-fts-flatcurve/issues/66#issuecomment-2294396247
which I'll quote here:
> This is not a flatcurve bug - it is a bug within Dovecot's indexing/fts
> handling.
>
> (Detailed debug below, so that core team can understand the problem.)
>
> As discussed above, this bug is triggered with virtual mailboxes on
> subsequent accesses. The initial access will work fine; the assert crash only
> happens when the Dovecot lib-storage index code reads the indexes created for
> the virtual mailboxes.
>
> `index_search_result_update_appends()` in
> lib-storage/index/index-search-result.c
>
> * Inside this function, a "temporary search parameter to limit the
> search only to the new messages" is added to the mail search args. Later, it
> is checked that the mail_search args points to this same search parameter
> after the search is executed. It is this assert check that is failing,
> because the mail args is changing during the course of the search.
>
> * `i_assert(result->search_args->args == &search_arg);`
>
>
> The search args are being altered in plugins/fts/fts-search.args.c,
> `fts_search_args_expand()`. Specifically, in that function, either
> `fts_search_args_expand_language_top_level()` or
> `fts_search_args_expand_tree()` is being called, which can replace the
> current args parameter with a new pointer.
>
> `fts_search_args_expand()` is being called in plugins//fts/fts-search.c by
> `fts_search_try_lookup()` when the FTS backend has the
> FTS_BACKEND_FLAG_TOKENIZED_INPUT flag set. Flatcurve has this flag set.
>
> This flag is defined in plugins/fts/fts-api-private.h. Comments say:
>
> ```
> /* Tokenize all the input. update_build_more() will be called a single
> directly indexable token at a time. Searching will modify the
> search
> args so that lookup() sees only tokens that can be directly
> searched. */
> ```
>
> Thus, the comments explicitly say the search args will be modified if this
> flag is set. This modification is incompatible with the assert check in
> `index_search_result_update_appends()`, which requires that the search args
> not be modified.
>
> The core team will have to determine whether this behavior is correct, or if
> the assert check is invalid.
You can find some more discussion at
https://github.com/slusarz/dovecot-fts-flatcurve/issues/66, but I think the
most important part (the very detailed analysis from Slusarz, thanks again!)
might be enough to understand and maybe fix the problem.
Relevant information about the mail system:
```
# 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.21 (f6cd4b8e)
# OS: Linux 6.5.0-45-generic x86_64 Ubuntu 22.04.4 LTS
[...]
namespace {
location = virtual:~/virtual:LAYOUT=maildir++
prefix = _/
separator = /
}
[...]
plugin {
acl = vfile
acl_shared_dict = file:/var/lib/dovecot/db/shared
fts = flatcurve
fts_autoindex = yes
fts_filters = normalizer-icu snowball stopwords
fts_filters_en = lowercase snowball stopwords
fts_languages = de en
fts_tokenizer_email_address = maxlen=100
fts_tokenizer_generic = algorithm=simple maxlen=30
fts_tokenizers = generic email-address
sieve = file:~/sieve;active=~/.dovecot.sieve
}
[...]
```
I can provide more information if you need. Thanks for looking into it!
_______________________________________________
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
