Re: Dovecot v2.3.13 released
Hello Aki, fts-solr is still crashing here. We have many X- headers from antispam, DKIM, and so on, I don't know if it has anything to do with it. The same configuration worked a couple of versions ago. Regards, Diego. Latest debian 10.7, binaries from repo.dovecot.org/ce-2.3-latest/debian/buster # dovecot --version 2.3.13 (89f716dc2) # for i in diego.liziero; do doveadm index -u $i \*; echo indexed $i; done doveadm(diego.liziero): Panic: file http-client-request.c: line 1240 (http_client_request_send_more): assertion failed: (req->payload_input != NULL) doveadm(diego.liziero): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7f9108b8561d] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f9108b8573e] -> /usr/lib/dovecot/libdovecot.so.0(+0xfa79b) [0x7f9108b9179b] -> /usr/lib/dovecot/libdovecot.so.0(+0xfa7d1) [0x7f9108b917d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x52e30) [0x7f9108ae9e30] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a868) [0x7f9108ae1868] -> /usr/lib/dovecot/libdovecot.so.0(http_client_connection_output+0xf2) [0x7f9108b36cc2] -> /usr/lib/dovecot/libdovecot.so.0(+0x120481) [0x7f9108bb7481] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f9108ba7599] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7f9108ba8b11] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f9108ba763c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f9108ba77b0] -> /usr/lib/dovecot/libdovecot.so.0(+0x9b610) [0x7f9108b32610] -> /usr/lib/dovecot/libdovecot.so.0(http_client_request_send_payload+0x30) [0x7f9108b326e0] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf15d) [0x7f910831415d] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_parser_more+0x27) [0x7f9108312f87] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc25f) [0x7f910831125f] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x4d) [0x7f910831198d] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x12060) [0x7f9108317060] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) [0x7f9108ca5a0e] -> doveadm(+0x368ff) [0x56138f8c98ff] -> doveadm(+0x30ee6) [0x56138f8c3ee6] -> doveadm(+0x31ada) [0x56138f8c4ada] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21a) [0x56138f8c587a] -> doveadm(doveadm_cmd_run_ver2+0x4df) [0x56138f8d5d2f] -> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x56138f8d5d87] -> doveadm(main+0x1ca) [0x56138f8b4e9a] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f910871e09b] -> doveadm(_start+0x2a) [0x56138f8b537a] Aborted On Mon, Jan 4, 2021 at 1:04 PM Aki Tuomi wrote: > [..] > - fts-solr: HTTP requests may have assert-crashed: > Panic: file http-client-request.c: line 1232 > (http_client_request_send_more): > assertion failed: (req->payload_input != NULL) > >
Re: SOLR 5
On Sat, Feb 28, 2015 at 8:17 PM, Robert Gierzinger wrote: > Hello, > > I just wanted to give SOLR 5 a try, however there probably have changed > quite some bits in the config files, did not even manage to create a core > with various solrconfig.xml and schema.xml files, but I am absolutely no > expert in solr. > Has anybody given it a try or are there some tips on how to get it running? > > regards, > Robert > Using solr-5.0.0 with a single core here, the last one I managed to get working with dovecot 1.1.18. Starting from 5.1.0 solr changed behaviour and complains with "Bad contentType for search handler :text/xml" (I've tried to change dovecot header requests to "application/x-www-form-urlencoded" but I got a "missing content stream" error). Here some further steps I did to get it working, not sure if it's the correct way of doing it, though (but I'm sure that someone will correct me where I'm wrong). I've created a new folder server/solr/dovecot/ with a file core.properties and two subdirs: data and conf. My core.properties contains these four lines: name=dovecot config=solrconfig.xml schema=schema.xml dataDir=data in the server/solr/dovecot/conf dir I initially copied the content of one of the sample dirs (server/solr/configsets/basic_configs/conf) and then I copied as schema.xml the solr-schema.xml from dovecot install. This changes the main url path of solr, so I had to point dovecot config to http://localhost:8983/solr/dovecot/ and the crontab became: # solr 0 0 * * *curl http://localhost:8983/solr/dovecot/update?optimize=true */10 * * * *curl http://localhost:8983/solr/dovecot/update?commit=true &>/dev/null If someone knows how to keep the usual main "http://localhost:8983/solr/"; url with solr-5, please let me know how to do it. In bin/init.d/solr there is an init.d script with some comments to get it correctly configured and some pointer to the other config steps (such as create a solr user, edit conf/solr.in.sh for other environment variables, and so on). Regards, Diego.
[Dovecot] [patch] make /./ a config option.
Hi,
I think that the wu-ftp style chroot /./ should be a configurable
option.
In our servers we have some home directories in /chroot-web/./username
(where web users can upload their web sites in a chrooted environment)
and all imap mail in /mail-disk/username.
We are planning a dovecot migration from our modified version of uw-imap
and we noticed that the chroot in /chroot-web/ can't be disabled.
This patch adds the bool option home_slash_dot_slash_chroot (feel free
to change this name to something easier to understand). Setting this to
"no" disables the wu-ftp style /./ chroot.
I hope this feature can be considered useful and soon included in
dovecot.
Regards,
Diego Liziero.
diff -dur dovecot-1.0.10/dovecot-example.conf dovecot-1.0.10-disable-slash-dot-slash-chroot/dovecot-example.conf
--- dovecot-1.0.10/dovecot-example.conf 2007-12-11 19:52:08.0 +0100
+++ dovecot-1.0.10-disable-slash-dot-slash-chroot/dovecot-example.conf 2008-02-15 10:44:39.0 +0100
@@ -354,6 +354,14 @@
# their mail directory anyway.
#mail_chroot =
+# Enable checking /./ in user's home directory for chrooting.
+# With this enabled (default), when user's home contains /./ (eg.
+# /newroot/./newhome/user) two things are changed:
+# - mail_chroot is overridden and set to the path before /./
+# - %h (home) is set to the path after /.
+#
+#home_slash_dot_slash_chroot = yes
+
##
## Mailbox handling optimizations
##
diff -dur dovecot-1.0.10/src/master/mail-process.c dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/mail-process.c
--- dovecot-1.0.10/src/master/mail-process.c 2007-12-20 21:51:23.0 +0100
+++ dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/mail-process.c 2008-02-15 09:42:53.0 +0100
@@ -477,7 +477,7 @@
}
}
- if (*chroot_dir == '\0' && (p = strstr(home_dir, "/./")) != NULL) {
+ if (set->home_slash_dot_slash_chroot && *chroot_dir == '\0' && (p = strstr(home_dir, "/./")) != NULL) {
/* wu-ftpd like /./ */
chroot_dir = t_strdup_until(home_dir, p);
home_dir = p + 2;
diff -dur dovecot-1.0.10/src/master/master-settings-defs.c dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/master-settings-defs.c
--- dovecot-1.0.10/src/master/master-settings-defs.c 2007-12-11 19:52:09.0 +0100
+++ dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/master-settings-defs.c 2008-02-15 09:13:30.0 +0100
@@ -50,6 +50,7 @@
/* mail */
DEF(SET_STR, valid_chroot_dirs),
DEF(SET_STR, mail_chroot),
+ DEF(SET_BOOL, home_slash_dot_slash_chroot),
DEF(SET_INT, max_mail_processes),
DEF(SET_BOOL, verbose_proctitle),
diff -dur dovecot-1.0.10/src/master/master-settings.c dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/master-settings.c
--- dovecot-1.0.10/src/master/master-settings.c 2007-12-21 16:10:24.0 +0100
+++ dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/master-settings.c 2008-02-15 09:12:21.0 +0100
@@ -199,6 +199,7 @@
/* mail */
MEMBER(valid_chroot_dirs) "",
MEMBER(mail_chroot) "",
+ MEMBER(home_slash_dot_slash_chroot) TRUE,
MEMBER(max_mail_processes) 1024,
MEMBER(verbose_proctitle) FALSE,
diff -dur dovecot-1.0.10/src/master/master-settings.h dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/master-settings.h
--- dovecot-1.0.10/src/master/master-settings.h 2007-12-11 19:52:09.0 +0100
+++ dovecot-1.0.10-disable-slash-dot-slash-chroot/src/master/master-settings.h 2008-02-15 09:12:36.0 +0100
@@ -60,6 +60,7 @@
/* mail */
const char *valid_chroot_dirs;
const char *mail_chroot;
+ bool home_slash_dot_slash_chroot;
unsigned int max_mail_processes;
bool verbose_proctitle;
Re: [Dovecot] [patch] make /./ a config option.
On Fri, 2008-02-15 at 14:53 +0200, Timo Sirainen wrote:
> On Fri, 2008-02-15 at 13:40 +0100, Diego Liziero wrote:
> > This patch adds the bool option home_slash_dot_slash_chroot (feel free
> > to change this name to something easier to understand). Setting this to
> > "no" disables the wu-ftp style /./ chroot.
>
> There are already too many options, but I guess valid_chroot_dirs could
> be used for this. Committed to v1.1:
> http://hg.dovecot.org/dovecot-1.1/rev/17c65dfdac2a
Great, but this patch solves partially what we would like to have:
it allows chroot options to be completely disabled, but it doesn't allow
to override /./ chroot with a global mail_chroot option.
This happens because to have mail_chroot config option working, we have
to add its entry in valid_chroot_dirs, too.
This should not be necessary.
In this case validate_chroot should be called before checking for
mail_chroot (see the patch below).
Thank you for your quick answer,
Regards,
Diego Liziero.
diff -dur dovecot-1.0.10/src/master/mail-process.c dovecot-1.0.10-chroot/src/master/mail-process.c
--- dovecot-1.0.10/src/master/mail-process.c 2007-12-20 21:51:23.0 +0100
+++ dovecot-1.0.10-chroot/src/master/mail-process.c 2008-02-16 13:26:16.0 +0100
@@ -492,9 +492,6 @@
return FALSE;
}
- if (*chroot_dir == '\0' && *set->mail_chroot != '\0')
- chroot_dir = set->mail_chroot;
-
if (*chroot_dir != '\0') {
if (!validate_chroot(set, chroot_dir)) {
i_error("Invalid chroot directory '%s' (user %s) "
@@ -502,6 +499,12 @@
chroot_dir, user);
return FALSE;
}
+ }
+
+ if (*chroot_dir == '\0' && *set->mail_chroot != '\0')
+ chroot_dir = set->mail_chroot;
+
+if (*chroot_dir != '\0') {
if (set->mail_drop_priv_before_exec) {
i_error("Can't chroot to directory '%s' (user %s) "
"with mail_drop_priv_before_exec=yes",
Only in dovecot-1.0.10-chroot/src/master: mail-process.c.orig
[Dovecot] suspect valgrind error in mail-index-map.c
Hi
At line 1118 of src//lib-index/mail-index-map.c, inside the function
mail_index_map_move_to_memory, there is:
mail_index_map_copy_header(map, map);
Valgrind is stating that "Source and destination overlap in memcpy".
I'm wondering if this code is just coping the same memory over itself,
or if it is doing something useful.
Regards,
Diego Liziero.
---
1104 void mail_index_map_move_to_memory(struct mail_index_map *map)
1105 {
1106struct mail_index_record_map *new_map;
1107
1108if (map->rec_map->mmap_base == NULL)
1109return;
1110
i_assert(map->rec_map->lock_id != 0);
1112
1113new_map = array_count(&map->rec_map->maps) == 1 ? map->rec_map :
1114mail_index_record_map_alloc(map);
1115
1116mail_index_map_copy_records(new_map, map->rec_map,
1117map->hdr.record_size);
1118mail_index_map_copy_header(map, map);
1119
1120if (new_map != map->rec_map) {
1121mail_index_record_map_unlink(map);
1122map->rec_map = new_map;
1123} else {
1124mail_index_unlock(map->index, &new_map->lock_id);
1125if (munmap(new_map->mmap_base, new_map->mmap_size) < 0)
1126mail_index_set_syscall_error(map->index, "munmap()");
1127new_map->mmap_base = NULL;
1128}
1129 }
Re: [Dovecot] suspect valgrind error in mail-index-map.c
On Sun, Mar 9, 2008 at 1:12 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > It's copying over itself, so it shouldn't break anything. But I fixed > the error anyway: http://hg.dovecot.org/dovecot-1.1/rev/8e014fd46e84 Thanks Timo. With this patch valgrind gives 0 errors most of the time. Forgive me if I'm asking this last valgrind trace. Can we simply ignore it? Diego. ==3921== 180 (124 direct, 56 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 5 ==3921==at 0x40046FF: calloc (vg_replace_malloc.c:279) ==3921==by 0x80D76DC: pool_system_malloc (mempool-system.c:74) ==3921==by 0x80B59CB: mail_transaction_log_file_alloc (mail-transaction-log-file.c:51) ==3921==by 0x80B3A86: mail_transaction_log_find_file (mail-transaction-log.c:385) ==3921==by 0x80B776F: mail_transaction_log_view_set (mail-transaction-log-view.c:160) ==3921==by 0x80B00B2: mail_index_sync_map (mail-index-sync-update.c:747) ==3921==by 0x80A9368: mail_index_map (mail-index-map.c:897) ==3921==by 0x80A63BB: mail_index_try_open (mail-index.c:290) ==3921==by 0x80A673C: mail_index_open (mail-index.c:352) ==3921==by 0x809AB78: index_storage_mailbox_open (index-storage.c:383) ==3921==by 0x807809B: mbox_alloc_mailbox (mbox-storage.c:573) ==3921==by 0x8078E2F: mbox_open (mbox-storage.c:591) ==3921==by 0x807900E: mbox_mailbox_open (mbox-storage.c:668) ==3921==by 0x809EB98: mailbox_open (mail-storage.c:459) ==3921==by 0x805E0E8: cmd_select_full (cmd-select.c:32) ==3921==by 0x805B5A8: cmd_examine (cmd-examine.c:8) ==3921==by 0x805F9A8: client_command_input (client.c:546) ==3921==by 0x805FA3D: client_command_input (client.c:595) ==3921==by 0x80601C4: client_handle_input (client.c:636) ==3921==by 0x80603DD: client_input (client.c:691) ==3921==by 0x80D5B9F: io_loop_handler_run (ioloop-epoll.c:201) ==3921==by 0x80D4E27: io_loop_run (ioloop.c:301) ==3921==by 0x8067DBB: main (main.c:293)
Re: [Dovecot] suspect valgrind error in mail-index-map.c
On Sun, Mar 9, 2008 at 2:07 AM, Diego Liziero <[EMAIL PROTECTED]> wrote:
> [..]
>
> 180 (124 direct, 56 indirect) bytes in 1 blocks are definitely lost in loss
> record 3 of 5
> [..]
> by 0x80B59CB: mail_transaction_log_file_alloc
> (mail-transaction-log-file.c:51)
> by 0x80B3A86: mail_transaction_log_find_file (mail-transaction-log.c:385)
I think that it's this last line causing the error.
mail-transaction-log.c
343 int mail_transaction_log_find_file(struct mail_transaction_log *log,
344uint32_t file_seq, bool nfs_flush,
345struct mail_transaction_log_file **file_r)
346 {
347 struct mail_transaction_log_file *file;
348 const char *path;
349 int ret;
[..]
382 /* see if we have it in log.2 file */
383 path = t_strconcat(log->index->filepath,
384MAIL_TRANSACTION_LOG_SUFFIX".2", NULL);
385 file = mail_transaction_log_file_alloc(log, path);
Here a new mail_transaction_log_file is allocated before getting lost.
Maybe I'm wrong, but, isn't here a path where
mail_transaction_log_file_free(&file); should be called before
returning without losing the memory pointed by file?
386 if ((ret = mail_transaction_log_file_open(file, TRUE)) <= 0)
387 return ret;
388
389 /* but is it what we expected? */
390 if (file->hdr.file_seq != file_seq)
391 return 0;
392
393 *file_r = file;
394 return 1;
395 }
Regards,
Diego.
[Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
To some users happens this assertion failure while deleting a message.
dovecot: Mar 10 08:40:44 Panic: IMAP(user): file index-sync.c: line 39
(index_mailbox_set_recent_uid): assertion failed: (seq_range_exists
(&ibox->recent_flags, uid))
dovecot: Mar 10 08:40:44 Error: IMAP(user): Raw backtrace: [see bleow]
dovecot: Mar 10 08:40:44 Error: child 17683 (imap) killed with signal 6
And the message doesn't get deleted.
Here one of the backtraces
(gdb) bt
#0 0x00ae9402 in __kernel_vsyscall ()
#1 0x00725ba0 in raise () from /lib/libc.so.6
#2 0x007274b1 in abort () from /lib/libc.so.6
#3 0x080ce02d in i_internal_fatal_handler (type=LOG_TYPE_PANIC,
status=0, fmt=0x80e1750 "file %s: line %d (%s): assertion failed:
(%s)",
args=0xbfd8c6a4 "=???\016\b'") at failures.c:424
#4 0x080cdc3c in i_panic (format=0x80e1750 "file %s: line %d (%s):
assertion failed: (%s)") at failures.c:187
#5 0x0809bb4a in index_mailbox_set_recent_uid (ibox=0x88ca768,
uid=1041) at index-sync.c:39
#6 0x0808085f in mbox_sync_loop (sync_ctx=0xbfd8c8d0,
mail_ctx=0xbfd8cac4, partial=true) at mbox-sync.c:457
#7 0x08081361 in mbox_sync (mbox=0x88ca768, flags=18) at mbox-sync.c:1504
#8 0x08079cf9 in mbox_transaction_commit (t=0x88c5550,
log_file_seq_r=0xbfd8cbc8, log_file_offset_r=0xbfd8cbc0) at
mbox-transaction.c:45
#9 0x0809c69e in index_transaction_commit (_t=0x88cb1c0,
uid_validity_r=0xbfd8cc48, first_saved_uid_r=0xbfd8cc44,
last_saved_uid_r=0xbfd8cc40) at index-transaction.c:105
#10 0x0805b0cf in cmd_copy (cmd=0x88b75c0) at cmd-copy.c:141
#11 0x0805f079 in cmd_uid (cmd=0x88b75c0) at cmd-uid.c:26
#12 0x0805f9a9 in client_command_input (cmd=0x88b75c0) at client.c:546
#13 0x0805fa3e in client_command_input (cmd=0x88b75c0) at client.c:595
#14 0x080601c5 in client_handle_input (client=0x88b7368) at client.c:636
#15 0x080603de in client_input (client=0x88b7368) at client.c:691
#16 0x080d5ba0 in io_loop_handler_run (ioloop=0x88b59b0) at ioloop-epoll.c:201
#17 0x080d4e28 in io_loop_run (ioloop=0x88b59b0) at ioloop.c:301
#18 0x08067dbc in main (argc=Cannot access memory at address 0x413a
) at main.c:293
(gdb) bt full
#0 0x00ae9402 in __kernel_vsyscall ()
No symbol table info available.
#1 0x00725ba0 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0x007274b1 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080ce02d in i_internal_fatal_handler (type=LOG_TYPE_PANIC,
status=0, fmt=0x80e1750 "file %s: line %d (%s): assertion failed:
(%s)",
args=0xbfd8c6a4 "=???\016\b'") at failures.c:424
backtrace = 0x88add50 "/usr/libexec/dovecot/imap [0x80ce024]
-> /usr/libexec/dovecot/imap [0x80cdc3c] -> /usr/libexec/dovecot/imap
[0x809bb4a] -> /usr/libexec/dovecot/imap [0x808085f] ->
/usr/libexec/dovecot/imap(mbox_sync+"...
#4 0x080cdc3c in i_panic (format=0x80e1750 "file %s: line %d (%s):
assertion failed: (%s)") at failures.c:187
args = 0xbfd8c6a4 "=???\016\b'"
#5 0x0809bb4a in index_mailbox_set_recent_uid (ibox=0x88ca768,
uid=1041) at index-sync.c:39
__PRETTY_FUNCTION__ = "index_mailbox_set_recent_uid"
#6 0x0808085f in mbox_sync_loop (sync_ctx=0xbfd8c8d0,
mail_ctx=0xbfd8cac4, partial=true) at mbox-sync.c:457
mail = {uid = 268, idx_seq = 133265, keywords = {arr = {buffer
= 0x88d3770, element_size = 3218655176}, v = 0x88d3770,
v_modifiable = 0x88d3770}, flags = 172 '???', uid_broken = 1,
expunged = 0, pseudo = 1, from_offset = 143457960,
body_size = 579466361970685888, offset = 143456608, space =
578778797692682360}
rec = (const struct mail_index_record *) 0xb7e6f008
uid = 1041
messages_count = 3518
offset = 563
ret =
expunged = false
skipped_mails = false
uids_broken = false
#7 0x08081361 in mbox_sync (mbox=0x88ca768, flags=18) at mbox-sync.c:1504
ret =
#8 0x08079cf9 in mbox_transaction_commit (t=0x88c5550,
log_file_seq_r=0xbfd8cbc8, log_file_offset_r=0xbfd8cbc0) at
mbox-transaction.c:45
mt = (struct mbox_transaction_context *) 0x88cb1c0
mbox = (struct mbox_mailbox *) 0x88ca768
lock_id = 3
ret = 0
#9 0x0809c69e in index_transaction_commit (_t=0x88cb1c0,
uid_validity_r=0xbfd8cc48, first_saved_uid_r=0xbfd8cc44,
last_saved_uid_r=0xbfd8cc40) at index-transaction.c:105
itrans = (struct mail_index_transaction *) 0x0
seq = 7
offset = 20544
#10 0x0805b0cf in cmd_copy (cmd=0x88b75c0) at cmd-copy.c:141
client = (struct client *) 0x88b7368
storage = (struct mail_storage *) 0x88b6c80
destbox = (struct mailbox *) 0x88ca768
t = (struct mailbox_transaction_context *) 0x0
search_arg =
messageset = 0x88bb6c0 "224"
mailbox = 0x88bb6c8 "Trash"
msg =
sync_flags =
imap_flags =
copy_count = 1
uid_validity =
uid1 =
uid2 =
ret = 1
__PRETTY_FUNCTION__ = "cmd_copy"
#11 0x0805f079 in cmd_uid (cmd=0x88b75c0) at cmd-uid.c:26
Re: [Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
On Mon, Mar 10, 2008 at 9:05 AM, Diego Liziero <[EMAIL PROTECTED]> wrote:
> To some users happens this assertion failure while deleting a message.
>
> dovecot: Mar 10 08:40:44 Panic: IMAP(user): file index-sync.c: line 39
> (index_mailbox_set_recent_uid): assertion failed: (seq_range_exists
> (&ibox->recent_flags, uid))
36 void index_mailbox_set_recent_uid(struct index_mailbox *ibox, uint32_t
uid)
37 {
38 if (uid <= ibox->recent_flags_prev_uid) {
39 i_assert(seq_range_exists(&ibox->recent_flags, uid));
40 return;
41 }
42 ibox->recent_flags_prev_uid = uid;
Here, when assert fails:
uid=1041
ibox->recent_flags_prev_uid = 4557
ibox->recent_flags->arr->element_size = 8
**ibox->recent_flags->v = {seq1 = 4557, seq2 = 4557}
(struct seq_range)(ibox->recent_flags->arr->buffer->data) = {seq1 =
143455672, seq2 = 8}
(gdb) print *ibox
$16 = {box = {name = 0x88ca8f8 "Trash", storage = 0x88b6c80, v =
{is_readonly = 0x809a750 ,
allow_new_keywords = 0x809a770
, close = 0x8077d50
,
get_status = 0x809a620 ,
list_index_has_changed = 0, list_index_update_sync = 0,
sync_init = 0x80826e0 , sync_next =
0x809bf00 ,
sync_deinit = 0x809bc30 , sync_notify
= 0, notify_changes = 0x8077d10 ,
transaction_begin = 0x809c6b0 ,
transaction_commit = 0x809c650 ,
transaction_rollback = 0x809c630 ,
keywords_create = 0x809a880 ,
keywords_free = 0x809a790 , get_uids =
0x80934d0 ,
mail_alloc = 0x8094320 , header_lookup_init =
0x8095d40 ,
header_lookup_deinit = 0x8095c90 ,
search_init = 0x8099140 ,
search_deinit = 0x8098b50 ,
search_next_nonblock = 0x8097e40 ,
search_next_update_seq = 0x8097c10
, save_init = 0x807cab0
,
save_continue = 0x807c740 , save_finish =
0x807c300 ,
save_cancel = 0x807c530 , copy = 0x809cbd0
,
is_inconsistent = 0x809aa00 },
pool = 0x88ca750, transaction_count = 0, private_flags_mask = 0,
file_create_mode = 384, dir_create_mode = 448, file_create_gid =
4294967295, notify_min_interval = 0, notify_callback = 0,
notify_context = 0x0, module_contexts = {arr = {buffer =
0x88ca900, element_size = 4}, v = 0x88ca900, v_modifiable =
0x88ca900},
opened = 1, mailbox_deleted = 0}, view_module_ctx = {reg = 0x0},
storage = 0x88b6c80, open_flags = 14, index = 0x88c09a8,
view = 0x88c3988, cache = 0x88caf50, mail_vfuncs = 0x810aa20,
md5hdr_ext_idx = 2, notify_to = 0x0, notify_files = 0x0,
notify_ios = 0x0, notify_last_check = 0, notify_last_sent = 0,
next_lock_notify = 1205134826,
last_notify_type = MAILBOX_LOCK_NOTIFY_NONE, commit_log_file_seq =
7, commit_log_file_offset = 20544, keyword_names = 0x88c0a84,
cache_fields = 0x88bf5d8, mail_cache_min_mail_count = 0,
recent_flags = {arr = {buffer = 0x88d3758, element_size = 8}, v =
0x88d3758,
v_modifiable = 0x88d3758}, recent_flags_prev_uid = 4557,
recent_flags_count = 1, sync_last_check = 0, readonly = 0, keep_recent
= 1,
keep_locked = 0, sent_diskspace_warning = 0,
sent_readonly_flags_warning = 0, notify_pending = 0, move_to_memory =
0, fsync_disable = 0}
--
Diego
[Dovecot] dovecot-1.1.rc3 segmentation fault in fetch_bodystructure
Hi,
another imap crash with latest dovecot.
segmentation fault in fetch_bodystructure
src/imap/imap-fetch.c
static int fetch_bodystructure(struct imap_fetch_context *ctx,
struct mail *mail, void *context ATTR_UNUSED)
{
const char *bodystructure;
if (mail_get_special(mail, MAIL_FETCH_IMAP_BODYSTRUCTURE,
&bodystructure) < 0)
return -1;
---> before the segfault here we have bodystructure=0 and
mail_get_special returns >=0
[..]
if (o_stream_send(ctx->client->output, "BODYSTRUCTURE (", 15) < 0 ||
/*line 461*/ o_stream_send_str(ctx->client->output, bodystructure) < 0 ||
---> here o_stream_send_str calls strlen(bodystructure=0), and strlen
tries to access "Address 0x0" causing a segfault
--
Address 0x0 is not stack'd, malloc'd or (recently) free'd
Process terminating with default action of signal 11 (SIGSEGV): dumping core
Access not within mapped region at address 0x0
at: strlen
by: o_stream_send_str (ostream.c:163)
by: fetch_bodystructure (imap-fetch.c:461)
by: imap_fetch (imap-fetch.c:309)
by: cmd_fetch (cmd-fetch.c:154)
by: client_command_input (client.c:546)
by: client_command_input (client.c:595)
by: client_handle_input (client.c:636)
by: client_input (client.c:691)
by: io_loop_handler_run (ioloop-epoll.c:201)
by: io_loop_run (ioloop.c:301)
by: main (main.c:293)
Re: [Dovecot] dovecot-1.1.rc3 segmentation fault in fetch_bodystructure
On Tue, Mar 11, 2008 at 9:09 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > Well, I'm not sure how you managed to cause this, but this should fix > it: http://hg.dovecot.org/dovecot-1.1/rev/7e27d67d3abe Thank you Timo for the quick fix, here we have latest rc3 in a production environment. It has been used by over 600 users in the last 2 days. The most failing assertion (9694 times in 2 days) is the one I posted yesterday: Panic: IMAP(username): file index-sync.c: line 39 (index_mailbox_set_recent_uid): assertion failed: (seq_range_exists(&ibox->recent_flags, uid)) It happens when users are moving messages to Trash folder with thunderbird. The workaround for the user is to delete directly the messages without moving them to Trash. We had also some trouble with pop3. A couple of users weren't able to get new mail (see log below) until we deleted completely their .imap dir. Diego. --- Error: POP3(username): Cached message offset lost for seq 1 in mbox file /maildir/username Error: POP3(username): Log synchronization error at seq=1,offset=7824 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field alignmentation 8 not used Error: POP3(username): Log synchronization error at seq=1,offset=7856 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field points outside record size (0+16 > 12) Error: POP3(username): Log synchronization error at seq=1,offset=7928 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field alignmentation 8 not used Error: POP3(username): Log synchronization error at seq=1,offset=8004 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field points outside record size (0+16 > 12) Warning: POP3(username): fscking index file /maildir/username/.imap/INBOX/dovecot.index Error: POP3(username): Cached message offset lost for seq 1 in mbox file /cl/e/spool-mail/username Error: POP3(username): Log synchronization error at seq=1,offset=8208 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field alignmentation 8 not used Error: POP3(username): Log synchronization error at seq=1,offset=8240 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field points outside record size (0+16 > 12) Error: POP3(username): Log synchronization error at seq=1,offset=8312 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field alignmentation 8 not used Error: POP3(username): Log synchronization error at seq=1,offset=8388 for /maildir/username/.imap/INBOX/dovecot.index: Broken extension introduction: Record field points outside record size (0+16 > 12) Warning: POP3(username): fscking index file /maildir/username/.imap/INBOX/dovecot.index Error: POP3(username): Sending log messages too fast, throttling.. Error: POP3(username): Couldn't init INBOX: Can't sync mailbox: Messages keep getting expunged
[Dovecot] imap sent-mail folder sometimes dosen't get updated when used by more than a mailer at the same time.
Hi, I'm collecting users feedback of latest dovecot 1.1.rc3 development release. Some users are complaining that their sent mails sometimes don't get written to imap Sent-mail folder. It seems that all these users were using multiple istances of imap processes to read their mail (thunderbird+horde-imp, evolution+horde-imp or multiple thunderbird istances). No error message is written in dovecot.log, no user feedback in horde-imp/evolution. Only thunderbird says it can't update Sent folder. Has anyone had similar issues? Diego.
Re: [Dovecot] imap sent-mail folder sometimes dosen't get updated when used by more than a mailer at the same time.
> On Wed, Mar 12, 2008 at 6:59 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: > > Has anyone had similar issues? Charles Marcus wrote: > Yes... I don't believe this is a dovecot issue, as this is an occasional > issue with Thunderbird I've seen on both dovecot and Courier-imap (in > fact it seems to happen more often on the sites that use Courier)... > [..] The fact is that with wu-imap none complained about missing sent mails, we just switced last week to dovecot (mainly because of really bad performance of wu-imap) and several users started complaining about this issue. Another (maybe related?) strange thing is that local mail (sent by sendmail) sometimes get delayed to the following queue run (that was set to 30 min). Even this fact never happened before the switch to dovecot. I thought it could be caused by a bad locking configuration, but Centos5 sendmail should use fcntl, wu-imap should have used it as well (am I wrong?), and dovecot is explicitly configured with fcntl, too. Perhaps, as it never happened with wu-imap, it could be fixed with dovecot, too. The hard part is to understand why it happens. Regards, Diego.
Re: [Dovecot] imap sent-mail folder sometimes dosen't get updated when used by more than a mailer at the same time.
On Wed, Mar 12, 2008 at 6:59 PM, Diego Liziero wrote: > [..] > Some users are complaining that their sent mails sometimes don't get > written to imap Sent-mail folder. Tim Alberts wrote: > [..] I'm finding that using multiple clients at the same time, changes are not > immediately posted so viewing the same account with thunderbird on one > machine and outlook on another, it appears to get out of sync. Seems > mostly the 'delete' or 'move' commands don't actually happen until the > client program is closed. Mmm.. the fact I'm reporting is different: you noticed that changes don't appear immediately when using multiple clients because of local caching. Here it happens that mail isn't actually written to an imap folder (sent mail). Closing all instances and opening a new one doesn't get the lost mail back. When it happens with thunderbird, the user gets an error stating that the mail can't be saved. With other mail clients nothing is said and the mail is just lost. Regards, Diego.
Re: [Dovecot] [SOLVED] imap sent-mail folder sometimes dosen't get updated when used by more than a mailer at the same time.
On Thu, Mar 13, 2008 at 3:36 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > Enabling rawlog (http://wiki.dovecot.org/Debugging/Rawlog) and looking > at what exactly gets sent at that time could be useful. I did it but I could't find anything useful. Then I decided to use wireshark on the client PC and here is what I found: Mail clients are creating new connections to save sent mail to "Sent" imap folder. At the end of the authentication process the following helpful message is sent to client: "NO Maximum number of connections from user+IP exceeded" Great. That's why. Our imap server is in dmz and all internal users are reaching this server natted from the same ip. The fact is that error message doesn't appear in dovecot logs, and email clients don't say anything useful, either. (maybe dovecot should write it to /var/log/dovecot.log) Increasing mail_max_userip_connections value should fix it. Thank you all for helping me in solving this issue. Regards, Diego.
Re: [Dovecot] [SOLVED] imap sent-mail folder sometimes dosen't get updated when used by more than a mailer at the same time.
On Thu, Mar 13, 2008 at 11:44 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > It is written as a disconnect reason: > Sorry Timo, you're right, my fault, it was there on info_log among tons of other debug messages, and I was watching the error log. > I thought Thunderbird was using max. 5 connections as default though, so > why is this happening? Is it not 5 as default or did your users change > it? The issue happens when multiple instances of the mail client are opened with the same username (and here dovecot sees everyone natted with the same ip). We have some office mail accounts that are read by more than a user. I have still to check how many connections are used by horde-imp webmail. So, again, my fault, and sorry for the noise. Diego.
[Dovecot] A message, when moved by multiple clients with same account and same filters, can be duplicated by filters on destination imap folder.
I'm still collecting user feedback after moving from uw-imap to dovecot development release 1.1.rc3. What said in the subject seems the obvious result of an improper configuration. Office emails are configured and accessed on multiple clients with the same automatic filters, and sometimes the filtered emails get duplicated on the destination folder. The fact is that this was perfectly working with wu-imap (without any duplication/triplication of emails). I'm looking if there is some workaround to keep this client misuse working with dovecot. Could an email have a sort of "primary key" that forbids it's duplication in the same imap folder? Regards, Diego.
Re: [Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
Attached the rawlogs while this happens. I've also a tar.gz of .imap dir of the affected user while this was happening, should I post it, too? Regards, Diego. 20080313-094002-29657.in Description: Binary data 20080313-094002-29657.out Description: Binary data
Re: [Dovecot] dovecot-1.1.rc3 segmentation fault in fetch_bodystructure
On Thu, Mar 27, 2008 at 5:47 PM, Dean Brooks <[EMAIL PROTECTED]> wrote:
>
> Can you confirm that the patch Timo submited in the above link fixes
> this problem for 1.1rc3? If so, will this be committed for rc4 or beyond?
>
No, the above patch was about another issue (segmentation fault in
fetch_bodystructure) and I forgot to change the subject about the assertion
failure you are getting.
As Timo told you, I changed the assert with a i_error waiting for a proper
fix.
Diego.
--- ./src/lib-storage/index/index-sync.c-orig 2008-03-13 16:46:
36.0 +0100
+++ ./src/lib-storage/index/index-sync.c2008-03-13 16:51:
38.0 +0100
@@ -36,7 +36,9 @@
void index_mailbox_set_recent_uid(struct index_mailbox *ibox, uint32_t uid)
{
if (uid <= ibox->recent_flags_prev_uid) {
- i_assert(seq_range_exists(&ibox->recent_flags, uid));
+ /*i_assert(seq_range_exists(&ibox->recent_flags, uid));*/
+ if (!seq_range_exists(&ibox->recent_flags, uid))
+ i_error("seq_range_exists(&ibox->recent_flags,
uid)");
return;
}
ibox->recent_flags_prev_uid = uid;
[Dovecot] Imap sent folder sometimes dosen't get updated in dovecot-1.1-rc3(+some current hg patches)
Yes, again, some sent emails don't get copied to sent folder. This time I get the following error: Error: IMAP(username): UIDs broken with partial sync in mbox file /maildir/username/Sent This happened at least with thunderbird and with evolution. Dovecot is running on vith fcntl locking on Centos5, sendmail should use fcntl, too. Filesystem is ext3 on drbd partition. After a grep vith dovecot.log I see the same error message also with other imap folders as well (mainly Trash imap folder). Should I try to get a rawlog? Regards, Diego.
Re: [Dovecot] [partially solved] Imap sent folder sometimes dosen't get updated in dovecot-1.1-rc3(+some current hg patches)
On Thu, Mar 27, 2008 at 11:35 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: > Yes, again, some sent emails don't get copied to sent folder. Thist time the reporter reproduced the issue because of a client wrong configuration and without the related error (he was using a secondary account that was set to save Sent mails locally instead of using imap folder, and all the missing sent mails were saved locally) > Error: IMAP(username): UIDs broken with partial sync in mbox file > /maildir/username/Sent This error was in the log at the same minute (but different seconds) the mail was sent, and it is probably unrelated. Maybe while the user searched the mail in the Sent folder after sending it. Sorry for the confusion. > After a grep vith dovecot.log I see the same error message also with other > imap folders as well (mainly Trash imap folder). This is still one of the errors I see in dovecot-1.1.rc3 log. Regards, Diego.
Re: [Dovecot] [partially solved] Imap sent folder sometimes dosen't get updated in dovecot-1.1-rc3(+some current hg patches)
On Fri, Apr 25, 2008 at 1:31 AM, Timo Sirainen wrote: > On Sat, 2008-03-29 at 19:19 +0100, Diego Liziero wrote: > > > Error: IMAP(username): UIDs broken with partial sync in mbox file > > /maildir/username/Sent > > Do you use mbox_very_dirty_syncs=yes? Mmm... actually the line is commented out, and the default should be =no. > It can cause this and I haven't > yet bothered to fix it. Anyway I should probably just change it to a > warning since Dovecot can fix the situation itself Ok. Regards, Diego.
[Dovecot] [patch] let valgrind run on login process with GDB=1
Not sure if this should be included in dovecot, just posting if someone
feels like using valgrind.
Diego.
-
diff -r ba634d2c0ab9 src/master/login-process.c
--- a/src/master/login-process.cWed Apr 30 20:18:37 2008 +0300
+++ b/src/master/login-process.cThu May 01 00:59:10 2008 +0200
@@ -689,7 +689,8 @@ static pid_t create_login_process(struct
fd_limit = 16 + listen_count + ssl_listen_count +
2 * (group->set->login_process_per_connection ? 1 :
group->set->login_max_connections);
- restrict_fd_limit(fd_limit);
+ if (getenv("GDB") == NULL)
+ restrict_fd_limit(fd_limit);
/* make sure we don't leak syslog fd, but do it last so that
any errors above will be logged */
Re: [Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
On Wed, Apr 30, 2008 at 4:07 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Mon, 2008-03-10 at 13:17 +0100, Diego Liziero wrote: > > On Mon, Mar 10, 2008 at 9:05 AM, Diego Liziero <[EMAIL PROTECTED]> > wrote: > > > To some users happens this assertion failure while deleting a message. > > > > > > dovecot: Mar 10 08:40:44 Panic: IMAP(user): file index-sync.c: line > 39 > > > (index_mailbox_set_recent_uid): assertion failed: (seq_range_exists > > > (&ibox->recent_flags, uid)) > > Wonder if this fixes it? > http://hg.dovecot.org/dovecot-1.1/rev/abc88e664e63 > Just updated to current tree. I'll let you know if it happens again. Here are some more errors I got in dovecot.log with 1.1.rc4: Corrupted index cache file /path/to/dovecot.index.cache: Broken MIME parts for mail UID uidnumber Corrupted index cache file /path/to/dovecot.index.cache: Broken virtual size for mail UID uidnumber Corrupted index cache file /path/to/dovecot.index.cache: used_file_size too large FETCH for mailbox mailboxname UID uidnumber got too little data: a vs a+b UIDs broken with partial sync in mbox file Corrupted transaction log file /path/to/dovecot.index.log: record size too small (type=0x40, offset=7576, size=0) Log synchronization error at seq=1,offset=24 for /path/to/dovecot.index: Broken extension introduction: Record field points outside record size (0+16 > 8) Diego.
Re: [Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
On Wed, Apr 30, 2008 at 4:07 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > On Mon, Mar 10, 2008 at 9:05 AM, Diego Liziero wrote: > > > To some users happens this assertion failure while deleting a message. > > > > > > dovecot: Mar 10 08:40:44 Panic: IMAP(user): file index-sync.c: line > 39 > > > (index_mailbox_set_recent_uid): assertion failed: (seq_range_exists > > > (&ibox->recent_flags, uid)) > > Wonder if this fixes it? > http://hg.dovecot.org/dovecot-1.1/rev/abc88e664e63 > Unfortunately not. Got it again this morning (I only changed i_assert with i_error): dovecot: May 02 09:40:39 Error: IMAP(username): seq_range_exists(&ibox->recent_flags, uid)
Re: [Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
Timo, I was wondering If I can help you in spotting the cause of this assertion failure (got this morning with rc5) adding some i_info/debug and other seq_range_exists tests. This morning all assertion failures were caused by users that deleted with thunderbird many emails from inbox and thunderbird tried to move them to imap trash folder. seq_range_exists(&ibox->recent_flags, uid) Regards, Diego.
Re: [Dovecot] mbox empty messages in Sent folder
On Mon, May 26, 2008 at 3:34 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Mon, 2008-05-26 at 15:08 +0200, Diego Liziero wrote: >> I'm talking about mbox Sent folder, where some mailers append through >> imap server >> a copy of each message they send. >> >> Sometimes, just three header lines got appended instead of the whole >> mail message, such as: >> >> >From [EMAIL PROTECTED] Fri May 23 12:30:14 2008 >> X-UID: 2852 >> Status: RO > > Are there other messages around it with the same From-line? Could this > From-line be part of the previous message's body? The empty mail header is part of a sent email that is missing in the Sent mbox file. The previous From header and the following one are correctly the ones of the previous and following sent emails. Here is the mbox lines starting fom the previous mail, and ending to the header of the following one (in this case it seems that the actual previous mail has been deleted as UID 2851 is missing, but I have other examples where no UID is missing): --- >From [EMAIL PROTECTED] Fri May 23 12:15:17 2008 Message-ID: <[EMAIL PROTECTED]> Date: Fri, 23 May 2008 12:15:06 +0200 From: xxx xxx <[EMAIL PROTECTED]> User-Agent: xxx x.x.x.xx (xxx/) MIME-Version: x.x To: xxx <[EMAIL PROTECTED]>, xxx <[EMAIL PROTECTED]> Subject: x x xxx xxx xxx x xx ... Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-UID: 2850 Status: RO X-Keywords: Content-Length: 336 xxx, xx xxx xx xxx xxx x.xx - xxx xx x xx xxx 'xx . -- xx xx x - xx xx xxx xxx xxx xxx , x - x x (xx) xxx.: xxx.xx xxx: xxx xxx.xxx .: xxx.xxx x: [EMAIL PROTECTED] >From [EMAIL PROTECTED] Fri May 23 12:30:14 2008 X-UID: 2852 Status: RO >From [EMAIL PROTECTED] Sat May 24 08:24:09 2008 Message-ID: <[EMAIL PROTECTED]> Date: Sat, 24 May 2008 08:24:02 +0200 From: xxx xxx <[EMAIL PROTECTED]> User-Agent: xxx x.x.x.xx (xxx/) MIME-Version: x.x To: <[EMAIL PROTECTED]> Subject: xxx xxx x ,xx (x xxx ) xxx xxx Content-Type: multipart/alternative; ="" X-UID: 2853 Status: R X-Keywords: Content-Length: 4591 Regards, Diego
[Dovecot] mbox empty messages in Sent folder
I'm talking about mbox Sent folder, where some mailers append through imap server a copy of each message they send. Sometimes, just three header lines got appended instead of the whole mail message, such as: >From [EMAIL PROTECTED] Fri May 23 12:30:14 2008 X-UID: 2852 Status: RO This happened in the past (dovecot-1.1-beta/rc with Evolution and with Thunderbird), and happened last week with dovecot-1.1-rc5 (+ last week hg patches). Anyone had a similar issue? Regards, Diego.
Re: [Dovecot] mbox empty messages in Sent folder
On Mon, May 26, 2008 at 11:17 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > Could it be that the connection died client was trying to save it, so it > never got there? Although it still shouldn't have left the From-line there. The mailer didn't crashed. When it happened to me I thought about a bug of Evolution. When the helpdesk told me that is happening also to other users (with thunderbird), I wrote to this list to ask if someone had a similar issue. > Could Dovecot have crashed there? I don't know, however nothing unusual is written to the log. > Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/dd9d344ba140 Thanks Timo, I'll let you know if it happens again with this patch. Diego.
Re: [Dovecot] dovecot's deliver and SELinux
On Thu, 2008-05-29 at 16:48 +0200, Dan Horák wrote: > Hello, > > I am the new maintainer of dovecot for Fedora and Red Hat and so I am > trying to cleanup some old reported bugs. > [..] Mmm.. I was wondering if it's worth to have a look at the various dovecot patches used by main distributions before releasing dovecot-1.1. I mean, having a look if there is something besides the changes of default configuration options or installation path that can be merged upstream. Regards, Diego
[Dovecot] Panic: Trying to close mailbox INBOX with open transactions
Got this using dovecot-1.1 (hg version of 2 days ago). I mean with http://hg.dovecot.org/dovecot-1.1/rev/c4342385d696 included. Regards, Diego. - Panic: IMAP(username): Trying to close mailbox INBOX with open transactions (gdb) bt #0 0x007c3402 in __kernel_vsyscall () #1 0x00138ba0 in raise () from /lib/libc.so.6 #2 0x0013a4b1 in abort () from /lib/libc.so.6 #3 0x080d49f0 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:152 #4 0x080d523a in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80f6c34 "Trying to close mailbox %s with open transactions", args=0xbf8648c4 "8^\\\b�H\206���\005\b\210�[\bN�\016\b�H\206�NI\016\b��[\bh�[\b\030I\206��\023\006\b��[\b��[\b\bI\206�2�\016\b��[\b�$") at failures.c:418 #5 0x080d4b2a in i_panic (format=0x80f6c34 "Trying to close mailbox %s with open transactions") at failures.c:193 #6 0x080a5126 in mailbox_close (_box=0x6) at mail-storage.c:471 #7 0x0805edb8 in cmd_logout (cmd=0x85bd7d8) at cmd-logout.c:18 #8 0x080613dc in client_command_cancel (_cmd=0x85bd3c4) at client.c:75 #9 0x08061662 in client_destroy (client=0x85bd368, reason=0x80eee5b "Connection closed") at client.c:134 #10 0x0806285e in client_input (client=0x85bd368) at client.c:697 #11 0x080df05a in io_loop_handler_run (ioloop=0x85bb9b0) at ioloop-epoll.c:201 #12 0x080de2a2 in io_loop_run (ioloop=0x85bb9b0) at ioloop.c:308 #13 0x0806dd14 in main (argc=1, argv=0xbf864ab4, envp=0xbf864abc) at main.c:293
[Dovecot] dovecot-1.1.rc7 Panic: POP3(username): file index-mail.c: line 1007: unreached
And got this regression with dovecot-1.1.rc7 with each pop3 connection, hg version of 2 days ago was working fine. Now I'm back again to hg before "Message sort index handling rewrite". Regards, Diego. --- Panic: POP3(username): file index-mail.c: line 1007: unreached (gdb) bt #0 0x00b91402 in __kernel_vsyscall () #1 0x00725ba0 in raise () from /lib/libc.so.6 #2 0x007274b1 in abort () from /lib/libc.so.6 #3 0x080c3f0d in default_fatal_finish (type=, status=0) at failures.c:152 #4 0x080c3f5a in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80d8210 "file %s: line %d: unreached", args=0xbfeec684 "3�\r\b�\003") at failures.c:418 #5 0x080c37fc in i_panic (format=0x80d8210 "file %s: line %d: unreached") at failures.c:193 #6 0x08089a60 in index_mail_get_special (_mail=0x963cbe0, field=6, value_r=0xbfeec70c) at index-mail.c:1007 #7 0x0805aac5 in list_uids_iter (client=0x9636f68, ctx=0x96389a0) at commands.c:566 #8 0x0805ad31 in client_command_execute (client=0x9636f68, name=0x963b37c "UIDL", args=0x80dc0c2 "") at commands.c:642 #9 0x08059f92 in client_input (client=0x9636f68) at client.c:413 #10 0x080cb7c0 in io_loop_handler_run (ioloop=0x96359b0) at ioloop-epoll.c:201 #11 0x080ca9c8 in io_loop_run (ioloop=0x96359b0) at ioloop.c:308 #12 0x0805bbfc in main (argc=Cannot access memory at address 0x4c52 ) at main.c:275
Re: [Dovecot] dovecot-1.1.rc7 Panic: POP3(username): file index-mail.c: line 1007: unreached
On Sat, May 31, 2008 at 12:47 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Fri, 2008-05-30 at 23:19 +0200, Diego Liziero wrote: >> And got this regression with dovecot-1.1.rc7 >> --- >> Panic: POP3(username): file index-mail.c: line 1007: unreached > > Broke with non-maildir formats. :( Fixed: > http://hg.dovecot.org/dovecot-1.1/rev/8e7a15987428 Tested. Now it works again, thanks. Diego.
[Dovecot] mbox: extra linefeed after Content-Length header in 1.1.rc8
mbox messages gets header corruption caused by an extra linefeed after Content-Length Users sees their mails in Sent mbox folder without the from and to fields, without attachments and with the date of 1/1/1970 Diego. --- Here is an anonymized header: >From [EMAIL PROTECTED] Tue Jun 03 09:14:33 2008 Message-ID: <[EMAIL PROTECTED]> X-UID: 3913 Status: RO X-Keywords: Content-Length: 6817 : xxx, xx xxx xx:xx:xx + : xxx <[EMAIL PROTECTED]> -x: xxx x.x.x.x (xxx/) -xxx: x.x xx: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> xx: [EMAIL PROTECTED], xx x <[EMAIL PROTECTED]>, xxx xxx <[EMAIL PROTECTED]> xxx: xx: x: xx: x xx: <[EMAIL PROTECTED]> xx-x-xx: <[EMAIL PROTECTED]> xxx-: /x; xxx=xxx-x; xx=xx xxx--:
Re: [Dovecot] mbox empty messages in Sent folder
On Tue, May 27, 2008 at 5:43 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: >> > Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/dd9d344ba140 >> >> Thanks Timo, I'll let you know if it happens again with this patch. > > That only causes the empty message not to be written at all, it doesn't > help about losing saved messages. Amoung the various emails with header corruption I got with 1.1.rc8, one of them is similar to the one of this bug report. Just the first 5 lines of header without body, and without the remaining lines of the header, either. But this time there are two header lines more: X-Keywords and Content-Length. --- >From [EMAIL PROTECTED] Mon Jun 02 23:53:29 2008 X-UID: 7863 Status: R X-Keywords: Content-Length: 0 --- Actually this is the Content-Length after having selected the mail with evolution, and having this error in dovecot.log: Error: IMAP(username): FETCH for mailbox sent-mail UID 7863 got too little data: 2 vs 615 Error: IMAP(username): Corrupted index cache file /mailhome/username/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 7863 I don't know if the original email content length was correctly written in the header, or in the index. Diego.
Re: [Dovecot] mbox: extra linefeed after Content-Length header in 1.1.rc8
On Tue, Jun 3, 2008 at 3:05 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Tue, 2008-06-03 at 10:34 +0200, Diego Liziero wrote: >> mbox messages get header corruption caused by an extra linefeed after >> Content-Length > > Fixed: http://hg.dovecot.org/dovecot-1.1/rev/e043135e971d Works, thank you. Now I have to fix users mbox files. As the extra linefeed is between Content-Length and Subject headers, I'm thinking about using a regexp based replace such as s/(Content-Length: [0-9]+)\n\n(Subject: )/$1\n$2/s but I can't find how to make multiple lines matching work. Any suggestion? Regards, Diego.
Re: [Dovecot] [solved] mbox: extra linefeed after Content-Length header in 1.1.rc8
> On Wed, 2008-06-04 at 23:59 +0200, Diego Liziero wrote:
> As the extra linefeed is between Content-Length and Subject headers,
> I'm thinking about using a regexp based replace such as
> s/(Content-Length: [0-9]+)\n\n(Subject: )/$1\n$2/s
> but I can't find how to make multiple lines matching work.
>
> Any suggestion?
Thank you everyone for your help.
After some quick tries, and following your suggestions, I ended up in
writing a silly perl script that matched one by one each of the three
lines and printed only the first and third one.
On Thu, Jun 5, 2008 at 12:07 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote:
>
> Perl maybe? Something like (not tested):
>
> perl -pe 'BEGIN { $/ = ""; } s/^(Content-Length: [0-9]+)\n\n(Subject:
> )/$1\n$2/g' < mbox > mbox2
>
> $/ changes the line separator.
Almost right. But this deletes all empty lines, not just the ones in
the header. I didn't try to have a deeper look.
On Thu, Jun 5, 2008 at 8:16 AM, <[EMAIL PROTECTED]> wrote:
> That would be the s modifier for a Perl regexp (treat string as a single
> line):
>
> $x =~ /.../s
This should be the right way.. see below.
On Thu, Jun 5, 2008 at 12:03 AM, Asheesh Laroia <[EMAIL PROTECTED]> wrote:
>Python has an re.MULTILINE option you can pass to the regular expression so
>that it can cross lines. Perhaps Perl >or your favorite regular expression
>toolkit has something similar?
Yes, but with perl I didn't find quickly a solution to read multiple
lines from a file without filling all system memory when files are
some gigabytes big.
Regards,
Diego.
Re: [Dovecot] While searching: Assertion failed (offset > = ctx- >input->v_offset)
On Friday 20 June 2008, Timo Sirainen wrote: > On Fri, 2008-06-20 at 09:42 +0200, Diego Liziero wrote: > > > > Timo, > > here is an anonymized mbox file that causes it at every body search > > (tested with rc12). > > Did you test it without index files? I couldn't reproduce the crash. Deleting index files solved the crash. I sent broken index files privately to Timo. Diego.
Re: [Dovecot] v1.1.rc13 released
Any chance to have this assert converted to error as last patch before 1.1?
Or am I the only one that is still getting this in rc13?
Regards,
Diego
--- ./src/lib-storage/index/index-sync.c-orig 2008-03-13
16:46:36.0 +0100
+++ ./src/lib-storage/index/index-sync.c2008-03-13
16:51:38.0 +0100
@@ -36,7 +36,9 @@
void index_mailbox_set_recent_uid(struct index_mailbox *ibox, uint32_t uid)
{
if (uid <= ibox->recent_flags_prev_uid) {
- i_assert(seq_range_exists(&ibox->recent_flags, uid));
+ /*i_assert(seq_range_exists(&ibox->recent_flags, uid));*/
+ if (!seq_range_exists(&ibox->recent_flags, uid))
+ i_error("seq_range_exists(&ibox->recent_flags,
uid) uid=%d",uid);
return;
}
ibox->recent_flags_prev_uid = uid;
Re: [Dovecot] mbox empty messages in Sent folder
On Tue, May 27, 2008 at 3:48 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: > On Mon, May 26, 2008 at 11:17 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: >> >> Could it be that the connection died client was trying to save it, so it >> never got there? Although it still shouldn't have left the From-line there. > > The mailer didn't crashed. > When it happened to me I thought about a bug of Evolution. > When the helpdesk told me that is happening also to other users > (with thunderbird), I wrote to this list to ask if someone had a similar > issue. > >> Could Dovecot have crashed there? > > I don't know, however nothing unusual is written to the log. > >> Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/dd9d344ba140 > > Thanks Timo, I'll let you know if it happens again with this patch. Unfortunately it happened again with dovecot-1.1.rc13 (=1.1.0) This time using horde/imp Diego.
[Dovecot] dovecot 1.1.2, another assertion failure in index-mail.c: (!mail->data.destroying_stream)
Here is the log. --- dovecot: Jul 25 07:18:35 Panic: IMAP(user): file index-mail.c: line 1091 (index_mail_close): assertion failed: (!mail->data.destroying_stream) dovecot: Jul 25 07:18:35 Error: IMAP(user): Raw backtrace: /usr/libexec/dovecot/imap [0x80f5fd4] -> /usr/libexec/dovecot/imap [0x80f6877] -> /usr/libexec/dovecot/imap(i_fatal+0) [0x80f6130] -> /usr/libexec/dovecot/imap(index_mail_close+0xf9) [0x80a8129] -> /usr/libexec/dovecot/imap [0x80a8148] -> /usr/libexec/dovecot/imap(index_mail_set_seq+0x50) [0x80a8302] -> /usr/libexec/dovecot/imap [0x8087e3b] -> /usr/libexec/dovecot/imap(mail_set_seq+0x21) [0x80b391c] -> /usr/libexec/dovecot/imap(index_storage_search_next_nonblock+0xcb) [0x80ad3e8] -> /usr/libexec/dovecot/imap(mailbox_search_next_nonblock+0x26) [0x80b6a97] -> /usr/libexec/dovecot/imap(mailbox_search_next+0x2c) [0x80b6a63] -> /usr/libexec/dovecot/imap [0x805b757] -> /usr/libexec/dovecot/imap(cmd_copy+0x1c4) [0x805b98e] -> /usr/libexec/dovecot/imap(cmd_uid+0xbb) [0x80610d3] -> /usr/libexec/dovecot/imap [0x8062494] -> /usr/libexec/dovecot/imap [0x80626c9] -> /usr/libexec/dovecot/imap [0x80627c7] -> /usr/libexec/dovecot/imap [0x8062803] -> /usr/libexec/dovecot/imap(client_input+0xb7) [0x8062991] -> dovecot: Jul 25 07:18:35 Error: IMAP(user): /usr/libexec/dovecot/imap(io_loop_handler_run+0x17d) [0x8100630] -> /usr/libexec/dovecot/imap(io_loop_run+0x35) [0x80ff8de] -> /usr/libexec/dovecot/imap(main+0xe4) [0x806de1d] -> /lib/libc.so.6(__libc_start_main+0xdc) [0x42bdec] -> /usr/libexec/dovecot/imap [0x805a1a1] dovecot: Jul 25 07:18:35 Error: child 27074 (imap) killed with signal 6 dovecot: Jul 25 07:18:35 Error: IMAP(user): Corrupted transaction log file /mailhome/user/.imap/Trash/dovecot.index.log: record size too small (type=0x40, offset=56, size=0) dovecot: Jul 25 07:18:36 Error: IMAP(user): Corrupted transaction log file /mailhome/user/.imap/Trash/dovecot.index.log: record size too small (type=0x40, offset=56, size=0) dovecot: Jul 25 07:19:33 Error: IMAP(user): Corrupted transaction log file /mailhome/user/.imap/Trash/dovecot.index.log: record size too small (type=0x40, offset=56, size=0) dovecot: Jul 25 07:19:42 Error: IMAP(user): Corrupted transaction log file /mailhome/user/.imap/Trash/dovecot.index.log: record size too small (type=0x40, offset=56, size=0) dovecot: Jul 25 07:19:48 Warning: IMAP(user): fscking index file (in-memory index) dovecot: Jul 25 07:19:48 Error: IMAP(user): Transaction log got desynced for index (in-memory index)
[Dovecot] bugzilla or other similar bug tracking systems
I was wondering if it could be useful to use such tools to keep track of users bugs. I find somehow harder to search the mailing list if a bug is known, if it's being worked on, if it needs more feedback, in witch release it has been eventually solved, and so on. Any thoughts? Regards, Diego.
Re: [Dovecot] mbox empty messages in Sent folder
On Mon, Jun 23, 2008 at 11:03 AM, Diego Liziero <[EMAIL PROTECTED]> wrote: > On Tue, May 27, 2008 at 3:48 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: >> On Mon, May 26, 2008 at 11:17 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: >> >>> Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/dd9d344ba140 >> >> Thanks Timo, I'll let you know if it happens again with this patch. > > Unfortunately it happened again with dovecot-1.1.rc13 (=1.1.0) And many users complains about this also with latest dovecot 1.1.2 In the last 2 days it happened 7 times to my user. I've the rawlog and valgrind logs. Nothing wrong according to valgrind, and nothing strange in the rawlog. Is there any further debug I can use?
Re: [Dovecot] Dovecot 1.1.1 killed with SIGABRT
On Mon, Jul 21, 2008 at 8:13 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Mon, 2008-07-21 at 18:08 +0200, Andreas M. Kirchwitz wrote: >> >> Jul 12 01:04:45 linux dovecot: Panic: IMAP(user2): file index-sync.c: >> line 39 (index_mailbox_set_recent_uid): assertion failed: >> (seq_range_exists(&ibox->recent_flags, uid)) > > I think I finally managed to fix this: > http://hg.dovecot.org/dovecot-1.1/rev/48bbaf0c3e4d Unfortunately just got it again with dovecot 1.1.2 seq_range_exists(&ibox->recent_flags, uid) ibox->recent_flags=164556344 uid=8
Re: [Dovecot] bugzilla or other similar bug tracking systems
On Fri, Aug 1, 2008 at 1:12 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Aug 1, 2008, at 12:51 AM, Diego Liziero wrote: > > I was wondering if it could be useful to use such tools to keep track >> of users bugs. >> >> I find somehow harder to search the mailing list if a bug is known, if >> it's being worked on, if it needs more feedback, in witch release it >> has been eventually solved, and so on. >> > > I'd like to have a bug tracking system that basically keeps track of > threads in this mailing list that have bugs and with some extra metadata > assigned to them about the bug states etc. But that'd require 1) ANNOTATE > extension for Dovecot and 2) Writing the bug tracking system itself.. > > I don't really want to have a separate bug tracking system where I'd be the > only one handling the bug reports. Ok, I'm waiting for that. My belief is that a public bug tracking system, whatever you choose, could bring more people into the bug fixing process. Who knows, maybe during the summer holidays someone that would like to spend some weeks to help this project could find an easy way to start.
Re: [Dovecot] mbox empty messages in Sent folder
On Fri, Aug 1, 2008 at 1:09 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Aug 1, 2008, at 1:49 AM, Diego Liziero wrote: > > On Mon, Jun 23, 2008 at 11:03 AM, Diego Liziero <[EMAIL PROTECTED]> >> wrote: >> >>> On Tue, May 27, 2008 at 3:48 PM, Diego Liziero <[EMAIL PROTECTED]> >>> wrote: >>> >>>> On Mon, May 26, 2008 at 11:17 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: >>>> >>>> Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/dd9d344ba140 >>>>> >>>> >>>> Thanks Timo, I'll let you know if it happens again with this patch. >>>> >>> >>> Unfortunately it happened again with dovecot-1.1.rc13 (=1.1.0) >>> >> >> And many users complains about this also with latest dovecot 1.1.2 >> >> In the last 2 days it happened 7 times to my user. I've the rawlog and >> valgrind logs. >> Nothing wrong according to valgrind, and nothing strange in the rawlog. >> >> Is there any further debug I can use? >> > > You mean the messages show up as APPENDed in the rawlog with an OK reply? It seems so, I sent you privately the rawlog and the last part of a sent-mail mailbox. Anyone besides me is seeing this? (random empty mails in Sent folder) In the last two weeks users are complainig almost every day about lost sent mails. Regards, Diego.
Re: [Dovecot] mbox empty messages in Sent folder
On Mon, Aug 4, 2008 at 12:07 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Aug 3, 2008, at 10:31 PM, Diego Liziero wrote: > >> It seems so, >> I sent you privately the rawlog and the last part of a sent-mail mailbox. > > The interesting thing about that rawlog was that it shows the APPEND > returning it saved the message with UID x, but in the mbox file there's no > UID x, but there is the empty message with UID x+1. > >> Anyone besides me is seeing this? (random empty mails in Sent folder) > > I'm using mboxes all the time, never seen this.. > > Perhaps if you put all processes through strace (-s 100) and when it > again happens for some user send me the strace? Although I'd guess it shows > that the message was properly written to the mbox file. The real question is > then what truncates the mbox file.. Just sent to Timo the strace of the same mail sent twice, the fist time appeared empty, the second time correctly written to sent-mail folder.
Re: [Dovecot] mbox empty messages in Sent folder
On Mon, Aug 4, 2008 at 4:17 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/8ab845d3c96d > It seems so, thanks Timo. With this patch, by now, all sent mails are correctly written in "Sent" folder, I'let you know if I've just been lucky :) BTW I didn't succeed in reproducing this issue with imaptest, what was the trick to trigger it?
Re: [Dovecot] Dovecot 1.1.1 killed with SIGABRT
On Fri, Aug 1, 2008 at 12:00 PM, Diego Liziero <[EMAIL PROTECTED]> wrote:
> On Mon, Jul 21, 2008 at 8:13 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote:
>> On Mon, 2008-07-21 at 18:08 +0200, Andreas M. Kirchwitz wrote:
>>> >> Jul 12 01:04:45 linux dovecot: Panic: IMAP(user2): file index-sync.c:
>>> line 39 (index_mailbox_set_recent_uid): assertion failed:
>>> (seq_range_exists(&ibox->recent_flags, uid))
>>
>> I think I finally managed to fix this:
>> http://hg.dovecot.org/dovecot-1.1/rev/48bbaf0c3e4d
>
> Unfortunately just got it again with dovecot 1.1.2
> seq_range_exists(&ibox->recent_flags, uid)
Got it again, I've exec and core file, if you need them.
BTW, I'm getting this assertion failure definetly less frequently than before.
Could it be just a leftover from the previous unpatched version?
Regards,
Diego.
Panic: IMAP(user): file index-sync.c: line 42
(index_mailbox_set_recent_uid): assertion failed:
(seq_range_exists(&ibox->recent_flags, uid))
Error: IMAP(user): Raw backtrace: /usr/libexec/dovecot/imap
[0x80f609c] -> /usr/libexec/dovecot/imap [0x80f693f] ->
/usr/libexec/dovecot/imap(i_fatal+0) [0x80f61f8] ->
/usr/libexec/dovecot/imap(index_mailbox_set_recent_uid+0xbc)
[0x80b201f] -> /usr/libexec/dovecot/imap(index_mailbox_set_recent_seq+0x33)
[0x80b2095] -> /usr/libexec/dovecot/imap [0x808f02b] ->
/usr/libexec/dovecot/imap [0x808f64d] -> /usr/libexec/dovecot/imap
[0x808ffb7] -> /usr/libexec/dovecot/imap(mbox_sync+0x2b) [0x8090243]
-> /usr/libexec/dovecot/imap [0x80851a1] ->
/usr/libexec/dovecot/imap(mail_index_transaction_commit+0x59)
[0x80c6df2] -> /usr/libexec/dovecot/imap(index_transaction_commit+0x65)
[0x80b3901] ->
/usr/libexec/dovecot/imap(mailbox_transaction_commit_get_uids+0x50)
[0x80b6bce] -> /usr/libexec/dovecot/imap [0x805a994] ->
/usr/libexec/dovecot/imap [0x805a3a0] ->
/usr/libexec/dovecot/imap(io_loop_handler_run+0x17d) [0x81006e8] ->
/usr/libexec/dovecot/imap(io_loop_run+0x35) [0x80ff996] ->
/usr/libexec/dovecot/imap(main+0xe4) [0x806de3d] -> /lib/li
Error: IMAP(user): bc.so.6(__libc_start_main+0xdc) [0x42bdec] ->
/usr/libexec/dovecot/imap [0x805a1a1]
Error: child 14026 (imap) killed with signal 6
(gdb) bt full
No symbol table info available.
#1 0x0043ed20 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0x00440631 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080f60be in default_fatal_finish (type=LOG_TYPE_PANIC, status=0)
at failures.c:149
backtrace = 0x9f34738 "/usr/libexec/dovecot/imap [0x80f609c]
-> /usr/libexec/dovecot/imap [0x80f693f] ->
/usr/libexec/dovecot/imap(i_fatal+0) [0x80f61f8] ->
/usr/libexec/dovecot/imap(index_mailbox_set_recent_uid+0xbc) [0x80"...
#4 0x080f693f in i_internal_fatal_handler (type=LOG_TYPE_PANIC,
status=0, fmt=0x8119ea0 "file %s: line %d (%s): assertion failed:
(%s)",
args=0xbfe8da34 "\223\236\021\b*") at failures.c:423
No locals.
#5 0x080f61f8 in i_panic (format=0x8119ea0 "file %s: line %d (%s):
assertion failed: (%s)") at failures.c:190
args = 0xbfe8da34 "\223\236\021\b*"
#6 0x080b201f in index_mailbox_set_recent_uid (ibox=0x9f46e48,
uid=14) at index-sync.c:42
__PRETTY_FUNCTION__ = "index_mailbox_set_recent_uid"
#7 0x080b2095 in index_mailbox_set_recent_seq (ibox=0x9f46e48,
view=0x9f50c10, seq1=2, seq2=7) at index-sync.c:59
uid = 14
#8 0x0808f02b in mbox_sync_update_index_header (sync_ctx=0xbfe8dc04)
at mbox-sync.c:1424
view = (struct mail_index_view *) 0x9f50c10
st = (const struct stat *) 0x9f580c0
first_recent_uid = 0
seq = 2
seq2 = 7
__PRETTY_FUNCTION__ = "mbox_sync_update_index_header"
#9 0x0808f64d in mbox_sync_do (sync_ctx=0xbfe8dc04, flags=18) at
mbox-sync.c:1560
mail_ctx = {sync_ctx = 0xbfe8dc04, mail = {uid = 19, idx_seq =
7, keywords = {arr = {buffer = 0x0, element_size = 0}, v = 0x0,
v_modifiable = 0x0}, flags = 40 '(', uid_broken = 0, expunged =
0, pseudo = 0, from_offset = 7366, body_size = 414, offset = 8025,
space = 70}, seq = 7, hdr_offset = 7433, body_offset = 8117,
header_first_change = 4294967295, header_last_change = 0,
header = 0x9f57c80, hdr_md5_sum =
"){\023??\\??y\226???Z\221?", content_length = 414,
hdr_pos = {578, 4294967295, 592, 4294967295,
567}, parsed_uid = 19, last_uid_updated_value = 0,
last_uid_value_start_pos = 0, have_eoh = 1, need_rewrite = 0,
seen_imapbase = 0,
updated = 0, recent = 1, dirty = 0, imapbase_rewrite = 0,
imapbase_updated = 0}
st = (const struct stat *) 0x9f580c0
i = 0
ret = 1
partial = 1
#10 0x0808ffb7 in mbox_sync_int (mbox=0x9f46e48, flags=18) at mbox-sync.c:1803
index_sync_ctx = (struct mail_index_sy
Re: [Dovecot] mbox empty messages in Sent folder
On Wed, Aug 6, 2008 at 4:26 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Aug 6, 2008, at 6:11 AM, Diego Liziero wrote: >> On Mon, Aug 4, 2008 at 4:17 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: >>> >>> Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/8ab845d3c96d >>> >> >> It seems so, thanks Timo. >> >> With this patch, by now, all sent mails are correctly written in >> "Sent" folder, I'let you know if I've just been lucky :) Definitely solved. I asked the most complainig users to test if it's fixed and they say "yes". The most affected client was horde/imp webmail. Thanks again Timo. >> BTW I didn't succeed in reproducing this issue with imaptest, what was >> the trick to trigger it? > > I'm not sure if there's an easy way to reproduce it. You'd have to cause the > first read to return EAGAIN but the second read that comes only microseconds > later to return the entire message. Perhaps if imaptest sent first the > APPEND command, then did a small pause and after that sent the message. Mmm.. I tried to comment out the "cork" part and added a 10% random sleep after sending the command if (!(rand()%9)) usleep(rand()%500); and I started getting the famous "Error: IMAP(testdove): FETCH for mailbox INBOX UID xxx got too little data: yyy vs zzz" instead. Regards, Diego.
[Dovecot] assertion failure in current hg, file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (1.1.3 was working fine)
Today I updated to current dovecot-1.1 hg tree and I got many of these
assertion failures:
file istream.c: line 303 (i_stream_read_data): assertion failed:
(stream->stream_errno != 0)
(gdb) bt full
#0 0x00352402 in __kernel_vsyscall ()
No symbol table info available.
#1 0x0043ed20 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0x00440631 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080f6968 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0)
at failures.c:150
backtrace = 0x906d950 "/usr/libexec/dovecot/imap [0x80f6946]
-> /usr/libexec/dovecot/imap [0x80f7207] ->
/usr/libexec/dovecot/imap(i_fatal+0) [0x80f6ac0] ->
/usr/libexec/dovecot/imap(i_stream_read_data+0.
#4 0x080f7207 in i_internal_fatal_handler (type=LOG_TYPE_PANIC,
status=0, fmt=0x8124778 "file %s: line %d (%s): assertion failed:
(%s)", args=0xbfe643a4 "mG\022\b/\001") at failures.c:430
No locals.
#5 0x080f6ac0 in i_panic (format=0x8124778 "file %s: line %d (%s):
assertion failed: (%s)") at failures.c:197
args = 0xbfe643a4 "mG\022\b/\001"
#6 0x080fcb64 in i_stream_read_data (stream=0x9094a08,
data_r=0xbfe6, size_r=0xbfe64440, threshold=0) at istream.c:303
ret = -1
read_more = false
__PRETTY_FUNCTION__ = "i_stream_read_data"
#7 0x08065ba1 in imap_fetch_send (ctx=0x90716e0, output=0x90714ec,
input=0x9094a08, cr_skipped=false, virtual_size=169,
add_missing_eoh=false, last_cr=0x9071734) at imap-fetch-body.c:132
msg = (const unsigned char *) 0x0
i = 3219539032
size = 0
vsize_left = 169
sent = 0
ret = 581237648214344776
add = 16 '\020'
blocks = false
#8 0x08065d7f in fetch_stream_send (ctx=0x90716e0) at imap-fetch-body.c:217
ret = 151603720
#9 0x080660e8 in fetch_stream (ctx=0x90716e0, size=0x908efc4) at
imap-fetch-body.c:293
input = (struct istream *) 0x10b8df
#10 0x08066225 in fetch_data (ctx=0x90716e0, body=0x9071cd8,
size=0x908efc4) at imap-fetch-body.c:318
str = (string_t *) 0x906cc18
#11 0x08066ac7 in fetch_body_mime (ctx=0x90716e0, mail=0x908e9e8,
body=0x9071cd8) at imap-fetch-body.c:555
part = (const struct message_part *) 0x908efb0
section = 0x9071d12 "MIME"
__PRETTY_FUNCTION__ = "fetch_body_mime"
#12 0x08064bad in imap_fetch_more (ctx=0x90716e0) at imap-fetch.c:309
h = (const struct imap_fetch_context_handler *) 0x90718c8
_data_stack_cur_id = 4
client = (struct client *) 0x9071368
handlers = (const struct imap_fetch_context_handler *) 0x9071850
count = 7
ret = 1
__PRETTY_FUNCTION__ = "imap_fetch_more"
#13 0x08064dd4 in imap_fetch (ctx=0x90716e0) at imap-fetch.c:361
ret = 0
__PRETTY_FUNCTION__ = "imap_fetch"
#14 0x0805c792 in cmd_fetch (cmd=0x9071600) at cmd-fetch.c:152
ctx = (struct imap_fetch_context *) 0x90716e0
args = (const struct imap_arg *) 0x9075688
search_arg = (struct mail_search_arg *) 0x9071678
messageset = 0x9075750 "4068"
ret = 151455048
#15 0x08061173 in cmd_uid (cmd=0x9071600) at cmd-uid.c:26
command = (struct command *) 0x907068c
cmd_name = 0x9075738 "fetch"
#16 0x08062534 in client_command_input (cmd=0x9071600) at client.c:580
client = (struct client *) 0x9071368
command = (struct command *) 0x23
__PRETTY_FUNCTION__ = "client_command_input"
#17 0x08062769 in client_command_input (cmd=0x9071600) at client.c:629
client = (struct client *) 0x9071368
command = (struct command *) 0x9070668
__PRETTY_FUNCTION__ = "client_command_input"
#18 0x08062867 in client_handle_next_command (client=0x9071368,
remove_io_r=0xbfe64765) at client.c:670
size = 132
#19 0x080628a3 in client_handle_input (client=0x9071368) at client.c:680
_data_stack_cur_id = 3
ret = 7
remove_io = false
handled_commands = false
#20 0x08062a31 in client_input (client=0x9071368) at client.c:725
cmd = (struct client_command_context *) 0x4e7408
output = (struct ostream *) 0x90714ec
bytes = 132
__PRETTY_FUNCTION__ = "client_input"
#21 0x08101115 in io_loop_handler_run (ioloop=0x906f9b0) at ioloop-epoll.c:203
ctx = (struct ioloop_handler_context *) 0x906faa8
events = (struct epoll_event *) 0x906fae8
event = (const struct epoll_event *) 0x906fae8
list = (struct io_list *) 0x9071568
io = (struct io_file *) 0x9071548
tv = {tv_sec = 1799, tv_usec = 999874}
events_count = 3
t_id = 2
msecs = 180
ret = 1
i = 0
j = 0
call = true
#22 0x081003ac in io_loop_run (ioloop=0x906f9b0) at ioloop.c:320
No locals.
#23 0x0806debd in main (argc=1, argv=0xbfe648c4, envp=0xbfe648cc) at main.c:293
No locals.
[Dovecot] another assertion failure in current 1.1 hg (1.1.3 was working fine) - file message-address.c: line 43 (parse_local_part): assertion failed: (ctx->parser.data != ctx->parser.end)
file message-address.c: line 43 (parse_local_part): assertion failed:
(ctx->parser.data != ctx->parser.end)
#0 0x001b3402 in __kernel_vsyscall ()
No symbol table info available.
#1 0x0043ed20 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0x00440631 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080f6968 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0)
at failures.c:150
backtrace = 0x90e9ed8 "/usr/libexec/dovecot/imap [0x80f6946]
-> /usr/libexec/dovecot/imap [0x80f7207] ->
/usr/libexec/dovecot/imap(i_fatal+0) [0x80f6ac0] ->
/usr/libexec/dovecot/imap [0x80eaee7] -> /usr/.
#4 0x080f7207 in i_internal_fatal_handler (type=LOG_TYPE_PANIC,
status=0, fmt=0x8122bf0 "file %s: line %d (%s): assertion failed:
(%s)", args=0xbfe92c54 "�+\022\b+") at failures.c:430
No locals.
#5 0x080f6ac0 in i_panic (format=0x8122bf0 "file %s: line %d (%s):
assertion failed: (%s)") at failures.c:197
args = 0xbfe92c54 "�+\022\b+"
#6 0x080eaee7 in parse_local_part (ctx=0xbfe92d78) at message-address.c:43
ret = 0
__PRETTY_FUNCTION__ = "parse_local_part"
#7 0x080eb373 in parse_addr_spec (ctx=0xbfe92d78) at message-address.c:163
ret = 24
ret2 = -1
#8 0x080eb49d in parse_mailbox (ctx=0xbfe92d78) at message-address.c:200
start = (const unsigned char *) 0x90fba9f ""
ret = -1
#9 0x080eb503 in parse_mailbox_list (ctx=0xbfe92d78) at message-address.c:214
ret = 151951304
#10 0x080eb5c9 in parse_group (ctx=0xbfe92d78) at message-address.c:246
ret = 152025736
#11 0x080eb64c in parse_address (ctx=0xbfe92d78) at message-address.c:268
start = (const unsigned char *) 0x90fba88 "undisclosed-recipients:"
ret = 151951000
#12 0x080eb68b in parse_address_list (ctx=0xbfe92d78,
max_addresses=4294967294) at message-address.c:283
ret = 0
#13 0x080eb78f in message_address_parse_real (pool=0x90e9500,
data=0x90fba88 "undisclosed-recipients:", size=23,
max_addresses=4294967295, fill_missing=true) at message-address.c:320
ctx = {pool = 0x90e9500, parser = {data = 0x90fba9f "", end =
0x90fba9f "", last_comment = 0x90e9570}, first_addr = 0x90e97c8,
last_addr = 0x90e97c8, addr = {next = 0x0, name = 0x0, route = 0x0,
mailbox = 0x0, domain = 0x0, invalid_syntax = false}, str =
0x90e9698, fill_missing = true}
ret = 1
#14 0x080eb7e0 in message_address_parse (pool=0x90e9500,
data=0x90fba88 "undisclosed-recipients:", size=23,
max_addresses=4294967295, fill_missing=true) at message-address.c:331
addr = (struct message_address *) 0x90e9500
#15 0x080ac097 in search_header_arg (arg=0x90ef6c0, ctx=0xbfe92f80) at
index-search.c:424
addr = (struct message_address *) 0x90ff31c
str = (string_t *) 0x17
_data_stack_cur_id = 5
msg_search_ctx = (struct message_search_context *) 0x91032f0
block = {part = 0x0, hdr = 0xbfe92e38, data = 0x0, size = 0}
hdr = {name = 0x811a0f0 "", name_len = 0, value = 0x90fba88
"undisclosed-recipients:", value_len = 23, full_value = 0x90fba88
"undisclosed-recipients:", full_value_len = 23, middle = 0x90fdd53 ":
",
middle_len = 0, name_offset = 0, full_value_offset = 4, continues =
0, continued = 0, eoh = 0, no_newline = 0, crlf_newline = 0,
use_full_value = 0}
ret = 0
#16 0x080b537a in search_arg_foreach (arg=0x90ef6c0,
callback=0x80abe09 , context=0xbfe92f80) at
mail-search.c:85
subarg = (struct mail_search_arg *) 0x0
__PRETTY_FUNCTION__ = "search_arg_foreach"
#17 0x080b53a4 in mail_search_args_foreach (args=0x90ef6c0,
callback=0x80abe09 , context=0xbfe92f80) at
mail-search.c:98
result = 1
#18 0x080ac2f7 in search_header (hdr=0x90fec80, ctx=0xbfe92f80) at
index-search.c:491
No locals.
#19 0x080ed97e in message_parse_header (input=0x90fa8f8, hdr_size=0x0,
flags=MESSAGE_HEADER_PARSER_FLAG_CLEAN_ONELINE, callback=0x80ac22f
, context=0xbfe92f80) at message-header-parser.c:389
hdr_ctx = (struct message_header_parser_ctx *) 0x90fec80
hdr = (struct message_header_line *) 0x90fec80
ret = 1
__PRETTY_FUNCTION__ = "message_parse_header"
#20 0x080ac691 in search_arg_match_text (args=0x90ef678,
ctx=0x90fb7a8) at index-search.c:580
hdr_ctx = {index_context = 0x90fb7a8, args = 0x90ef678, hdr =
0x90fec80, parse_headers = 0, custom_header = 1, threading = 0}
input = (struct istream *) 0x90fa8f8
headers_ctx = (struct mailbox_header_lookup_ctx *) 0x90fb068
headers = (const char * const *) 0x90e9458
have_headers = true
have_body = false
__PRETTY_FUNCTION__ = "search_arg_match_text"
#21 0x080ad3e1 in search_match_next (ctx=0x90fb7a8) at index-search.c:967
arg = (struct mail_search_arg *) 0x90fb7a8
ret = -1
#22 0x080ad703 in index_storage_search_next_nonblock (_ctx=0x90fb7a8,
mail=0x90f8f68, tryagain_r=0xbfe93083) at index-search.c:1042
_data_stack_cur_id = 4
ctx = (struct index_search_cont
Re: [Dovecot] assertion failure in current hg, file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (1.1.3 was working fine)
On Wed, Sep 17, 2008 at 9:35 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote:
>
> Could you do:
>
>> #6 0x080fcb64 in i_stream_read_data (stream=0x9094a08,
>> data_r=0xbfe6, size_r=0xbfe64440, threshold=0) at istream.c:303
>> ret = -1
>> read_more = false
>> __PRETTY_FUNCTION__ = "i_stream_read_data"
>
> fr 6
> p *stream
> p *stream.real_stream
(gdb) fr 6
#6 0x080fcb64 in i_stream_read_data (stream=0x9094a08,
data_r=0xbfe6, size_r=0xbfe64440, threshold=0) at istream.c:303
303 i_assert(stream->stream_errno != 0);
(gdb) p *stream
$3 = {v_offset = 155312, stream_errno = 0, mmaped = 0, blocking = 1,
closed = 0, seekable = 1, eof = 0, real_stream = 0x90949e0}
(gdb) p *stream.real_stream
$4 = {iostream = {refcount = 2, close = 0x810f75c
, destroy = 0x80e8a2c
,
set_max_buffer_size = 0x80e8acb
, destroy_callback =
0x80a771c , destroy_context =
0x908e9e8},
read = 0x80e95c9 , seek = 0x80e989a
, sync = 0x80e9a51
, stat = 0x80e9a63
, istream = {
v_offset = 155312, stream_errno = 0, mmaped = 0, blocking = 1,
closed = 0, seekable = 1, eof = 0, real_stream = 0x90949e0}, fd = -1,
abs_start_offset = 109832723, statbuf = {st_dev = 0, __pad1 = 0,
__st_ino = 0, st_mode = 0, st_nlink = 0, st_uid = 0, st_gid = 0,
st_rdev = 0, __pad2 = 0, st_size = -1, st_blksize = 0, st_blocks = 0,
st_atim = {tv_sec = 1221544863, tv_nsec = 0}, st_mtim = {
tv_sec = 1221544863, tv_nsec = 0}, st_ctim = {tv_sec =
1221544863, tv_nsec = 0}, st_ino = 0}, buffer = 0x0, w_buffer = 0x0,
buffer_size = 0, max_buffer_size = 8192, skip = 0, pos = 0, parent =
0x9094580,
parent_start_offset = 0, line_str = 0x0}
(gdb)
[Dovecot] lately pop3 with #define DEBUG needs GDB=1
I'm not sure when this happened.
In yesterday dovecot-1.1 hg if pop3 is compiled with DEBUG defined, it
needs GDB=1 otherwise it ends with:
Panic: Leaked file fd 4: dev 104.2 inode 3342766
Not sure if this can be caused by the fact that I call pop3 with a bash script.
protocol pop3 {
mail_executable = /usr/libexec/dovecot/pop3.sh
inside I've something like
if [ x`id -un` = "xmyuser" -o x`id -un` = "xtestdovecot" ]
then
exec /usr/libexec/dovecot/pop3-new-test-release
else
exec /usr/libexec/dovecot/pop3
[..]
(gdb) bt full
#0 0x006ed402 in __kernel_vsyscall ()
No symbol table info available.
#1 0x0043ed20 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0x00440631 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080e44a8 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0)
at failures.c:150
backtrace = 0x982b160 "/usr/libexec/dovecot/pop3 [0x80e4486]
-> /usr/libexec/dovecot/pop3(default_error_handler+0) [0x80e4506] ->
/usr/libexec/dovecot/pop3(i_fatal+0) [0x80e4600] ->
/usr/libexec/dovecot/.
#4 0x080e4506 in default_fatal_handler (type=LOG_TYPE_PANIC,
status=0, format=0x810f568 "Leaked file fd %d: dev %s.%s inode %s",
args=0xbfee57a4 "\004") at failures.c:162
No locals.
#5 0x080e4600 in i_panic (format=0x810f568 "Leaked file fd %d: dev
%s.%s inode %s") at failures.c:197
args = 0xbfee57a4 "\004"
#6 0x080e519d in fd_debug_verify_leaks (first_fd=4, last_fd=1024) at
fd-close-on-exec.c:71
addr = {family = 58360, u = {ip6 = {in6_u = {u6_addr8 =
"\225�\017\b\a\000\000\000�XGD��\017\b", u6_addr16 = {58005,
2063, 7, 0, 22776, 17479, 58362, 2063}, u6_addr32 = {135258773, 7,
1145526520,
135259130}}}, ip4 = {s_addr = 135258773}}}
raddr = {family = 2, u = {ip6 = {in6_u = {u6_addr8 =
"\031^��Y��X�\005\021D", u6_addr16 = {24089, 49134, 23028,
49134, 22744, 49134, 4357, 68}, u6_addr32 = {3220069913, 3220068852,
3220068568,
4460805}}}, ip4 = {s_addr = 3220069913}}}
port = 4274100
rport = 3220068600
st = {st_dev = 26626, __pad1 = 0, __st_ino = 3342766, st_mode
= 33152, st_nlink = 1, st_uid = 0, st_gid = 0, st_rdev = 0, __pad2 =
0, st_size = 6654265, st_blksize = 4096, st_blocks = 13032, st_atim =
{
tv_sec = 1221681411, tv_nsec = 0}, st_mtim = {tv_sec = 1221683098,
tv_nsec = 0}, st_ctim = {tv_sec = 1221683098, tv_nsec = 0}, st_ino =
3342766}
old_errno = 9
#7 0x0805c817 in main (argc=1, argv=0xbfee5994, envp=0xbfee599c) at main.c:257
No locals.
Re: [Dovecot] lately pop3 with #define DEBUG needs GDB=1
On Wed, Sep 17, 2008 at 11:10 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > a) make the pop3 process sleep at startup so you can do ls > -l /proc/pid/fd/4 Pop3 is fine, it was my fault, I left a file descriptor open. Sorry for the noise. Diego.
Re: [Dovecot] Dovecot performance on GFS clustered filesystem
I've read somewhere that one of gfs2 goals was to improve performance for directory access with many files. I've tested it doing a simple ls in a directory with many test empty files in gfs and it was _really_ slow, doing the ls on a gfs2 with the same amount of emtpy files is actually faster. But when I tested gfs2 with bonnie++ I got fewer sequential I/O speed than in gfs (consider that I tested a beta version of gfs2 some months ago, maybe things are better now). So my conclusion of the tests was that gfs is best with mbox, gfs2 beta with maildir. But, again, I haven't tested gfs2 improvements recently. Regards, Diego.
[Dovecot] FETCH for mailbox mailboxname UID #1 got too little data: #2 vs #3
I got it with multiple imaptest instances even with current dovecot-1.1 hg tree. I checked the emails with that UIDs and they are actually truncated. Some things I noted on these mails: - they are all with MIME multipart attachments. - the last multipart attachment is truncated - the truncated last line is not complete - the following line is the beginning of a new mail (From line) without any empty line after the truncated attachment. - in "got too little data: #2 vs #3" 0 <= #2 < #3 (zero included) - Content-Length: #4 is sometimes bigger than both #2 and #3, sometimes smaller. Regards, Diego.
Re: [Dovecot] assertion in dovecot imap 1.1.1 to 1.1.3
2008/10/1 Rene Luria <[EMAIL PROTECTED]>: > Dovecot dies with signal 11 (segfault) when doing some commands with a > specific message Could you post a backtrace (bt full) of the core file? See: http://dovecot.org/bugreport.html Regards, Diego.
Re: [Dovecot] assertion failure in current hg, file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (1.1.3 was working fine)
On Mon, Sep 22, 2008 at 9:56 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > On Wed, 2008-09-17 at 22:01 +0200, Diego Liziero wrote: > > On Wed, Sep 17, 2008 at 9:35 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > > > > > > Could you do: > > > > > >> #6 0x080fcb64 in i_stream_read_data (stream=0x9094a08, > > >> data_r=0xbfe6, size_r=0xbfe64440, threshold=0) at istream.c:303 > > >> ret = -1 > > >> read_more = false > > >> __PRETTY_FUNCTION__ = "i_stream_read_data" > > > > > > fr 6 > > > p *stream > > > p *stream.real_stream > > > > (gdb) fr 6 > > #6 0x080fcb64 in i_stream_read_data (stream=0x9094a08, > > data_r=0xbfe6, size_r=0xbfe64440, threshold=0) at istream.c:303 > > 303 i_assert(stream->stream_errno != 0); > > I'm still not really sure why this is happening. I recently added some > more asserts to v1.2 code tree. Now that I've fixed several bugs in them > maybe they're stable enough for v1.1 too. Does this move the assert to > come earlier? http://hg.dovecot.org/dovecot-1.1/rev/a1a14d67b15d Not sure if it's related, but the only assertion that is failing now is this one (sorry, this time I've no core file): file index-mail.c: line 1091 (index_mail_close): assertion failed: (!mail->data.destroyi ng_stream) Raw backtrace: /usr/libexec/dovecot/imap [0x80f6a6a] -> /usr/libexec/dovecot/imap [0x80f732b] -> /usr/libexec/dovecot/imap(i_fatal+0) [0x80f6be4] -> /usr/libexec/dovecot/imap(index_mail_close+0xf9) [0x80a8489] -> /usr/libexec/dovecot/imap(index_mail_free+0x1a) [0x80a8abd] -> /usr/libexec/dovecot/imap(mail_free+0x1e) [0x80b3cb0] -> /usr/libexec/dovecot/imap [0x805b7f0] -> /usr/libexec/dovecot/imap(cmd_copy+0x1c4) [0x805ba0e] -> /usr/libexec/dovecot/imap(cmd_uid+0xbb) [0x8061173] -> /usr/libexec/dovecot/imap [0x8062534] -> /usr/libexec/dovecot/imap [0x8062769] -> /usr/libexec/dovecot/imap [0x8062867] -> /usr/libexec/dovecot/imap [0x80628a3] -> /usr/libexec/dovecot/imap(client_input+0xb7) [0x8062a31] -> /usr/libexec/dovecot/imap(io_loop_handler_run+0x17d) [0x8101485] -> /usr/libexec/dovecot/imap(io_loop_run+0x35) [0x810071c] -> /usr/libexec/dovecot/imap(main+0xe4) [0x806debd] -> /lib/libc.so.6(__libc_start_main+0xdc) [0x42bdec] -> /usr/libexec/dovecot/imap [0x805a221] Regards, Diego.
Re: [Dovecot] assertion failure in current hg, file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (1.1.3 was working fine)
On Sun, Oct 5, 2008 at 4:22 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: > > Not sure if it's related, but the only assertion that is failing now > is this one (sorry, this time I've no core file): > > file index-mail.c: line 1091 (index_mail_close): assertion failed: > (!mail->data.destroyi > ng_stream) Sorry for the noise, forget my previous mail. It's again my fault. This happened because some users were over fs quota (without the quota plugin). And that's why I've no core file (I use the same file system for mail & core files) Regards, Diego.
Re: [Dovecot] assertion failure in current hg, file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (1.1.3 was working fine)
On Sun, Oct 5, 2008 at 5:14 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Sun, 2008-10-05 at 17:00 +0200, Diego Liziero wrote: >> > >> > file index-mail.c: line 1091 (index_mail_close): assertion failed: >> > (!mail->data.destroying_stream) > > Hmm. So that assert could be because Dovecot ran out of disk space? I've > been trying to figure out several times why it would happen, but I > haven't been able to. Not the asset in the subject, but "file index-mail.c: line 1091 (index_mail_close): assertion failed: (!mail->data.destroying_stream)" means "Dovecot ran out of disk space". The assert in the subject here didn't happen any longer, maybe it has been solved with one of your recent fixes. Regards, Diego.
Re: [Dovecot] FETCH for mailbox mailboxname UID #1 got too little data: #2 vs #3
On Sun, Oct 5, 2008 at 3:38 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Sun, 2008-10-05 at 15:27 +0300, Timo Sirainen wrote: >> On Wed, 2008-09-24 at 22:35 +0200, Diego Liziero wrote: >> > I got it with multiple imaptest instances even with current dovecot-1.1 hg >> > tree. >> > >> > I checked the emails with that UIDs and they are actually truncated. >> > >> > Some things I noted on these mails: >> > - they are all with MIME multipart attachments. >> > - the last multipart attachment is truncated >> > - the truncated last line is not complete >> > - the following line is the beginning of a new mail (From line) >> > without any empty line after the truncated attachment. >> > - in "got too little data: #2 vs #3" 0 <= #2 < #3 (zero included) >> > - Content-Length: #4 is sometimes bigger than both #2 and #3, sometimes >> > smaller. >> >> Any idea how the messages got there? Via deliver or IMAP APPEND, IMAP >> COPY or something else like procmail? Here procmail uses just INBOX folder, this error happens also to Sent and Trash folders that are used just by dovecot. > Maybe this helps: http://hg.dovecot.org/dovecot-1.1/rev/f55e58146f77 Just got this with current hg and imaptest: Error: IMAP(username): FETCH for mailbox Trash UID 215597 got too little data: 469042 vs 470768 But I'm not using a new user. I'm using the usual test user without deleting any mbox or index. Maybe that UID got corrupted by a previous dovecot version. Now I start testing with a clean user and I'll let you know if it happens again.
Re: [Dovecot] FETCH for mailbox mailboxname UID #1 got too little data: #2 vs #3
On Sun, Oct 5, 2008 at 6:34 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: > On Sun, Oct 5, 2008 at 3:38 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: >> On Sun, 2008-10-05 at 15:27 +0300, Timo Sirainen wrote: >>> On Wed, 2008-09-24 at 22:35 +0200, Diego Liziero wrote: >>> > I got it with multiple imaptest instances even with current dovecot-1.1 >>> > hg tree. >>> > >>> > I checked the emails with that UIDs and they are actually truncated. >>> > >>> > Some things I noted on these mails: >>> > - they are all with MIME multipart attachments. >>> > - the last multipart attachment is truncated >>> > - the truncated last line is not complete >>> > - the following line is the beginning of a new mail (From line) >>> > without any empty line after the truncated attachment. >>> > - in "got too little data: #2 vs #3" 0 <= #2 < #3 (zero included) >>> > - Content-Length: #4 is sometimes bigger than both #2 and #3, sometimes >>> > smaller. > >> Maybe this helps: http://hg.dovecot.org/dovecot-1.1/rev/f55e58146f77 > [..] > Now I start testing with a clean user and I'll let you know if it happens > again. Unfortunately it happens also with a new user and using just multiple imaptest instances (no procmail involved here): dovecot: Oct 05 19:25:24 Error: IMAP(newuser): FETCH for mailbox INBOX UID 1636 got too little data: 322644 vs 324628
Re: [Dovecot] Disabling global content_filter with an empty filter specified with an access table
On Wed, Oct 8, 2008 at 7:36 PM, mouss <[EMAIL PROTECTED]> wrote: > > an alternative is > > content_filter = > smtpd_sender_restrictions = >check_sender_access hash:/etc/postfix/sender_ok >check_sender_access pcre:/etc/postfix/filter > > == dsn_ok > <> OK > > == filter > /./ FILTER filter:[1.2.3.4]:10024 > > where "filter:[1.2.3" is what you used to put in content_filter. Yes, that's what I was thinking when I wrote "workaround" in my first mail: On Wed, Oct 8, 2008 at 5:08 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: >[..] >I think that this solution is more readable than the only workaround I >can imagine now (that is disabling global filtering and enabling it in >a pcre table for everything except that particular case). So it's a bit less readable, I've to remember to disable sender restrictions in the content filter return transport to avoid filter loops, there will be one more line for each mail in the log stating that the filter is triggered, but apart from that it should work as I need. Thank you to everyone that helped me, tomorrow I'm going to test both methods. Regards, Diego.
Re: [Dovecot] Dovecot performance on GFS clustered filesystem
On Wed, Sep 24, 2008 at 9:18 PM, Diego Liziero <[EMAIL PROTECTED]> wrote: > I've read somewhere that one of gfs2 goals was to improve performance > for directory access with many files. > > I've tested it doing a simple ls in a directory with many test empty > files in gfs and it was _really_ slow, doing the ls on a gfs2 with the > same amount of emtpy files is actually faster. > > But when I tested gfs2 with bonnie++ I got fewer sequential I/O speed > than in gfs (consider that I tested a beta version of gfs2 some months > ago, maybe things are better now). This seems true only for sequential write speed. > So my conclusion of the tests was that gfs is best with mbox, gfs2 > beta with maildir. > > But, again, I haven't tested gfs2 improvements recently. > > Regards, > Diego. I lauched bonnie++ on the same lvm2 partition formatted with different filesystems to test their speed. Consider that I launced the test just one time per filesystem, so the numbers should not be considered sharp, but with a certain percentage of error. With clustered filesystems (gfs and gfs2) the test has been launched on the same box, but with the fillesystem mounted on two nodes. Option used to format gfs and gfs2: "-r 2048 -p lock_dlm -j 4" The test box has two dual-core Opteron processors, 8 Gb of ram, two 4Gb fiber channel HBA, gigabit ethernet (for the distributed lock manager connection) and Centos 5.2 x86_64 installed. With this configuration it seems that: gfs is faster than gfs2 when writing sequential blocks, much faster in creating and deleting files; gfs2 seems to have a faster read speed. Regards, Diego. bonnie++ -s 16g Version 1.03c --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Filesystem Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP ext216G 69230 95 149430 47 54849 25 71681 90 215568 54 473.3 1 xfs 16G 62828 94 135482 54 64010 28 71841 92 238351 51 632.3 2 ext316G 56485 93 115398 68 56051 32 73536 92 211219 48 552.0 2 gfs 16G 47079 98 124123 82 42651 53 65692 91 189533 65 431.5 3 gfs216G 40203 77 74620 53 52596 39 73187 93 226909 58 496.2 2 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP ext2 16 3810 99 + +++ + +++ 2395 99 + +++ 10099 99 xfs 16 2446 51 + +++ 1073 24 2019 59 + +++ 796 21 ext3 16 5597 77 + +++ 16764 99 19450 99 + +++ + +++ gfs 16 882 40 + +++ 4408 82 1001 56 + +++ 2734 69 gfs2 16 457 0 + +++ 994 0 477 0 + +++ 995 29
[Dovecot] dovecot 1.1.4 maildir imap segfault in message_parse_header_next
I've tried to stress test dovecot 1.1.4 with imaptest for days without
any assertion failure or crash.
Just some "got too little data" messages.
So far it's the most stable 1.1.x version.
Today a user got this imap segfault with vanilla 1.1.4 (I don't know
if it's something you have already fixed in current tree).
The user didn't complain of anything, I've just found the error in the
logs and the core file.
Regards,
Diego.
Core was generated by `/usr/libexec/dovecot/imap'.
Program terminated with signal 11, Segmentation fault.
#0 0x080c8d41 in message_parse_header_next (ctx=0x8774fa0,
hdr_r=0xbfa438e0) at message-header-parser.c:114
114 if (msg[0] == '\n' ||
(gdb) bt full
#0 0x080c8d41 in message_parse_header_next (ctx=0x8774fa0,
hdr_r=0xbfa438e0) at message-header-parser.c:114
msg = (const unsigned char *) 0x0
i =
size = 0
startpos =
colon_pos = 4294967295
parse_size = 0
value_pos =
ret = -2
continued = false
continues =
crlf_newline = false
#1 0x080c62f5 in read_header (mstream=0x877d6d0) at istream-header-filter.c:163
hdr = (struct message_header_line *) 0x0
highwater_offset =
pos =
ret =
matched = false
hdr_ret =
__PRETTY_FUNCTION__ = '\0'
#2 0x080c6a17 in i_stream_header_filter_read (stream=0x877d6d0) at
istream-header-filter.c:293
mstream = (struct header_filter_istream *) 0xfffe
ret =
pos =
#3 0x080d4fa8 in i_stream_read (stream=0x877d6f8) at istream.c:73
_stream = (struct istream_private *) 0xfffe
ret =
__PRETTY_FUNCTION__ = '\0'
#4 0x080d505d in i_stream_read_data (stream=0x877d6f8,
data_r=0xbfa439a8, size_r=0xbfa439a4, threshold=0) at istream.c:299
ret = 0
read_more = false
__PRETTY_FUNCTION__ = '\0'
#5 0x080cb8ec in message_get_body_size (input=0x877d6f8,
body=0xbfa439d8, has_nuls=0x0) at message-size.c:76
msg =
i =
size =
missing_cr_count =
__PRETTY_FUNCTION__ = '\0'
#6 0x08064178 in fetch_body_header_fields (ctx=0x875e668,
mail=0x8776138, body=0x875e958) at imap-fetch-body.c:458
size = {physical_size = 0, virtual_size = 0, lines = 0}
old_offset = 0
#7 0x08062218 in imap_fetch (ctx=0x875e668) at imap-fetch.c:309
_data_stack_cur_id = 4
ret =
__PRETTY_FUNCTION__ = "\000\000\000\000\000\000\000\000\000\000"
#8 0x0805bd9e in cmd_fetch (cmd=0x875e5c0) at cmd-fetch.c:152
ctx = (struct imap_fetch_context *) 0x875e668
args = (const struct imap_arg *) 0x8762638
search_arg = (struct mail_search_arg *) 0x875e610
messageset =
ret =
#9 0x0805fe8c in client_command_input (cmd=0x875e5c0) at client.c:580
client = (struct client *) 0x875e368
command =
__PRETTY_FUNCTION__ = '\0'
#10 0x0805ff35 in client_command_input (cmd=0x875e5c0) at client.c:629
client = (struct client *) 0x875e368
command = (struct command *) 0x0
__PRETTY_FUNCTION__ = '\0'
#11 0x080606f5 in client_handle_input (client=0x875e368) at client.c:670
_data_stack_cur_id = 3
ret =
remove_io =
handled_commands = false
#12 0x0806090e in client_input (client=0x875e368) at client.c:725
cmd =
output = (struct ostream *) 0x875e4ec
bytes = 197
__PRETTY_FUNCTION__ = '\0'
#13 0x080d8710 in io_loop_handler_run (ioloop=0x875c9b0) at ioloop-epoll.c:203
ctx =
event = (const struct epoll_event *) 0x875cae8
list = (struct io_list *) 0x875e568
io = (struct io_file *) 0x875e548
tv = {tv_sec = 4, tv_usec = 926334}
t_id = 2
msecs =
ret = 1
i = 0
j = 0
call =
#14 0x080d77f8 in io_loop_run (ioloop=0x875c9b0) at ioloop.c:320
No locals.
#15 0x0806848c in main (argc=Cannot access memory at address 0x0
) at main.c:293
No locals.
Re: [Dovecot] assertion failure in current hg, file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (1.1.3 was working fine)
On Sun, Oct 5, 2008 at 5:44 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote: > On Sun, 2008-10-05 at 17:23 +0200, Diego Liziero wrote: >> >> Not the asset in the subject, but "file index-mail.c: line 1091 >> (index_mail_close): assertion failed: (!mail->data.destroying_stream)" >> means "Dovecot ran out of disk space". > > Right. And that had been annoying me for a long time now. Fixed finally: > http://hg.dovecot.org/dovecot-1.1/rev/3b0d23902a32 Another "Dovecot ran out of disk space" (if you feel like it's worth fixing it) (sorry, no core file as user was out of fs quota) file mbox-sync-rewrite.c: line 590 (mbox_sync_rewrite): assertion failed: (mails[idx].from_offset == start_offset) Raw backtrace: /usr/libexec/dovecot/imap [0x80d0ae0] -> /usr/libexec/dovecot/imap [0x80d0b3a] -> /usr/libexec/dovecot/imap [0x80d03cc] -> /usr/libexec/dovecot/imap(mbox_sync_rewrite+0xe05) [0x8086675] -> /usr/libexec/dovecot/imap(mbox_sync+0x140c) [0x80845ec] -> /usr/libexec/dovecot/imap(mbox_storage_sync_init+0x47) [0x8084b87] -> /usr/libexec/dovecot/imap(imap_sync_init+0x49) [0x8066889] -> /usr/libexec/dovecot/imap(cmd_sync_delayed+0x1a9) [0x8066ad9] -> /usr/libexec/dovecot/imap [0x806083e] -> /usr/libexec/dovecot/imap(client_input+0x5e) [0x806090e] -> /usr/libexec/dovecot/imap(io_loop_handler_run+0x100) [0x80d8710] -> /usr/libexec/dovecot/imap(io_loop_run+0x28) [0x80d77f8] -> /usr/libexec/dovecot/imap(main+0x4ac) [0x806848c] -> /lib/libc.so.6(__libc_start_main+0xdc) [0x42bdec] -> /usr/libexec/dovecot/imap [0x805a1f1] Diego
Re: [Dovecot] dovecot 1.1.4 maildir imap segfault in message_parse_header_next
On Thu, Oct 16, 2008 at 11:39 AM, Timo Sirainen <[EMAIL PROTECTED]> wrote:
> On Oct 16, 2008, at 11:33 AM, Diego Liziero wrote:
>
>> Today a user got this imap segfault with vanilla 1.1.4 (I don't know
>
> Hmm. And Maildir as topic says?
No, sorry, wrong subject, mbox
>> #0 0x080c8d41 in message_parse_header_next (ctx=0x8774fa0,
>> hdr_r=0xbfa438e0) at message-header-parser.c:114
>
> p *ctx.input
> p *ctx.input.real_stream
(gdb) p *ctx.input
$1 = {v_offset = 0, stream_errno = 0, mmaped = 0, blocking = 1, closed
= 0, seekable = 1, eof = 0, real_stream = 0x8771538}
(gdb) p *ctx.input.real_stream
$2 = {iostream = {refcount = 3, close = 0x80e3f10
,
destroy = 0x80c6d50 ,
set_max_buffer_size = 0x80c6d20
,
destroy_callback = 0x8094630 ,
destroy_context = 0x8776138},
read = 0x80c6940 , seek = 0x80c6be0
,
sync = 0x80c5fa0 , stat = 0x80c6b40
, istream = {v_offset = 0,
stream_errno = 0, mmaped = 0, blocking = 1, closed = 0, seekable =
1, eof = 0, real_stream = 0x8771538}, fd = -1,
abs_start_offset = 374333755, statbuf = {st_dev = 0, __pad1 = 0,
__st_ino = 0, st_mode = 0, st_nlink = 0, st_uid = 0, st_gid = 0,
st_rdev = 0, __pad2 = 0, st_size = -1, st_blksize = 0, st_blocks =
0, st_atim = {tv_sec = 1224104682, tv_nsec = 0}, st_mtim = {
tv_sec = 1224104682, tv_nsec = 0}, st_ctim = {tv_sec =
1224104682, tv_nsec = 0}, st_ino = 0}, buffer = 0x0, w_buffer = 0x0,
buffer_size = 0, max_buffer_size = 8192, skip = 0, pos = 0, parent =
0x8770fe0, parent_start_offset = 0, line_str = 0x0}
>> size = 0
>
> i_stream_read_data() returned 0 bytes, but
>
>> ret = -2
>
> it also returned that the input buffer is full. That shouldn't be happening.
> http://hg.dovecot.org/dovecot-1.1/rev/82d4756f43cc should catch it earlier.
Ok thanks.
[Dovecot] Proxying pop3 sessions into an imap one.
I know it seems at least unusual. But I would like to know if someone knows a software that can proxy multiple pop3 and imap connections to the same account and use only imap connections to the real server. I need it because I've to deal with two pieces of software that are out of my control: - 1 - A certified mail provider that states something that I would like to know if it's really true, or if it's worth keeping like a funny signature: "multiple imap sessions are possible. When an imap session is active, no pop3 connections are possible on the same account; when no imap connection is active, only one pop3 connection at a time is possible on the same account. This is a limitation of the pop3 protocol." - 2 - A software written to deal with certified mails that supports only pop3 used by many people on the same account, (and the same account is used also by multiple imap mail readers). Regards, Diego.
Re: [Dovecot] Proxying pop3 sessions into an imap one.
On Wed, Oct 22, 2008 at 12:49 PM, Sotiris Tsimbonis <[EMAIL PROTECTED]> wrote: > > I don't think that you'll find any piece of software that internally > 'translates' pop3 commands to imap .. Unless it saves a local copy of the mailbox... > Pop3 sessions are usually locked, so multiple sessions are not possible.. > Unless something like the following is set in dovecot.conf > > # Keep the mailbox locked for the entire POP3 session. > pop3_lock_session = no > > but this is dovecot specific (does this mail provider use dovecot?). No idea about the provider. > Also, I believe that behaviour will probably by uncertain if, say, multiple > pop3 clients try to modify the same (single) mbox file.. So it also depends > on the mailbox format you are trying to access, maybe Maildir or dbox are > better.. > > [..] > > Again.. Many people using pop3.. Depends on mail server software and mailbox > format.. IMAP does multiple sessions better. Just thinking aloud... Could be a solution if I keep a local copy of the mailbox fetching it somehow through imap or pop3 from the provider and then use dovecot with "pop3_lock_session = no" to serve the clients? I'm not sure of it just because I don't know how the certified email internally works. Thank you for your help, Diego.
[Dovecot] assertion failed in 1.1.7 file mbox-sync.c: line 1305 (mbox_sync_handle_eof_updates)
Dovecot 1.1.7 is running so smoothly that I gave up checking its log
files daily. :)
I've just had a look, and among the usual
"IMAP(username): FETCH for mailbox Sent UID xx got too little data: xx vs xx"
messages (that means that unfortunately sometimes some messages are
still written truncated) I saw this assertion failure:
file mbox-sync.c: line 1305 (mbox_sync_handle_eof_updates): assertion
failed: (file_size >=
sync_ctx->expunged_space + trailer_size)
Regards,
Diego.
(gdb) bt full
#0 0x008a9402 in __kernel_vsyscall ()
No symbol table info available.
#1 0x00a2fd20 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0x00a31631 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080f73c0 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0)
at failures.c:150
backtrace = 0x97df358 "/usr/libexec/dovecot/imap [0x80f739e] ->
/usr/libexec/dovecot/imap [0x80f7c5f] ->
/usr/libexec/dovecot/imap(i_fatal+0) [0x80f7518] ->
/usr/libexec/dovecot/imap [0x80911de] -> /usr/libexec/dovecot/imap"...
#4 0x080f7c5f in i_internal_fatal_handler (type=LOG_TYPE_PANIC,
status=0, fmt=0x811771c "file %s: line %d (%s): assertion failed:
(%s)",
args=0xbfbabf14 "\020w\021\b\031\005") at failures.c:430
No locals.
#5 0x080f7518 in i_panic (format=0x811771c "file %s: line %d (%s):
assertion failed: (%s)") at failures.c:197
args = 0xbfbabf14 "\020w\021\b\031\005"
#6 0x080911de in mbox_sync_handle_eof_updates (sync_ctx=0xbfbac0f4,
mail_ctx=0xbfbac008) at mbox-sync.c:1305
st = (const struct stat *) 0x97f3420
file_size = 242
offset = 242
padding = 684287899293748052
trailer_size = 0
__PRETTY_FUNCTION__ = "mbox_sync_handle_eof_updates"
#7 0x08091d77 in mbox_sync_do (sync_ctx=0xbfbac0f4, flags=0) at
mbox-sync.c:1547
mail_ctx = {sync_ctx = 0xbfbac0f4, mail = {uid = 0, idx_seq = 1,
keywords = {arr = {buffer = 0x0, element_size = 0}, v = 0x0,
v_modifiable = 0x0}, flags = 12 '\f', uid_broken = 0, expunged =
1, pseudo = 0, from_offset = 0, body_size = 0, offset = 0,
space = 243}, seq = 1, hdr_offset = 66, body_offset = 151,
header_first_change = 4294967295, header_last_change = 0,
header = 0x9800878, hdr_md5_sum =
"�\035\214�\217\000�\004�\200\t\230��B~", content_length =
18446744073709551615, hdr_pos = {69, 12,
4294967295, 82, 32}, parsed_uid = 2, last_uid_updated_value = 0,
last_uid_value_start_pos = 2, have_eoh = 1, need_rewrite = 0,
seen_imapbase = 1, updated = 0, recent = 0, dirty = 0,
imapbase_rewrite = 0, imapbase_updated = 0}
st = (const struct stat *) 0x97f3420
i = 0
ret = 1
partial = 1
#8 0x0809277e in mbox_sync_int (mbox=0x97f0c28, flags=0) at mbox-sync.c:1806
index_sync_ctx = (struct mail_index_sync_ctx *) 0x9808b38
sync_view = (struct mail_index_view *) 0x97f1648
trans = (struct mail_index_transaction *) 0x97f3168
sync_ctx = {mbox = 0x97f0c28, flags = 0, input = 0x97f34e8,
file_input = 0x97f3400, write_fd = 8, orig_mtime = 1227954336,
orig_atime = 1227954336, orig_size = 242, last_stat = {st_dev =
37637, __pad1 = 0, __st_ino = 4751377, st_mode = 33152, st_nlink = 1,
st_uid = 631, st_gid = 508, st_rdev = 0, __pad2 = 0, st_size =
242, st_blksize = 4096, st_blocks = 16, st_atim = {
tv_sec = 1227960259, tv_nsec = 0}, st_mtim = {tv_sec =
1227954336, tv_nsec = 0}, st_ctim = {tv_sec = 1227954336, tv_nsec =
0},
st_ino = 4751377}, index_sync_ctx = 0x9808b38, sync_view =
0x97f1648, t = 0x97f3168, reset_hdr = {major_version = 0 '\0',
minor_version = 0 '\0', base_header_size = 0, header_size = 0,
record_size = 0, compat_flags = 0 '\0', unused = "\000\000",
indexid = 0, flags = 0, uid_validity = 0, next_uid = 0,
messages_count = 0, unused_old_recent_messages_count = 0,
seen_messages_count = 0, deleted_messages_count = 0,
first_recent_uid = 0, first_unseen_uid_lowwater = 0,
first_deleted_uid_lowwater = 0, log_file_seq = 0,
log_file_tail_offset = 0, log_file_head_offset = 0, sync_size = 0,
sync_stamp = 0,
day_stamp = 0, day_first_uid = {0, 0, 0, 0, 0, 0, 0, 0}}, hdr =
0x97f1300, header = 0x9800878, from_line = 0x9800858,
base_uid_validity = 1, base_uid_last = 2, base_uid_last_offset = 0,
mails = {arr = {buffer = 0x97f8c40, element_size = 52},
v = 0x97f8c40, v_modifiable = 0x97f8c40}, sync_changes =
0x97f8c60, mail_keyword_pool = 0x9800168, saved_keywords_pool =
0x97f50d8,
prev_msg_uid = 2, next_uid = 3, idx_next_uid = 3, seq = 1, idx_seq =
2, need_space_seq = 0, last_nonrecent_uid = 0,
expunged_space = 243, space_diff = 0, dest_first_mail = 1,
first_mail_crlf_expunged = 0, delay_writes = 1, renumber_uids = 0,
moved_offsets = 0, ext_modified = 0, index_reset = 0, errors = 0}
sync_flags = MAIL_INDEX_SYNC_FLAG_DROP_RECENT
lock_id = 23
ret = 1
changed = 1
delay_writes = true
__PRETTY_FUNCTION__ = "mbox_sync_int"
#9 0x08092a0a in mbox_sync (mbox=0x97f
Re: [Dovecot] assertion failed in 1.1.7 file mbox-sync.c: line 1305 (mbox_sync_handle_eof_updates)
On Thu, Dec 4, 2008 at 8:26 AM, Diego Liziero wrote: > Dovecot 1.1.7 is running so smoothly that I gave up checking its log > files daily. :) > > I've just had a look, and among the usual > "IMAP(username): FETCH for mailbox Sent UID xx got too little data: xx vs xx" > messages (that means that unfortunately sometimes some messages are > still written truncated) I saw this assertion failure: > > file mbox-sync.c: line 1305 (mbox_sync_handle_eof_updates): assertion > failed: (file_size >= > sync_ctx->expunged_space + trailer_size) Btw: (gdb) fr 6 #6 0x080911de in mbox_sync_handle_eof_updates (sync_ctx=0xbfbac0f4, mail_ctx=0xbfbac008) at mbox-sync.c:1305 1305i_assert(file_size >= sync_ctx->expunged_space + trailer_size); (gdb) print file_size $1 = 242 (gdb) print sync_ctx->expunged_space $2 = 243 (gdb) print trailer_size $3 = 0 Regards, Diego.
[Dovecot] another assertion failure in dovecot 1.1.7 mbox-sync-rewrite: (mails[idx].from_offset == start_offset)
Sorry, this time I've no core file, (I forgot to set ulimit -c unlimited before starting dovecot) Regards, Diego. --- dovecot: Dec 09 08:26:52 Panic: IMAP(user): file mbox-sync-rewrite.c: line 590 (mbox_sync_rewrite): assertion failed: (mails[idx].from_offset == start_offset) dovecot: Dec 09 08:26:52 Error: IMAP(user): Raw backtrace: /usr/libexec/dovecot/imap [0x80f739e] -> /usr/libexec/dovecot/imap [0x80f7c5f] -> /usr/libexec/dovecot/imap(i_fatal+0) [0x80f7518] -> /usr/libexec/dovecot/imap(mbox_sync_rewrite+0x6e8) [0x8094e45] -> /usr/libexec/dovecot/imap [0x809105c] -> /usr/libexec/dovecot/imap [0x8091d77] -> /usr/libexec/dovecot/imap [0x809277e] -> /usr/libexec/dovecot/imap(mbox_sync+0x2b) [0x8092a0a] -> /usr/libexec/dovecot/imap [0x8084ff3] -> /usr/libexec/dovecot/imap(mailbox_close+0x47) [0x80b701c] -> /usr/libexec/dovecot/imap(cmd_close+0xc0) [0x805b5d8] -> /usr/libexec/dovecot/imap [0x80625c4] -> /usr/libexec/dovecot/imap [0x80627f9] -> /usr/libexec/dovecot/imap [0x80628f7] -> /usr/libexec/dovecot/imap [0x8062933] -> /usr/libexec/dovecot/imap(client_input+0xb7) [0x8062ac1] -> /usr/libexec/dovecot/imap(io_loop_handler_run+0x17d) [0x8101ecd] -> /usr/libexec/dovecot/imap(io_loop_run+0x35) [0x8101164] -> /usr/libexec/dovecot/imap(main+0xb0) [0x806df19] -> /lib/libc.so.6(__libc_start_main+0xdc) [0x7a5dec] -> /usr/libexec/dovecot/imap [0x805a2b1]
Re: [Dovecot] another assertion failure in dovecot 1.1.7 mbox-sync-rewrite: (mails[idx].from_offset == start_offset)
On Sat, Dec 13, 2008 at 5:16 PM, Timo Sirainen wrote: > > I did several mbox fixes today. Wonder if they could have fixed also > some of these other mbox bugs you've reported? I'm stress-testing current 1.1.x mercurial head with imaptest and with my account. So far everything seems fine. It sounds great if you managed to fix these last rare bugs. Thanks for your work Timo, I'll let you know if anything wrong occurs. btw, did you have time to have a look at multiappend valgrind stuff I sent you? regards, Diego.
Re: [Dovecot] another assertion failure in dovecot 1.1.7 mbox-sync-rewrite: (mails[idx].from_offset == start_offset)
On Sat, Dec 13, 2008 at 4:45 PM, Diego Liziero wrote: > Sorry, this time I've no core file, (I forgot to set ulimit -c > unlimited before starting dovecot) No, I didn't forget, I got it again without core file because this is another "disk full" assertion failure. Both users that got it were over quota. So probably nothing to worry about. Regards, Diego. > --- > dovecot: Dec 09 08:26:52 Panic: IMAP(user): file mbox-sync-rewrite.c: > line 590 (mbox_sync_rewrite): assertion failed: > (mails[idx].from_offset == start_offset) > dovecot: Dec 09 08:26:52 Error: IMAP(user): Raw backtrace: > /usr/libexec/dovecot/imap [0x80f739e] -> /usr/libexec/dovecot/imap > [0x80f7c5f] -> /usr/libexec/dovecot/imap(i_fatal+0) [0x80f7518] -> > /usr/libexec/dovecot/imap(mbox_sync_rewrite+0x6e8) [0x8094e45] -> > /usr/libexec/dovecot/imap [0x809105c] -> /usr/libexec/dovecot/imap > [0x8091d77] -> /usr/libexec/dovecot/imap [0x809277e] -> > /usr/libexec/dovecot/imap(mbox_sync+0x2b) [0x8092a0a] -> > /usr/libexec/dovecot/imap [0x8084ff3] -> > /usr/libexec/dovecot/imap(mailbox_close+0x47) [0x80b701c] -> > /usr/libexec/dovecot/imap(cmd_close+0xc0) [0x805b5d8] -> > /usr/libexec/dovecot/imap [0x80625c4] -> /usr/libexec/dovecot/imap > [0x80627f9] -> /usr/libexec/dovecot/imap [0x80628f7] -> > /usr/libexec/dovecot/imap [0x8062933] -> > /usr/libexec/dovecot/imap(client_input+0xb7) [0x8062ac1] -> > /usr/libexec/dovecot/imap(io_loop_handler_run+0x17d) [0x8101ecd] -> > /usr/libexec/dovecot/imap(io_loop_run+0x35) [0x8101164] -> > /usr/libexec/dovecot/imap(main+0xb0) [0x806df19] -> > /lib/libc.so.6(__libc_start_main+0xdc) [0x7a5dec] -> > /usr/libexec/dovecot/imap [0x805a2b1] >
Re: [Dovecot] mbox "Next message unexpectedly lost" bug fixed
First the good news: with these patches, the "Next message unexpectedly lost" bugs I got so far were all caused reading a mbox written by a previous version of dovecot. So I think this time you really fixed the last annoying 1.1.x bug :) Then another doubt: with these patches I'm still getting some: FETCH for mailbox Sent UID 1077 got too little data: 1380 vs 3541 Corrupted index cache file /mailhome/username/.imap/Sent/dovecot.index.cache: Broken virtual size for mail UID 1077 These mails were written by a hg version pulled Dec 14 (that includes the mbox patches). Any suggestion for the kind of debug I can add to help solving these ones? Regards, Diego.
Re: [Dovecot] PDF corruption issue
On Sun, Jan 11, 2009 at 10:56 PM, Stefan Klatt wrote: > > Hi Ed, > >> I can basically confirm the same intermittent issue, but not where it's >> coming from > Same to me. > I found it sometimes at JPGs included at emails. > The most of the image shows correct but about the last 30 percent is > only grey. Could you please have a look if you are seeing messages like this in your dovecot.log? Error: IMAP(username): FETCH for mailbox Sent UID 2375 got too little data: 4096 vs 5247 (with different numbers) Thanks, Diego Liziero.
Re: [Dovecot] Corrupted index cache file
On Thu, Feb 12, 2009 at 5:39 PM, Charles Marcus wrote: > On 2/12/2009 3:31 AM, Frank Bonnet wrote: >> dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent >> UID 7139 got too little data: 2 vs 11160 >> dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): Corrupted index cache >> file /user/dumontj/.imap/Sent/dovecot.index.cache: Broken virtual size >> for mail UID 7139 >> >> Does remove the cache will cure the problem ? > > Version? There have been lots of mbox fixes in recent versions... I'm still getting this error in the 1.1.x hg tree (using a post 1.1.11 snapshot taken 6 days ago), happy to see I'm not the only one. It's really _rare_ and difficult to reproduce. Usually, it happens when a thunderbird user tells dovecot to store a mail with multipart mime attachments in a mbox folder (mostly Sent or Trash folder). When this error appears, the last mail attachment is actually truncated. Filesystem here is ext3 on a two node drbd active/passive cluster (Centos 5.2) Regards, Diego.
Re: [Dovecot] Corrupted index cache file
On Fri, Feb 13, 2009 at 12:32 AM, Timo Sirainen wrote: > On Fri, 2009-02-13 at 00:10 +0100, Diego Liziero wrote: >> >> dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent >> >> UID 7139 got too little data: 2 vs 11160 > .. >> When this error appears, the last mail attachment is actually truncated. > > Any idea if the saved data simply isn't written / is truncated, instead > of the following message overwriting it? i.e. has this happened to the > last message in the mbox file (with uid = next_uid-1 to make sure the > last message wasn't actually expunged)? Sorry, no idea. There is always a delay between when the error is written in dovecot.log and when I and ask the user what happened. The truncated message is always one of the latest in the mbox, but I can't tell if it happens when the message is the last one. Maybe a debug message that logs the latest mbox uid together with the "FETCH for mailbox Sent UID 7139 got too little data : 2 vs 11160" message could help. Or maybe a debug extra check that reads every newly written mbox mail if it's actually written up to the last byte... Diego.
