Re: Dovecort-2.2.22

2016-03-08 Thread Ralf Zimmermann
Here is the backtrace:

#0  0x7f23b5484067 in __GI_raise (sig=sig@entry=6) at 
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x7f23b5485448 in __GI_abort () at abort.c:89
#2  0x7f23b587b9e6 in default_fatal_finish (type=, 
status=status@entry=0) at failures.c:201
#3  0x7f23b587badc in i_internal_fatal_handler (ctx=0x7ffdc070e470, 
format=, args=) at failures.c:670
#4  0x7f23b582433d in i_panic (format=format@entry=0x7f23b481a158 "file %s: 
line %d (%s): assertion failed: (%s)") at failures.c:275
#5  0x7f23b48159f8 in virtual_backend_box_close (mbox=mbox@entry=0x1e068f0, 
bbox=0x1e5f6a8) at virtual-storage.c:403
#6  0x7f23b4815f94 in virtual_mailbox_close_internal 
(mbox=mbox@entry=0x1e068f0) at virtual-storage.c:445
#7  0x7f23b4815fe9 in virtual_mailbox_close (box=0x1e068f0) at 
virtual-storage.c:507
#8  0x7f23b5b3e5da in mailbox_close (box=0x3a5c) at mail-storage.c:1240
#9  0x7f23b5b3e663 in mailbox_free (_box=_box@entry=0x7ffdc070e5e8) at 
mail-storage.c:1260
#10 0x0042038a in imap_status_get (cmd=cmd@entry=0x1e02d20, 
ns=ns@entry=0x1df4dc0, mailbox=mailbox@entry=0x1dd02e8 "virtual/last_48_hours",
items=items@entry=0x7ffdc070e640, result_r=result_r@entry=0x7ffdc070e660) 
at imap-status.c:96
#11 0x00414498 in cmd_status (cmd=0x1e02d20) at cmd-status.c:40
#12 0x0041947c in command_exec (cmd=cmd@entry=0x1e02d20) at 
imap-commands.c:180
#13 0x00417ac2 in client_command_input (cmd=cmd@entry=0x1e02d20) at 
imap-client.c:958
#14 0x00417b50 in client_command_input (cmd=0x1e02d20) at 
imap-client.c:1018
#15 0x00417ec5 in client_handle_next_command (remove_io_r=, client=0x1e02120) at imap-client.c:1058
#16 client_handle_input (client=0x1e02120) at imap-client.c:1070
#17 0x00418362 in client_input (client=0x1e02120) at imap-client.c:1117
#18 0x7f23b588e9dc in io_loop_call_io (io=0x1e24780) at ioloop.c:564
#19 0x7f23b588fd01 in io_loop_handler_run_internal 
(ioloop=ioloop@entry=0x1dd8730) at ioloop-epoll.c:220
#20 0x7f23b588ea65 in io_loop_handler_run (ioloop=ioloop@entry=0x1dd8730) 
at ioloop.c:612
#21 0x7f23b588ec08 in io_loop_run (ioloop=0x1dd8730) at ioloop.c:588
#22 0x7f23b5829a23 in master_service_run (service=0x1dd85d0, 
callback=callback@entry=0x4239b0 ) at master-service.c:640
#23 0x0040c3c7 in main (argc=1, argv=0x1dd8390) at main.c:454

Mit freundlichen Grüßen

Ralf Zimmermann
Senior Security Engineer
State Certified Engineer

SIEGNETZ.IT GmbH
Einheitsstrasse 2, D-57076 Siegen
Telefon: +4927168193130 Fax: +492716819329
Mobil  : +491735360015
http://www.siegnetz.de
http://rz.siegnetz.de

Amtsgericht Siegen HRB4838
Geschäftsführer: Oliver Seitz
Sitz der Gesellschaft ist Siegen

> Am 04.03.2016 um 21:02 schrieb Timo Sirainen :
> 
> On 04 Mar 2016, at 17:46, Ralf Zimmermann  wrote:
>> 
>> With Dovecot-2.2.22 and enabled virtual plugin I get following error 
>> messages:
>> 
>> Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x819f0) 
>> [0x7f12330bf9f0] ->
> 
> The raw backtrace isn't very helpful unfortunately. What was the panic log 
> message before this? Also it could be helpful to have gdb backtrace: 
> http://dovecot.org/bugreport.html
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Dovecort-2.2.22

2016-03-08 Thread Ralf Zimmermann
I have activated the virtual plugin with dovecot-2.2.22 on a Debian Jessie with 
Linux 3.2.0-4-amd64.

In this example I opened the virtual folder Trash.

# ~Maildir/virtual/Trash/dovecot-virtual
*
  deleted

Here the mail debug log entry:

Panic: file virtual-storage.c: line 403 (virtual_backend_box_close): assertion 
failed: (bbox->open_tracked)
Mar  8 11:32:49 lab dovecot: imap(rzimmerm...@lab.de): Error: Raw backtrace: 
/usr/local/lib/dovecot/libdovecot.so.0(+0x819f0) [0x7f26e3c4c9f0] -> 
/usr/local/lib/dovecot/libdovecot.so.0(+0x81adc) [0x7f26e3c4cadc] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f26e3bf533d] -> 
/usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_backend_box_close+0x178) 
[0x7f26e2be69f8] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9f94) 
[0x7f26e2be6f94] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9fe9) 
[0x7f26e2be6fe9] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) 
[0x7f26e3f0f5da] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13) 
[0x7f26e3f0f663] -> dovecot/imap(imap_status_get+0x9a) [0x42038a] -> 
dovecot/imap(cmd_status+0x148) [0x414498] -> dovecot/imap(command_exec+0x8c) 
[0x41947c] -> dovecot/imap() [0x417ac2] -> dovecot/imap() [0x417b50] -> 
dovecot/imap(client_handle_input+0x155) [0x417ec5] -> 
dovecot/imap(client_input+0x72) [0x418362] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f26e3c5f9dc] 
-> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe1) 
[0x7f26e3c60d01] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) 
[0x7f26e3c5fa65] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) 
[0x7f26e3c5fc08] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f26e3bfaa23] -> dovecot/imap(main+0x2d7) [0x40c3c7] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f26e3841b45] -> 
dovecot/imap() [0x40c530]
Mar  8 11:32:49 lab dovecot: imap(rzimmerm...@lab.de): Fatal: master: 
service(imap): child 5737 killed with signal 6 (core dumped)

Here the backtrace:

Core was generated by `dovecot/imap'.
Program terminated with signal SIGABRT, Aborted.
#0  0x7f192af9a067 in __GI_raise (sig=sig@entry=6) at 
../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x7f192af9a067 in __GI_raise (sig=sig@entry=6) at 
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x7f192af9b448 in __GI_abort () at abort.c:89
#2  0x7f192b3919e6 in default_fatal_finish (type=, 
status=status@entry=0) at failures.c:201
#3  0x7f192b391adc in i_internal_fatal_handler (ctx=0x7ffccfe2b340, 
format=, args=) at failures.c:670
#4  0x7f192b33a33d in i_panic (format=format@entry=0x7f192a330158 "file %s: 
line %d (%s): assertion failed: (%s)") at failures.c:275
#5  0x7f192a32b9f8 in virtual_backend_box_close (mbox=mbox@entry=0x23582f0, 
bbox=0x2359288) at virtual-storage.c:403
#6  0x7f192a32bf94 in virtual_mailbox_close_internal 
(mbox=mbox@entry=0x23582f0) at virtual-storage.c:445
#7  0x7f192a32bfe9 in virtual_mailbox_close (box=0x23582f0) at 
virtual-storage.c:507
#8  0x7f192b6545da in mailbox_close (box=0x3ac5) at mail-storage.c:1240
#9  0x7f192b654663 in mailbox_free (_box=_box@entry=0x7ffccfe2b4a8) at 
mail-storage.c:1260
#10 0x00412dae in close_selected_mailbox 
(client=client@entry=0x234a120) at cmd-select.c:375
#11 0x00412ed3 in close_selected_mailbox (client=0x234a120) at 
cmd-select.c:368
#12 cmd_select_full (cmd=0x234ad20, readonly=) at 
cmd-select.c:418
#13 0x0041947c in command_exec (cmd=cmd@entry=0x234ad20) at 
imap-commands.c:180
#14 0x00417ac2 in client_command_input (cmd=cmd@entry=0x234ad20) at 
imap-client.c:958
#15 0x00417b50 in client_command_input (cmd=0x234ad20) at 
imap-client.c:1018
#16 0x00417ec5 in client_handle_next_command (remove_io_r=, client=0x234a120) at imap-client.c:1058
#17 client_handle_input (client=0x234a120) at imap-client.c:1070
#18 0x00418362 in client_input (client=0x234a120) at imap-client.c:1117
#19 0x7f192b3a49dc in io_loop_call_io (io=0x36ef230) at ioloop.c:564
#20 0x7f192b3a5d01 in io_loop_handler_run_internal 
(ioloop=ioloop@entry=0x2320730) at ioloop-epoll.c:220
#21 0x7f192b3a4a65 in io_loop_handler_run (ioloop=ioloop@entry=0x2320730) 
at ioloop.c:612
#22 0x7f192b3a4c08 in io_loop_run (ioloop=0x2320730) at ioloop.c:588
#23 0x7f192b33fa23 in master_service_run (service=0x23205d0, 
callback=callback@entry=0x4239b0 ) at master-service.c:640
#24 0x0040c3c7 in main (argc=1, argv=0x2320390) at main.c:454


Mit freundlichen Grüßen

Ralf Zimmermann
Senior Security Engineer
State Certified Engineer

SIEGNETZ.IT GmbH
Einheitsstrasse 2, D-57076 Siegen
Telefon: +4927168193130 Fax: +492716819329
Mobil  : +491735360015
http://www.siegnetz.de
http://rz.siegnetz.de

Amtsgericht Siegen HRB4838
Geschäftsführer: Oliv

Reappearing emails - IMAP trace

2016-03-08 Thread Ron Cleven
A few days back, I sent an overview of this problem, but received no 
responses.   Since then, I have run dozens of traces to isolate the 
problem, difficult because there are timing issues involved.  I have 
finally nailed it down.  If this is not the proper place to report such 
bugs or if someone knows that this bug has been fixed, please let me 
know.  As I noted in my earlier post, we have been running Dovecot 
2.2.10 with a pair of CentOS 7 boxes with replications for the past 
year.  We have been quite happy with the performance and reliability.


Recently we received a report that emails could reappear in the INBOX 
after being deleted.  After running a pile of traces, I determined that 
the problem was strangely related to replications.  For the purposes of 
this discussion, I will refer to the two symmetric replicating servers 
as A and B.  Further, let us assume that during "normal" operation, all 
the emails are delivered to A via SMTP and are replicated to B.  Under 
those assumptions, if the IMAP user connects to A (where the messages 
were originally delivered), there is no problem, at least no problem I 
was able to find.  The problem I am describing only arises if the IMAP 
user connects to B.  Connecting to B has never presented any other 
problems that I am aware of.


The test for which I have provided the trace starts with a test mailbox 
containing only 3 unread messages in the INBOX.  Moving 1 of the unread 
messages to Trash is all that is needed to reproduce the problem.  
Remember this is ONLY a problem if the IMAP sessions do not connect to 
the server to which the messages were originally delivered.  Also, I 
found that there is a timing window.  The critical IMAP commands are:


   UID STORE xxx +FLAGS.SILENT (\Seen)
   UID MOVE xxx Trash

If you introduce a large enough delay (I arbitrarily chose 5 seconds) 
between those two commands, there is no problem. Presumably this has to 
do with the two boxes syncing up some critical data structure.


It takes a short time for the message that was moved to Trash to 
reappear in the INBOX.   The trace initially shows the number of 
messages going from 3 to 2 when the message is moved to Trash, but a 
second or so later, the message count goes back up to 3. Interestingly, 
the reappearing message shows back up as Unseen in the INBOX but a 
duplicate copy of it stays in Trash as Seen.


Dragging unread messages to Trash may not be polite, but it is 
technically acceptable.


Regarding the trace below , CR's show up as \r, LF's show up as \n, and 
each line starts with S or C depending upon whether it is from the 
Server or Client.  There is blank line following each connection.


Thanks,
Ron



S * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE I
S DLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.\r\n
C 10001 LOGIN testi...@usgo.net ***\r\n
S 10001 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENAB
S LE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDS
S UBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UID
S PLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRE
S S WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in
S \r\n
C 10002 STATUS INBOX (MESSAGES UNSEEN)\r\n
S * STATUS INBOX (MESSAGES 3 UNSEEN 3)\r\n
S 10002 OK Status completed.\r\n
C 10003 SELECT INBOX\r\n
S * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)\r\n
S * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*
S )] Flags permitted.\r\n
S * 3 EXISTS\r\n
S * 1 RECENT\r\n
S * OK [UNSEEN 1] First unseen.\r\n
S * OK [UIDVALIDITY 1457030049] UIDs valid\r\n
S * OK [UIDNEXT 129] Predicted next UID\r\n
S * OK [HIGHESTMODSEQ 417] Highest\r\n
S 10003 OK [READ-WRITE] Select completed (0.000 secs).\r\n
C 10004 UID SEARCH 3\r\n
S * SEARCH 128\r\n
S 10004 OK Search completed (0.000 secs).\r\n
C 10005 LIST "" Trash\r\n
S * LIST (\HasNoChildren \Trash) "." Trash\r\n
S 10005 OK List completed.\r\n
C 10006 UID STORE 128 +FLAGS.SILENT (\Seen)\r\n
S 10006 OK Store completed.\r\n
C 10007 UID MOVE 128 Trash\r\n
S * OK [COPYUID 1457030331 128 127] Moved UIDs.\r\n
S * 3 EXPUNGE\r\n
S * 0 RECENT\r\n
S 10007 OK Move completed.\r\n
C 10008 STATUS INBOX (MESSAGES UNSEEN)\r\n
S * STATUS INBOX (MESSAGES 2 UNSEEN 2)\r\n
S 10008 OK [CLIENTBUG] Status on selected mailbox completed.\r\n
C 10009 STATUS Trash (MESSAGES UNSEEN)\r\n
S * STATUS Trash (MESSAGES 1 UNSEEN 0)\r\n
S 10009 OK Status completed.\r\n
C 10010 LOGOUT\r\n
S * BYE Logging out\r\n
S 10010 OK Logout completed.\r\n

S * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE I
S DLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.\r\n
C 10001 LOGIN testi...@usgo.net ***\r\n
S 10001 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENAB
S LE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDS
S UBJECT MULTIAPPEND UR

Re: dovecot: imap-login: Panic: Trying to allocate 0 bytes

2016-03-08 Thread Ron Garret
Figured it out.  Turned out to be a corrupted ssl parameter file.

On Mar 7, 2016, at 1:01 PM, Ron Garret  wrote:

> My ISP had a hard drive crash.  After the dust settled, my dovecot 
> installation started failing with the following error every time a client 
> tries to connect:
> 
> dovecot: imap-login: Panic: Trying to allocate 0 bytes
> 
> My installation has otherwise been stable and rock-solid for years.  Both I 
> and the techs at my ISP are stumped.  Any advice on how to debug this would 
> be greatly appreciated.
> 
> Vital info:
> 
> [ron@vm1:/etc/dovecot]$ /usr/sbin/dovecot -n
> # 1.2.15: /etc/dovecot/dovecot.conf
> # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.10 
> log_timestamp: %Y-%m-%d %H:%M:%S 
> ssl_ca_file: /etc/ssl/local-certs/startssl.ca.pem
> ssl_cert_file: /etc/ssl/local-certs/...
> ssl_key_file: /etc/ssl/local-keys/...
> login_dir: /var/run/dovecot/login
> login_executable: /usr/lib/dovecot/imap-login
> first_valid_uid: 100
> mail_privileged_group: mail
> mbox_write_locks: fcntl dotlock
> auth default:
>  user: postfix
>  passdb:
>driver: sql
>args: /etc/dovecot/dovecot-sql.conf
>  userdb:
>driver: prefetch
>  socket:
>type: listen
>client:
>  path: /var/spool/postfix/private/auth
>  mode: 432
>  user: postfix
>  group: postfix
> 
> I have checked the MySQL configuration and everything seems to be OK there.  
> In fact, I’ve checked everything that I know how to check, and everything 
> seems to be OK.  And yet it is not working.
> 
> Many thanks,
> rg


Re: Dovecot & Pigeon w/ MySQL

2016-03-08 Thread Stephan Bosch
Op 3/4/2016 om 6:17 PM schreef Jorge Bastos:
> Hi Stephan,
>
> Oh I see.
> Is there this feature request already to support the save on MySQL/database?

No need. I've this implemented now:

https://github.com/stephanbosch/pigeonhole-0.4-patches/blob/master/master/sieve-storage-dict-modifiable.patch

This patch still needs to be split up and cleaned up a bit. Also, there
is currently no documentation. I will merge this after the next release,
which will happen soon.

However, once merged, you should still consider this new feature
experimental for the time being.

Regards,

Stephan.

> Jorge,
>
>> -Original Message-
>> From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf Of Stephan
>> Bosch
>> Sent: sexta-feira, 4 de Março de 2016 11:32
>> To: Jorge Bastos; 'Dovecot Mailing List'
>> Subject: Re: Dovecot & Pigeon w/ MySQL
>>
>> Op 3/3/2016 om 4:03 PM schreef Jorge Bastos:
>>> Howdy,
>>>
>>>
>>>
>>> I'm looking for a good howto to have pigeon saving the sieve scripts
>>> on an mysql table.
>> Pigeonhole can currently only retrieve Sieve scripts from a database,
>> not store them there; .e.g., from ManageSieve.
>>
>>> Can some point me to a good one? Dr. google doesn't show me much
>> about it.
>>
>> http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration
>> http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration/Dict
>>
>> Regards,
>>
>> Stephan.