Re: Distributed File Systems
-- David Chait <[EMAIL PROTECTED]> is rumored to have mumbled on Freitag, 18. Oktober 2002 23:23 Uhr -0700 regarding Distributed File Systems: Hi, Has anyone here looked into or had experience with Distributed File Systems (AFS, NFS, CODA, etc) applied to mail partitions to allow for clusetering or fail over capability of Cyrus IMAP machines? I have seen docs for splitting the accounts between machines, however this doesn't seem like the best fault tollerant solution. distributed file systems don't work. Look here for a different approach: <http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg= 17132> -- Sebastian Hagedorn M.A. - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 msg08763/pgp0.pgp Description: PGP signature
Re: IMAPD: BUSY state: terminated abnormally
Hi, --On 29. Oktober 2007 18:01:20 +0100 Michael Plate <[EMAIL PROTECTED]> wrote: I have problem with a mailbox of one user: trying to select a mailboxs ends up in: master[1100]: service imap pid 3135 in BUSY state: terminated abnormally I tried to reconstruct the mailbox, but this did not help. Even deleting and creating an new one (same name of cause) still results in that problem. lmtpunix can not deliver to the box. Version is 2.2.13, no stuff relies on Berkeley DB (mostly skiplist). The server had a crash a week ago, but the fsck (xfs) did not mention any problem. We are using saslauthd and winbind to auth against ADS and and an Openldap used for some users via PAM. Changing the user from ADS to the local LDAP makes no changes.. Any ideas, please ? try removing the user's seen file. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpQHySi0JEQD.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAPD: BUSY state: terminated abnormally
--On 30. Oktober 2007 09:37:35 +0100 Michael Plate <[EMAIL PROTECTED]> wrote: When deleting the mailbox I recognize that cyradm still sees that - maybe because of mailboxes.db: lm user/theuser user/theuser (\NonExistent \Noselect \HasChildren) That's strange. How did you delete the mailbox? What does "ctl_mboxlist -d | grep theuser" tell you? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpFxBRevSZEy.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAPD: BUSY state: terminated abnormally
--On 30. Oktober 2007 15:11:01 +0100 Michael Plate <[EMAIL PROTECTED]> wrote: When deleting the mailbox I recognize that cyradm still sees that - maybe because of mailboxes.db: lm user/theuser user/theuser (\NonExistent \Noselect \HasChildren) That's strange. How did you delete the mailbox? dm user/theuser What does "ctl_mboxlist -d | grep theuser" tell you? user.theuser.Drafts default theuser lrswipcda cyrus lrswipcda user.theuser.Sent default theuser lrswipcda cyrus lrswipcda user.theuser.Trash default theuser lrswipcda cyrus lrswipcda I don't get that. In my experience a "dm user/..." deletes all mailboxes a user has: [EMAIL PROTECTED] imap]$ cyradm cyrus Password: cyrus.rrz.uni-koeln.de> cm user/nureintest cyrus.rrz.uni-koeln.de> cm user/nureintest/test cyrus.rrz.uni-koeln.de> lm user/nureintes* user/nureintest (\HasChildren) user/nureintest/test (\HasNoChildren) cyrus.rrz.uni-koeln.de> dm user/nureintest deletemailbox: Permission denied cyrus.rrz.uni-koeln.de> lam user/nureintes* user/nureintest: nureintest lrswipkxtecda user/nureintest/test: nureintest lrswipkxtecda cyrus.rrz.uni-koeln.de> sam user/nureintes* cyrus lrswipkxtecda Setting ACL on user/nureintest...OK. Setting ACL on user/nureintest/test...OK. cyrus.rrz.uni-koeln.de> lam user/nureintes* user/nureintest: nureintest lrswipkxtecda cyrus lrswipkxtecda user/nureintest/test: nureintest lrswipkxtecda cyrus lrswipkxtecda cyrus.rrz.uni-koeln.de> dm user/nureintest cyrus.rrz.uni-koeln.de> lm user/nureintes* cyrus.rrz.uni-koeln.de> -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpaHr92YU6gN.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: OT: Re: How many people to admin a Cyrus system?
--On 13. November 2007 09:10:08 -0500 Joseph Brennan <[EMAIL PROTECTED]> wrote: Ian G Batten <[EMAIL PROTECTED]> wrote: However, people don't want calendaring, they want Outlook. This describes exactly the point of view of administrative staff. Fortunately that's not true of everyone. They live in Microsoft Office, and they need a server to support it. That is the assignment given. If they don't yet have Exchange there's still hope ;-) I was looking at Open-Xchange on the web <http://www.open-xchange.com/>. The server provides webmail and MAPI interfaces. The "Hosting Edition" (and maybe the others, it is not clear) can talk to Cyrus and includes ACL support. (We're still running both Exchange for admin staff and Cyrus for the much larger university community of faculty and students.) OK, to wean them off Exchange is probably impossible, but for new users Open-Xchange is (one) possibility. It so happens that they (OX) were here today because we (Cologne University) are considering the "Hosting Edition". The beauty of it is that it works as a pretty seamless add-on to our existing Cyrus infrastructure. We are now waiting for an offer. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpLBfbZJahzt.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
One more attempt: stuck processes
Hi,
I've brought up this topic before. We've been running cyrus-imapd very
happily for several years. Yet there's one issue that none of the updates
have resolved. The last time I reported it we were running 2.2.12. Now
we're running 2.3.8, but the issues is the same: POP and IMAP processes
that use TLS/SSL get "stuck". My observations point towards dropped dial-up
connections as the cause. Here's an example:
[EMAIL PROTECTED] root]# strace -p 4998
Process 4998 attached - interrupt to quit
read(0,
Process 4998 detached
In gdb:
0x003ed41e in __read_nocancel () from /lib/tls/libc.so.6
(gdb) bt
#0 0x003ed41e in __read_nocancel () from /lib/tls/libc.so.6
#1 0x00c422f7 in BIO_new_socket () from /lib/libcrypto.so.4
#2 0x00c402b2 in BIO_read () from /lib/libcrypto.so.4
#3 0x00d26c30 in ssl3_alert_code () from /lib/libssl.so.4
#4 0x00d26dcc in ssl3_alert_code () from /lib/libssl.so.4
#5 0x00d280cf in ssl3_read_bytes () from /lib/libssl.so.4
#6 0x00d28ffc in ssl3_get_message () from /lib/libssl.so.4
#7 0x00d1ecab in ssl3_accept () from /lib/libssl.so.4
#8 0x00d1e944 in ssl3_accept () from /lib/libssl.so.4
#9 0x00d2dc9a in SSL_accept () from /lib/libssl.so.4
#10 0x00d2980d in ssl23_get_client_hello () from /lib/libssl.so.4
#11 0x00d29712 in ssl23_accept () from /lib/libssl.so.4
#12 0x00d2dc9a in SSL_accept () from /lib/libssl.so.4
#13 0x080a0a03 in idle_notify ()
#14 0x0805ef21 in idle_update ()
#15 0x08051fc6 in shut_down ()
#16 0x0804ebe4 in ?? ()
#17 0x08d53458 in ?? ()
#18 0x08d61a98 in ?? ()
#19 0x in ?? ()
(gdb)
Ultimately all processes get stuck in libc, so this might be a library
issue. My problem is that I have no way to confirm that. I've compared that
stack trace to the source and I'm confused. idle_notify() is in idle.c, and
it's very simple:
/*
* Notify idled of a mailbox change
*/
void idle_notify(struct mailbox *mailbox)
{
/* We should try to determine if we need to send this
* (ie, is an imapd is IDLE on 'mailbox'?).
*/
idle_send_msg(IDLE_NOTIFY, mailbox);
}
So why doesn't the stack trace show a call to idle_send_msg()? Shouldn't
the SSL routines be called from a procedure that actually does I/O?
Anyway, if there is anybody who has seen something like this I would like
to know about it. Also if anyone has a clue how to further debug this,
please let me know!
Thanks, Sebastian Hagedorn
--
Sebastian Hagedorn - Postmaster - RZKR-R1 (Gebäude 52), Zimmer 18
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
Universität zu Köln / Cologne University - Tel. +49-221-478-5587
Skype: shagedorn
pgpzRU0VT9Nay.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 14. November 2007 09:30:45 -0600 Gary Mills <[EMAIL PROTECTED]> wrote: On Wed, Nov 14, 2007 at 04:15:13PM +0100, Sebastian Hagedorn wrote: I've brought up this topic before. We've been running cyrus-imapd very happily for several years. Yet there's one issue that none of the updates have resolved. The last time I reported it we were running 2.2.12. Now we're running 2.3.8, but the issues is the same: POP and IMAP processes that use TLS/SSL get "stuck". My observations point towards dropped dial-up connections as the cause. Here's an example: Have you tried setting `tls_session_timeout' to zero in imapd.conf to disable caching of TLS sessions? No. While it's possible that there's some connection, I don't see it from the stack trace. I think I'd like some indication from one of the developers that caching *could* be the cause before I try that. I believe that that solved the problem for me, although it happened rarely. I only use `berkeley-nosync' for `duplicate_db' and `tlscache_db', but I suspected that the problem was the result of a database deadlock. We use skiplist, which to my mind makes it even less likely that that's the cause ... thanks for the suggestion anyway. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpbKouWlbjeE.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 15. November 2007 06:55:44 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: OK. What version of OpenSSL? cyradm says: Built w/OpenSSL 0.9.7a Feb 19 2003 Running w/OpenSSL 0.9.7a Feb 19 2003 rpm says: openssl-0.9.7a-33.23 This is RHEL 3. Are they imaps/pop3s processes, or are your clients using STARTTLS? Both. It doesn't seem to matter which. Are you sure that you're not running out of entropy? Yes, because cyrus-sasl uses /dev/urandom. Have you tried adding the '-U 1' option to the offending services in cyrus.conf? No. Since this potentially affects all IMAP and POP processes I would have to do it for all entries. Do you recommend that I try that? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpUWKZDaLfag.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 14. November 2007 16:39:44 -0500 Ken Murchison <[EMAIL PROTECTED]>
wrote:
It looks to me like we are timing out the client while the client is
IDLEing, but we get a signal from idled in the middle of shutdown(). Try
this patch.
--- imapd.c.~1.535.~2007-11-14 16:16:21.0 -0500
+++ imapd.c 2007-11-14 16:22:59.0 -0500
@@ -836,6 +836,8 @@
{
int i;
+idle_done(imapd_mailbox);
+
proc_cleanup();
i = 0;
Thanks. I will try this patch as soon as I can, but it's clearly not the
only issue, because the same thing happens with POP processes. Here's an
example for one:
(gdb) bt
#0 0x0096441e in __read_nocancel () from /lib/tls/libc.so.6
#1 0x00ac02f7 in BIO_new_socket () from /lib/libcrypto.so.4
#2 0x00abe2b2 in BIO_read () from /lib/libcrypto.so.4
#3 0x00fcfe13 in ssl23_read_bytes () from /lib/libssl.so.4
#4 0x00fcec51 in ssl23_get_client_hello () from /lib/libssl.so.4
#5 0x00fce712 in ssl23_accept () from /lib/libssl.so.4
#6 0x00fd2c9a in SSL_accept () from /lib/libssl.so.4
#7 0x08052cb3 in shut_down ()
#8 0x0804e513 in shut_down ()
#9 0x0804d58c in ?? ()
#10 0x0001 in ?? ()
#11 0x08dd6848 in ?? ()
#12 0x in ?? ()
As you see it looks very much like the IMAP stack trace, but obviously the
IDLE routine is not a factor. OTOH, all the stuck IMAP processes I've
looked at so far *did* have those idle_xxx lines in their traces. I am
correct in assuming that that would only occur for clients that actually
use IDLE, right?
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgp6gqxRgy74J.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 15. November 2007 08:21:48 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: No. Since this potentially affects all IMAP and POP processes I would have to do it for all entries. Do you recommend that I try that? Since it looks like things are hanging when a process is being used, I'd like to see if the problem goes away if we don't reuse the processes. I'm just trying to do a bsearch() on the problem. OK. I've made the change and HUP'ed master. I can see the first processes with -U 1. So I guess I'll have to wait a while to see if any of those get stuck as well. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpgSvEePTrl0.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 15. November 2007 08:32:18 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: Since it looks like things are hanging when a process is being used, I'd like to see if the problem goes away if we don't reuse the processes. I'm just trying to do a bsearch() on the problem. OK. I've made the change and HUP'ed master. I can see the first processes with -U 1. So I guess I'll have to wait a while to see if any of those get stuck as well. OK, let me know. Didn't work :-( # ps -aef| grep U UIDPID PPID C STIME TTY TIME CMD ... cyrus25038 32385 0 15:44 ?00:00:00 pop3d -s -U 1 ... That's more than an hour ago. So: # strace -p 25038 Process 25038 attached - interrupt to quit read(0, # gdb -p 25038 GNU gdb Red Hat Linux (6.3.0.0-1.138.el3rh) ... (gdb) bt #0 0x0079f41e in __read_nocancel () from /lib/tls/libc.so.6 #1 0x00d0b2f7 in BIO_new_socket () from /lib/libcrypto.so.4 #2 0x00d092b2 in BIO_read () from /lib/libcrypto.so.4 #3 0x005dae13 in ssl23_read_bytes () from /lib/libssl.so.4 #4 0x005d9c51 in ssl23_get_client_hello () from /lib/libssl.so.4 #5 0x005d9712 in ssl23_accept () from /lib/libssl.so.4 #6 0x005ddc9a in SSL_accept () from /lib/libssl.so.4 #7 0x08052cb3 in shut_down () #8 0x0804e513 in shut_down () #9 0x0804d58c in ?? () #10 0x0001 in ?? () #11 0x082ee848 in ?? () #12 0x in ?? () Any other ideas? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpoJF9EbdufP.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 15. November 2007 11:00:39 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: (gdb) bt # 0 0x0079f41e in __read_nocancel () from /lib/tls/libc.so.6 # 1 0x00d0b2f7 in BIO_new_socket () from /lib/libcrypto.so.4 # 2 0x00d092b2 in BIO_read () from /lib/libcrypto.so.4 # 3 0x005dae13 in ssl23_read_bytes () from /lib/libssl.so.4 # 4 0x005d9c51 in ssl23_get_client_hello () from /lib/libssl.so.4 # 5 0x005d9712 in ssl23_accept () from /lib/libssl.so.4 # 6 0x005ddc9a in SSL_accept () from /lib/libssl.so.4 # 7 0x08052cb3 in shut_down () # 8 0x0804e513 in shut_down () # 9 0x0804d58c in ?? () # 10 0x0001 in ?? () # 11 0x082ee848 in ?? () # 12 0x in ?? () Any other ideas? Not at the moment. I don't understand how SSL_accept() gets called from shut_down(). That's what I'd been wondering about myself. Are you running a Murder? No, it's a pretty plain setup. The only reason I could imagine for the sequence of calls was signal handling. But let's be methodical. There's only one spot where SSL_accept() is called: in tls_start_servertls(). In pop3d.c that's only called in cmd_starttls(). That in turn is called either in cmdloop (for handling of STLS) or in service_main() for connections to port 995. shut_down() could conceivably be called by signals_poll(). The POP process above has the following lines in syslog at debug level: Nov 15 15:44:18 lvr13 master[25038]: about to exec /usr/lib/cyrus-imapd/pop3d Nov 15 15:44:18 lvr13 pop3s[25038]: executed Nov 15 15:44:20 lvr13 pop3s[25038]: accepted connection That's it. So I'm guessing that the connection (which was to some dial-up IP address) got dropped while the process was in service_main(). I don't understand why that doesn't show up in the trace, and why last/first four entries read '??'. But even that doesn't really make sense, because shut_down() isn't actually a signal handler, it's only called at certain points by signal_polls(). -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpjsGUbLBc4U.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 15. November 2007 18:14:05 +0100 Alain Spineux <[EMAIL PROTECTED]> wrote: # strace -p 25038 Process 25038 attached - interrupt to quit read(0, Do you know what is 0, if it was a socket it should timeout, isn't it ? It should, I guess, but it doesn't. # ls -l /proc/25038/fd should answer the question I did not know that. Usually I just use lsof. Interestingly, there's this: # ls -l /proc/25038/fd insgesamt 0 lrwx--1 cyrusmail 64 15. Nov 18:15 0 -> socket:[4230164633] ... It's blinking red, which normally means a broken link. I'm not sure how reliable that is in this case. Anyway, lsof reports: pop3d 25038 cyrus0u IPv4 -64802663 TCP cyrus.rrz.uni-koeln.de:pop3s->p50865F5D.dip.t-dialin.net:1064 (ESTABLISHED) It *thinks* the connections is still open. So does netstat: # LANG=C netstat -a|grep p50865F5D tcp0 0 cyrus.rrz.uni-koeln.d:pop3s p50865F5D.dip.t-dialin:1064 ESTABLISHED But obviously that connection is dead. I don't know what conclusions to draw from that ... -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpyEp2xvUdqN.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 15. November 2007 19:25:19 +0100 Simon Matter <[EMAIL PROTECTED]> wrote: It's blinking red, which normally means a broken link. I'm not sure how The file 0 is a symbolic symlink which doesn't really point to a file, that's why the shell shows it blinking. Everything okay here. Thanks. That's what I thought, but I wasn't sure. reliable that is in this case. Anyway, lsof reports: pop3d 25038 cyrus0u IPv4 -64802663 TCP cyrus.rrz.uni-koeln.de:pop3s->p50865F5D.dip.t-dialin.net:1064 (ESTABLISHED) It *thinks* the connections is still open. So does netstat: # LANG=C netstat -a|grep p50865F5D tcp0 0 cyrus.rrz.uni-koeln.d:pop3s p50865F5D.dip.t-dialin:1064 ESTABLISHED But obviously that connection is dead. I don't know what conclusions to draw from that ... Just two ideas come to mind: 1) Since it only happens on dialup connections, could it be that the dialin router at the providers end sends TCP/RST when a client hangs up and those packets are filtered somewhere, maybe on your firewall? OK, let's run with that one. a) We don't really have a firewall, we only use ACLs on the Cisco routers. You can't even filter TCP/RST there. b) Even *if* a TCP/RST had been dropped, lost or whatever, the server *still* should timeout eventually! 2) Could it be that SO_LINGER should be used as socket option in service_create() in master/master.c. I didn't remember that option, so I just read up on it. It seems as though SO_LINGER is very dependent on implementation. If I get your intention correctly SO_LINGER would have to be set with l_onoff set to non-zero and l_linger to zero, right? So close() would return immediately? That might make sense if the stack trace showed a call to close(). But if I understand the code correctly, close() isn't called at all. The socket is closed as a result of a call to exit(). And that defeats all use of SO_LINGER: "When the socket is closed as part of exit(2), it always lingers in the background." If it's complete nonsense, ignore it. I wouldn't know :-) -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpproVHc1y86.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 16:52:27 +0100 Gabor Gombas <[EMAIL PROTECTED]> wrote: On Fri, Nov 16, 2007 at 12:36:49PM +0100, Sebastian Hagedorn wrote: He suggested that the trace is unreliable. Perhaps a bug in RHEL 3's version of OpenSSL messes up the stack. That would also explain why nobody else seems to have this problem. FYI I also know a system that has problems with hung Cyrus processes. AFAIR they have problems with pop3s only, but that may be because there are more POP3 than IMAP users, I don't know. The system in question runs 2.3.8 on Debian Etch currently. That's a 2.6 kernel, right? I intend to help diagnose that system but had no time so far; they're now running a script that does a POP3 connection every couple of minutes and if that takes too long it restarts Cyrus. Hm, we don't suffer any actual slowdown, it's just that the number of processes increases over time. There is nothing interesting in the logs: Oct 15 02:39:31 host cyrus/master[6102]: about to exec /usr/local/cyrus/sbin/pop3d Oct 15 02:39:31 host cyrus/pop3s[6102]: executed Oct 15 02:39:31 host cyrus/pop3s[6102]: accepted connection That's what I'm seeing. Could you get a stack trace? OTOH there are a lot of messages like the following: Oct 16 14:13:10 host cyrus/master[26136]: about to exec /usr/local/cyrus/sbin/pop3d Oct 16 14:13:10 host cyrus/pop3s[26136]: executed Oct 16 14:13:10 host cyrus/pop3s[26136]: accepted connection Oct 16 14:13:10 host cyrus/pop3s[26136]: pop3s failed: [XX.XXX.XX.XXX] Oct 16 14:13:10 host cyrus/pop3s[26136]: Fatal error: tls_start_servertls() failed Oct 16 14:13:10 host cyrus/master[15923]: process 26136 exited, status 75 Oct 16 14:13:10 host cyrus/master[15923]: service pop3s pid 26136 in BUSY state: terminated abnormally Any idea what's causing that? I have many of those as well. I suppose that could be any number of things. Faulty protocol or dropped connections. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgp3H24eUgNSV.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 18:07:51 +0100 Gabor Gombas <[EMAIL PROTECTED]> wrote: Hm, we don't suffer any actual slowdown, it's just that the number of processes increases over time. It's not a slowdown - the client connects, and hangs. It never even gets to the authentication phase (at least it's not logged). Clients that happen to connect to a non-affected process run normally. Well, that just sounds like you're running out of entropy. That's a different issue. Recompile your cyrus-sasl to use /dev/urandom instead of /dev/random or disable apop in /etc/imapd.conf: allowapop: 0 Either of those things should get rid of that. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgp4g0isqF9Ha.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 11:27:09 +0100 Sebastian Hagedorn <[EMAIL PROTECTED]> wrote: 1) Since it only happens on dialup connections, could it be that the dialin router at the providers end sends TCP/RST when a client hangs up and those packets are filtered somewhere, maybe on your firewall? OK, let's run with that one. a) We don't really have a firewall, we only use ACLs on the Cisco routers. You can't even filter TCP/RST there. b) Even *if* a TCP/RST had been dropped, lost or whatever, the server *still* should timeout eventually! I just had a discussion with a colleague regarding this. He made two observations: 1. In the absence of the SO_KEEPALIVE option it is entirely possible that a TCP connection remains ESTABLISHED even when the other side has gone. This may not be a solution to this particular problem, but it made me wonder why Cyrus does *not* use SO_KEEPALIVE. Is there a downside to it? 2. The stack trace looks garbled: (gdb) bt #0 0x0079f41e in __read_nocancel () from /lib/tls/libc.so.6 #1 0x00d0b2f7 in BIO_new_socket () from /lib/libcrypto.so.4 #2 0x00d092b2 in BIO_read () from /lib/libcrypto.so.4 #3 0x005dae13 in ssl23_read_bytes () from /lib/libssl.so.4 #4 0x005d9c51 in ssl23_get_client_hello () from /lib/libssl.so.4 #5 0x005d9712 in ssl23_accept () from /lib/libssl.so.4 #6 0x005ddc9a in SSL_accept () from /lib/libssl.so.4 #7 0x08052cb3 in shut_down () #8 0x0804e513 in shut_down () #9 0x0804d58c in ?? () #10 0x0001 in ?? () #11 0x082ee848 in ?? () #12 0x in ?? () He suggested that the trace is unreliable. Perhaps a bug in RHEL 3's version of OpenSSL messes up the stack. That would also explain why nobody else seems to have this problem. I think I will try one more approach: I reverted cyrus.conf to not use "-U 1" anymore, so that processes should be reused. I will strace one of the pop3d processes in the hope that it gets stuck. That way I should be able to see where things go wrong. If the process terminates normally I will try with another one. If that doesn't go anywhere, I guess I'll drop this investigation. We will upgrade to RHEL 5 some time next year, so hopefully that will bring new bugs :-) -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpbrIUha0peZ.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 13:54:24 +0100 Alain Spineux <[EMAIL PROTECTED]>
wrote:
On Nov 16, 2007 12:36 PM, Sebastian Hagedorn <[EMAIL PROTECTED]>
wrote:
I just had a discussion with a colleague regarding this. He made two
observations:
1. In the absence of the SO_KEEPALIVE option it is entirely possible
that a TCP connection remains ESTABLISHED even when the other side has
gone.
I said that socket should timeout, but this is true only when the
protocol (TCP here)
require a response (usualy AK here) or at connection establishement.
Right.
On the contrary
it should stay open indefinitely util something happens. Router doing
NAT can drop
a too old connection, because it has to maintains a NAT table and make
some cleanup time to time, this where "KEEPALIVE" become usefull.
Not only there, but I think also in the case of unilaterally dropped
connections.
This may not be a solution to this particular problem, but it made me
wonder why Cyrus does *not* use SO_KEEPALIVE. Is there a downside to it?
Cyrus has already a built-in time out, it seems a lite conflicting to
actively maintains the connection until it drop it itself !
I'm not sure I understand that sentence.
This is the works of the client to actively maintains the connection,
if it want it !
Yes, but what if the client is gone? I realise that *normally* the server
keeps a built-in timeout, but I'm guessing that sometimes it doesn't work,
perhaps because something (in prot_fill() perhaps?) blocks.
I think I will try one more approach: I reverted cyrus.conf to not use
"-U 1" anymore, so that processes should be reused. I will strace one of
the pop3d processes in the hope that it gets stuck. That way I should be
able to see where things go wrong. If the process terminates normally I
will try with another one. If that doesn't go anywhere, I guess I'll
drop this
You could try to replace imapd by a home made script, something like .
mv imapd imapd_
echo exec strace -o /tmp/imapd.$$ imapd_ $* > imapd
chmod imapd a+x
Thanks for the suggestion. I'll think about it, although I'm wary of doing
that on a production server.
investigation. We will upgrade to RHEL 5 some time next year, so
hopefully that will bring new bugs :-)
Sorry but I dont understand what you are complaining about!
I'm not complaining ...
Is-it because the imap or pop client is loosing its connection and
this disturb the user
No.
or just because you are getting some sleeping processes ?
If it were "some" I wouldn't worry. I'm talking hundreds of processes! I
know I can kill them, in fact for the pop3d processes we run this command
once a month:
ps -C pop3d -o pid,start|grep [a-z]|awk '{print $1}'|xargs kill
(It kills pop3d processes that have the month in their start time, i.e. are
more than a day old)
But for imapd processes it's not as easy to tell if they are just
long-living or stuck.
Do you have a "timeout" option in your imapd.conf to force the
imap/pop server to autologout ?
No. But both POP and IMAP have default timeouts. They just don't work in my
case.
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpUj3SrktoJw.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 08:00:07 -0600 Gary Mills <[EMAIL PROTECTED]> wrote: This timeout doesn't work in some cases. We have lots of POP sessions that never terminate. That's interesting to hear! Especially since you are using Solaris. About 30 out of 40 are in that state now. Here's an example: cyrus 13075 708 0 Oct 14 ?0:05 pop3d -s cyrus 20023 708 0 Oct 29 ?0:00 pop3d cyrus 24560 708 1 07:38:03 ?0:03 pop3d cyrus 631 708 0 Oct 03 ?0:10 pop3d -s cyrus 6786 708 0 Oct 20 ?0:00 pop3d -s cyrus 29777 708 0 07:45:03 ?0:00 pop3d cyrus 19175 708 0 Oct 04 ?0:04 pop3d -s One I just checked is stuck in a read(): # truss -p 19175 read(0, 0x002316F0, 5) (sleeping...) ^?# pfiles 19175 19175: pop3d -s Current rlimit: 256 file descriptors 0: S_IFSOCK mode:0666 dev:271,0 ino:25813 uid:0 gid:0 size:0 O_RDWR sockname: AF_INET 130.179.16.23 port: 995 peername: AF_INET 130.179.188.184 port: 51771 Could you get a stack trace? If you have gdb you just call it with "gdb -p 19175". Then you can do "bt" at the prompt. I forget how to do it with Sun's debugger. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpvKBTMY4YQA.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 09:37:42 -0600 Gary Mills <[EMAIL PROTECTED]> wrote: Could you get a stack trace? If you have gdb you just call it with "gdb -p 19175". Then you can do "bt" at the prompt. I forget how to do it with Sun's debugger. Easy: # pstack 19175 19175: pop3d -s fef9f810 read (0, 2316f0, 5) fee1d2d0 read (0, 2316f0, 5, 0, 0, 0) + 5c ff06bb38 sock_read (1f0860, 2316f0, 5, 5, 0, 0) + 24 ff068af0 BIO_read (1f0860, 2316f0, 5, fef98b84, 0, 0) + 110 ff278488 ssl3_read_n (212798, 5, 8805, 0, 0, 203958) + 174 ff2785fc ssl3_get_record (204ce0, 8000, 8400, 4400, f1, f0) + d0 ff279424 ssl3_read_bytes (212798, 1000, 2000, 4, 0, ffbfe731) + 228 ff27a99c ssl3_get_message (ff2a259c, 2070a0, 0, , 19000, ffbfe7a0) + d0 ff27042c ssl3_accept (2150, 2160, 2180, 21e0, 2110, 2122) + 904ff27bd2c ssl23_get_client_hello (2316fb, 6c, 6c, 4, fe79, 0) + 828ff27b4b4 ssl23_accept (4000, 2000, 0, 0, 0, 0) + 2a4 00032d00 tls_start_servertls (0, 1, ffbfee24, ffbfee20, 1849a8, ff00) + 1980002c504 cmd_starttls (1, 1fd8b8, 0, 0, 0, 0) + 184 0002a638 service_main (2, 192198, ffbffce0, 1aec4, 3508c, 1) + 488 00035250 main (2, ffbffcd4, ffbffce0, 17c400, 0, 0) + e18 00029298 _start (0, 0, 0, 0, 0, 0) + 108 Thanks, that looks like progress! That stack trace looks similar enough to the one I'm seeing that I could imagine that it is what I *should* be seeing if the stack weren't garbled. Of course that's only speculation. Ken, is it possible that the call to SSL_accept() in tls_start_servertls() blocks when the client goes away? That could explain everything -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpldwycIAjiI.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
OK, now I got this: Nov 16 18:37:06 lvr13 pop3s[23089]: SSL_read() returned -1 But that process terminated normally. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpRJSjlsSCf8.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 14:23:17 +0100 Simon Matter <[EMAIL PROTECTED]>
wrote:
Did you ever see non SSL connections get stuck?
No.
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpAIZv7hfTCt.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 11:27:52 -0500 Ken Murchison <[EMAIL PROTECTED]>
wrote:
Sebastian Hagedorn wrote:
The only reason I could imagine for the sequence of calls was signal
handling. But let's be methodical. There's only one spot where
SSL_accept() is called: in tls_start_servertls(). In pop3d.c that's only
called in cmd_starttls(). That in turn is called either in cmdloop (for
handling of STLS) or in service_main() for connections to port 995.
Actually, now that I think about it, I believe SSL_accept() can be called
from SSL_read() at any time if a renegotiation is required. Since
shut_down() calls prot_fill(), which in turn can call SSL_read(), its
possible that we can get an SSL_accept() call. Before I start hacking
code, can you apply the following patch (sorry about the line breaks) and
see if I'm heading in the right direction? Let me know if you get any of
the WARNING messages in your logs.
--- prot.c.~1.93.~ 2007-11-16 11:21:56.0 -0500
+++ prot.c 2007-11-16 11:23:32.0 -0500
@@ -468,6 +468,7 @@
/* just do a SSL read instead if we're under a tls layer */
if (s->tls_conn != NULL) {
n = SSL_read(s->tls_conn, (char *) s->buf, PROT_BUFSIZE);
+ if (n <= 0) syslog(LOG_WARNING, "SSL_read() returned %d", n);
} else {
n = read(s->fd, s->buf, PROT_BUFSIZE);
}
Yes, I do:
Nov 16 17:59:34 lvr13 pop3s[3196]: SSL_read() returned 0
Nov 16 17:59:38 lvr13 pop3s[3196]: SSL_read() returned 0
Nov 16 18:00:09 lvr13 pop3s[3215]: SSL_read() returned 0
Nov 16 18:00:26 lvr13 pop3s[3847]: SSL_read() returned 0
Nov 16 18:00:34 lvr13 pop3s[3215]: SSL_read() returned 0
Nov 16 18:00:34 lvr13 pop3s[3199]: SSL_read() returned 0
Nov 16 18:00:39 lvr13 pop3s[3199]: SSL_read() returned 0
Nov 16 18:00:43 lvr13 pop3s[3229]: SSL_read() returned 0
Not all of these processes are stuck, though. (Maybe none are). Should I be
looking for something specific?
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpzM3I7B80P9.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 12:39:28 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: Sorry, my patch wasn't complete. It wasn't logging the value that I wanted. OK: Nov 16 18:48:17 lvr13 pop3s[1385]: SSL_read() returned 0:5 Nov 16 18:48:33 lvr13 pop3s[1375]: SSL_read() returned 0:5 Nov 16 18:48:50 lvr13 pop3s[1980]: SSL_read() returned 0:6 Nov 16 18:48:54 lvr13 pop3s[1376]: SSL_read() returned 0:5 Nov 16 18:49:03 lvr13 pop3s[1375]: SSL_read() returned 0:5 Nov 16 18:49:11 lvr13 pop3s[1375]: SSL_read() returned 0:5 Nov 16 18:49:38 lvr13 pop3s[1375]: SSL_read() returned 0:5 Nov 16 18:49:54 lvr13 pop3s[1404]: SSL_read() returned 0:5 I'm guessing that's still not enough: #define SSL_ERROR_SYSCALL 5 /* look at error stack/return value/errno */ #define SSL_ERROR_ZERO_RETURN 6 SSL_ERROR_SYSCALL Some I/O error occurred. The OpenSSL error queue may contain more information on the error. If the error queue is empty (i.e. ERR_get_error() returns 0), ret can be used to find out more about the error: If ret == 0, an EOF was observed that violates the pro- tocol. If ret == -1, the underlying BIO reported an I/O error (for socket I/O on Unix systems, consult errno for details). So should I add a call to ERR_get_error()? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpFMfHMrSvNV.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 16. November 2007 18:21:21 +0100 Gabor Gombas <[EMAIL PROTECTED]> wrote: On Fri, Nov 16, 2007 at 06:11:01PM +0100, Sebastian Hagedorn wrote: Well, that just sounds like you're running out of entropy. That's a different issue. Recompile your cyrus-sasl to use /dev/urandom instead of /dev/random or disable apop in /etc/imapd.conf: Debian uses /dev/urandom for a long time: # strings /usr/lib/libsasl2.so.2 | grep random /dev/urandom And according to the logs I have, after a pop3 process got stuck other IMAP users can still log in using TLS+PLAIN, so entropy can be ruled out. OK. Still the symptom seems to be different from what I'm seeing. Could it be that you have a process limit in /etc/cyrus.conf? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpvuf1O5cVWm.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 16. November 2007 12:58:49 -0500 regarding Re: One more attempt: stuck processes: So should I add a call to ERR_get_error()? Not yet. I'm assuming that none of these processes has hung. We're getting an I/O error most likely because the client has closed the connection immediately after sending QUIT. This is harmless. What I really want to see is if we get a SSL_ERROR_WANT_xxx return code when we're hung. I have both good and bad news. Bad news first: there is a stuck process that did *not* log that SSL_read line. Good news: the binary I'm running now isn't stripped and has much more detail in its stack trace: (gdb) bt #0 0x003d341e in __read_nocancel () from /lib/tls/libc.so.6 #1 0x0017f2f7 in BIO_new_socket () from /lib/libcrypto.so.4 #2 0x0017d2b2 in BIO_read () from /lib/libcrypto.so.4 #3 0x0089ec30 in ssl3_alert_code () from /lib/libssl.so.4 #4 0x0089edcc in ssl3_alert_code () from /lib/libssl.so.4 #5 0x008a00cf in ssl3_read_bytes () from /lib/libssl.so.4 #6 0x008a0ffc in ssl3_get_message () from /lib/libssl.so.4 #7 0x00896cab in ssl3_accept () from /lib/libssl.so.4 #8 0x00896944 in ssl3_accept () from /lib/libssl.so.4 #9 0x008a5c9a in SSL_accept () from /lib/libssl.so.4 #10 0x008a180d in ssl23_get_client_hello () from /lib/libssl.so.4 #11 0x008a1712 in ssl23_accept () from /lib/libssl.so.4 #12 0x008a5c9a in SSL_accept () from /lib/libssl.so.4 #13 0x08052cf3 in tls_start_servertls (readfd=-512, writefd=-512, layerbits=0xbfff7a78, authid=0xbfff7a74, ret=0x810bca0) at tls.c:803 #14 0x0804e553 in cmd_starttls (pop3s=1) at pop3d.c:1076 #15 0x0804d5cc in service_main (argc=2, argv=0x9e84008, envp=0xbfff9850) at pop3d.c:537 #16 0x08054550 in main (argc=2, argv=0x9, envp=0xbfff9850) at service.c:539 There's much less POP activity now, so I may have to wait until Monday for more results. -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpvINiK8adT6.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Bingo!
-- Sebastian Hagedorn <[EMAIL PROTECTED]> is rumored to have mumbled on 16. November 2007 22:03:21 +0100 regarding Re: One more attempt: stuck processes: The question is how pop3d knows that the connection is dropped. And maybe that's really where dial-up comes into play. In don't know if you're in a position to test that, but what happens if you telnet to port 995 from dial-up and then drop the dial-up connection? I guess I might try that from home now. That does it ... I disconnected my cable modem while having an open telnet connection to 995. Now that process is stuck. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpd0ZxYqLLmf.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Bingo!
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 16.
November 2007 16:29:20 -0500 regarding Re: Bingo!:
That does it ... I disconnected my cable modem while having an open
telnet connection to 995. Now that process is stuck.
Does the same thing happen if you telnet to port 110?
Actually yes - so far! But the stack trace and strace are instructive:
(gdb) bt
#0 0x006cf2e8 in ___newselect_nocancel () from /lib/tls/libc.so.6
#1 0x08073f76 in prot_fill (s=0x9f6bf48) at prot.c:439
#2 0x080757ad in prot_fgets (buf=0xbfff7a30 "quit", size=8191,
s=0x9f6bf48) at prot.c:1196
#3 0x0804da6b in cmdloop () at pop3d.c:762
#4 0x0804d516 in service_main (argc=1, argv=0x9f1f008, envp=0xbfffb80c) at
pop3d.c:543
#5 0x08054550 in main (argc=1, argv=0x9, envp=0xbfffb80c) at service.c:539
# strace -p 18432
Process 18432 attached - interrupt to quit
select(1, [0], NULL, NULL, {463, 9}
The select() will time out eventually, I'm sure. I'm currently waiting for
that to happen.
--
Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
Universität zu Köln / Cologne University - Tel. +49-221-478-5587
pgpG7EqedAZoN.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Bingo!
-- Sebastian Hagedorn <[EMAIL PROTECTED]> is rumored to have mumbled on
16. November 2007 22:36:09 +0100 regarding Re: Bingo!:
The select() will time out eventually, I'm sure. I'm currently waiting
for that to happen.
Here we go:
# strace -p 18432
Process 18432 attached - interrupt to quit
select(1, [0], NULL, NULL, {463, 9}) = 0 (Timeout)
time(NULL) = 1195249308
close(9)= 0
munmap(0xb47a4000, 4096)= 0
unlink("/var/lib/imap/proc/18432") = 0
munmap(0xb47a5000, 12214272)= 0
close(6)= 0
munmap(0xb41da000, 6070272) = 0
close(10) = 0
munmap(0xb534b000, 32768) = 0
munmap(0xb6953000, 2621440) = 0
munmap(0xb5353000, 23068672)= 0
munmap(0xb74a1000, 1318912) = 0
munmap(0xb735f000, 1318912) = 0
munmap(0xb721d000, 1318912) = 0
munmap(0xb70db000, 1318912) = 0
munmap(0xb6f99000, 1318912) = 0
munmap(0xb6e57000, 1318912) = 0
munmap(0xb6d15000, 1318912) = 0
munmap(0xb6bd3000, 1318912) = 0
munmap(0xb75f4000, 16384) = 0
exit_group(0) = ?
I suppose an alarm handler is in order?
--
Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
Universität zu Köln / Cologne University - Tel. +49-221-478-5587
pgp92aahzato9.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Bingo!
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 16. November 2007 16:55:27 -0500 regarding RE: Bingo!: It looks like it timed out properly, correct? Exactly. So the non-SSL code path is not affected, but SSL gets stuck. The manpage for SSL_read says: "If the underlying BIO is blocking, SSL_read() will only return, once the read operation has been finished or an error occurred, except when a renegotiation take place, in which case a SSL_ERROR_WANT_READ may occur. This behaviour can be controlled with the SSL_MODE_AUTO_RETRY flag of the SSL_CTX_set_mode(3) call. If the underlying BIO is non-blocking, SSL_read() will also return when the underlying BIO could not satisfy the needs of SSL_read() to con- tinue the operation. In this case a call to SSL_get_error(3) with the return value of SSL_read() will yield SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE. As at any time a re-negotiation is possible, a call to SSL_read() can also cause write operations! The calling process then must repeat the call after taking appropriate action to satisfy the needs of SSL_read(). The action depends on the underlying BIO. When using a non-blocking socket, nothing is to be done, but select() can be used to check for the required condition. When using a buffering BIO, like a BIO pair, data must be written into or retrieved out of the BIO before being able to continue." I haven't yet found what BIO stands for According to Wikipedia it's "an abstraction library used by OpenSSL to handle communication of various kinds, including files and sockets, both secure and not". I'm not sure what to make of that. I would assume that we've got a blocking BIO, because it is - d'oh - blocking. But I don't see how you influence what kind of BIO you use. -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpGbPVKbmHmE.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 16. November 2007 15:54:50 -0500 regarding Re: One more attempt: stuck processes: That's exactly what Gary is seeing. Right. Apparently stripped binaries aren't any good for straces. Its blocking in SSL_accept(). Apparently the client connects to port 995, and then either sends nothing, or goes away and leaves the socket open. I've reproduced the former by telneting to port 995 and doing nothing. Does it time out eventually, i.e. after 10 minutes? If not that in itself could be considered a bug. I have been unable to reproduce the latter because as soon as I QUIT the telnet session or kill() the telnet process, pop3d exits gracefully. The question is how pop3d knows that the connection is dropped. And maybe that's really where dial-up comes into play. In don't know if you're in a position to test that, but what happens if you telnet to port 995 from dial-up and then drop the dial-up connection? I guess I might try that from home now. -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpS9nwbHCdvp.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Bingo!
I'm not sure of you're aware of it, so I'll point it out: you tried
something different from what Ken and I tried. It doesn't explain
everything, but at least some of what you see.
-- Alain Spineux <[EMAIL PROTECTED]> is rumored to have mumbled on 17.
November 2007 00:46:43 +0100 regarding Re: Bingo!:
I tested the timeout using a SSL connection and a normal connection
and get two different result !
The SSL was working as expected but the normal one "timeout" in a
different way, it get a SIGTERM !
I will repeat the test
Here is the normal one :
# telnet localhost 110
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
+OK eg01.emailgency.loc Cyrus POP3 v2.3.9-openpkg server ready
user [EMAIL PROTECTED]
+OK Name is a valid mailbox
pass
+OK Mailbox locked and ready
list
+OK scan listing follows
1 1779
.
# I dont send anymore command for 10min and wait for the timeout !
Connection closed by foreign host.
That's OK. Just to make sure: the problem I had noticed on my server and
which we were trying to nail down did *not* occur for sessions where a
login had been done. It only happens for sessions where the client does
nothing or goes away *before* a login (or actually a successful SSL
negotiation) has happened.
Here is the strace from the list command
read(0, "list\r\n", 4096) = 6
time(NULL) = 1195253508
write(13, "<1195253508<", 12) = 12
write(13, "list\r\n", 6)= 6
open("/kolab/var/imapd/msg/shutdown", O_RDONLY) = -1 ENOENT (No such
file or directory)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
time(NULL) = 1195253508
write(13, ">1195253508>", 12) = 12
write(13, "+OK scan listing follows\r\n1 1779"..., 37) = 37
write(1, "+OK scan listing follows\r\n1 1779"..., 37) = 37
time(NULL) = 1195253508
select(1, [0], NULL, NULL, {600, 0}
)= ? ERESTARTNOHAND (To be restarted)
#
# HERE IT IS WAITING FOR 10min AS EXPECTED
--- SIGTERM (Terminated) @ 0 (0) ---
Process 5720 detached
I dont understand why it got a SIGTERM !
Neither do I. I tested the same thing and I got this trace:
Process 18811 attached - interrupt to quit
select(1, [0], NULL, NULL, {565, 38}) = 0 (Timeout)
time(NULL) = 1195297357
close(9) = 0
munmap(0xb47a7000, 4096)= 0
unlink("/var/lib/imap/proc/18811") = 0
...
So everything looks fine here.
--
Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
Universität zu Köln / Cologne University - Tel. +49-221-478-5587
pgp3STziVdp5B.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 17. November 2007 11:21:38 -0500 regarding Re: One more attempt: stuck processes: Here's a patch that seems to fix the problem. I did some basic testing (Linux only) to make sure that it doesn't break anything else, but its always possible that it has some unforseen side effects. Keep an eye on it and let me know if you see anything unusual. Thanks, it seems to be working fine so far! Enjoy your weekend now ... -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpkpMVeIVDp2.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Bingo!
-- Gabor Gombas <[EMAIL PROTECTED]> is rumored to have mumbled on 17. November 2007 14:34:02 +0100 regarding Re: Bingo!: I'm not sure what to make of that. I would assume that we've got a blocking BIO, because it is - d'oh - blocking. But I don't see how you influence what kind of BIO you use. A BIO is non-blocking if the underlying file (socket) descriptor has the O_NONBLOCK flag set, either during open or by a previous call to fcntl(). And yes, if O_NONBLOCK was not set on a socket, then any OpenSSL operations can block pretty much indefinitely. Thanks for the explanation. -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpV6drjZVE6e.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 19. November 2007 12:35:46 -0500 regarding Re: One more attempt: stuck processes: How are things looking today? Good! When I just checked I thought I'd found a new hanging pop3d process, because it's been around for 6 hours, but in fact it's still working: Nov 19 14:17:16 lvr13 pop3s[23493]: badlogin: p50865731.dip.t-dialin.net [80.134.87.49] LOGIN user not found ... Nov 19 19:04:23 lvr13 pop3s[23493]: badlogin: p50865731.dip.t-dialin.net [80.134.87.49] LOGIN user not found Nov 19 19:04:27 lvr13 pop3s[23493]: badlogin: p50865731.dip.t-dialin.net [80.134.87.49] LOGIN user not found Nov 19 19:04:30 lvr13 pop3s[23493]: badlogin: p50865731.dip.t-dialin.net [80.134.87.49] LOGIN user not found Sigh. But that's beside the matter. The only other potential downside the patch has is that stracing or gdb'ing it causes the timeout to trigger prematurely. AFAIK that's a common issue for signals. I'm not sure if there's something that could be done about that, i.e. catching ERESTART or something like that? -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpOw7IfeSb18.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 19. November 2007 13:17:07 -0500 regarding Re: One more attempt: stuck processes: The only other potential downside the patch has is that stracing or gdb'ing it causes the timeout to trigger prematurely. AFAIK that's a common issue for signals. I'm not sure if there's something that could be done about that, i.e. catching ERESTART or something like that? Is this only a problem with pop3s, or does it also effect the STLS command? Yes, it does: telnet cyrus 110 ... stls +OK Begin TLS negotiation now -ERR [SYS/PERM] Starttls failed That's when I straced the process ... -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpgMNLWdKmc1.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 20. November 2007 09:20:42 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: OK. Can you both try this alternate patch? It should be portable, and GDB shouldn't cause it to kick out. I've set it up so that for SSL-wrapped services it will timeout after 3 minutes, otherwise it uses the service-specific timeout. Thanks for working on this. I think you missed a spot: gcc -c -I.. -I./../sieve -I./../imap -I./../lib -I/usr/include/db4 -I/usr/kerberos/include -I/usr/include/et -DHAVE_CONFIG_H -I/usr/kerberos/include -O2 -g -pipe -march=i386 -mcpu=i686 -fPIC \ parser.c parser.c: In function `cmd_starttls': parser.c:834: warning: passing arg 3 of `tls_start_servertls' makes integer from pointer without a cast parser.c:834: warning: passing arg 4 of `tls_start_servertls' from incompatible pointer type parser.c:834: warning: passing arg 5 of `tls_start_servertls' from incompatible pointer type parser.c:834: too few arguments to function `tls_start_servertls' I can fix this myself, but it's probably easier if you do it. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpImfrnNAhAO.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
--On 20. November 2007 15:59:18 +0100 Sebastian Hagedorn
<[EMAIL PROTECTED]> wrote:
I can fix this myself, but it's probably easier if you do it.
Just FYI: I fixed it locally with a 3 minute timeout and it compiled fine.
I'll start testing it now.
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpoLn6l11ZH7.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Maybe too much of a good thing?
Well, the new patch works as intended (processes time out yet remain straceable), but looks like it might be overzealous: Nov 20 16:46:30 lvr13 pop3s[25622]: accepted connection Nov 20 16:46:30 lvr13 pop3s[25622]: error or timeout in SSL_accept() -> done Nov 20 16:46:30 lvr13 pop3s[25622]: pop3s failed: ug-out-1314.google.com [66.249.92.171] Nov 20 16:46:30 lvr13 pop3s[25622]: Fatal error: tls_start_servertls() failed Nov 20 16:46:30 lvr13 master[32385]: process 25622 exited, status 75 Nov 20 16:46:30 lvr13 master[32385]: service pop3s pid 25622 in BUSY state: terminated abnormally I understand that the timeout is not the only possible error condition, but failing immediately seems strange. There are definitely more "error or timeout in SSL_accept() -> done" messages than there used to be stuck processes! Perhaps we should log the error code returned by SSL_get_error()? I think I'll add some more logging locally. -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Gebäude 52), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 Skype: shagedorn pgpXc94aPjRDt.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Maybe too much of a good thing?
--On 20. November 2007 11:08:30 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: OK, let me know what you find out. Working on it. I didn't change the logic if/when SSL_accept() fails, because if its an SSL_wrapped process, there is nothing to fall back on (the application protocol hasn't started yet). Perhaps your dial-in clients take longer than 3 minutes to complete the handshake. No, because it fails *immediately*. Hmm, from this particular log, I don't see the debug message tellinng us that we're waiting for more input. When I test locally (over localhost), I always get at least in "-> waiting" log message before the "-> done" messsage. Sure, for the most part I get those as well. But there are exceptions. And of course in principle SSL_get_error() can return something unexpected the first time around. So that's what I'll be looking for. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpWjBeSg04KI.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: lmtp timed out while sending MAIL FROM
-- Gerard <[EMAIL PROTECTED]> is rumored to have mumbled on 27. November 2007 12:22:42 -0500 regarding Re: lmtp timed out while sending MAIL FROM: Some DBERROR in the logs somewhere ? There were, but not anymore. In troubleshooting the issues we converted deliver.db to skiplist rather then BDB which eliminated the errors. They looked like this. Nov 26 19:24:49 imap06 lmtpunix[28264]: DBERROR db4: 15 lockers No that it matters, but those aren't actually errors. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpcSQJchj03G.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus upgrade from 2.1.18 to 2.2.13 moved email messages
-- Steinar Bang <[EMAIL PROTECTED]> is rumored to have mumbled on 28. November 2007 20:55:16 +0100 regarding Cyrus upgrade from 2.1.18 to 2.2.13 moved email messages: What previously was mail/s/user/sb/ is now mail/u/s/user/sb/ That means you are using fulldirhash. It's an option in /etc/imapd.conf. And my restored mailbox.db probably points to the old structure...? So how to I create one that points to the new structure? It might be easier to reverse the process. Disable the feature and run rehash. No promises, though. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpLTyPdhZ1To.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.3.11 STARTTLS broken if tls_ca_file is defined
--On 16. Dezember 2007 15:08:46 +0100 Wolfgang Breyha <[EMAIL PROTECTED]> wrote: I always had tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt defined in my imapd.conf. FWIW: I have a tls_ca_file defined as well. Since I updated to 2.3.11 yesterday STARTTLS didn't work anymore because negotiation failed and timed out. $CLIENT was waiting for more packets from server AFAIS in a tcpdump, where $CLIENT is Thunderbird, gnutls-cli, apple-mail. Hm, I don't run 2.3.11 proper, but my locally built version contains the modified tls.c etc. So I would think that it should behave the same way as 2.3.11, but of course I can't be sure. And here STARTTLS works fine. IMAPS always worked...so I searched for differences in the code and found the "client cert verfication" code triggered by askcert == 1 in tls.c:738 Hm, do you use client certificates? We don't ... Log always showed: 00:00 imap[8508]: accepted connection +02 imap[8508]: SSL_accept() incomplete -> wait <- here the client waits +23 imap[8508]: EOF in SSL_accept() -> fail <- here client sent FIN That code is where all the changes were made. It's conceivable that there are cases where the new approach breaks. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgppshY1gVTVI.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: imapd signal 11 blues
--On 19. Dezember 2007 02:04:57 -0500 John Crawford <[EMAIL PROTECTED]> wrote: Well, I did a gdb run for the first time with cyrus, using the debug_command option in imapd.conf (and CFLAGS="--debug -g -Wall") for cyrus-imap23's work directory. Here's the first bits of the traceback for my "imtest" invoked blowup ... debugging symbols found)...(no debugging symbols found)...0x2854af87 in write () from /li\ b/libc.so.6 Program received signal SIGSEGV, Segmentation fault. 0x08163533 in ?? () # 0 0x08163533 in ?? () # 1 0x0246 in ?? () # 2 0x297edbdd in berkeleydb_open () from # /usr/local/lib/sasl2/libsasldb.so.2 3 0x297edf89 in _sasldb_putdata () # from /usr/local/lib/sasl2/libsasldb.so.2 4 0x297ec335 in # sasldb_auxprop_store () from /usr/local/lib/sasl2/libsasldb.so.2 # 5 0x281804a5 in sasl_auxprop_store () from /usr/local/lib/libsasl2.so.2 # 6 0x28188f77 in sasl_setpass () from /usr/local/lib/libsasl2.so.2 # 7 0x28189ef1 in _sasl_transition () from /usr/local/lib/libsasl2.so.2 # 8 0x2818b60a in _sasl_checkpass () from /usr/local/lib/libsasl2.so.2 # 9 0x2818b742 in sasl_checkpass () from /usr/local/lib/libsasl2.so.2 I'm only guessing here, but based on similar issues I had with (other) libraries it's perhaps a conflict in Berkeley versions. Is it possible that cyrus-imapd and cyrus-sasl were compiled/linked against different versions? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpObEjGchM22.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 4xusers imap processes?
-- Kenneth Marshall <[EMAIL PROTECTED]> is rumored to have mumbled on 19. Dezember 2007 17:09:35 -0600 regarding Re: 4xusers imap processes?: Many mail clients keep open multiple IMAP connections. If they are staying around, I would leave them alone. Clients get upset when their connections go away on them, and then they just open a new one so the problem remains. OTOH, cyrus-imapd prior to 2.3.11 had a bug where SSL-based processes could hang around indefinitely. Use lsof and strace (or whatever your platform offers) to determine whether they are actually still alive. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpRnt48a8RNq.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 4xusers imap processes?
--On 20. Dezember 2007 07:11:28 -0600 Mike Eggleston <[EMAIL PROTECTED]>
wrote:
OTOH, cyrus-imapd prior to 2.3.11 had a bug where SSL-based processes
could hang around indefinitely. Use lsof and strace (or whatever your
platform offers) to determine whether they are actually still alive.
I have cyrus-imapd 2.3.1 on Fedora Core 5. I also had a seen file
corruption incident recently. Could the two be related somehow?
No.
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpBxzknzUvaS.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: quota corruption
-- Erol YILDIZ <[EMAIL PROTECTED]> is rumored to have mumbled on 29. Dezember 2007 11:44:25 +0200 regarding Re: quota corruption: But now some of the users still having the problems for which I have started this thread. One of the symptoms is, users can't purge deleted items, when they press purge it says the client lost connection to the server and that maybe becuase either the folder doesnt exist or this is a limitation of the imap server. Another one is when the user selects a mail to view, client says it needs to be downloaded from the server and asks the user if he wants to mark it for download. Even if the user marks it for download still cant view the mail. Now you know when I disable the quota for the user the problem was solved so that I tought it may be because of the quota files. But when I did 'quota -f' this didnt help to solve the problem. Maybe I need to delete the quota files and recreate them. Any further ideas? This may not be the answer you're looking for, but based on anecdotal evidence I'd say it's an Outlook bug. Here's what you could try if you're adventurous: mkdir the directory /var/lib/imap/log/username as user cyrus, replacing "username" with one of the users that have the problems. Wait until the problem occurs. Look at the log files created. If you can correlate a sequence of IMAP commands to the problem, it way give a hint what's wrong. And to verify that it's an Outlook issue you could switch the affected users to a different client, at least temporarily. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpvTlNFCvFYr.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.3.11 STARTTLS broken if tls_ca_file is defined
-- [EMAIL PROTECTED] is rumored to have mumbled on 2. Januar 2008 17:46:11 +0100 regarding 2.3.11 STARTTLS broken if tls_ca_file is defined: Since I upgraded to 2.3.11, It's seems i've got the same problem. I can use TLS via SSL via imaps on port 993 when i disable the tls_ca_file : imaps[45635]: TLS server engine: cannot load CA data Jan 2 17:34:47 imaptest imaps[45635]: starttls: TLSv1 with cipher AES256-SHA (256/256 bits reused) no authentication Jan 2 17:34:47 imaptest imaps[45635]: login: [x.x.x.x] toto2 plain+TLS User logged in but I can't connect to TLS via imapd on port 143 : TLS server engine: cannot load CA data Jan 2 17:35:37 imaptest imap[45653]: TLS server engine: No CA file specified. Client side certs may not work Jan 2 17:35:37 imaptest imap[45653]: STARTTLS negotiation failed: [ x.x.x.x ] I use client certificate. Not in the first example. "plain+TLS" means that a password was used. So does logging in via TLS on port 143 work if you use a password instead of a certificate? It's possible that authenticating with a certificate was broken with the changes from 2.3.10 to 2.3.11. What can i do to solve it? Revert to 2.3.10? If you want to help find out what's going in, please increase logging for Cyrus to the debug level (in syslog.conf). That should show much more detailed logging. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpJl3Mr9Xbhc.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.3.11 STARTTLS broken if tls_ca_file is defined
Hi,
please don't write to me personally but keep this on the list instead.
--On 15. Januar 2008 10:32:16 +0100 [EMAIL PROTECTED] wrote:
Here is my log, when i try to open a connection in TLS.
Jan 15 10:29:54 imaptest master[1024]: about to exec
/usr/local/cyrus/bin/imapd Jan 15 10:29:54 imaptest imap[1024]: executed
Jan 15 10:29:54 imaptest imap[1024]: accepted connection
Jan 15 10:29:54 imaptest imap[1024]: imapd:Loading hard-coded DH
parameters Jan 15 10:29:54 imaptest imap[1024]: wrong version number in
SSL_accept() -> fail Jan 15 10:29:54 imaptest imap[1024]: STARTTLS
negotiation failed: [10.1.45.1] Jan 15 10:29:55 imaptest imap[1024]:
accepted connection
Jan 15 10:29:55 imaptest imap[1024]: wrong version number in SSL_accept()
-> fail Jan 15 10:29:55 imaptest imap[1024]: STARTTLS negotiation failed:
[10.1.45.1]
Thanks a lot for further information.
OK, I guess that's helpful. The reason for the failure is this line:
wrong version number in SSL_accept() -> fail
Now the question is why that happens. This is the code that logs the line:
case SSL_ERROR_SSL:
err = ERR_get_error();
if (err == 0) {
syslog(LOG_DEBUG, "protocol error in SSL_accept() -> fail");
} else {
syslog(LOG_DEBUG, "%s in SSL_accept() -> fail",
ERR_reason_error_string(err));
}
break;
So the server notes an SSL error, logs it and drops the connection. The
cause for the error seems to be something like this:
"Versions in client/server SSL records do not agree.
Probably your client sends SSL2 client_hello handshake
message and server is configured only for SSL3/TLS1.
In this situation server does not accept SSL2
client_hello what is being manifested by "wrong version
number" error.
To resolve this error you may disable SSL2 on client
or enable SSL2 handshake on server.
tcpdump output from wrong session handshake
may be helpful too."
What I don't understand is how it could've worked in earlier versions.
Anyway, could this be a client issue? Can you try other clients to see if
they handle this differently? Can you disable SSLv2 in your client?
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpucDoVRS9Hr.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.3.11 STARTTLS broken if tls_ca_file is defined
-- [EMAIL PROTECTED] is rumored to have mumbled on 16. Januar 2008 18:03:50 +0100 regarding Re: 2.3.11 STARTTLS broken if tls_ca_file is defined: It works on SSL (port 993). It doesn't works on port 143 with TLS. That makes sense, because AFAIK port 143 is for TLSv1 only. If the client tries anything but TLSv1 on port 143, that should cause an error. Doing some change on ssl in about:config of thunderbird gave me differents logs : Jan 16 17:53:27 imaptest imap[35698]: accepted connection Jan 16 17:53:27 imaptest imap[35698]: imapd:Loading hard-coded DH parameters Jan 16 17:53:27 imaptest imap[35698]: SSL_accept() incomplete -> wait Jan 16 17:53:59 imaptest imap[35698]: EOF in SSL_accept() -> fail Jan 16 17:53:59 imaptest imap[35698]: STARTTLS negotiation failed: [10.1.45.1] OK, but what did you change? I can't verify if it works because I don't have client certificates, but looking at my copy of Thunderbird the following account settings *should* work: Port 143, connect via TLS Make sure that encryption is set to TLS and *not* to SSL in the account settings ... -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgp2oOvZmDoxF.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cannot get sieve working with 2.3.11
-- Albert Chin <[EMAIL PROTECTED]> is rumored to have mumbled on 2. Februar 2008 08:21:44 -0600 regarding Cannot get sieve working with 2.3.11: $ find /ext/mail/sieve /ext/mail/sieve /ext/mail/sieve/global /ext/mail/sieve/global/china.sieve.bc /ext/mail/sieve/global/china.sieve.script /ext/mail/sieve/global/defaultbc $ cd /ext/mail/sieve/global $ ls -l -rw--- 1 cyrus cyrus 88 Feb 2 14:08 china.sieve.bc -rw--- 1 cyrus cyrus 48 Feb 2 14:08 china.sieve.script lrwx-- 1 cyrus cyrus 14 Feb 2 14:03 defaultbc -> china.sieve.bc The sieve script above should discard all mail with a Subject header but it does not. Any ideas? I don't use them myself, but AFAIK global scripts need to be imported into local scripts. It's not possible to just install a script and have it run for each user. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpHzQ3eS4dVH.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cannot get sieve working with 2.3.11
-- Albert Chin <[EMAIL PROTECTED]> is rumored to have
mumbled on 3. Februar 2008 02:18:23 -0600 regarding Re: Cannot get sieve
working with 2.3.11:
-rw--- 1 cyrus cyrus 88 Feb 2 14:08 china.sieve.bc
-rw--- 1 cyrus cyrus 48 Feb 2 14:08 china.sieve.script
lrwx-- 1 cyrus cyrus 14 Feb 2 14:03 defaultbc -> china.sieve.bc
The sieve script above should discard all mail with a Subject header
but it does not. Any ideas?
Ok, found the problem. In timsieved/actions.c we have:
if (sieved_userisadmin) {
strlcat(sieve_dir, "/global", size);
}
else {
char hash = (char) dir_hash_c(user, config_fulldirhash);
snprintf(sieve_dir+len, size-len, "/%c/%s", hash, user);
}
imapd.conf has:
admins: china
Removing myself from admins gets sieve working. But, why the above? It
would seem that Cyrus assumes users with administrative rights would
not be using sieve.
You should *never* use a user with admin rights for "normal" purposes. So
it's wrong to give yourself admin rights. Always use the default admin user
and use it only for admin tasks. Admin users are supposed to use sieve only
to manage global scripts.
--
Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
Universität zu Köln / Cologne University - Tel. +49-221-478-5587
pgp6N1JpKXSvl.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Non-ASCII chars in MAIL FROM envelope
--On 13. Februar 2008 09:50:58 +0100 Simon Matter <[EMAIL PROTECTED]> wrote: See http://archives.neohapsis.com/archives/postfix/2008-01/1321.html Yes, but that will block possibly valid mail. Of course I don't accept mail with non-ASCII RCPT TO addresses because Cyrus doesn't allow it, but I should accept non-ASCII MAIL FROM addresses if they are valid. But Cyrus also refuses them. That's the real problem. Isn't that controlled by reject8bit and munge8bit? munge8bit: 1 If enabled, lmtpd munges messages with 8-bit characters in the headers. The 8-bit characters are changed to ‘X’. If reject8bit is enabled, setting munge8bit has no effect. (A proper solution to non-ASCII characters in headers is offered by RFC 2047 and its predecessors.) reject8bit: 0 If enabled, lmtpd rejects messages with 8-bit characters in the headers. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgp4l8WLFAMAr.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Reject large emails
--On 13. Februar 2008 12:27:11 +0200 Nikos Gatsis <[EMAIL PROTECTED]> wrote:
We use sendmail. Do you know how to set up sendmail.mc to reject those
emails?
define(`confMAX_MESSAGE_SIZE', `5000')dnl
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpwuO9LJicin.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: STARTTLS on Cyrus IMAPd 2.3.11
-- Jorey Bump <[EMAIL PROTECTED]> is rumored to have mumbled on 19. März 2008 01:09:31 -0400 regarding Re: STARTTLS on Cyrus IMAPd 2.3.11: Can anyone confirm that STARTTLS connections to port 143 work with 2.3.11? Of course they do. We've been running 2.3.11 for a few months now and haven't had any issues with it. There must be another variable somewhere - perhaps the OpenSSL version. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpILl0ocDulh.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: reconstruct doing nothing
-- Alain Spineux <[EMAIL PROTECTED]> is rumored to have mumbled on 22. März 2008 11:29:36 +0100 regarding Re: reconstruct doing nothing: Conclusion - -r looks to be useless - -f discover yet unknow folder, recursively too, but only inside new folder, not if already known, use * to for a full discovery in two time user/[EMAIL PROTECTED] and user/first.lastname/[EMAIL PROTECTED] - '*' and '%' allow to walk around the mailbox tree, but only inside already know folder This was tested on a 2.3.11 FWIW, I think -r works just fine unless you're using virtual domains ... -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgp3W7ZEb9K4W.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Can Cyrus deliver to a default mailbox?
Hi, --On 25. März 2008 06:26:32 + "J.J. Day" <[EMAIL PROTECTED]> wrote: I am using Cyrus Imap 2.3 with Sendmail 8.14. I am new to both programs so I may not be interpreting the results correctly. But it appears that all mail to the system is received by sendmaiil as if the recipient is valid regardless of whether or not a user & mailbox exists in Cyrus. that's correct. The sendmail output from the sending system looks like: >>> RCPT To:<[EMAIL PROTECTED]> >>> DATA 250 2.1.5 <[EMAIL PROTECTED]>... Recipient okThe message ends up queued for delivery and a non delivery message is generated after the retry period expires. All mail to unknown users needs to be delivered to a "catchall" mailbox. Currently sendmail has a define for LUSER_RELAY and the imapd.conf file has an entry of soft_nouath: 0. So, I have two questions (I think either one will solve the problem) 1 - can Cyrus accept and deliver all messages to a default mailbox if the user / mailbox doesn't exist? See below. 2 - Is there a way for Cyrus to tell sendmail that the user doesn't exist? Yes. In fact there are several ways. We're using sendmail's virtusertable feature. You need an entry for each user. We've added the following entry: @uni-koeln.de error:5.1.1:550 User unknown So all addresses not explicitly listed will be handled as errors. You could just as well redirect all mail to unknown users to a specific catch-all user. Another approach is this one: <http://jmaimon.com/sendmail/anfi.homeunix.net/sendmail/rtcyrus2.html> Take your pick ... -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpUiTB2O0Ahl.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: fud
Hi,
--On 17. April 2008 08:05:56 +0200 Martin Schweizer
<[EMAIL PROTECTED]> wrote:
There is a fud-client binary, though it's availability varies by distro.
See this thread:
http://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-January/028273.ht
ml
Sorry for the delay but I was busy in the past few days. I checked the
above thread and find the file fud-client.c. I did compile it. As far
as I can see the client works but until now I did not find the right
command parameters for execute.
did you look at the code? Here's what it says right at the top:
/* This is an example client for the experimental fud service.
For this to work, the mailbox must have the local ACL bit 0 enabled
for user 'anonymous' ('anyone' will also work)
For example, from cyradm: 'sam anonymous 0'
*/
...
void
usage()
{
fprintf(stderr,"usage: fud-client [-p port] host user mailbox\n");
exit(EX_USAGE);
}
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgp9783wCRPNO.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
--On 23. April 2008 15:37:19 +0400 Dmitriy Kirhlarov <[EMAIL PROTECTED]> wrote:
Attached patch add to log information about moving messages between
folders. I am using this information from logs for relaunch dspam.
Any chances for add this patch to project tree?
FWIW, logging this at LOG_ERR level certainly isn't the right way to do
that ... I'd say it should be INFO at best, if not DEBUG.
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpHwdZftQnTl.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Hello cyrus users
--On 24. April 2008 09:06:33 +0200 Simon Matter <[EMAIL PROTECTED]> wrote: Holy moly. Looking for cyrus on machine was interesting as almost all of the files would not come up with a 'locate' command. And still do not. I am manually going through all the doc folders and etc and sys folders since locate cannot find them (and they are there) maybe rpm -ql cyrus-imapd True. "locate" only works after updatedb has been run. There is only one book on amazon about cyrus and it is not in print yetone. And it has nothing to do with sendmail. Redhat has little info on it although it calls it the 'default' pop3 for the enterprise 5.1 (centos in my case) All I am looking to do is have sendmail put the mail into users mailboxes (users who will be from virtual hosts like [EMAIL PROTECTED] and [EMAIL PROTECTED]) and just allow me to grab the emails with good old outlookand of course send the mails (but that is sendmail I believe.) And you think that's *easy*?? Whenever you work with virtual hosts, things get tricky. Are there any good tutorials just for that, cause by god the program does everything under the sun and is rather confusing at first glance... How many users do you have, how do you want to authenticate them? Things are quite different if you have 10 or 1 users on a server. Cyrus can do alot which is why it has quite some features you may have to configure. I can't help with sendmail, I didn't touch it in the last 8 years but found postfix much easier for me. I can help with the sendmail integration if neccesary. The authentication is probably the most difficult part, though. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpCwKf1vUa7K.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Return-Path in Sieve Scripts
Hi,
--On 16. Mai 2008 14:20:09 +0200 Nikolaus Rath <[EMAIL PROTECTED]> wrote:
I am delivering mail to cyrus from exim using lmtpd.
Apparently lmtpd adds the Return-Path header only after the sieve
scripts have been processed, because in the scripts I cannot match
this header.
If I add the return-path using exim before the message is delivered to
lmtpd, the sieve script works. However, lmtpd then adds a second
return-path header. This is does not cause any actual problems, but I
do not like it nevertheless.
Is there a way to forbid lmtpd to add a second return-path header or,
alternatively, to have it add the header earlier, so that I can use it
in sieve scripts?
I can't answer your question, but I don't think it's a good idea to even
try it like that. That's what the "envelope" test in RFC 5228 is for ...
require "envelope";
if envelope :all :is "from" "[EMAIL PROTECTED]" {
discard;
}
<http://tools.ietf.org/html/rfc5228#section-5.4>
So the better question would be IMO if and when Cyrus will support that
test.
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgprEVeCu7Fpq.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: How to repair a broken seen state file
--On 7. Juli 2008 08:31:59 -0500 Gary Mills <[EMAIL PROTECTED]> wrote: I'm seeing errors like this regularly in our messages log: Jul 4 11:43:37 castor imap[16398]: [ID 514311 local6.error] DBERROR: skiplist recovery: 058C should be INORDER Jul 4 11:43:37 castor imap[16398]: [ID 729713 local6.error] DBERROR: opening /imap/conf/user/O/inqarts.seen: cyrusdb error It's always for this one file. All the others are fine. It's a skiplist database. The ownership is correct. A reconstruct on the mailbox has no effect on the file. That's to be expected. Can I just remove the file, with no ill effects? Yes, but of course all "seen" information is lost in that case. Can I fix it somehow? Simon Matter has posted a script to the list: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/2006-April/021522.html> I've never used it. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpMgAyEhBAln.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Reconstructing mailboxes when using delayed expunge
Hi, we're running Cyrus 2.3.11 with "delete_mode: delayed". Today, for the first time since using that feature, we had to undelete an entire hierarchy of mailboxes a user had inadvertently deleted. We found that it was more tricky than we had anticipated. This is an anecdotal report of how we managed to do it eventually. I hope this might help other admins in the future, because I don't think this is really documented anywhere. The first issue was finding the deleted mailboxes. It seems they can be anywhere, more or less. We have three partitions. We managed to find them by doing this: ctl_mboxlist -d | grep DELETED | grep username Then we had to look for them with mbpath, replacing the '.' with '/', because we use unixhierarchysep. The next surprise was that the actual mails were in subdirectories with funny names, e.g.: DELETED/user/xxx/blabla/bla/48873884 (That's when the original mailbox was called user/xxx/blabla/bla) I found out that a timestamp is added by reading the source. I suppose that's so that multiple deletions of a mailbox of the same name don't clobber prior versions. We then copied the contents of the timestamped folders to a newly created folder in the user's mailbox. Our internal documentation said not to copy the metadata (i.e. the cyrus.xxx files), but I don't remember why. Is that the right thing to do? It caused us some headaches later on. Then we tried our usual reconstruct approach, i.e. 'reconstruct -rf user/xxx'. However, the restored mailboxes weren't added. The reason is that -f only works when a cyrus.header file exists. Then we tried touching cyrus.header in the folders. But that's no good in itself, either. There was an error message that the mailbox had an invalid format. A look at the manpage suggested that -x should help. That sort of worked, but two mailboxes still weren't added because there was no cyrus.header file. So without the trial & error, the correct approach seems to be: - find the mailbox(es) - copy their contents (with or without metadata) - touch cyrus.header in *all* mailboxes if you did not copy the metadata - run 'reconstruct -rfx' on the user Has anybody else found a better way? Does everybody agree that there should be an 'unexpunge' for mailboxes as well?? I guess an unexpunge for mailboxes would have to let you specify a target folder, because simply writing everything to the original position might not be correct ... Cheers, Sebastian Hagedorn -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpuDMp9dYrAQ.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Reconstructing mailboxes when using delayed expunge
Hi, --On 23. Juli 2008 12:44:13 -0500 Paul Engle <[EMAIL PROTECTED]> wrote: You can specify the prefix for deleted mailboxes with the 'deletedprefix' option in imapd.conf. It defaults to DELETED, so anything deleted will go into a tree starting with that, preserving the same structure as on the live partition. I was aware of that, but with three partitions you won't know on which it is and which hash letter was used (we are using fulldirhash as well). It's been our understanding that the delayed delete process is really just a rename under the hood. It just renames the mailboxes to somewhere outside the 'normal' area so that users no longer see them. As such, we've generally had success by just doing another rename to get the mailbox back. If the user hadn't gone and created a new mailbox with the same name, then it would just be: cyradm> rename DELETED/user/foo/bar/baz/TIMECODE user/foo/bar/baz [partition] If the mailbox was indeed recreated in the interim, then we usually just add a suffix to the mailbox and tell the user where it is: cyradm> rename DELETED/user/foo/bar/baz/TIMECODE user/foo/bar/baz_RESTORE [partition] That's great! I wish I had known that before :-) --On 23. Juli 2008 20:41:52 -0400 Wesley Craig <[EMAIL PROTECTED]> wrote: On 23 Jul 2008, at 13:44, Paul Engle wrote: You do have to do the ctl_mboxlist -d to get the specific timecode, though. You can also use, e.g.: cyradm> lm DELETED.user.wc2263.* DELETED.user.wc2263.EG.487BF83F (\HasNoChildren) DELETED.user.wc2263.XXX.487BF841 (\HasNoChildren) DELETED.user.wc2263.XXX.4887CA8E (\HasNoChildren) cyradm> That's another great tip! A little convoluted, but you can get the human readable time by subtracting the time code above (which is in hex) from "now". E.g.: : ; date +"%s" 1216859626 : ; dc 1216859626 16i 487BF841 -p 1216084033 775593 : ; date --date="-775593 seconds" Mon Jul 14 21:11:28 EDT 2008 : ; OK, maybe more than a little convoluted... I'm note really a dc fiend, so I'm not 100% sure what you are doing there, but doesn't this work just as well? % perl -e 'print scalar localtime(hex "487BF841")."\n"' Tue Jul 15 03:07:13 2008 The difference in time is only because of our different timezones ... Thanks for all the help, Sebastian -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpbmwQ2w7lqo.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: snmp monitoring two cyrus instances on one machine
--On 8. September 2008 16:52:37 +0200 Rudy Gevaert <[EMAIL PROTECTED]> wrote: Some time ago I tried to get snmp monitoring of my cyrus installation. I got stuck back then and gave up. Today I tried again and surprisingly I managed to get a lot farther. I can read out the snmp values of 'a' cyrus instances. On each machine I run two cyrus instances. Using different IP addresses or just different ports? One master and one replica. However it is not clear how I can distinguish between the master and the replica. If they are using different IP addresses I would expect that to work. I'ne never done that myself, though. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpcZWI6tKZp4.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Which 2.3.x version to match 2.2.12 in stability? :-)
--On 16. September 2008 14:17:57 +0200 Ciprian Marius Vizitiu <[EMAIL PROTECTED]> wrote: As much as I hate it time has come to upgrade my very well behaved Cyrus imapd so I was wondering: given the rock solid stability I have experienced with 2.2.12 is there any 2.3.x Cyrus with some close record of stability? Sure! 2.3.x is at least as stable as 2.2.x, I'd say even more so. We're currently running 2.3.11, but I'd use the latest version. We're planning on upgrading soon. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgphRqXeP0oZO.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [OT] m$ activesync with cyrus backend
--On 16. September 2008 18:31:31 + Andy Fiddaman <[EMAIL PROTECTED]> wrote: ; is there a way to mimic exchange server in the linux environment where i ; can keep my cyrus/postfix/amavis install -- is there a linux app that ; speaks "exchange"? Z-Push (http://z-push.sourceforge.net/) does the job nicely. I use that for my iPhone with Cyrus as the backend. After I read that, I am now trying to get that to work. I don't have any ActiveSync-capable clients myself, but a colleague of mine has a Windows Mobile cell phone. He managed to configure the account, the login to our Cyrus server is successful, but nothing is synchronized. I've enabled debug.txt, but there's nothing enlightening to be found there. What's more, without having gotten it to work, I fail to see what the advantage over a proper IMAP account is. I don't see how there could be an actual push component with such a setup. Maybe I'm missing something? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpG3hRtwgoJz.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [OT] m$ activesync with cyrus backend
--On 18. September 2008 16:35:03 +0200 lartc <[EMAIL PROTECTED]> wrote: essentially what's happening here is that apple's sdk has no api connections to calender/tasks/notes/etc -- only contacts. I get that ... so without jailbreaking the iphone, the only way to sync without wires is MobileME or exchange. Not quite, unless I'm mistaken. If all you're interested in is mail, you could just as well configure an IMAP account for your Cyrus server, couldn't you? I thought the only advantage to ActiveSync for mail was push. Am I wrong? Is there more to it? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpc901P74X0N.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Sieve: UTF-7 changes in 2.3.12p2?
--On 25. September 2008 12:21:33 +0200 Jan Schneider <[EMAIL PROTECTED]> wrote: Maybe this only a theoretical problem. Is there any desktop client out there that *generates* Sieve scripts, i.e. not simply download and upload complete scripts like KMail? Yes, Mulberry does that. But it's write-only, i.e. it can't list or edit scripts that are on the server, so IMHO it's not very useful. I suppose we'll need to patch our Smartsieve installation before we upgrade to 2.3.12 ... -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpctsykZUvOe.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Folder allowed character
-- Alain Spineux <[EMAIL PROTECTED]> is rumored to have mumbled on 20. November 2008 07:57:13 +0100 regarding Re: Folder allowed character: On Tue, Nov 18, 2008 at 7:38 PM, Wesley Craig <[EMAIL PROTECTED]> wrote: On 18 Nov 2008, at 05:28, Antonio Talarico wrote: Where i can found a list with allowed character for a folder name? ./imap/mboxname.c:#define GOODCHARS " #$'+,-. 0123456789:[EMAIL PROTECTED]" These characters are about the mailbox name. Imap clients must use a modified UTF-7 encoding for Folders name. This allow you to use any characters in any language (if your client is compliant on this part of imap spec). Actually not *any* character. You can't have a '/' or a '!', for example. Those characters aren't encoded any differently with modified UTF-7. AFAICT you can use any character that's outside of US-ASCII, but only those from US-ASCII that are in GOODCHARS. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpBhEk2XmxQp.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
AW: Cyrus-IMAPD Hot Standby Configuration
-- Helmut Weigel <[EMAIL PROTECTED]> is rumored to have mumbled on 24. November 2008 22:19:58 +0100 regarding AW: Cyrus-IMAPD Hot Standby Configuration: Replication requires 2.3.x and 2.3.13 is the best replicating cyrus imapd available. RedHat ships 2.3.7 with RHEL Enterprise Server 5. Is that sufficient? Does anybody have experiences with that version? There's no need to use Red Hat's stock RPMs, because Simon Matter provides excellent more current ones. We (Universität zu Köln) have been using them successfuly for years: <http://invoca.ch/pub/packages/cyrus-imapd/> Initially we staid with Red Hat's because we thought they'd give us better support if we did, but their support isn't worth much. Believe me, you're much better off with the community support you're getting here! -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpRfG62Kmsoo.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: choosing a file system
-- Nik Conwell is rumored to have mumbled on 31. Dezember 2008 07:47:31 -0500 regarding Re: choosing a file system: Just curious - how do stop people from accessing their mailboxes during the time they are being renamed and moved to another partition? I just do a grep on the username in the proc directory - if there is no process for that user, I figure it's safe enough to move the mailbox. This approach has worked well so far. I experimented with accessing a mailbox while it was being moved and that seemed to be OK as well, i.e. it failed while the operation was in progress. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpPU72K0BOGZ.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: login issues
-- dick hoogendijk is rumored to have mumbled on 2. Januar 2009 21:04:37 +0100 regarding login issues: All mail users are in sasldb. But now the problem is that I get more and more outlook (MS) clients that can't do CRAM-MD5. So, I guess the imapd.conf I use needs some tweaking. What do I have to change so that outlook express users can login? We use the following for maximum compatibility, but it requires that all those SASL modules are actually installed: allowanonymouslogin: no allowplaintext: no sasl_pwcheck_method: auxprop sasl_mech_list: DIGEST-MD5 CRAM-MD5 PLAIN NTLM LOGIN That way Outlook can either use NTLM or some SSL/TLS secured connection. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpZ3G9tQxn82.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: different Cert for POP/IMAP
-- mno is rumored to have mumbled on 10. Januar 2009 09:58:29 +0100 regarding different Cert for POP/IMAP: When configuring the individual parameter pop3_tls_cert_file and pop3_tls_key_file it seems these params are ignord. Looking at the manpage (man imapd.conf) are not mentioned. Though I' not a programmer, I had a look at the source itself and did not find any hint for the Use of pop3_tls_cert_file and pop3_tls_key_file. These params are useless - can anybody confirm this? No. How can I use different Certificates for pop and imap ? Yes. We've got this in imapd.conf and it works just fine: sieve_tls_cert_file: /var/lib/imap/imapd.cert sieve_tls_key_file: /var/lib/imap/imapd.key imaps_tls_cert_file: /var/lib/imap/imapd.cert imap_tls_cert_file: /var/lib/imap/imapd.cert imaps_tls_key_file: /var/lib/imap/imapd.key imap_tls_key_file: /var/lib/imap/imapd.key pop3s_tls_cert_file: /var/lib/imap/popd.cert pop3_tls_cert_file: /var/lib/imap/popd.cert pop3s_tls_key_file: /var/lib/imap/popd.key pop3_tls_key_file: /var/lib/imap/popd.key tls_ca_file: /var/lib/imap/chain.pem Perhaps you need pop3s? -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgp52LGwY0TMf.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.2.12 to 2.3.13 upgrade :
Hello Jeff,
--On 16. Januar 2009 06:01:48 -0500 Jeff Blaine
wrote:
Our happy and functioning 2.2.12 server setup shows the
following when switched over to 2.3.13. The odd thing is
that we're using skiplist, not berkeley, for our dbs.
If anyone has comments, I'd love to hear them.
bash-2.05# cat /etc/imapd.conf
configdirectory:/var/imap
defaultpartition: default
partition-default: /var/spool/imap
imap_admins:root cyrus
sasl_pwcheck_method:saslauthd
sieveusehomedir:true
autocreatequota:20
duplicate_db: skiplist
tlscache_db:skiplist
annotation_db: skiplist
tls_cert_file: /var/imap/ca.crt
tls_key_file: /var/imap/server.pem
tls_ca_file:/var/imap/ca.crt
bash-2.05#
what about mboxlist_db? And even if all your dbs *are* skiplist, you might
still have remants from previous Berkeley dbs in your /var/lib/imap/db
directory. If you are sure you don't need those, just delete them.
Obviously the new Cyrus has found a Berkeley environment somewhere ...
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpDVoENV5UEa.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
Hello Jeff, --On 16. Januar 2009 06:38:27 -0500 Jeff Blaine wrote: Maybe we're doing something wrong in the process, but it seems that every time we perform offline maintenance (upgrade, whatever) on Cyrus IMAPd ... our users complain that TLS breaks afterward, but then fixes itself in time. I've demonstrated this to myself just now with the upgrade to 2.3.13 from 2.2.12. My TLS session is cached but broken with the new setup (or for whatever other reason). That is, even after restarting Thunderbird, I get the following: Jan 16 06:31:50 imapsrv imap[19690]: [ID 239158 local6.notice] STARTTLS negotiation failed: bva-172.our.com Is there a way to zero/flush all TLS cached sessions? I have to imagine there is, but I don't know how. as before: just delete the tls_sessions files before you start cyrus-imapd. They will be recreated automatically. You could even make that part of the initscript, because those session don't survive a restart anyway. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgp2ADvpcojIT.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
--On 16. Januar 2009 06:58:06 -0500 Jeff Blaine
wrote:
Hmm. That's not working for me.
bash-2.05# pwd
/var/imap
# rm tls_sessions.db
# /etc/init.d/imap start
#
Jan 16 06:54:36 imapsrv imap[20300]: [ID 286863 local6.notice]
imapd:Loading hard-coded DH parameters
Jan 16 06:54:36 imapsrv imap[20300]: [ID 239158 local6.notice] STARTTLS
negotiation failed: bva-172.our.com
bash-2.05# pwd
/var/imap
bash-2.05# ls -l tls*
tls*: No such file or directory
bash-2.05#
Strange. I know for a fact that it has worked for us in the past. I can't
verify that easily right now, because I don't currently have a test set-up.
What happens if you touch the file prior to starting imap?
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpd0Ta8ee4lJ.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
--On 16. Januar 2009 07:48:18 -0500 Jeff Blaine wrote: More info after increasing local6.info to local6.debug for syslog: accepted connection imapd:Loading hard-coded DH parameters SSL_accept() incomplete -> wait decryption failed or bad record mac in SSL_accept() -> fail STARTTLS negotiation failed: bva-172.our.com Our TLS all worked fine before the upgrade :( I'm pretty sure the tls_cache is a red herring. The SSL/TLS code changed a lot between 2.2 and 2.3. My guess would be that there lies the actual problem. I wonder where the line "Loading hard-coded DH parameters" comes from. I haven't seen that before. Anyway, I guess you need an SSL expert to make sense of that. How old is your certificate? Maybe the new code doesn't like it? Did you build the binary yourself or where did you get it? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpIxKGHpc25t.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
--On 16. Januar 2009 09:43:02 -0500 Jeff Blaine wrote: A new cert did not solve the problem: Jan 16 09:41:30 imapsrv imap[12264]: [ID 921384 local6.debug] accepted connection Jan 16 09:41:30 imapsrv imap[12264]: [ID 192010 local6.debug] wrong version number in SSL_accept() -> fail But it results in a different error message. Jan 16 09:41:30 imapsrv imap[12264]: [ID 239158 local6.notice] STARTTLS negotiation failed: bva-172.our.com That reminds me of something. Try removing this line from your config: tls_ca_file:/var/imap/ca.crt Also, try using different clients. IIRC, there is an issue specifically with Thunderbird and that setting. I don't remember the details, but you should be able to find them in the archives. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgp9eUvstlcAX.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
-- Jeff Blaine is rumored to have mumbled on 16. Januar 2009 10:12:00 -0500 regarding Re: Expire (manually) TLS sessions?: With the tls_ca_file line removed, Thunderbird asked me to specify a client certificate, I chose my cert and entered my password to access it. It's quite possible that authentication with a certificate is broken. I helped fix a few SSL-related bugs that affected us, but we don't use certificates. That means that the changes that were made weren't tested using such a setup. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpjvYOUFzVzb.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
-- Jeff Blaine is rumored to have mumbled on 16. Januar 2009 10:19:51 -0500 regarding Re: Expire (manually) TLS sessions?: Outlook 2007 works. Using a password, that is. I guess you didn't try it with a certificate? (I assume Outlook supports that, but I'm not sure) Unfortunately, this is not an option for us as our users use Thunderbird. How about Thunderbird using a password for authentication? Is that an option at all? -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpEgLvR32NtC.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
-- Bron Gondwana is rumored to have mumbled on 17. Januar 2009 13:24:41 +1100 regarding Re: Expire (manually) TLS sessions?: just delete the tls_sessions files before you start cyrus-imapd. They will be recreated automatically. You could even make that part of the initscript, because those session don't survive a restart anyway. In that case maybe Cyrus should do this itself? Sounds like a candidate for a fix. I'm not sure. Right now it seems to me that Jeff's problem didn't have anything to do with that file. The problems we were having with it were years ago, back when we used Berkeley DB. We have since switched to skiplist and haven't had any trouble anymore. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpkdAH47TUFv.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Expire (manually) TLS sessions?
Hi Jeff, --On 21. Januar 2009 11:19:31 -0500 Jeff Blaine wrote: Sorry for the delay -- I had my wedding and a brief mini-honeymoon to attend to ;) congrats! How about Thunderbird using a password for authentication? Is that an option at all? I realize this is a little "all over the road" here, but bear with me as I am just trying to get something working at this point for our users who are now without secure IMAP :( With "TLS" selected in Thunderbird, I am given no choice but to select a client certificate. See attached images. I wonder why that is. The only reason that comes to mind is that you *have* a certificate. I don't and so I'm never asked to use it. So why don't you try removing your certificate? Honestly, I would expect the same to happen that happens when you use SSL, but you never know. Another user reports that GNU Emacs with the Gnus client works with SSL and port 993. I've confirmed this in the log: Jan 21 11:11:03 imapsrv imaps[14170]: [ID 277583 local6.notice] login: jimbo-host.our.com [xx.xx.50.67] jimbo plaintext+TLS User logged in If I configure Thunderbird to do that (SSL via 993), I get the following: Jan 21 11:10:19 imapsrv imaps[14104]: [ID 636471 local6.notice] TLS server engine: cannot load CA data Jan 21 11:10:19 imapsrv imaps[14104]: [ID 286863 local6.notice] imapd:Loading hard-coded DH parameters Jan 21 11:10:19 imapsrv imaps[14104]: [ID 798856 local6.notice] imaps TLS negotiation failed: myclient.our.com Jan 21 11:10:19 imapsrv imaps[14104]: [ID 637875 local6.error] Fatal error: tls_start_servertls() failed I have no idea why that happens. I just tried it myself and got the following in our log: Jan 21 18:17:48 lvr13 imaps[9855]: accepted connection Jan 21 18:17:48 lvr13 imaps[9855]: SSL_accept() incomplete -> wait Jan 21 18:17:48 lvr13 imaps[9855]: SSL_accept() succeeded -> done Jan 21 18:17:48 lvr13 imaps[9855]: starttls: TLSv1 with cipher AES256-SHA (256/256 bits new) no authentication Jan 21 18:17:53 lvr13 imaps[9855]: login: [redacted] User logged in Could it be that your OpenSSL version or your certificate somehow don't support features that Thunderbird requires? I'm really no expert, but I know that client and server *negotiate* about these things. And the error reads "negotiation failed" ... If your server is accessible over the Internet, perhaps I could try connecting to it with "openssl s_client". That might tell us something. You can try that as well, of course. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpEqVfGk4Rwe.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Question about CYRUS-IMAP and FETCH BODY[]
Hi, --On 18. März 2009 21:14:36 -0600 Paulino Calderon wrote: Hey guys, We recently upgraded cyrus-imapd in one of our production servers (debian btw) to 2.2.13-14+b3. After that, one of our programs stopped working complaining about not being able to fetch the messages, so I took a look at the traffic between our imap server and the program and I saw this: http://calder0n.com/cyrus-imap-traffic.png As you can see, the connection is being established succesfully but our program ( it was running OK for almost 2 years btw ) is sending a: IMAP FETCH BODY[1] actually that's not true. It's sending a *UID* FETCH, which is something quite different. And the server is only responding: IMAP Ok Completed (0.000 sec) That's quite correct if no message with UID 2 exists ... you've given no indication that it *does* exist. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpbncXOdmHi6.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Question about CYRUS-IMAP and FETCH BODY[]
-- Paulino Calderon is rumored to have mumbled on 19. März 2009 11:40:53 -0600 regarding Re: Question about CYRUS-IMAP and FETCH BODY[]: As you can see, the connection is being established succesfully but our program ( it was running OK for almost 2 years btw ) is sending a: IMAP FETCH BODY[1] actually that's not true. It's sending a *UID* FETCH, which is something quite different. And the server is only responding: IMAP Ok Completed (0.000 sec) That's quite correct if no message with UID 2 exists ... you've given no indication that it *does* exist. Umm, well after a: IMAP SELECT INBOX The server is responding: http://calder0n.com/cyrus-imap-selectinbox-response.png So? I still see no evidence that a message with *UID* 2 exists. The response only shows that a message with *sequence number* 2 exists! That's why I said that the actual command is quite different from what you claimed it was. This command will give you a response whenever there are at least two messages: 10 FETCH 2 BODY[1] This command won't give you a response even if there are 100,000 messages in the mailbox if none of them has UID 2: 10 UID FETCH 2 BODY[1] -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Regionales Rechenzentrum (RRZK) Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpJcPlV4nqeo.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Question about CYRUS-IMAP and FETCH BODY[]
Hi, --On 19. März 2009 22:43:43 -0600 Paulino Calderon wrote: Yeah I've realized it would be for the best to replace that old library. It's going to fail again if we leave it there. Anyway, I'm still curious why it broke after the upgrade, what do you guys think? Is it just not following a newer RFC as it's supposed to? no, nothing has changed in that respect. The library we're using is: http://www.codeproject.com/KB/IP/imaplibrary.aspx Looking at the documentation it strikes me that there appears to be no way to actually list the contents of a mailbox. If those commands are really the only ones that exist, the only way to use it would appear to be a SearchMessage followed by a FetchMessage. If you don't do a SeachMessage, you have no way of knowing what you can fetch. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpNrrgnR5EZg.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Does Cyrus benefit greatly from increased FS buffer cache?
--On 16. April 2009 10:58:15 +1000 Rob Mueller wrote: http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernel s-cache-vs-inode-memory/ Anyone have any specific thoughts? Is there any other benefit we might see from large memory allocation in 64-bit architecture? Given that I wrote that blog post, I can only tell you that in our environment, 64-bit kernels made a big difference. I wonder if ext3 behaves differently, Red Hat's 32-bit behaves differently, or if something altogether different is going on. We are currently running RHEL 3 in 32-bit mode, our servers have 16 GB, and most of it is used for caching: # free total used free sharedbuffers cached Mem: 16214344 16197612 16732 0 86944 13415172 -/+ buffers/cache:2695496 13518848 Swap: 4192944 84364184508 So it would seem that a 64-bit kernel wouldn't improve on that, right? Or is that a difference between 2.4 and 2.6? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpYjennlWZCV.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Does Cyrus benefit greatly from increased FS buffer cache?
-- Andrew Morgan is rumored to have mumbled on 16. April 2009 09:52:15 -0700 regarding Re: Does Cyrus benefit greatly from increased FS buffer cache?: So it would seem that a 64-bit kernel wouldn't improve on that, right? Or is that a difference between 2.4 and 2.6? That's interesting, and not what I expected. :) What does "cat /proc/meminfo" show? # cat /proc/meminfo total:used:free: shared: buffers: cached: Mem: 16603488256 16582602752 208855040 96661504 14362886144 Swap: 4293574656 8638464 4284936192 MemTotal: 16214344 kB MemFree: 20396 kB MemShared: 0 kB Buffers: 94396 kB Cached: 14018624 kB SwapCached: 7632 kB Active: 12502776 kB ActiveAnon:2275396 kB ActiveCache: 10227380 kB Inact_dirty: 2378304 kB Inact_laundry: 453980 kB Inact_clean:284768 kB Inact_target: 3123964 kB HighTotal:15597440 kB HighFree: 5724 kB LowTotal: 616904 kB LowFree: 14672 kB SwapTotal: 4192944 kB SwapFree: 4184508 kB Committed_AS: 2391184 kB HugePages_Total: 0 HugePages_Free: 0 Hugepagesize: 2048 kB -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Regionales Rechenzentrum (RRZK) Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgppvjY80Iq6y.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: finding by message id
--On 5. Mai 2009 15:45:36 +0200 Gabriele Bulfon wrote: I've got a message logged by posfix with a specific message id, and stated as correctly sent to cyrus. But the user of the imap mailbox is sure that he never received that message. All users lie. No filter is setup to move the message elsewhere, and no error is visible anywhere. So, I've got the message id from the postfix log. I know that cyrus checks for duplicate ids, so he's got quick knoweledge of every id in the imap folders. How can I ask cyrus for an ID and see if it actually exists somewhere? And how to find it without having to scan all imap folders? Cyrus can log the message id's it delivers, e.g.: May 5 15:49:24 lvr13 lmtpunix[1219]: Delivered: <101915.23.1241531136265.javamail.gbul...@pgbulfon> to mailbox: user.a0620 I'm not sure which log level you need to get these messages. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgp1vhETgtkLS.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: unexpunge segfault part 2
--On 6. Mai 2009 10:20:39 +1000 Bron Gondwana wrote: I've just enabled CONDSTORE for a sacrificial few thousand users to see what happens :) Including me of course! The new Thunderbird beta supports using it, so I want it on! Interesting. I'd ignored that so far, but now I got curious. I have enabled it on my INBOX and managed to do so on all my subfolders like this: mboxcfg user/xxx/* condstore true Is that how you did it or is there a better way? And how exactly does the new mailbox_default_options work? The manpage only says: Default "options" field for the mailbox on create. You’ll want to know what you’re doing before setting this, but it can apply some default annotations like condstore or duplicate supression So how would I enable condstore for all new mailboxes? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpPmwjckmBV9.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: misterious duplicate message-id
--On 7. Mai 2009 13:13:24 +0200 Gabriele Bulfon wrote: as a follow up to my "finding by message id" message, I found the reason why that message was silently discarded: for 3 times that Outlook user could send an email with the exact same message-id generated by Outlook! How is it possible?! That's a known Outlook bug. I don't recall the precise circumstances, but I've seen that again and again. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpJCXPjJlpC8.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: misterious duplicate message-id
--On 7. Mai 2009 14:11:36 +0200 Gabriele Bulfon wrote:
I supposed it was a known bugbut it seems to be unknown to google!
I looked for "outlook duplicate message-id" in various formsno result!
Not so here ... e.g.:
<http://www.mail-archive.com/info-cyrus@lists.andrew.cmu.edu/msg36497.html>
<http://support.microsoft.com/kb/289593>
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
.:.Regionales Rechenzentrum (RRZK).:.
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
.:.:.:.Skype: shagedorn.:.:.:.
pgpYUHXjoYdw4.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: imapd locked
--On 18. Mai 2009 14:00:33 +0200 Gerald Nowitzky wrote: I had a strange problem with my cyrus here. I am starting imapd with imapcmd="imapd -U 50" listen="imap" prefork=0 maxchild=100 imaps cmd="imapd -s -U 50" listen="imaps" prefork=0 maxchild=100 Three times this morning I found that imap was stalled. What dou you mean when you write "stalled"? I had 100 instances of the imap imapd-process. ... It might well be that I hit the maxchild limit on imap, but that shouldnt stall the complete imap, should it? Depends on what you mean. New connections to imapd block block indefinitely until the number of processes goes below maxchild. If that's what you call "stall", it's to be expected. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. pgphfqmmw9uof.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: imapd locked
--On 18. Mai 2009 08:41:42 -0400 Adam Tauno Williams wrote: Depends on what you mean. New connections to imapd block block indefinitely until the number of processes goes below maxchild. If that's what you call "stall", it's to be expected. If connections are forced into a wait-state by a maxchild limitation is this condition logged or any kind of notice raised? AFAIK no. That got me in the early days of our using Cyrus. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. pgpd1xnNkRBF4.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Sieve: Vacation not working (solved)
--On 27. Mai 2009 22:42:30 -0400 "Gottschalk, David" wrote: Well, I tried to enable this same feature this evening in sendmail, but ran into some problems. Forgive me as I am complete novice when it comes to enabling this feature in sendmail. I enabled this in my sendmail.cf, You didn't actually edit the .cf file, did you? That's not how it works ... you need to edit the .mc and generate the .cf. then got the following error: unknown mailer cyrusv2 -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. pgpGH6Rpkaax6.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Sieve: Vacation not working (solved)
--On 28. Mai 2009 11:39:03 +0200 Garry wrote: You didn't actually edit the .cf file, did you? That's not how it works ... you need to edit the .mc and generate the .cf. He didn't, he put it in the .mc and m4'd it ... Wouldn't the m4 command have thrown an error in that case if it didn't know the cyrusv2 mailer? Unless you use the m4 files of a newer sendmail than the one you're actually running, that is. anyway, I suppose the cause is the rather old sendmail 8.12 he is running, which most likely doesn't have the appropriate files for cyrusv2 as a mailer yet ... Actually it does, at least with 8.12.6 and higher. 8.12.2 doesn't seem to have it. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. pgpNlKAiwoQRI.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.3.14: posting to shared mailbox results in 550 Permission denied
--On 29. Mai 2009 10:12:32 +0200 Simon Matter wrote: I can confirm that anonymous posting to a shared folder with 'anonymous p' rights works on the 64bit version but not on the 32bit version. So there must be a change between 2.3.13 and 2.3.14 which breaks something on 32bit systems but not on 64bit. Is there anybody around who can confirm this? We've always used the 'anyone p' right for this purpose, that's why I didn't notice anything ... I just set up a mailbox to test this: cyrus.rrz.uni-koeln.de> lam posting anonymous lrsp anyone lrs Not surprisingly, the result is the same one you observed: it only works when 'anyone' has the 'p' right. (we're running a 32bit system) So where does it say that the 'anonymous p' right is *supposed* to work? The only reference I've been able to find is overview.html, and that isn't particularly clear. What's the harm in granting 'anyone p' instead? -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. pgpzUjPOLbEdy.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Self-healing mailbox?
Today I noticed the following in our Logwatch script's output for Cyrus (2.3.14): May 30 04:51:06 lvr13 cyr_expire[562]: IOERROR: reading cache record for user.aqg04: initial bogus offset 11091472 of 75256 for 6/10; mailbox needs a reconstruct May 30 04:51:06 lvr13 cyr_expire[562]: failure expiring user.aqg04: System I/O error That's the first time I've seen one of these new messages, but I'd read about them and wasn't surprised. I didn't reconstruct the mailbox immediately, though. And when I looked at today's log, I found: May 31 04:46:40 lvr13 cyr_expire[28829]: Expunged 7 messages from user.aqg04 What gives? Can I assume that the error from yesterday somehow healed itself? Or should I run reconstruct anyway? -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Regionales Rechenzentrum (RRZK) Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpUbFdPZ9yr4.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Self-healing mailbox?
-- "Ciprian Marius Vizitiu (GBIF)" is rumored to have mumbled on 31. Mai 2009 09:27:41 +0200 regarding Re: Self-healing mailbox?: What gives? Can I assume that the error from yesterday somehow healed itself? Or should I run reconstruct anyway? I for one 'd worry more about that "System I/O error ". It can be that there was a storage error which didn't happen during a second attempt. I doubt it, because the same error occurred one day earlier: May 29 04:53:38 lvr13 cyr_expire[2602]: IOERROR: reading cache record for user.aqg04: initial bogus offset 11090136 of 32488 for 5/7; mailbox needs a reconstruct May 29 04:53:38 lvr13 cyr_expire[2602]: failure expiring user.aqg04: System I/O error I interpret the "System I/O error" to be the IOERROR from the line before, i.e. not actually an I/O error but rather a corrupt file. The error does not show on any of the previous days, nor does it show today. -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Regionales Rechenzentrum (RRZK) Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpfs8eEFkuy9.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Self-healing mailbox?
--On 31. Mai 2009 22:41:52 +1000 Bron Gondwana wrote:
I interpret the "System I/O error" to be the IOERROR from the line
before, i.e. not actually an I/O error but rather a corrupt file. The
error does not show on any of the previous days, nor does it show today.
Sounds to me like it was copying the record to the cyrus.expunge.NEW
file each time for the earlier days (and failing, so skipping the
entire mailbox), but now the record has actually expired, so it
doesn't need to copy the cache record to the cyrus.cache.NEW file,
and hence never looks at it.
Meaning: yes, it did heal itself!
Thanks for the explanation!
--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
.:.Regionales Rechenzentrum (RRZK).:.
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.
pgpUMYHxuRFYC.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
