Re: avelsieve problem switching mail servers

[Maria McKinley]

Simon Matter wrote:
>> Simon Matter wrote:
>> snipped a bunch of stuff to keep things readable<<
>> I seem to have scripts created by both websieve and avelsieve, but I
>> do
>> know that we have been managing sieve through squirrelmail for a very
>> long time. Looks like the websieve scripts haven't been touched since
>> 2007. So, I think I'm back to where I was. Avelsieve was working on
>> the
>> last machine, but isn't working now. I'm pretty sure I have copied
>> all
>> configs over. How can I increase logging, so I can see what is
>> happening
>> when I get the time-out?
>>
>> thanks,
>> maria
> Which Versions of cyrus do you have installed on the new
> and the old mailserver? And does Avelsieve use Net_Sieve?
> If yes which version of Net_Sieve do you have installed.
>
 old
cyrus-admin-2.2   2.2.13-13
avelsieve 1.9.7-2
 new
cyrus-admin-2.2 2.2.13-14+lenny3
avelsieve   1.9.7-6+lenny1

 avelsieve does not appear to use Net_Sieve.
>>> Which version of PHP do you have on old and new?
>>>
>>> Simon
>>>
>> Well, this is interesting.
>>
>> old:
>> montoya:~# dpkg -l |grep php
>> ii  libapache-mod-php46:4.4.4-9+lenny1
>> server-side, HTML-embedded scripting languag
>> ii  libapache2-mod-php5   5.2.6.dfsg.1-1+lenny2
>> server-side, HTML-embedded scripting languag
>> ii  libphp-phpmailer  1.73-6 full
>> featured email transfer class for PHP
>> ii  libphp-snoopy 1.2.4-1Snoopy
>> is a PHP class that simulates a web b
>> ii  php4-common   6:4.4.4-9+lenny1   Common
>> files for packages built from the php
>> ii  php4-ldap 6:4.4.4-9+lenny1   LDAP
>> module for php4
>> ii  php5  5.2.4-2
>> server-side, HTML-embedded scripting languag
>> ii  php5-common   5.2.6.dfsg.1-1+lenny2  Common
>> files for packages built from the php
>> ii  php5-gd   5.2.6.dfsg.1-1+lenny2  GD
>> module for php5
>> ii  php5-ldap 5.2.6.dfsg.1-1+lenny2  LDAP
>> module for php5
>> ii  php5-mysql5.2.6.dfsg.1-1+lenny2  MySQL
>> module for php5
>>
>>
>> new:
>> ella:~# dpkg -l |grep php
>> ii  libapache2-mod-php5 5.2.6.dfsg.1-1+lenny3
>> server-side, HTML-embedded scripting language (Apache 2 module
>> ii  php5-common 5.2.6.dfsg.1-1+lenny3
>> Common files for packages built from the php5 source
>> ii  php5-ldap   5.2.6.dfsg.1-1+lenny3  LDAP
>> module for php5
>>
>> I'm not sure how to figure out which php packages avelsieve was actually
>> using. There is no executable to check libraries, and there is not a
>> version of php listed in dependencies for the package.
> 
> I'm afraid I can't help you much with the debian specific things, I have
> no idea how debian handles such parallel installs of multiple PHP
> versions.
> 
> What I know is that there is a difference when using avelsieve with PHP4
> or 5 concerning STARTTLS. With PHP4, no STARTTLS is used, while with PHP5
> it is used - and I think that's where changes were made in avelsieve-1.9.8
> so you can configure what to do with STARTTLS when using PHP5.
> 
> Now, if you have everything installed from debian packages, I think it
> should work an if it doesn't, you may contact the package maintainers
> about it. We don't know which patches have been included in your avelsieve
> package.
> 
> Regards,
> Simon
> 

Yup, I found a bug.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=547648

cheers,
maria

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Murder confusion -- two mupdate slaves, lmtpproxyd's always connecting to master

[Michael Bacon]

First off, thank you to everyone who's chimed in with extremely helpful 
suggestions in response to my previous message.  Some of them have helped 
performance, and others mysteriously haven't when I really thought they 
would, but it's been very helpful for me in untying this knot.  I've been 
too busy with a cranky service to respond to everyone personally, but 
please know it's appreciated.

I have two questions regarding the Murder architecture.  The first is just 
a general annoyance at the way master starts up the mupdate processes.  In 
order to get master to fire off anything, you have to set prefork=1. 
However, the result of this is that you generate one mupdate, it reports as 
unavailable to the master, and so the master fires off another one.  These 
two seem to fight with each other over connections to the mupdate master, 
over locks to the mailboxes database, and over who gets to serve 
connections.  I don't think more than one is needed from what I can tell, 
but you get two just out of the master architecture.  Do I really need two, 
and if not, is there some way to keep more than one from starting up? 
(maxchild=1 does not do the trick for whatever reason.  I'm still 
deciphering code on this one.)

The second one is that the code for lmtpproxyd very explicitly connects to 
the mupdate master rather than the local slave.  I can't really figure out 
why it would do this, but here are the two relevant snippets from lmtpd.c:

This is from service_init:

if (config_mupdate_server &&
(config_mupdate_config == IMAP_ENUM_MUPDATE_CONFIG_STANDARD) &&
!config_getstring(IMAPOPT_PROXYSERVERS)) {
/* proxy only -- talk directly to mupdate master */
r = mupdate_connect(config_mupdate_server, NULL, &mhandle, NULL);
if (r) {
syslog(LOG_ERR, "couldn't connect to MUPDATE server %s: %s",
   config_mupdate_server, error_message(r));
fatal("error connecting with MUPDATE server", EC_TEMPFAIL);
}
}


and it appears that this runs every time a message delivery is attempted:

static int mlookup(const char *name, char **server, char **aclp, void *tid)
{
int r, type;

if (server) *server = NULL;

if (mhandle) {
/* proxy only, so check the mupdate master */
struct mupdate_mailboxdata *mailboxdata;

/* find what server we're sending this to */
r = mupdate_find(mhandle, name, &mailboxdata);

In short, it looks like, unlike proxyd and pop3proxyd, lmtpproxyd never 
even bothers to check the local mailboxes database, and when it wants an 
answer, feels the need to go directly to the mupdate master, rather than 
querying the handy dandy local slave.  Is this intentional?  Why can't it 
use the local cache?

Thanks much,
Michael Bacon
UNC Chapel Hill

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Murder confusion -- two mupdate slaves, lmtpproxyd's always connecting to master

[Gary Mills]

On Tue, Nov 10, 2009 at 11:51:42AM -0500, Michael Bacon wrote:
> 
> I have two questions regarding the Murder architecture.  The first is just 
> a general annoyance at the way master starts up the mupdate processes.  In 
> order to get master to fire off anything, you have to set prefork=1. 
> However, the result of this is that you generate one mupdate, it reports as 
> unavailable to the master, and so the master fires off another one.  These 
> two seem to fight with each other over connections to the mupdate master, 
> over locks to the mailboxes database, and over who gets to serve 
> connections.  I don't think more than one is needed from what I can tell, 
> but you get two just out of the master architecture.  Do I really need two, 
> and if not, is there some way to keep more than one from starting up? 
> (maxchild=1 does not do the trick for whatever reason.  I'm still 
> deciphering code on this one.)

I finally fixed that one, but it took a long time to find the reason.
I always had two copies of the mupdate master running, but one of them
did almost nothing...

# ps -fp "$(pgrep mupdate)"
 UID   PID  PPID   CSTIME TTY TIME CMD
   cyrus  3024   700   0   Apr 03 ?   0:01 mupdate -C 
/etc/mupdate/imapd.conf -m
   cyrus  3026   700   0   Apr 03 ?  49:02 mupdate -C 
/etc/mupdate/imapd.conf -m

It turned out that one was listening on an IPv4 port and the other on
an IPv6 port.  Changing cyrus.conf from `listen=3905 prefork=1' to
`listen=3905 proto="tcp4" prefork=1' solved the problem for me.

I did the same thing for other Cyrus services as well.

-- 
-Gary Mills--Unix Group--Computer and Network Services-

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Murder confusion -- two mupdate slaves, lmtpproxyd's always connecting to master

[Brian Awood]

You can avoid the behavior in your second question by setting 
mupdate_config: unified
in your proxies imapd.conf.  That was a long standing issue with 
murder in a large scale environment with cyrus 2.2.

On Tuesday 10 November 2009 @ 11:51, Michael Bacon wrote:
> The second one is that the code for lmtpproxyd very explicitly
> connects to the mupdate master rather than the local slave.  I
> can't really figure out why it would do this, but here are the two
> relevant snippets from lmtpd.c:
>
> This is from service_init:
>
> if (config_mupdate_server &&
> (config_mupdate_config ==
> IMAP_ENUM_MUPDATE_CONFIG_STANDARD) &&
> !config_getstring(IMAPOPT_PROXYSERVERS)) {
> /* proxy only -- talk directly to mupdate master */
> r = mupdate_connect(config_mupdate_server, NULL, &mhandle,
> NULL); if (r) {
> syslog(LOG_ERR, "couldn't connect to MUPDATE server %s:
> %s", config_mupdate_server, error_message(r)); fatal("error
> connecting with MUPDATE server", EC_TEMPFAIL); }
> }
>
>
> and it appears that this runs every time a message delivery is
> attempted:
>
> static int mlookup(const char *name, char **server, char **aclp,
> void *tid) {
> int r, type;
>
> if (server) *server = NULL;
>
> if (mhandle) {
> /* proxy only, so check the mupdate master */
> struct mupdate_mailboxdata *mailboxdata;
>
> /* find what server we're sending this to */
> r = mupdate_find(mhandle, name, &mailboxdata);
>
> In short, it looks like, unlike proxyd and pop3proxyd, lmtpproxyd
> never even bothers to check the local mailboxes database, and when
> it wants an answer, feels the need to go directly to the mupdate
> master, rather than querying the handy dandy local slave.  Is this
> intentional?  Why can't it use the local cache?
>
> Thanks much,
> Michael Bacon
> UNC Chapel Hill
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html



Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Murder confusion -- two mupdate slaves, lmtpproxyd's always connecting to master

[Michael Bacon]

--On November 10, 2009 11:11:41 AM -0600 Gary Mills  
wrote:

> I finally fixed that one, but it took a long time to find the reason.
> I always had two copies of the mupdate master running, but one of them
> did almost nothing...
>
> # ps -fp "$(pgrep mupdate)"
>  UID   PID  PPID   CSTIME TTY TIME CMD
>cyrus  3024   700   0   Apr 03 ?   0:01 mupdate -C
> /etc/mupdate/imapd.conf -mcyrus  3026   700   0   Apr 03 ?
> 49:02 mupdate -C /etc/mupdate/imapd.conf -m
>
> It turned out that one was listening on an IPv4 port and the other on
> an IPv6 port.  Changing cyrus.conf from `listen=3905 prefork=1' to
> `listen=3905 proto="tcp4" prefork=1' solved the problem for me.

Ah ha!  Thank you!

-M


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Murder confusion -- two mupdate slaves, lmtpproxyd's always connecting to master

[Michael Bacon]

I saw this recommendation elsewhere, but I don't understand why one would 
set mupdate_config: unified if I'm not, in fact, running a unified murder. 
What's the gain here?  We have dedicated front-end boxes and dedicated 
back-end boxes.  Is this just sort of a, "yeah, it's not for that, but it 
does what you want anyway" kind of things?

Thanks,
-Michael

--On November 10, 2009 12:26:22 PM -0500 Brian Awood  
wrote:

>
> You can avoid the behavior in your second question by setting
> mupdate_config: unified
> in your proxies imapd.conf.  That was a long standing issue with
> murder in a large scale environment with cyrus 2.2.
>
> On Tuesday 10 November 2009 @ 11:51, Michael Bacon wrote:
>> The second one is that the code for lmtpproxyd very explicitly
>> connects to the mupdate master rather than the local slave.  I
>> can't really figure out why it would do this, but here are the two
>> relevant snippets from lmtpd.c:
>>
>> This is from service_init:
>>
>> if (config_mupdate_server &&
>> (config_mupdate_config ==
>> IMAP_ENUM_MUPDATE_CONFIG_STANDARD) &&
>> !config_getstring(IMAPOPT_PROXYSERVERS)) {
>> /* proxy only -- talk directly to mupdate master */
>> r = mupdate_connect(config_mupdate_server, NULL, &mhandle,
>> NULL); if (r) {
>> syslog(LOG_ERR, "couldn't connect to MUPDATE server %s:
>> %s", config_mupdate_server, error_message(r)); fatal("error
>> connecting with MUPDATE server", EC_TEMPFAIL); }
>> }
>>
>>
>> and it appears that this runs every time a message delivery is
>> attempted:
>>
>> static int mlookup(const char *name, char **server, char **aclp,
>> void *tid) {
>> int r, type;
>>
>> if (server) *server = NULL;
>>
>> if (mhandle) {
>> /* proxy only, so check the mupdate master */
>> struct mupdate_mailboxdata *mailboxdata;
>>
>> /* find what server we're sending this to */
>> r = mupdate_find(mhandle, name, &mailboxdata);
>>
>> In short, it looks like, unlike proxyd and pop3proxyd, lmtpproxyd
>> never even bothers to check the local mailboxes database, and when
>> it wants an answer, feels the need to go directly to the mupdate
>> master, rather than querying the handy dandy local slave.  Is this
>> intentional?  Why can't it use the local cache?
>>
>> Thanks much,
>> Michael Bacon
>> UNC Chapel Hill
>> 
>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>
>
>





Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Murder confusion -- two mupdate slaves, lmtpproxyd's always connecting to master

[Brian Awood]

On Tuesday 10 November 2009 @ 12:37, Michael Bacon wrote:
> I saw this recommendation elsewhere, but I don't understand why one
> would set mupdate_config: unified if I'm not, in fact, running a
> unified murder. What's the gain here?  We have dedicated front-end
> boxes and dedicated back-end boxes.  Is this just sort of a, "yeah,
> it's not for that, but it does what you want anyway" kind of
> things?

Unfortunately it's not well documented, but the unified murder config 
currently only works on a proxy host.  Don't try to configure it on 
a "backend" machine that has local mailboxes!!!  Unless you want to 
manually fix up the mailboxes.db and mupdate master. :)  There are a 
few reports in the bugzilla from people who have tried to do that and 
mucked things up.  

So to answer your question, basically yes.  The gain is that using it 
on a proxy gives you the behavior you want,  which is lmtpd checks 
it's local db copy first instead of asking mupdate on every delivery.  
It seems to me like the behavior anyone running murder would want, 
but for some reason that isn't apparently true.

-Brian

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Quota question

[Nybbles2Byte]

Hello,

If I do a SETQUOTA and GETQUOTA should I be using bytes or kilobytes? Sorry, 
this is probably obvious to most people but I am rather new at it.

I'm connecting with "fsockopen" since I'm right on the server... just in case 
how you connect makes a difference.

-- 
Nybbles2Byte  mailto:nybbles2b...@gmail.com
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Murder confusion -- two mupdate slaves, lmtpproxyd's always connecting to master

[Michael Bacon]

--On November 10, 2009 1:23:33 PM -0500 Brian Awood  
wrote:

> Unfortunately it's not well documented, but the unified murder config
> currently only works on a proxy host.  Don't try to configure it on
> a "backend" machine that has local mailboxes!!!  Unless you want to
> manually fix up the mailboxes.db and mupdate master. :)  There are a
> few reports in the bugzilla from people who have tried to do that and
> mucked things up.
>
> So to answer your question, basically yes.  The gain is that using it
> on a proxy gives you the behavior you want,  which is lmtpd checks
> it's local db copy first instead of asking mupdate on every delivery.
> It seems to me like the behavior anyone running murder would want,
> but for some reason that isn't apparently true.

Thanks very much for this -- the mupdate server is now not sweating nearly 
as much.  We still have some outstanding performance issues to stomp, but 
we were locking up the mupdate master pretty badly, and it was making 
everything worse.  With this, I think we can focus on single-server issues 
from here on out.

-Michael




Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Quota question

[Adam Tauno Williams]

On Tue, 2009-11-10 at 11:10 -0800, Nybbles2Byte wrote:
> Hello,
> If I do a SETQUOTA and GETQUOTA should I be using bytes or kilobytes?
> Sorry, this is probably obvious to most people but I am rather new at
> it.

KB.  So "sq user.me 384000" is 384MB.

> I'm connecting with "fsockopen" since I'm right on the server... just
> in case how you connect makes a difference.

Eh?
-- 
openSUSE 
Linux for human beings who need to get things done.


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: VMware for Cyrus?

[David Lang]

On Mon, 9 Nov 2009, Sebastian Hagedorn wrote:


--On 9. November 2009 08:37:46 -0300 Reinaldo de Carvalho
 wrote:


You need analyse the I/O consumition.

# iostat -d 1


I trust real world experiences more than benchmarks ... either people on
this list have successfully run Cyrus under ESX or they haven't. I don't
want to be the first to try it.


iostat is not a benchmark, it's a tool to measure your I/O patterns (number of 
I/O's, size of transfers, etc)


while I understand your desire to have real-world experiance, you do need to 
know your usage patterns to size the system accordingly.


David Lang
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: VMware for Cyrus?

[David Lang]

On Mon, 9 Nov 2009, Sebastian Hagedorn wrote:


--On 9. November 2009 14:10:54 +0100 Simon Matter 
wrote:


While virtualization has advantages it has also disadvantages. One thing
is that it introduces an additional layer of complexity into the game.
It's my impression that in many areas virtualization gets introduced not
because of technical reasons but because of political pressure.


In our case I wouldn't necessarily call it political pressure ... it's more
like organizational pressure. We have fewer personnel resources than we
used to, and have to run more systems with them!


and every time you introduce virtualization you introduce an additional system 
that you need to run.


remember that you need to admin all the virtual machines, just like you would if 
they were on their own physical boxes, plus you now need to admin the host OS.


David Lang
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

ipurge syntax

[Patrick Boutilier]

How can you get ipurge to only work on the INBOX? If I try this nothing 
happens:

ipurge -d 31 user.spam


If I put in -f then the INBOX and every folder underneath gets processed.


man page seems to indicate that only processing the INBOX is not possible:

Ipurge by default only deletes mail below shared folders, which means 
that  mails  in  mailbox(es)  below  INBOX.*  and  user.*  stay
untouched. Use the option -f to also delete mail in mailbox(es) 
below these folders.

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html