Re: How to improve mail search from iPad and other mobile devices

2014-09-23 Thread Robert Mueller
> According to Apple's manual, Mail searches all folders like this: > "Searching looks at the address fields, the subject, and the message > body." FYI when we found out about this change, we made all BODY searches from iOS become FUZZY BODY searches to make them work reasonably. http://blog.fa

Re: Cyrus 2.5, xapian, Sphinx and index sizes

2014-09-23 Thread Robert Mueller
> # Any time the disk gets over 50%, compress -o single down to data > 13 * * * * /home/mod_perl/hm/scripts/xapian_compact.pl -a -o -d 50 temp > data > # Copy the temporary search databases down to data during the week > 43 1 * * 1,2,3,4,5,6 /home/mod_perl/hm/scripts/xapian_compact.pl -a > temp,

Re: Cannot see entries in folders when using MS Outlook 2013

2013-04-15 Thread Robert Mueller
> When I use outlook 2013, I can see the subfolders in my inbox, but NO entries > are shown. In Outlook, go to the Advanced settings tab for the account, make sure your root folder path is "Inbox" not "INBOX". Rob Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.

Re: problems with quota files

2013-02-20 Thread Robert Mueller
> I realized that quota shown for users with quota command is much less > than real filesystem usage This is expected. cyrus quota is sum of size in bytes of each email. Disk quota used might be much higher because minimum size of a file on lots of filesystems is 1 block. Rob Cyrus Home

Re: UTF7 mailbox codification error

2012-02-28 Thread Robert Mueller
> It looks like internal debugging junk from our system leaked > into a public build to me. > > Rob? Should we do a release without the 'contact rjlov'? Richard fixed this, and I've pushed the latest code to: https://github.com/robmueller/mail-imaptalk/ You can get it from there for now. Rob

Re: UTF7 mailbox codification error

2012-02-23 Thread Robert Mueller
> $IMAPs1->set_unicode_folders(1); > $IMAPs2->set_unicode_folders(1); ... > How can I avoid this error? Or force that the method do not "convert" the > strings. You explicitly said you want unicode folder support, but then explicitly pass IMAP UTF-7 folder names rather than a perl unicode string.

Re: Cyrus 2.4.13 memory use

2012-02-08 Thread Robert Mueller
> > It looks like 3000 IMAP sessions are going to take around 8 GBytes > > of RAM just to run, and we will need to buy additional RAM for > > buffer cache. This isn't the end of the world: memory is cheap. I'm > > just curious if anyone else saw a similar increase when upgrading > > from 2.3 to 2.

RE: disable IMAP IDLE

2010-11-23 Thread Robert Mueller
> I was asked by IT to not permit IDLE since the current server went down > after 4-500 blackberries ate up all the (limited) capabilities of that > machine. I'd really be surprised if that was a problem these days. We have machines that have 1 connections quite fine. Yes they're fairly load

Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

2010-11-16 Thread Robert Mueller
> > This is depends on what filesystem you are useing, I have mailboxes with > > hundreds > > of thousands of messages in them on XFS and have no problems, but on ext3 I > > start seeing slowdowns with a bit over ten thousand messages. > > Was dir_index enabled on that ext3 filesystem? Prior

Re: how to debug what happens inside mailboxes

2010-04-12 Thread Robert Mueller
> > That is exactly what I need. Thanks. Is there some sort of documentation for > > such features in cyrus? I can't seem to find anything beyond basic setup and > > what is in manual pages. Stuff like annotations (through which I managed to > > delete 30GB of emails), global sieve skripts, snmp c

Re: Nginx configuration for imap

2010-03-17 Thread Robert Mueller
> But I thought a memcache lookup will be much more inexpenisve than > connecting to a mysql db to do lookup for every cyrus connection Probably slightly. But what happens if the value isn't in memcached? Where do you get the value from? Anyway, it's still WAY better than doing: > > > $user['u

Re: Nginx configuration for imap

2010-03-17 Thread Robert Mueller
> $user['user1'] = 10.1.1.1; > $user['user2'] = 10.1.1.2; > > $user[user15000]=10.1.1.1; > For 15k users this method becomes very heavy. There are too many httpd > processes running that suck the resources on the machine. I want to > store the userlist in a memcache and look it up through ng

Re: what happens when cyrus.squat is old?

2010-01-21 Thread Robert Mueller
> > Are you using the new incremental mode david carter added? > > > > -i Incremental updates where squat indexes already exist. > > I'm not. This is a very old install. However, we're planning a > migration to a new server and I'll possibly try it then. Can you just > compile squatter se

cyrus replication over a WAN

2009-12-09 Thread Robert Mueller (web)
I just wanted to check if any people have any experience or reports about doing cyrus replication over a WAN? A few potential issues spring to mind: 1. I know the replication protocol is pretty "chatty", so does the higher latency over a WAN (ms vs us) cause it to get behind more regularly? 2. Th

Re: cyrus replication : master to replica and replica to master

2009-10-22 Thread Robert Mueller (web)
> What are the particular bits that could conflict and have undesirable > results? Metadata, messages, entire mailboxes? In this hypothetical > active/active configuration, what exactly what could an IMAP client > potentially do to create undesirable results? Simple. Client A: upload message to

Re: Incorrect sort order when sorting by date

2009-05-18 Thread Robert Mueller (web)
I bet if you look at the Date headers of the problem emails, you'll find that they're not RFC compliant. The cyrus date parser is very strict, and if the header isn't RFC compliant, you'll get a bad value, and bad sorting. Looking at the RFC http://www.faqs.org/rfcs/rfc5256.html If the sent

Re: UC Davis Cyrus Incident September 2007

2007-10-18 Thread Robert Mueller
> Actually, I don't see a deadlock situation at all... I am guessing that > theorettically, it is possible... but the "ln -sf" option makes the > overwriting of the symlink an atomic action (as much as it can), which Not a "deadlock" situation, but a possible "file doesn't exist" error. In the Un

Re: huge mail hangs lmtpd

2007-04-11 Thread Robert Mueller
> I had again problems with an huge mail and lmtpd. I believe this is > caused by the sieve regex filtering of the huge mail. > > The lmtp uses about twice the size of the email of mem and all the > cpu-time > it can get. Then postfix gets a time out, but the lmtpd still keeps > running > and d

Re: Recomendations for a 15000 Cyrus Mailboxes

2007-04-10 Thread Robert Mueller
> 1. Linux LVM over a 600 GB RAID 10 ( 4 x 300 GB) > 2. Which filesystem seems to be the better ? ext3 ? xfs ? reiserfs ? > 3. Which options to format the filesystem ? acording to the chosed > filesystem > 4. Which pop3 / imap proxy to use ? > 5. Single instance or multiple instance

Re: BODYSTRUCTURE question

2007-02-27 Thread Robert Mueller
> Nice! Is this code available for public consumption? I'd love to use > something like this to strip out \0s in messages too. Most of the other > "solutions" for cyrus/postfix that I've seen required an additional > exec() in the delivery pipeline, which I would like to avoid. It's all done i

Re: lmtpd memory leak?

2007-02-22 Thread Robert Mueller
> all my lmtpd-processes are growing in size and are consuming more and > more CPU-time and increasing it's priority. > > As far as i can see, mail gets deliverd fine, but only from the small > lmtpd-processes. The huge processes are not used for delivery, and new > lmtp processes get starte

Re: Bounces on local delivery

2006-12-04 Thread Robert Mueller
This usually happens because the lmtp connection got out of sync somehow. There was a bug in older postfix's about this: http://archives.neohapsis.com/archives/postfix/2004-04/1731.html I haven't seen the problem since then since Wietse fixed it, but I'd try setting lmtp_cache_connect=no in yo

Bug with rename INBOX -> INBOX.blah and replication

2006-12-04 Thread Robert Mueller
Hi Ken There's a bug with replication and renaming INBOX -> INBOX.blah. From http://www.ietf.org/rfc/rfc3501.txt: Renaming INBOX is permitted, and has special behavior. It moves all messages in INBOX to a new mailbox with the given name, leaving INBOX empty. If the server imp

Re: performance on large inboxes

2006-11-12 Thread Robert Mueller
I'm using ext3 with dir_hash. I considered using XFS, but there are a lot of benchmarks that show that XFS is not faster in general, also the XFS development seems to be stucked at the moment and from my own experience as well as from other people in a recent thread on this mailinglist there

Re: problems on folders containing @

2006-11-12 Thread Robert Mueller
why does this give an error? x create [EMAIL PROTECTED] x NO Invalid mailbox name Why isn't it allowed at Cyrus that folders contain an @-sign? It was allowed with dovecot and courier-imap and it is possible to create folders with several other characters like Umlauts and spaces. Why is the

Re: Too slow

2006-10-25 Thread Robert Mueller
I turned the mail partition now to xfs and it's terribly fast. WOW!!! did you use an ext3 partition with dir_index before? I'm just asking We've just had some experience with filesystems ourselves. Previously we've used reiserfs exclusively for the last 5 years. All up, it's been really go

Re: Performance and cheap storage

2006-08-03 Thread Robert Mueller
not sure if we qualify as big enough, but here goes: we typically have 3000 concurrent TLS/SSL connections on each Perdition server during peak hours (although we occasionally see 5000), but the CPU impact is negligible[1]. at peak, 8% system and 12% user out of 400% CPU available (this is Del

Re: Cyrus 2.3.7 Replication Question

2006-07-12 Thread Robert Mueller
Interesting. This process also implies you want to stop incoming mail for a period of time to both servers. Sound right? I would say the process for "failover" is: 1. Server A is master (sync_client) replicating to Server B (sync_server) 2. Server A dies/is stopped 3. All IMAP/POP/LMTP connec

Re: Cyrus 2.3.7 Replication Question

2006-07-12 Thread Robert Mueller
Can you detail this A <-> B replication setup more? I've been trying to figure out how to make use of the new 2.3.7 replication features outside of a murder. The thing I can't wrap my head around is the following situation: 1) Server A dies 2) Users are sent to Server B via DNS redirects 3)

Re: Cyrus 2.3.7 Replication Question

2006-07-12 Thread Robert Mueller
You can make it work (we do), but you need 2 separate instances of cyrus on each machine, which basically means 2 start/stop scripts, 2 different ports/ips for each service, 2 imapd.confs, 2 cyrus.confs, and lots of -C command line params to everything so it uses the right conf file. to avoid a

Re: Cyrus 2.3.7 Replication Question

2006-07-12 Thread Robert Mueller
It won't. Cyrus currently only supports unidirectional replication. You can make it work (we do), but you need 2 separate instances of cyrus on each machine, which basically means 2 start/stop scripts, 2 different ports/ips for each service, 2 imapd.confs, 2 cyrus.confs, and lots of -C comm

Re: Does anyone else see skiplist recovery errors?

2006-06-15 Thread Robert Mueller
However, it may also depend on the way cyrus-imapd is stopped by the system. At least on RedHat/Fedora, the function used by the init scripts send a TERM to the master, and if it doesn't die for some time, it sends KILL which _could_ result in corrupt ondisk data if I understand it correctly. Mayb

Does anyone else see skiplist recovery errors?

2006-06-14 Thread Robert Mueller
I'm trying to find out if anyone else sees intermittent skiplist recovery problems as sometimes we do, usually after a cyrus restart. Our particular setup is cyrus 2.3.6, but we've seen this problem with everything from 2.1 onwards. This is on on x86 with linux 2.6.x, various versions. We use

Re: What version of BDB are people using?

2006-06-12 Thread Robert Mueller
Actually been over this on the list before. It's completely unrelated to that issue. It's not like some connections get through and some don't. It's completely unreproducible and *NO* POP3 connections get through at all until a reboot. One thing to try, when you connect, see if a new pop3

What version of BDB are people using?

2006-06-09 Thread Robert Mueller
I'm just trying to get an informal survey of which version or Berkeley DB people are using successfully in large cyrus environments. We're currently using: db4-4.2.52-3.1 - old redhat based machines libdb4.2.52-18 - newer debian based machines Both of them seem to be a bit "flakey". We only us

Re: Cyrus Patches used at FastMail.FM

2006-05-28 Thread Robert Mueller
accept the message, store it "as is" and ignore the stuff you don't understand when building indexes. And ignore it when using the indexes. Yes, but where's the patch to do that? I'll have a guess that something like this already happens. My guess is that the charset mapping tables ignore 8-

Re: Cyrus Patches used at FastMail.FM

2006-05-26 Thread Robert Mueller
Comments on the non-local ones: - cyrus-8bit-2.3.3.diff Was it not the position held by CMU in the last five years or so that one such patch would *have* do to it properly and re-encode the header to some default charset, in order not to corrupt the store with lack of charset information (for

Re: Cyrus IMAPd 2.3.4 Released

2006-05-25 Thread Robert Mueller
I agree that a test suite is something that we need. I'll add this to our TODO list. As always, code contributions are always welcome. See below and attached, a copy of what I sent april '04. It's not great, but it's a start... Rob Hi Ken & Rob Given that there's currently no regre

Re: Cyrus IMAPd 2.3.4 Released

2006-05-25 Thread Robert Mueller
Is this a good spot to suggest branching from the last stable release (2.3.3 in this case) and just applying the bugfixes if HEAD is only half finished? 2.3 is beta branch. You want stable you go for 2.2. Beta is may be, but still 1. There's no regression testing with cyrus at all. I di

Some part of bug from 2.3.4 still in 2.3.5

2006-05-25 Thread Robert Mueller
Hi Ken There's still a serious bug in 2.3.5 that causes copied messages to be pseudo invisible in the moved to folder. I can reproduce it as follows: . select inbox.Archive * FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected medeleted hasatt) * OK [PERMANENTFLAGS (\Answered \F

Re: Cyrus Patches used at FastMail.FM

2006-05-23 Thread Robert Mueller
I know we discussed this in the past, but I can't seem to find the thread. What part of the existing STATUS code causes the bottleneck? Is it STATUS_RECENT and STATUS_UNSEEN? It's a combination of both. The main things are: 1. Status UNSEEN and RECENT both have to loop over the cyrus.index fi

Re: Cyrus Patches used at FastMail.FM

2006-05-23 Thread Robert Mueller
Statuscache also piqued my interestDoes it give any win for POP3 clients? We've a *HUGE* number of Outlook users that have the terribly wrong idea that they just MUST poll every minute. That along with the seen state stuff would be a good thing for us. No, it's only for IMAP clients an

Re: Cyrus Patches used at FastMail.FM

2006-05-23 Thread Robert Mueller
Cool, some of the patches look really interesting and I'm considering to include one or the other into my rpm packages. For example the statuscache patch seems very nice. Just to be sure, are there any license restrictions on the patches? No, no license restrictions. We'd love these to get bac

Re: cyrus 2.3.3 change + solved: permission denied setting flags

2006-05-18 Thread Robert Mueller
Its a bug. This make have creeped in with the updated ACL code. What is the ACL on user.xyz for the authorized user? Was logged in as admin user... . myrights user.xyz * MYRIGHTS user.xyz lrswipkxtecda . OK Completed . getacl user.xyz * ACL user.xyz xyz lrswipkxtecd admin lrswipkxtecda anyone

cyrus 2.3.3 change + solved: permission denied setting flags

2006-05-18 Thread Robert Mueller
Ok, we've tracked down the permissions problem. It has to do with what I believe is a change in behaviour between 2.3 CVS 6 months ago and 2.3.3. Cyrus 2.3 from CVS about 6 months ago... . select user.xyz ... . OK [READ-WRITE] Completed . examine user.xyz ... . OK [READ-ONLY] Completed Cyrus 2

Cyrus virtual domain & replication...

2006-04-30 Thread Robert Mueller
I'm interested in the new replication stuff, which is why I want to do the upgrade ... I'm not entirely sure that replication plays nice with virtdomains yet. IIRC, there is a bug in bugzilla for this. So just checking, is there anyone out there currently using cyrus with virtual domains and

Re: plus addressing to a mailbox with a space in its name

2006-04-05 Thread Robert Mueller
Has anyone had success with using plus addressing to send to a specific mail folder when the folder name has a space in it? With cyrus 2.3 you can enable: lmtp_fuzzy_mailbox_match: 1 From imapd.conf lmtp_fuzzy_mailbox_match: 0 If enabled, and the mailbox specified in the

Re: BDB and errors...

2006-03-14 Thread Robert Mueller
> I have no real idea what could cause this but I have the following > sequence in my db conversion script which is used by the init script > in my rpms. The procedure is the best according do lots of my tests > using different version of db3 and db4 with cyrus-imapd. As you can > see I first try

BDB and errors...

2006-03-14 Thread Robert Mueller
We're using cyrus 2.3 and everything works fine, except we seem to have intermittent problems with BDB 4.2 (specifically the RPM db4-4.2.52-3.1). We only use BDB for the delivery db. In general it works fine, however if for some reason a server has crashed and we reboot the server, we then see

Re: improving concurrency/performance

2005-11-06 Thread Robert Mueller
> >> In our experience FS-wise, ReiserFS is the worst performer between ext3, > >> XFS e ReiserFS (with tailBLAH turned on or off) for a Cyrus Backend (>1M > >> mailboxes in 3 partitions per backend, 0.5TB each partition). > > > > Interesting ... can you provide some numbers, even from memory? > >

Difference between ". list * %" and ". list * *"

2004-01-30 Thread Robert Mueller
This is cyrus 2.2.3 with skiplist mailboxes db. Someone pointed this strange behaviour out to me... . list * * * LIST (\HasChildren) "." "INBOX" * LIST (\HasNoChildren) "." "INBOX.Folder1" * LIST (\HasChildren) "." "INBOX.Folder2" * LIST (\HasChildren) "." "INBOX.Folder2.SubFolder1" . list * %

Slow "status" calls...

2003-09-18 Thread Robert Mueller
I was noticing that IMAP 'status' calls to update the message counts in our web interface seemed to be taking quite a bit of time. I ran a stand alone script to check this, and found: Time: 0.002, msgs:12, unseen:10, folder: INBOX.spam Time: 0.004, msgs: 18978, unseen: 18978, folder: INBOX