Re: Not able to delete mails in 2.4.6
On Thursday, November 24, 2011 1:05 PM, "Ram" wrote: > Hello > > I had upgraded my imap server to 2.4.6 That's not an upgrade ;) Seriously, 2.4.6 has known bugs. > Some users keep complaining that delete is not working. > Even If I use raw Imap commands and delete the message .. the delete > flag is not being set > > As a fix , If I remove cyrus.* from all the folders in the users > directory and reconstruct then the problem gets solved > But why does this happen ? How can I debug ? God that's an awful way to "fix" problems. > I have noticed that these users have a line "NonJunk" in the > cyrus.header file , Is that any indication of anything ? That's just a flag. Bron. -- Bron Gondwana br...@fastmail.fm Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Huge load after update from 2.3.16 to 2.4.12
On Wed, November 23, 2011 10:40 pm, pl...@bibliothek.uni-kassel.de wrote: > [...] > >> >> When performing an upgrade from earlier than 2.4.3, all messages are >> reparsed, and your cache files are re-generated, when a mailbox is first >> opened. You may be able to increase you syslogging levels to get a better >> view into what's going on: > [...] > > > Thanx, > > > four weeks ago I made a dump and tried a new installation on a test machine, > but I had no problems. Michael, You probably tested the system's integrity and functionality, and all appeared fine. What you probably did not generate was the equivalent of a realistic production load (POP + IMAP + LMTP). We found ourselves in a similar situation two years ago (with eternal thanks to Simon Matter who helped us *enormously* with a suggestion written on this mailing list). Regards, Eric Luyten (now running 2.3 for +70k users with 50M messages, wondering what server and I/O load we will observe when upgrading to 2.4) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Strange cyrus quota problem
I am rather new to cyrus imap. We took over a client who uses it and has a
strange quota problem. One single user keeps getting "Over quota" alerts
although cyrus (as well as the client show very small quota usage (24%...)
The cyrus log file shows sending the alert, a few lines below it shows the
correct quota:
1 -- Thu Nov 24 10:33:21 2011
2
3 >1322127201>3 OK User logged in^M
4 <1322127201<4 namespace^M
5 >1322127201>* NAMESPACE (("INBOX." ".")) (("user." ".")) ((""
"."))^M
6 4 OK Completed^M
7 <1322127201<5 ID ("name" "Thunderbird" "version" "8.0")^M
8 >1322127201>* ID ("name" "Cyrus IMAPD" "version" "v2.2.12 2005/02/14
16:43:51" "vendor" "Project Cyrus" "support-url" "http://asg.web.cmu.edu
/cyrus" "os" "Linux" "os-version" "2.6.13-15.16-smp" "environment" "Built
w/Cyrus SASL 2.1.21; Running w/Cyrus SASL 2.1.21; Built w/Sleepycat
Software: Berkeley DB 4.3.27: (September 9, 2005); Running w/Sleepycat
Software: Berkeley DB 4.3.27: (September 9, 2005); Built w/OpenSSL
0.9.7g 11 Apr 2005; Running w/OpenSSL 0.9.7g 11 Apr 2005; CMU Sieve 2.2;
DRAC; TCP Wrappers; mmap = shared; lock = fcntl; nonblock = fcntl; a
uth = unix; idle = idled")^M
9 5 OK Completed^M
10 <1322127201<6 lsub "" "INBOX.*"^M
11 >1322127201>* LSUB (\HasChildren) "." "INBOX.Archives"^M
181 * LSUB () "." "INBOX.privat"^M
182 8 OK Completed (0.010 secs 85 calls)^M
183 <1322127201<9 list "" "INBOX"^M
184 >1322127201>* LIST (\HasChildren) "." "INBOX"^M
185 9 OK Completed (0.000 secs 85 calls)^M
186 <1322127201<10 select "INBOX"^M
187 >1322127201>* FLAGS (\Answered \Flagged \Draft \Deleted \Seen
$Forwarded $label1 $MDNSent wartet wartet_auf_entscheidung $label4 zur_info
Junk $label2 erledigt $label3 NonJunk $label5)^M
188 * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
$Forwarded $label1 $MDNSent wartet wartet_auf_entscheidung $label4 zur_info
Junk $label2 erledigt $label3 NonJunk $label5 \*)] ^M
189 * 77 EXISTS^M
190 * 0 RECENT^M
191 * OK [UNSEEN 77] ^M
192 * OK [UIDVALIDITY 1307090395] ^M
193 * OK [UIDNEXT 2680] ^M
194 * NO [ALERT] Mailbox is over quota^M
195 10 OK [READ-WRITE] Completed^M
196 <1322127201<11 myrights "INBOX"^M
197 >1322127201>* MYRIGHTS INBOX lrswipcda^M
198 11 OK Completed^M
199 <1322127201<12 getacl "INBOX"^M
200 >1322127201>* ACL INBOX lrswipcda^M
201 12 OK Completed^M
202 <1322127201<13 getquotaroot "INBOX"^M
203 >1322127201>* QUOTAROOT INBOX user.^M
204 * QUOTA user. (STORAGE 2441822 1000)^M
205 13 OK Completed^M
I gather from this that the server sends the "over quota" alert (194 *
NO [ALERT] Mailbox is over quota) and a few lines down the server finds that
the user uses 24% of his quota (204 * QUOTA user. (STORAGE
2441822 1000))
I tried setting the quota to several different values increasing, decreasing
it from the 1, even setting it to none but the result is always as
above. I tried running quota -f and restarting cyrus several times without
any result.
I'd really appreciate any suggestions on this.
Regards
Thomas
PS.: I am aware, that the cyrus version in use is old, yet we cannot change
this at the moment.
--
=
iPodion GmbH
Rotensterngasse 20/3
A-1020 Wien, Austria
Mobil: +43-660-216 32 98
Tel.:+43-1-216 32 985-00 mailto:off...@ipodion.at
Fax: +43-1-216 32 985-99 http://www.iPodion.at
=
Neu und gratis für iPodion-Kunden: E-Mail-SSL-Zertifikate für Digitale
Signaturen.
smime.p7s
Description: S/MIME cryptographic signature
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: rooling replication and many pop3d
On 24/11/2011 02:07, Andy Bennett wrote: > Hi, > I enabled replication between two servers with version 2.4.10 cyrus. I set the option for the rolling replication, and it works fine but obviously I have a high CPU load. Unfortunately after 10 minutes of running processes pop3d increasing from 50 to over 200, making the server unusable for customers. Can you tell me why this increase is abnormal? >>> Can you use something like 'top' to work out which processes are >>> consuming most of the CPU time? > Thanks for the screenshots. > > >> This screenshot of top before : >> >> http://www.digicolor.net/cyrus/img1.jpg > This shows a load average of around 1. That means that, at any given > point in time over the past 1 and 5 minutes, 1 process has been waiting > in the run queue, ready to go. > This therefore not an entirely idle machine. I see you're running a > nameserver and a few other things: it looks they've been busy on the CPU > but not excessively so. > > What's more worrying is the 4.5% of CPU time spent "waiting". This time > is accrued when processes are unable to run due to outstanding IO. > > > >> and after 20 minutes of rooling replication : >> >> http://www.digicolor.net/cyrus/img5.jpg > This shows a load average of around 7 and 30% of CPU time spent in > iowait. This machine does not seem to be managing well with the IO load > of rolling replication. > > > >>> Can you use something like 'vmstat 1' to show us how much I/O there is >>> on the system? >> This screenshot of top before : >> >> http://www.digicolor.net/cyrus/img3.jpg > This shows a system that is not reading anything from disk (bi). A small > number of blocks are being written out to disk (bo). Each line > represents activity for a period of 1 second, as specified by the > parameter to 'vmstat'. > > >> and after 20 minutes of rooling replication : >> >> http://www.digicolor.net/cyrus/img6.jpg > This system is writing to disk but it's very choppy. Sometimes it's > getting 7,000 blocks out per second and other times it's only 1,000. > Depending on your block size, this probably represents only a few MB per > second. The last column shows iowait CPU percentage.. and it's rather large. > > What IO subsystem do you have on this machine? What filesystem are you > using? I have 4 HDD 500GB S SATA II with LSI 8704 ELP SAS RAID 6 filesystem ext3 mounted with noatime, nodiratime > The IO on this machine appears to be struggling significantly. > > I did a quick test on my laptop. I have a 2.5", 7,200rpm 200GB disk. > > I ran this in my home directory to cause every file to be read from disk: > > - > $ find -type f | xargs cat> /dev/null > - > > 'vmstat 1' gives lines like this: > > - > procs ---memory-- ---swap-- -io -system-- > cpu > r b swpd free buff cache si sobibo in cs us sy > id wa > 1 1 1680 30600272 257568800 12376 576 2141 3424 3 6 > 47 43 > 1 1 1680 32396272 257673600 29988 0 1882 3632 5 7 > 47 41 > 0 1 1680 33868272 257666000 46820 0 2304 4443 4 8 > 48 39 > 1 0 1680 33416272 257860000 36716 0 2067 3733 3 7 > 48 42 > 0 1 1680 34000272 258194400 50432 0 1164 2983 3 6 > 50 42 > 0 1 1680 31876272 258532000 4646464 1223 2964 3 8 > 49 40 > 1 1 1680 30288272 258867200 51712 0 1380 3658 3 7 > 46 43 > 0 1 1680 29836272 259055200 59776 0 1288 3549 4 7 > 47 42 > 0 1 1680 30324272 259294800 58368 0 1287 3568 2 7 > 49 41 > 1 1 1680 30308272 259310800 1280018 917 1673 2 2 > 49 46 > - > > They're an order of magnitude greater than what you're seeing. As you > can see, I drop a few bi when I start to do bo but that's because I've > only got a single spindle. > > Please can you run the same test? this my vmstat after I run the command find -type f | xargs cat > /dev/null : http://www.digicolor.net/cyrus/img7.jpg > > Can you track the source of all those writes in img3? Please can you tell us more about the type of machine you are trying to run this on? Thanks for the info and screenshots so far. Is a : n. 2 XEON Clov. 2,33 GHz - X5345 8GB RAM With debian 5.0 Regards, Obe. -- Oberdan Albertoni --- DIGICOLOR.NET s.r.l. Via Ala Ponzone, 30 - 26100 CREMONA (CR) Tel. +39 (0372) 30669 - 1965101 - 1951509 Fax +39 (0372) 464840 E-mail i...@digicolor.net - Website http://www.digicolor.net C.F / P.IVA 01227520192 - R.E.A. CR n. 153382 - Capitale Sociale € 10.400 i.v. Filiale di Codogno Via Galilei, 17 – 26845 CODOGNO (LO) Tel. +39 (0377) 741006 – 431931 Fax +39 (0372) 464840 E-mail info.codo...@digicolor.net - Website http://www.digicolor.net ---
Re: rooling replication and many pop3d
On Thu, November 24, 2011 11:52 am, Oberdan Albertoni wrote: > I have 4 HDD 500GB S SATA II > with LSI 8704 ELP SAS RAID 6 > filesystem ext3 mounted with noatime, nodiratime RAID 6 over four disks. FYI, RAID 6 carries the highest 'write penalty' of all RAID levels. http://www.techrepublic.com/blog/datacenter/calculate-iops-in-a-storage-array/2182 Rearranging your disks is most likely not a short term solution. (can probably be done in a weekend's downtime but beware, restoring a few million small files takes a LOT more time than dumping them to another medium, be it disk or tape) *If* your system has a fifth drive bay, RAID 1+0 and one hot spare would have given you a much, much faster mail spool of equal size, at little extra cost. Eric Luyten. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: rooling replication and many pop3d
Hi, >> What IO subsystem do you have on this machine? What filesystem are you >> using? > > I have 4 HDD 500GB S SATA II > with LSI 8704 ELP SAS > RAID 6 > filesystem ext3 mounted with noatime, nodiratime > this my vmstat after I run the command find -type f | xargs cat > > /dev/null : > > http://www.digicolor.net/cyrus/img7.jpg This is a very sad set of numbers. :-( It's still really, really choppy and the dead spots don't seem to correlate particularly well with bo activity. Have you got someone who can help you debug your storage array and ensure that it's not degraded in some way? Is the battery empty? Are there any disks which are failing? Do all the disks have their write caches and retry handling set up correctly? How old is your file system? Have you been running it near capacity (>70% full) for long periods of time? You might be able to tell whether you have a file system problem or an array problem by comparing your previous vmstat with the results obtained from 'sudo /sbin/haparm -tT /dev/'. For reference, on my disk I get: - $ sudo /sbin/hdparm -tT /dev/sda /dev/sda: Timing cached reads: 3272 MB in 2.00 seconds = 1637.05 MB/sec Timing buffered disk reads: 194 MB in 3.02 seconds = 64.18 MB/sec - I'd certainly not expect anything less from your array. I'd hope for at least 100MB/sec if not a lot more. Regards, @ndy -- andy...@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Strange cyrus quota problem
Le 24/11/2011 11:09, Thomas Kirchtag a écrit : > I am rather new to cyrus imap. We took over a client who uses it and has a > strange quota problem. One single user keeps getting "Over quota" alerts > although cyrus (as well as the client show very small quota usage (24%...) > > ... > > > I gather from this that the server sends the "over quota" alert (194 * > NO [ALERT] Mailbox is over quota) and a few lines down the server finds that > the user uses 24% of his quota (204 * QUOTA user. (STORAGE > 2441822 1000)) > > I tried setting the quota to several different values increasing, decreasing > it from the 1, even setting it to none but the result is always as > above. I tried running quota -f and restarting cyrus several times without > any result. > Hi, The problem with versions older than 2.3 is that quota data are handled through 32-bits variables: signed for limit, unsigned for usage. In your case quota usage is handled correctly (under the 2^32-1 limit), but your quota limit being 1000 KiB triggers the quota alert because it is compared to the quota usage in bytes: 1000*1024 exceeds the capacity of the 32-bit signed variable. Actually I believe you are experiencing the same thing that we did with 2.1.x versions some years ago: when overflowing a 32-bit value, the sign bit would be flipped (e.g. 2^31 in a signed variable would become -2^31). So for example in your case cyrus would believe the actual limit is 1000*1024 - 2*(2^32) (variable overflowed two times): the usage would thus be around 150%, that is over quota. So in those old versions you will experience all kind of issues when dealing with quota over 2GiB. Of course the code was reworked since cyrus 2.3 and now works as expected. Regards Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Huge load after update from 2.3.16 to 2.4.12
Am 24.11.2011 10:37, schrieb Eric Luyten: > On Wed, November 23, 2011 10:40 pm, pl...@bibliothek.uni-kassel.de wrote: [...] > Michael, > > > You probably tested the system's integrity and functionality, and all appeared > fine. What you probably did not generate was the equivalent of a realistic > production load (POP + IMAP + LMTP). > We found ourselves in a similar situation two years ago (with eternal thanks > to Simon Matter who helped us *enormously* with a suggestion written on this > mailing list). We've tested just with a few people, and the machine has much more power - also it was ONLY IMAP, so this might have been the reason. Everything runs OK by now, BTW. > > > Regards, > Eric Luyten (now running 2.3 for +70k users with 50M messages, wondering > what server and I/O load we will observe when upgrading to 2.4) Go do some yoga before ;-) CU Michael Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Enabling IDLE with murder configuration
Hi, I am testing IDLE feature in our murder configuration (2 murder hosts + 4 backend servers). I understand I have to compile imapd with --enable-idled option. But I wonder on which server(s) i have to start the idled daemon in cyrus.conf ? on murder frontend only or imap backend only or both ? Of course mail clients are all connected to murder frontends only. Thank you, J-Christophe. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
