Re: [Dovecot] Performance-Tuning

2011-11-17 Thread Henrique Santos Fernandes
[]'sf.rique How many servers have access to your maildir on ext4 so that you could > switch from ocfs2 to ext4? > I use ocfs2 in my test environment for four servers (2 MX and 2 IMAP) I used have 3 serves one mailman and 2 imaps.I experence that if take my loadbalancer and takes it all to j

Re: [Dovecot] Performance-Tuning

2011-11-17 Thread Jan-Frode Myklebust
On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote: > > Wich clustered filesytem do you have? We use IBM GPFS, with currently 7 servers working against shared LUNs from an IBM DS4800. > > My ocfs2 setup had some problems... but still.. > Some numbers: > > OCFS2 > 1TB of

Re: [Dovecot] Performance-Tuning

2011-11-16 Thread Henrique Santos Fernandes
Jan-Frode Wich clustered filesytem do you have? I used to have ocfs2 but had problems with performance. So had to get back to ext4 and it solve the performance problem... My ocfs2 setup had some problems... but still.. Some numbers: OCFS2 1TB of maildir files. Full backup 36 Hours Incremental 1

Re: [Dovecot] Performance-Tuning

2011-11-14 Thread Stan Hoeppner
On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote: > Agree. A non-clustered fs should give you better performance, and > probably also be more reliable, if you can live with the SPoF and > full downtime during patching/upgrades/maintenance. But I would expect > xfs to be a better choice than ext*.

Re: [Dovecot] Performance-Tuning

2011-11-14 Thread Jan-Frode Myklebust
On Mon, Nov 14, 2011 at 10:34:02AM +0100, Peer Heinlein wrote: > > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > > HP EVA. > > > > You have 11 TB of mails on a non cluster filesystem? > > Yes. > > I don't believe a clustered filesystem would have more performance and

Re: [Dovecot] Performance-Tuning

2011-11-14 Thread Peer Heinlein
Am Montag, 14. November 2011, 00:31:24 schrieb Patrick Westenberg: > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > HP EVA. > > You have 11 TB of mails on a non cluster filesystem? Yes. I don't believe a clustered filesystem would have more performance and would be m

Re: [Dovecot] Performance-Tuning

2011-11-13 Thread Patrick Westenberg
Peer Heinlein schrieb: I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. You have 11 TB of mails on a non cluster filesystem? Is it only accessed from one server or how does it work?

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Stan Hoeppner
On 11/8/2011 7:50 AM, Peer Heinlein wrote: > I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. That's a lot of mail (likely a large user base--not given), on a filesystem not designed for such, on a decent SAN controller--LUN RAID configuration not given. > I always want

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Eric Rostetter
Quoting Peer Heinlein : The problem is: You're running in problems with shared folders. You can't read your neighbors storage-engine from ldap. Yes, but I didn't have any shared folders, so it worked. Your milage may vary, as I said... :) -- Eric Rostetter The Department of Physics The Univ

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Peer Heinlein
Am Dienstag, 8. November 2011, 17:53:32 schrieb Eric Rostetter: > May not work for you, but... > > The way I did this when I migrated was to run two dovecot instances, and > have perdition software on a front-end (could be on the same machine > instead of a front-end, I just happen to have a fro

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Eric Rostetter
Quoting Peer Heinlein : It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP attributes. In this case the whole migration process could be split up into groups. Unfortunately we have shared folders and I don't know a way to read the *remote* mailbox-format from LDAP... So h

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Timo Sirainen
On 8.11.2011, at 15.50, Peer Heinlein wrote: > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). A few more ideas for Maildir if you haven't done yet: -

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Jahnke-Zumbusch, Dirk
Hi there, >I never tried it, but it should be possible to provide the mail_location >from the user repsoitory (LDAP, SQL, whatever) Actually this works :-) Our userdb looks similar to: account1:xyz:000:000::/account1s/home/dir::userdb_mail=maildir:/account1s/home/dir/Maildir account2:xyz:000:00

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Timo Sirainen
On 8.11.2011, at 16.03, Morten Stevens wrote: > We have switched our mailbox storage format from maildir to mdbox! > > Maildir is a disaster. (too many small files) After the migration to mdbox > the performance has improved significantly. > > Conclusion: mdbox is great and much better performa

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Timo Sirainen
On 8.11.2011, at 16.16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! > > I wonder how I can incrementally change over from Maildir to mdbox? > I can of course use dsync to mirror Maildir: to mdbox:, but how can I > make dove

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Timo Sirainen
On 8.11.2011, at 16.34, Peer Heinlein wrote: >> I can of course use dsync to mirror Maildir: to mdbox:, but how can I >> make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or >> vice versa). > > I wonder about that problem too. Even the last-last-last-quick sync would be > so much

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Ralf Hildebrandt
* Peer Heinlein : > Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: > > Hi, > > > I wonder how I can incrementally change over from Maildir to mdbox? > > If you have double diskspace: haha :) no. I thought of a per-user migration, that way I don't need extra space. -- Ralf

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Oliver Eales
On 08.11.2011 15:16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! I never tried it, but it should be possible to provide the mail_location from the user repsoitory (LDAP, SQL, whatever) So you can keep your global config, and

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Peer Heinlein
Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: Hi, > I wonder how I can incrementally change over from Maildir to mdbox? If you have double diskspace: Just use "dsync mirror" in the background to prepare the change. After that it's just a short downtime to migrate incrementa

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Javier de Miguel Rodríguez
Other important thing to consider is message expunging. With mdbox you are "delaying" the I/O associated with deleting e-mails. We have a nightly cronjob that expunge messages from mdboxes. If you have en EVA (wich one? 4.400? 6.400? ) you also can consider RAID 1+0 or SSD for indexe

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Peer Heinlein
Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodríguez: Hi, > If you have CPU to spare, consider using zlib with mdbox. You are > trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Hey. This point is great. I hadn't realized that. Sure. zlib will sa

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Ralf Hildebrandt
* Morten Stevens : > We have switched our mailbox storage format from maildir to mdbox! I wonder how I can incrementally change over from Maildir to mdbox? I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vi

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Javier de Miguel Rodríguez
We are very happy with mdbox+zlib+ext4 + iSCSI SAN (HP Lefthand in our setup) If you have CPU to spare, consider using zlib with mdbox. You are trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Mdbox has halved our backup windows (2,8 TB uncompressed mailboxes, 2 T

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Morten Stevens
On 08.11.2011 14:50, Peer Heinlein wrote: *) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to? Hi Peer, We have switched our mailbox storage format from maildir to mdbox! Maildir is a disas

Re: [Dovecot] Performance-Tuning

2011-11-08 Thread Ricardo Branco
What is the setup on the EVA, FC or iSCSI? Sent from my BlackBerry® wireless device -Original Message- From: Peer Heinlein Sender: dovecot-boun...@dovecot.org Date: Tue, 8 Nov 2011 14:50:25 To: Subject: [Dovecot] Performance-Tuning Hi, I have > 11 TB hard used Mailstorage, saved

[Dovecot] Performance-Tuning

2011-11-08 Thread Peer Heinlein
Hi, I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that. At the moment I *need* more speed, we have too much wa