Re: LARGE single-system Cyrus installs?

2007-10-06 Thread Wesley Craig
Personally, I've seen Solaris bottlenecking on file opens in large directories. This was a while ago, but it was one of the major reason we switched to Linux -- the order of magnitude improvement in directory scale was sure handy for 80-90K users with no quota. The kind of blocking I'm ta

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread Vincent Fox
Rob Mueller wrote: >> We are in the process of moving from reiserfs to ext3 (with dir_index). >> >> ZFS with mirrors across 2 separate storage devices, means never having to say you're sorry. I sleep very well at night. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ:

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread David Carter
On Sat, 6 Oct 2007, Rob Mueller wrote: That's strange. What mount options are/were you using? We use/used: reiserfs - rw,noatime,nodiratime,notail,data=journal ext3 - noatime,nodiratime,data=journal Same, but data=ordered in both cases If you weren't using "notail" on reiserfs, that would de

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread Rob Mueller
>> Are you comparing an "old" reiserfs partition with a "new" ext3 one where >> you've just copied the email over to? If so, that's not a fair >> comparison. > > No, a newly created partitions in both cases. Fragmented partitions are > slower still of course. That's strange. What mount options

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread David Carter
On Sat, 6 Oct 2007, Rob Mueller wrote: > Are you comparing an "old" reiserfs partition with a "new" ext3 one where > you've just copied the email over to? If so, that's not a fair comparison. No, a newly created partitions in both cases. Fragmented partitions are slower still of course. > Give

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread Rob Mueller
> I think what truly scares me about reiser is those rather regular > posts to various mailing lists I'm on saying "my reiser fs went poof > and lost all my data, what should I do?" I've commented on this before. I believe it's absolutely hardware related rather than reiserfs related. http://ww

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread Rob Mueller
> The iostat and sar data disagrees with it being an I/O issue. > > 16 gigs of RAM with about 4-6 of it being used for Cyrus > leaves plenty for ZFS caching. Our hardware seemed more than > adequate to anyone we described it to. > > Yes beyond that it's anyone guess. If it wasn't IO limit relate

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread Rob Mueller
> A data point regarding reiserfs/ext3: > > We are in the process of moving from reiserfs to ext3 (with dir_index). > > ext3 seems to do substantially better than reiserfs for us, especially for > read heavy loads (squatter runs at least twice as fast as it used do). Are you comparing an "old" re

Re: LARGE single-system Cyrus installs?

2007-10-06 Thread Rob Mueller
> I suppose that 8 SATA disks for the data and four 15k SAS disks for the > metadata would be a good mix. Yes. As I mentioned, our iostat data shows that meta-data is MUCH hotter than email spool data. --- Checking iostat, a rough estimate shows meta data get 2 x the rkB/s and 3 x the wkB/s vs