On Jan 26, 2007, at 09:16, Jeffery Malloch wrote:
Hi Folks,
I am currently in the midst of setting up a completely new file
server using a pretty well loaded Sun T2000 (8x1GHz, 16GB RAM)
connected to an Engenio 6994 product (I work for LSI Logic so
Engenio is a no brainer). I have configured a couple of zpools
from Volume groups on the Engenio box - 1x2.5TB and 1x3.75TB. I
then created sub zfs systems below that and set quotas and
sharenfs'd them so that it appears that these "file systems" are
dynamically shrinkable and growable.
ah - the 6994 is the controller we use in the 6140/6540 if i'm not
mistaken .. i guess this thread will go down in a flaming JBOD vs
RAID controller religious war again .. oops, too late :P
yes - the dynamic LUN expansion bits in ZFS is quite nice and handy
for managing dynamic growth of a pool or file system. so going back
to Jeffery's original questions:
1. How stable is ZFS? The Engenio box is completely configured
for RAID5 with hot spares and write cache (8GB) has battery backup
so I'm not too concerned from a hardware side. I'm looking for an
idea of how stable ZFS itself is in terms of corruptability, uptime
and OS stability.
I think the stability issue has already been answered pretty well ..
8GB battery backed cache is nice .. performance wise you might find
some odd interactions with the ZFS adaptive cache integration and the
way in which the intent log operates (O_DSYNC writes can potentially
impose a lot of in flight commands for relatively little work) -
there's a max blocksize of 128KB (also maxphys), so you might want to
experiment with tuning back the stripe width .. i seem to recall the
the 6994 controller seemed to perform best with 256KB or 512KB stripe
width .. so there may be additional tuning on the read-ahead or write-
behind algorithms.
2. Recommended config. Above, I have a fairly simple setup. In
many of the examples the granularity is home directory level and
when you have many many users that could get to be a bit of a
nightmare administratively. I am really only looking for high
level dynamic size adjustability and am not interested in its built
in RAID features. But given that, any real world recommendations?
Not being interested in the RAID functionality as Roch points out
eliminates the self-healing functionality and reconstruction bits in
ZFS .. but you still get other nice benefits like dynamic LUN expansion
As i see it, since we seem to have excess CPU and bus capacity on
newer systems (most applications haven't quite caught up to impose
enough of a load yet) .. we're back to the mid '90s where host based
volume management and caching makes sense and is being proposed
again. Being proactive, we might want to consider putting an
embedded Solaris/ZFS on a RAID controller to see if we've really got
something novel in the caching and RAID algorithms for when the
application load really does catch up and impose more of a load on
the host. Additionally - we're seeing that there's a big benefit in
moving the filesystem closer to the storage array since most users
care more about their consistency of their data (upper level) than
the reliability of the disk subsystem or RAID controller.
Implementing a RAID controller that's more intimately aware of the
upper data levels seems like the next logical evolutionary step.
3. Caveats? Anything I'm missing that isn't in the docs that
could turn into a BIG gotchya?
I would say be careful of the ease at which you can destroy file
systems and pools .. while convenient - there's typically no warning
if you or an administrator does a zfs or zpool destroy .. so i could
see that turning into an issue. Also if a LUN goes offline, you may
not see this right away and you would have the potential to corrupt
your pool or panic your system. Hence the self-healing and scrub
options to detect and repair failure a little bit faster. People on
this forum have been finding RAID controller inconsistencies .. hence
the religious JBOD vs RAID ctlr "disruptive paradigm shift"
4. Since all data access is via NFS we are concerned that 32 bit
systems (Mainly Linux and Windows via Samba) will not be able to
access all the data areas of a 2TB+ zpool even if the zfs quota on
a particular share is less then that. Can anyone comment?
Doing 2TB+ shouldn't be a problem for the NFS or Samba mounted
filesystem regardless if the host is 32bit or not. The only place
where you can run into a problem is if the size of an individual file
crosses 2 or 4TB on a 32bit system. I know we've implemented file
systems (QFS in this case) that were samba shared to 32bit windows
hosts in excess of 40-100TB without any major issues. I'm sure
there's similar cases with ZFS and thumper .. i just don't have that
data.
a little late to the discussion, but hth
---
.je
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
