On Nov 17, 2009, at 12:50 PM, Ian Allison wrote:
Hi,
I know (from the zfs-discuss archives and other places [1,2,3,4])
that a lot of people are looking to use zfs as a storage server in
the 10-100TB range.
I'm in the same boat, but I've found that hardware choice is the
biggest issue. I'm struggling to find something which will work
nicely under solaris and which meets my expectations in terms of
hardware. Because of the compatibility issues, I though I should ask
here to see what solutions people have already found.
I'm learning as I go here, but as far as I've been able to
determine, the basic choices for attaching drives seem to be
1) SATA Port multipliers
2) SAS Multilane Enclosures
3) SAS Expanders
In option 1, the controller can only talk to one device at a time,
in option 2 each miniSAS connector can talk to 4 drives at a time
but in option 3 the expander can allow for communication with up to
128 drives. I'm thinking about having ~8-16 drives on each
controller (PCI-e card) so I think I want option 3. Additionally,
because I might get greedier in the future and decide to add more
drives on each controller I think option 3 is the best way to go. I
can have a motherboard with a lot of PCIe slots and have one
controller card for each expander.
Cases like the Supermicro 846E1-R900B have 24 hot swap bays
accessible via a single (4u) LSI SASX36 SAS expander chip, but I'm
worried about controller death and having the backplane as a single
point of failure.
There will be dozens of single point failures in your system. Don't
worry about
controllers or expanders because they will be at least 10x more
reliable than
your disks. If you want to invest for better reliability, invest in
enterprise class
disks, preferably SSDs.
-- richard
I guess, ideally, I'd like a 4u enclosure with 2x2u SAS expanders.
If I wanted hardware redundancy, I could then use mirrored vdevs
with one side of each mirror on one controller/expander pair and
the other side on a separate pair. This would allow me to survive
controller or expander death as well hard drive failure.
Replace motherboard: ~500
Replace backplane: ~500
Replace controller: ~300
Replace disk (SATA): ~100
Does anyone have any example systems they have built or any thoughts
on what I could do differently?
Best regards,
Ian.
[1] http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg27234.html
[2] http://www.avsforum.com/avs-vb/showthread.php?p=17543496
[3] http://www.stringliterals.com/?p=53
[4] http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg22761.html
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
