Hi,
I know (from the zfs-discuss archives and other places [1,2,3,4]) that a
lot of people are looking to use zfs as a storage server in the 10-100TB
range.
I'm in the same boat, but I've found that hardware choice is the biggest
issue. I'm struggling to find something which will work nicely under
solaris and which meets my expectations in terms of hardware. Because of
the compatibility issues, I though I should ask here to see what
solutions people have already found.
I'm learning as I go here, but as far as I've been able to determine,
the basic choices for attaching drives seem to be
1) SATA Port multipliers
2) SAS Multilane Enclosures
3) SAS Expanders
In option 1, the controller can only talk to one device at a time, in
option 2 each miniSAS connector can talk to 4 drives at a time but in
option 3 the expander can allow for communication with up to 128 drives.
I'm thinking about having ~8-16 drives on each controller (PCI-e card)
so I think I want option 3. Additionally, because I might get greedier
in the future and decide to add more drives on each controller I think
option 3 is the best way to go. I can have a motherboard with a lot of
PCIe slots and have one controller card for each expander.
Cases like the Supermicro 846E1-R900B have 24 hot swap bays accessible
via a single (4u) LSI SASX36 SAS expander chip, but I'm worried about
controller death and having the backplane as a single point of failure.
I guess, ideally, I'd like a 4u enclosure with 2x2u SAS expanders. If I
wanted hardware redundancy, I could then use mirrored vdevs with one
side of each mirror on one controller/expander pair and the other side
on a separate pair. This would allow me to survive controller or
expander death as well hard drive failure.
Replace motherboard: ~500
Replace backplane: ~500
Replace controller: ~300
Replace disk (SATA): ~100
Does anyone have any example systems they have built or any thoughts on
what I could do differently?
Best regards,
Ian.
[1] http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg27234.html
[2] http://www.avsforum.com/avs-vb/showthread.php?p=17543496
[3] http://www.stringliterals.com/?p=53
[4] http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg22761.html
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
