Nicholas Lee wrote:
On 4/19/07, *Adam Lindsay* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
16x hot swap SATAII hard drives (plus an internal boot drive)
Tyan S2895 (K8WE) motherboard
Dual GigE (integral nVidia ports)
2x Areca 8-port PCIe (8-lane) RAID drivers
2x AMD Opteron 275 CPUs (2.2GHz, dual core)
8 GiB RAM
...
I guess my questions are:
- Does anyone out there have a clue where the potential bottlenecks
might be?
Get INTEL GigE!!!!!!! NVidia aren't as fast and Intel drivers are very
good.
yes, I've been made aware of a lot of pushback on the nVidia
drivers/hardware recently, and so I'll aim to put a better networking
card in, as well.
If it is just storage then it might not matter whether you use Opterons
or Xeons. Since you are just feeding stuff from the disk to the nic,
things like HyperTransport are probably not so much of a win.
Not sure if Tyan have Intel 3U systems, maybe look at Supermicro instead.
It's just the mobo, at the moment, from Tyan. The local vendor has his
standard 3U chassis that he uses.
- Is there anywhere where I can save a bit of money? (For example,
might the SuperMicro AOC-SAT2-MV8 hanging off the PCI-X slots
provide enough bandwidth to the disks?)
I'm using this card, seems to work fine. I posted some numbers on this
at [1].
That's good, interesting stuff. Here are a few thoughts going through my
head:
- The bus interconnect is more critical with software RAID than
hardware, no? For each block, more data has to go to the CPU.
- I'm very lured by the performance offered by two 8-lane PCIe cards
(full duplex, mind...) over PCI-X cards running (with the selected mobo)
at 133MHz and 100MHz.
- I'll be trying to wring as much IO performance as possible from 15
drives -- most likely 3x 5-disk RAIDZ sets. While your figures are
encouraging, I'm not sure they wouldn't hit a ceiling before they scale
up to where I want to be. (It looks like they won't, but I'm not sure.)
This is a [2] 2U 12 disk system with two Dual Core 2GHz Xeons, 4Gb.
2x2x500Gb mirror and 5xRaid2z with Seagate ES 500Gb drives. Rellings
reply might be useful as well.
Raid cards a definitely a waste of money if you are using zfs.
Which is exactly what I thought going into this, but I am not aware of
that many choices in JBOD controllers, especially running off the PCIe bus.
Thanks for the time and pointers,
adam
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
