On Wed, May 26, 2010 at 8:35 PM, Marc Bevand <m.bev...@gmail.com> wrote: > The Supermicro X8DTi mobo and LSISAS9211-4i HBA are both PCIe 2.0 compatible, > so the max theoretical PCIe x4 throughput is 4GB/s aggregate, or 2GB/s in each > direction, well above the 800MB/s bottleneck observed by Giovanni.
I only looked at the Megaraid 8888 that he mentioned, which has a PCIe 1.0 4x interface, or 1000MB/s. I wonder if its performance was slightly lower than the LSI 9211. The board also has a PCIe 1.0 4x electrical slot, which is 8x physical. If the card was in the PCIe slot furthest from the CPUs, then it was only running 4x. > A single 3Gbps link provides in theory 300MB/s usable after 8b-10b encoding, > but practical throughput numbers are closer to 90% of this figure, or 270MB/s. > 6 disks per link means that each disk gets allocated 270/6 = 45MB/s. ... except that a SFF-8087 connector contains four 3Gbps connections. It may depend on how the drives were connected to the expander. You're assuming that all 18 are on 3 channels, in which case moving drives around could help performance a bit. If the expander is able to use all four channels, which it should be able to do, there would be 1200MB/s theoretical, or 1080MB/s using your 90% figure of actual bandwidth available. > So with 18 disks striped, this gives a max usable throughput of 18*45 = > 810MB/s, > which matches exactly what Giovanni observed. QED! Giovani will have to confirm the layout of the drives and which expansion slot he was using to be sure. Until then, I declare myself the winner in this pissing contest. -B -- Brandon High : bh...@freaks.com _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
