On Wed, May 26, 2010 at 6:09 PM, Giovanni Tirloni <gtirl...@sysdroid.com> wrote: > On Wed, May 26, 2010 at 9:22 PM, Brandon High <bh...@freaks.com> wrote: >> >> I'd wager it's the PCIe x4. That's about 1000MB/s raw bandwidth, about >> 800MB/s after overhead. > > Makes perfect sense. I was calculating the bottlenecks using the > full-duplex bandwidth and it wasn't apparent the one-way bottleneck.
Actually both of you guys are wrong :-) The Supermicro X8DTi mobo and LSISAS9211-4i HBA are both PCIe 2.0 compatible, so the max theoretical PCIe x4 throughput is 4GB/s aggregate, or 2GB/s in each direction, well above the 800MB/s bottleneck observed by Giovanni. This bottleneck is actually caused by the backplane: Supermicro "E1" chassis like Giovanni's (SC846E1) include port multipliers that degrade performance by putting 6 disks behind a single 3Gbps link. A single 3Gbps link provides in theory 300MB/s usable after 8b-10b encoding, but practical throughput numbers are closer to 90% of this figure, or 270MB/s. 6 disks per link means that each disk gets allocated 270/6 = 45MB/s. So with 18 disks striped, this gives a max usable throughput of 18*45 = 810MB/s, which matches exactly what Giovanni observed. QED! -mrb _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
