> My testing has shown some serious problems with the > iSCSI implementation for OpenSolaris. > > I setup a VMware vSphere 4 box with RAID 10 > direct-attached storage and 3 virtual machines: > - OpenSolaris 2009.06 (snv_111b) running 64-bit > - CentOS 5.3 x64 (ran yum update) > - Ubuntu Server 9.04 x64 (ran apt-get upgrade) > > I gave each virtual 2 GB of RAM, a 32 GB drive and > setup a 16 GB iSCSI target on each (the two Linux vms > used iSCSI Enterprise Target 0.4.16 with blockio). > VMware Tools was installed on each. No tuning was > done on any of the operating systems. > > I ran two tests for write performance - one one the > server itself and one from my Mac connected via > Gigabit (mtu of 1500) iSCSI connection using > globalSAN’s latest initiator. > > Here’s what I used on the servers: > time dd if=/dev/zero of=/root/testfile bs=1048576k > count=4 > and the Mac OS with the iSCSI connected drive > (formatted with GPT / Mac OS Extended journaled): > time dd if=/dev/zero of=/Volumes/test/testfile > bs=1048576k count=4 > > The results were very interesting (all calculations > using 1 MB = 1,084,756 bytes) > > For OpenSolaris, the local write performance averaged > 86 MB/s. I turned on lzjb compression for rpool (zfs > set compression=lzjb rpool) and it went up to 414 > MB/s since I’m writing zeros). The average > performance via iSCSI was an abysmal 16 MB/s (even > with compression turned on - with it off, 13 MB/s). > > For CentOS (ext3), local write performance averaged > 141 MB/s. iSCSI performance was 78 MB/s (almost as > fast as local ZFS performance on the OpenSolaris > server when compression was turned off). > > Ubuntu Server (ext4) had 150 MB/s for the local > write. iSCSI performance averaged 80 MB/s. > > One of the main differences between the three virtual > machines was that the iSCSI target on the Linux > machines used partitions with no file system. On > OpenSolaris, the iSCSI target created sits on top of > ZFS. That creates a lot of overhead (although you do > get some great features). > > Since all the virtual machines were connected to the > same switch (with the same MTU), had the same amount > of RAM, used default configurations for the operating > systems, and sat on the same RAID 10 storage, I’d say > it was a pretty level playing field. > > While jumbo frames will help iSCSI performance, it > won’t overcome inherit limitations of the iSCSI > target’s implementation.
cross-posting with zfs discuss. -- This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss