Here are some more findings... The Nexenta box has 3 pools: syspool: made of 2 mirrored (hardware RAID) local SAS disks pool_sas: made of 22 15K SAS disks in ZFS mirrors on 2 JBODs on 2 controllers pool_sata: made of 42 SATA disks in 6 RAIDZ2 vdevs on a single controller
When we copy data from any linux box to either the pool_sas or pool_sata, it is painfully slow. When we copy data from any linux box directly to the syspool, it is plenty fast When we copy data locally on the Nexenta box from the syspool to either the pool_sas or pool_sata, it is crazy fast. We also see the same pattern whether we use iSCSI or NFS. We've also tested using different NICs (some at 1GbE, some at 10GbE) and even tried bypassing the switch by directly connecting the two boxes with a cable- and it didn't made any difference. We've also tried not using the SSD for the ZIL. So... We've ruled out iSCSI, the networking, the ZIL device, even the HBAs as it is fast when it is done locally. Where should we look next? Thank you all for your help! Ian -- This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
