Ross Asks: So on that note, ZFS should disable the disks' write cache, not enable them despite ZFS's COW properties because it should be resilient.
No, because ZFS builds resiliency on top of unreliable parts. it's able to deal with contained failures (lost state) of the disk write cache. It can then export LUNS that have WC enabled or disabled. But if we enable the WC on the exported LUNS, then the consumer of these LUNS must be able to say the same. The discussion at that level then needs to focus on failure groups. Ross also Said : I asked this question earlier, but got no answer: while an iSCSI target is presented WCE does it respect the flush command? Yes. I would like to say "obviously" but it's been anything but. -r Ross Walker writes: > On Aug 4, 2010, at 3:52 AM, Roch <roch.bourbonn...@sun.com> wrote: > > > > > Ross Walker writes: > > > >> On Aug 3, 2010, at 12:13 PM, Roch Bourbonnais <roch.bourbonn...@sun.com> > >> wrote: > >> > >>> > >>> Le 27 mai 2010 à 07:03, Brent Jones a écrit : > >>> > >>>> On Wed, May 26, 2010 at 5:08 AM, Matt Connolly > >>>> <matt.connolly...@gmail.com> wrote: > >>>>> I've set up an iScsi volume on OpenSolaris (snv_134) with these > >>>>> commands: > >>>>> > >>>>> sh-4.0# zfs create rpool/iscsi > >>>>> sh-4.0# zfs set shareiscsi=on rpool/iscsi > >>>>> sh-4.0# zfs create -s -V 10g rpool/iscsi/test > >>>>> > >>>>> The underlying zpool is a mirror of two SATA drives. I'm connecting > >>>>> from a Mac client with global SAN initiator software, connected via > >>>>> Gigabit LAN. It connects fine, and I've initialiased a mac format > >>>>> volume on that iScsi volume. > >>>>> > >>>>> Performance, however, is terribly slow, about 10 times slower than an > >>>>> SMB share on the same pool. I expected it would be very similar, if > >>>>> not faster than SMB. > >>>>> > >>>>> Here's my test results copying 3GB data: > >>>>> > >>>>> iScsi: 44m01s 1.185MB/s > >>>>> SMB share: 4m27 11.73MB/s > >>>>> > >>>>> Reading (the same 3GB) is also worse than SMB, but only by a factor of > >>>>> about 3: > >>>>> > >>>>> iScsi: 4m36 11.34MB/s > >>>>> SMB share: 1m45 29.81MB/s > >>>>> > >>> > >>> <cleaning up some old mail> > >>> > >>> Not unexpected. Filesystems have readahead code to prefetch enough to > >>> cover the latency of the read request. iSCSI only responds to the > >>> request. > >>> Put a filesystem on top of iscsi and try again. > >>> > >>> For writes, iSCSI is synchronous and SMB is not. > >> > >> It may be with ZFS, but iSCSI is neither synchronous nor asynchronous is > >> is simply SCSI over IP. > >> > > > > Hey Ross, > > > > Nothing to do with ZFS here, but you're right to point out > > that iSCSI is neither. It was just that in the context of > > this test (and 99+% of iSCSI usage) it will be. SMB is > > not. Thus a large discrepancy on the write test. > > > > Resilient storage, by default, should expose iSCSI channels > > with write caches disabled. > > > So on that note, ZFS should disable the disks' write cache, not enable them > despite ZFS's COW properties because it should be resilient. > > > >> It is the application using the iSCSI protocol that > > determines whether it is synchronous, issue a flush after > > write, or asynchronous, wait until target flushes. > >> > > > > True. > > > >> I think the ZFS developers didn't quite understand that > > and wanted strict guidelines like NFS has, but iSCSI doesn't > > have those, it is a lower level protocol than NFS is, so > > they forced guidelines on it and violated the standard. > >> > >> -Ross > >> > > > > Not True. > > > > > > ZFS exposes LUNS (or ZVOL) and while at first we didn't support > > DKIOCSETWCE, we now do. So a ZFS LUN can be whatever you > > need it to be. > > I asked this question earlier, but got no answer: while an iSCSI target is > presented WCE does it respect the flush command? > > > Now in the context of iSCSI luns hosted by a resilient > > storage system, enabling write caches is to be used only in > > very specific circumstances. The situation is not symmetrical > > with WCE in disks of a JBOD since that can be setup with > > enough redundancy to deal with potential data loss. When > > using a resilient storage, you need to trust the storage for > > persistence of SCSI commands and building a resilient system > > on top of write cache enabled SCSI channels is not trivial. > > Not true, advertise WCE, support flush and tagged command queuing and the > initiator will be able to use the resilient storage appropriate for it's > needs. > > -Ross > _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
