On Fri, Apr 2, 2010 at 8:03 AM, Edward Ned Harvey <solar...@nedharvey.com> wrote: >> > Seriously, all disks configured WriteThrough (spindle and SSD disks >> > alike) >> > using the dedicated ZIL SSD device, very noticeably faster than >> > enabling the >> > WriteBack. >> >> What do you get with both SSD ZIL and WriteBack disks enabled? >> >> I mean if you have both why not use both? Then both async and sync IO >> benefits. > > Interesting, but unfortunately false. Soon I'll post the results here. I > just need to package them in a way suitable to give the public, and stick it > on a website. But I'm fighting IT fires for now and haven't had the time > yet. > > Roughly speaking, the following are approximately representative. Of course > it varies based on tweaks of the benchmark and stuff like that. > Stripe 3 mirrors write through: 450-780 IOPS > Stripe 3 mirrors write back: 1030-2130 IOPS > Stripe 3 mirrors write back + SSD ZIL: 1220-2480 IOPS > Stripe 3 mirrors write through + SSD ZIL: 1840-2490 IOPS > > Overall, I would say WriteBack is 2-3 times faster than naked disks. SSD > ZIL is 3-4 times faster than naked disk. And for some reason, having the > WriteBack enabled while you have SSD ZIL actually hurts performance by > approx 10%. You're better off to use the SSD ZIL with disks in Write > Through mode. > > That result is surprising to me. But I have a theory to explain it. When > you have WriteBack enabled, the OS issues a small write, and the HBA > immediately returns to the OS: "Yes, it's on nonvolatile storage." So the > OS quickly gives it another, and another, until the HBA write cache is full. > Now the HBA faces the task of writing all those tiny writes to disk, and the > HBA must simply follow orders, writing a tiny chunk to the sector it said it > would write, and so on. The HBA cannot effectively consolidate the small > writes into a larger sequential block write. But if you have the WriteBack > disabled, and you have a SSD for ZIL, then ZFS can log the tiny operation on > SSD, and immediately return to the process: "Yes, it's on nonvolatile > storage." So the application can issue another, and another, and another. > ZFS is smart enough to aggregate all these tiny write operations into a > single larger sequential write before sending it to the spindle disks.
Hmm, when you did the write-back test was the ZIL SSD included in the write-back? What I was proposing was write-back only on the disks, and ZIL SSD with no write-back. Not all operations hit the ZIL, so it would still be nice to have the non-ZIL operations return quickly. -Ross _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
