On Mon, 28 Sep 2009, Richard Connamacher wrote:
Thanks for the detailed information. When you get the patch, I'd
love to hear if it fixes the problems you're having. From my
understanding, a working prefetch would keep video playback from
stuttering whenever the drive head moves — is this right?
For me, agressive prefetch is most important in order to schedule
reads from enough disks in advance to produce a high data rate. This
is because I am using mirrors. When using raidz or raidz2 the
situation should be a bit different because raidz is striped. The
prefetch bug which is specifically fixed is when using thousands of
files in the 5MB-8MB range which is typical for film postproduction.
The bug is that prefetch becomes disabled if the file had been
accessed before but its data is no longer in cache.
When doing video playback, it is typical to be reading from several
files at once in order to avoid the potential for read "stutter".
The inability to read and write simultaneously (within reason) would
be frustrating for a shared video editing server. I wonder if ZFS
needs more parallelism? If any software RAID ends up having a
ZFS has a lot of parallelism since it is optimized for large data
servers.
similar problem, then we might have to go with the hardware RAID
setups I'm trying to avoid. I wonder if there's any way to work
around that. Would a bigger write cache help? Or adding an SSD for
the cache (ZFS Intent Log)? would Linux software RAID be any better?
The problem seems to be that ZFS uses a huge write cache by default
and it delays flushing it (up to 30 seconds) so that when the write
cache is flushed, it maximally engages the write channel for up to 5
seconds. Decreasing the size of the write cache diminishes the size
of the problem.
Assuming they fix the prefetch performance issues you talked about,
do you think ZFS would be able to keep up with uncompressed 1080p HD
or 2K?
That is not clear to me yet. With my setup, I can read up to
550MB/second from a large file. That is likely the hardware limit for
me. But when reading one-at-a-time from individual 5 or 8MB files,
the data rate is much less (around 130MB/second).
I am using Solaris 10. OpenSolaris performance seems to be better
than Solaris 10.
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
