Hi All,
I'd like to ask about whether there is a method to enforce a certain txg
commit frequency on ZFS. I'm doing a large amount of video streaming
from a storage pool while also slowly continuously writing a constant
volume of data to it (using a normal file descriptor, *not* in O_SYNC).
When reading volume goes over a certain threshold (and average pool load
over ~50%), ZFS thinks it's running out of steam on the storage pool and
starts committing transactions more often which results in even greater
load on the pool. This leads to a sudden spike in I/O utilization on the
pool in roughly the following method:
# streaming clients pool load [%]
15 8%
20 11%
40 22%
60 33%
80 44%
--- around here txg timeouts start to shorten ---
85 60%
90 70%
95 85%
My application does a fair bit of caching and prefetching, so I have
zfetch disabled and primarycache set to only metadata. Also, reads
happen (on a per client basis) relatively infrequently, so I can easily
take it if the pool stops reading for a few seconds and just writes
data. The problem is, ZFS starts alternating between reads and writes
really quickly, which in turn starves me on IOPS and results in a huge
load spike. Judging on load numbers up to around 80 concurrent clients,
I suspect I could go up to 150 concurrent clients on this pool, but
because of this spike I top out at around 95-100 concurrent clients.
Regards,
--
Saso
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
