> On Wed, 24 Jun 2009, Lejun Zhu wrote: > > > There is a bug in the database about reads blocked > by writes which may be related: > > > > > http://bugs.opensolaris.org/view_bug.do?bug_id=6471212 > > > > The symptom is sometimes reducing queue depth makes > read perform better. > > I have been banging away at this issue without > resolution. Based on > Roch Bourbonnais's blog description of the ZFS write > throttle code, it > seems that I am facing a perfect storm. Both the > storage write > bandwidth (800+ MB/second) and the memory size of my > system (20 GB) > result in the algorithm batching up 2.5 GB of user > data to write.
With ZFS write throttle, the number 2.5GB is tunable. From what I've read in the code, it is possible to e.g. set zfs:zfs_write_limit_override = 0x8000000 (bytes) to make it write 128M instead. > Since I am using mirrors, this results in 5 GB of > data being written > at full speed to the array on a very precise schedule > since my > application is processing fixed-sized files with a > fixed algorithm. > The huge writes lead to at least 3 seconds of read > starvation, > resulting in a stalled application and a square-wave > of system CPU > utilization. I could attempt to modify my > application to read ahead > by 3 seconds but that would require gigabytes of > memory, lots of > complexity, and would not be efficient. > > Richard Elling thinks that my array is pokey, but > based on write speed > and memory size, ZFS is always going to be batching > up data to fill > the write channel for 5 seconds so it does not really > matter how fast > that write channel is. If I had 32GB of RAM and 2X > the write speed, > the situation would be identical. > > Hopefully someone at Sun is indeed working this read > starvation issue > and it will be resolved soon. > > Bob > -- > Bob Friesenhahn > bfrie...@simple.dallas.tx.us, > http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, > http://www.GraphicsMagick.org/ > ____________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ss -- This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
