On Thu, 3 Sep 2009, Ivan Voras wrote:
2009/9/3 Scott Long <sco...@samsco.org>:
On Thu, 3 Sep 2009, Alexander Motin wrote:
It would be nice if every level would do it's own job.
It's the job of the driver to handle the limitations of the hardware, yes.
Again, if you want to experiment with pushing this functionality into GEOM,
be my guest. But until then, consider following my advice.
Speaking as an user who goes "huh, well" every time he sees a RAID
card with a GB of cache talking to the OS in 64 kB chunks, eventually
removing this limitation seems a Nice Thing to Have.
I don't know how things look at the driver side, but GEOM by itself
has no problems passing around requests of at least "long" in length.
Specifically, it cares about struct bio, where bio_bcount is a "long"
and bio_length and bio_completed are off_t. So, ssize_t looks ok as a
high boundary.
There was a time (apparently much of it was a bug in reporting and is
now fixed :( ) when MAXPHYS could be manually redefined to be 256K or
more and iostat would nicely state the higher value. I think the
concern raised at the topic was that it doesn't play nice with
bufcache, and I think the specific problem was possible out-of-memory
situations. Now that kernel limits on AMD64 are much increased (to
values not longer fitting in uint32_t) I wonder if the problem is so
serious?
The problem is lack of kernel address space, not lack of RAM, but that's
just semantics in this discussion. I've tested with increasing MAXPHYS in
increments to 1M. Performance increases logrithmically, and effectively
hits a max at 512K for the variety of controllers that I tested. The gain
from 64K to 128K is huge, the gain from 128K to 256K is ok, the gain from
256k to 512k is measurable but less significant, and the gain from 512k to
1m is almost not measurable.
I have simple patches to increase MAXPHYS. The introduction of the the
maxio paramter in the CAM SIM interface is there in preparation for this.
However, a _LOT_LOT_LOT_ of drivers in the tree falsely assume that
MAXPHYS and DEFLTPHYS are 128k and 64k respectively, and size their data
structures accordingly. Changing these values will cause the drivers to
fail in bad ways. So an audit needs to be done. Also, MAXPHYS is abused
by the swapper in the struct-buf, so that needs to be reviewed as well.
Even though kernel address space is less restricted on 64bit platforms,
it's still not free and limitless. Large I/O's requires more work in the
VM to assign address space, and in turn causes more lock contention. I
haven't done any practical measurements of this on common workloads, but I
can anecdotally say that I see increased lock contention from it in
locking profiles. If FreeBSD wants to seriously increase MAXPHYS, this
needs to be looked at and either proven to not be important, or fixed.
Scott
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
