You might want to have a look at my blog on filesystem cache
tuning... It will probably help
you to avoid memory contention between the ARC and your apps.
http://www.thezonemanager.com/2009/03/filesystem-cache-optimization.html
Brad
Brad Diggs
Senior Directory Architect
Virtualization Architect
xVM Technology Lead
Sun Microsystems, Inc.
Phone x52957/+1 972-992-0002
Mail bradley.di...@sun.com
Blog http://TheZoneManager.com
Blog http://BradDiggs.com
On Jul 4, 2009, at 2:48 AM, Phil Harman wrote:
ZFS doesn't mix well with mmap(2). This is because ZFS uses the ARC
instead of the Solaris page cache. But mmap() uses the latter. So if
anyone maps a file, ZFS has to keep the two caches in sync.
cp(1) uses mmap(2). When you use cp(1) it brings pages of the files
it copies into the Solaris page cache. As long as they remain there
ZFS will be slow for those files, even if you subsequently use
read(2) to access them.
If you reboot, your cpio(1) tests will probably go fast again, until
someone uses mmap(2) on the files again. I think tar(1) uses
read(2), but from my iPod I can't be sure. It would be interesting
to see how tar(1) performs if you run that test before cp(1) on a
freshly rebooted system.
I have done some work with the ZFS team towards a fix, but it is
only currently in OpenSolaris.
The other thing that slows you down is that ZFS only flushes to disk
every 5 seconds if there are no synchronous writes. It would be
interesting to see iostat -xnz 1 while you are running your tests.
You may find the disks are writing very efficiently for one second
in every five.
Hope this helps,
Phil
blogs.sun.com/pgdh
Sent from my iPod
On 4 Jul 2009, at 05:26, Bob Friesenhahn
<bfrie...@simple.dallas.tx.us> wrote:
On Fri, 3 Jul 2009, Bob Friesenhahn wrote:
Copy Method Data Rate
==================================== ==================
cpio -pdum 75 MB/s
cp -r 32 MB/s
tar -cf - . | (cd dest && tar -xf -) 26 MB/s
It seems that the above should be ammended. Running the cpio based
copy again results in zpool iostat only reporting a read bandwidth
of 33 MB/second. The system seems to get slower and slower as it
runs.
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
