On Fri, 7 May 2010, Kris Kasner wrote:
One thing my customers noticed immediately was a reduction in "free" memory as reported by 'top'. By way of explaining that ZFS keeps it's cache in kernel and not in the freelist, it became apparent that memory is being used disproportionally to the filesystems.
Strange.
Is this a problem? solarisinternals.com ZFS Best Practices seems to indicate reducing the ARC in the presence of another filesystem is a good idea.
That would be if they are found to be competing for memory, with both under memory pressure. Under normal conditions, your zfs-based root filesystem should be idling and not require very much memory, but it will cache data in memory if memory is available.
In this scenario (boot on ZFS, everything else on VxFS) what would be a reasonable value to limit the ZFS ARC without impacting performance?
Depending on what ZFS is being used for, it should be fine to limit to a couple of GB.
Is ZFS swap cached in the ARC? I can't account for data in the ZFS filesystems to use as much ARC as is in use without the swap files being cached.. seems a bit redundant?
With zfs boot, the default system paging device ("swap") is based on the zfs filesystem. Usually this is not a problem since the paging device is rarely used. Something to be aware of is that zfs does not use the "unified page cache" like UFS does. This means that a memory mapped file or device may end up using twice as much memory, with reconciliation in zfs if a memory-mapped page is updated. Also something to be aware of is that Solaris /tmp is mapped to swap. Based on this, I recommend looking for large use of /tmp and/or applications which memory map files (in swap, /tmp, /var/tmp) or devices (e.g. /dev/zero).
With ZFS ARC in kernel, does this reduce the memory that vxfs sees as available for it's own buffer cache? From teh Evil Tuning Guide and other reading at solarisinternals, ZFS is supposed to be a good citizen when it comes to giving up memory for userland apps that need memory, but is vxfs asking for memory in a way that zfs is pushing it into the corner?
Zfs does not push very hard (although it likes to consume). Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
