>On May 9, 2010 at 5:29PM, Richard Elling wrote: >>On May 9, 2011, at 12:29 PM, Chris Forgeron wrote: >>[..] >> Q1 - Doesn't this behavior mean that the L2ARC can never get data objects if >> the ARC doesn't hold them? > >Yes. > >> Is setting primary to metadata and secondary to all an impossible request? > >Today, this is not how it is implemented. However, it is open source :-) The >current architecture allows the L2ARC fill to not impact the normal ARC >operations. > If you implement a policy that says "only use the L2ARC to cache data" then > you are implicitly requiring the write bandwidth of the L2ARC to be faster > than > the read bandwidth of the pool -- not likely to be a winning combination. > This isn't a problem for > the ARC because it has memory bandwidth, which is, > of > course, always greater than I/O bandwidth.
>> I believe the user data still goes through the ARC, it's just not kept when >> primarycache=metadata. >Correct. Ah, thanks, that saves me a lot of time banging by head against the desk trying to figure out why I can't make this setting work. ZFS is open source, but we're in a bit of a stall with ZFS development, until the behind-the-scenes committee finishes deliberation and we have new direction to start working with. IMO, anything other than a very small patch is dangerous to work on, as we don't know what's in the cooker for the next ZFS OpenSource release. I do recognize my status as a ZFS leech at the moment (not committing any code, but asking for changes), but it's not easy to transition. I have gone through the ARC code in detail - I'll have to review again and see if there are any quick tweaks I can perform that may give me the options (and thus the performance) I'm looking for. Of course, I think there's nothing small and quick to be done to the ZFS code, or it would have already been done. :-) >> Q2 is basically: What's the best way to keep as much of the DDT live in ARC >> and L2ARC without buying more RAM? >> >> Hopefully I'm not dismissed with "Just buy more RAM", which I can see as a >> valid point in a lot of situations where people are running on 4 and 8 gig >> systems and trying to access 12+ TB of data in a DeDupe situation. > > :-) Thanks for not saying it. :-) >> >> Keeping my RAM at 24 gigs isn't just a budget request, but it also the max >> RAM you can get in most workstation boards these days, and it's moving in >> the direction of better energy usage and heat generation, as my 120 Gig >> SSD's burn a fraction of the >>power than another 24 gigs of DDR3 will. >Perhaps use the tool designed for the task? > -- richard Which tool is this? >> >> Even if I added more RAM, I'd still have this problem. Say I have 196 Gigs >> of RAM in this server, and I want to dedicate the ARC just to metadata, and >> the L2ARC to user data. From my experiments, this wouldn't work. As I keep >> scaling up this server, it will keep eroding the metadata as userdata pours >> through the cache system. >> >> I think the ultimate would be cache priority - Setting metadata to be the >> most important, and then user data as secondary importance, so that it never >> evicts any metadata from ARC/L2ARC for user data, however I don't think this >> is possible today, and I'm unsure if it's on the drawing board. >> After looking at the ARC code, I wonder if the easier method wouldn't be a arc_data_limit much like the arc_meta_limit. It would work exactly the same way as meta_limit, but now we'd be able to save the precious ARC for as much metadata as we desired. When primarycache=all, ARC would contain user data, and thus would be available to the L2ARC for caching. Because we can set the max amount of user data to live in the cache, we would be able to control the erosion of the ARC's metadata. Eventually if you have your secondarycache=all , both metadata and userdata would enter the L2ARC. You'd be able to achieve many levels of fine-grained control over the caching. As it stands right now from my tests, the user data will _always_ erode the metadata from the ARC, and that can't be good for speed in a dedup situation, no matter how much RAM you have available. This means that the general assertion that dedup only impacts a small hit on speed due to the lookup is assuming that you're not going to use your ARC for anything other than metadata, and that the entire dedup table fits in ARC. The reality is that the DDT needs to be fetched from L2ARC or even disk more frequently than you'd expect. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
