I'm doing my own digging to figure out what seems to make sense. Josh had mentioned Puppet::push_context, set in the configurer. We push and pop context for each apply run; however that's a private API that doesn't seem to be meant for general use. Piggybacking on it looks like it would get messy.
There's also Puppet::Util::Storage, which superficially looks appropriate for this kind of caching ( http://www.rubydoc.info/gems/puppet/Puppet/Util/Storage). I'm still trying to wrap my head around what side-effects might occur. On Tue, Dec 16, 2014 at 6:27 PM, Trevor Vaughan <[email protected]> wrote: > > Part of my other heartburn with using a file was revisited hard upon me as > I recalled the original extdata function implementation. > > In the case of extdata, one large extdata file + a lot of extlookups = > massive catalog compile times on the server. > > So, every time I want to call the cache, across potentially large numbers > of providers and/or other things requiring state, I *really* don't want to > read a file. Particularly, when I don't know what's going to be in it. > > In this case, we would have to contend with slower client run times and > more CPU overhead as well as disk I/O requirements. Indicating that people > should change the way their OS is configured inasmuch as using tmpfs when > they may not have this choice does not seem ideal unless, of course, it > ships with puppet and doesn't require a system reboot. If, for some reason, > I have 50 providers that want to use this, this is 50 file reads and writes > that could be avoided. > > Giving people the choice of Disk vice Memory overhead would be ideal if > you want both for some reason. > > I'm honestly not seeing what would be so bad about scope.cache where cache > is some top level Puppet::Cache object that holds hashes that expire at the > end of a run. You would have to do things very politely in terms of > namespacing but you have to do that anyway. > > I am, of course, not opposed to saving cache state to disk for debugging > purposes, and think that should be an option when the --debug flag is used. > > Trevor > > Trevor > > On Tue, Dec 16, 2014 at 7:37 PM, Felix Frank < > [email protected]> wrote: >> >> Hey, >> >> good points - state retention at whatever granular level would be a good >> general purpose tool to have. If it's built in a pervasive fashion (i.e., >> any provider might use the cache for whetever it deems appropriate), it >> gains added visibility and becomes more opaque to the user - which is a >> good thing, and addresses one of the major concerns I'm having with this. >> The other being that it needs to be tunable for the user in some fashion. >> >> I have no qualms about disk I/O - after all, the user can choose whatever >> block backend they want. Users who depend on low latency or need to save >> IOPS can employ a tmpfs, for example. >> >> Cheers, >> Felix >> >> On 12/17/2014 12:56 AM, Trevor Vaughan wrote: >> >> I'm happy with catalog lifetime. >> >> I'm really not happy with doing anything that involves disk I/O. >> >> This would be key to getting providers to be able to save state in a >> non-hacky way as well. >> >> Trevor >> >> On Tue, Dec 16, 2014 at 6:45 PM, Michael Smith < >> [email protected]> wrote: >>> >>> I don't like any of the ideas I raised, but this will take some digging. >>> We need to determine what life-time the cache should have, and what >>> interface. I'm leaning towards either a cached read API in the FileSystem >>> utilities, or a cache tied to the catalog lifetime. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/5490D048.7020702%40Alumni.TU-Berlin.de >> <https://groups.google.com/d/msgid/puppet-dev/5490D048.7020702%40Alumni.TU-Berlin.de?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > Trevor Vaughan > Vice President, Onyx Point, Inc > (410) 541-6699 > [email protected] > > -- This account not approved for unencrypted proprietary information -- > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUCo4FmT9QGk_P1kYg0CdEWA9pqhU%3D6jeXjBAr9z7fD9w%40mail.gmail.com > <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUCo4FmT9QGk_P1kYg0CdEWA9pqhU%3D6jeXjBAr9z7fD9w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CABy1mM%2B2rP9wE%3D6bea91B5J_e6PLfNqjMB4KLWTLS6gT5iuxXg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
