I don't like any of the ideas I raised, but this will take some digging. We need to determine what life-time the cache should have, and what interface. I'm leaning towards either a cached read API in the FileSystem utilities, or a cache tied to the catalog lifetime.
On Tue, Dec 16, 2014 at 11:08 AM, Michael Smith < [email protected]> wrote: > > I'm trying to get my head around this, and I don't understand Henrik's > worry about unbound class variables causing memory leaks. If you have more > background on that, it might help. > > Avoiding globals in general means you have to have a file on disk. Any of > the solutions I can think of to speed that up - using an in-memory database > like SQLite or using a memory mapped file with > https://rubygems.org/gems/mmap (I have no experience with that module, so > not exactly a recommendation) - introduce their own global memory to keep > track of in-memory objects. However, they may be less problematic than > unbound class variables. > > Looking at how read_mounts is called, it looks pretty intrusive to try to > introduce a method argument to carry the cache. The other solution I think > is possible, but don't know off the top of my head how to do, is to change > the global variables to class variables that are bound when the module is > included; but that sounds like 'unbound class variables' to me. > > On Sun, Dec 7, 2014 at 3:12 PM, Trevor Vaughan <[email protected]> > wrote: >> >> So, in PUP-3116, I proposed a (very bad) solution for reading >> /prod/mounts every time a file was opened during a run. >> >> The original goal was to get around cases where the sandbox service had >> not been properly configured and /proc/mounts grew to a few thousand lines. >> >> What I would *love* is a proper top-level queueing mechanism both for >> global items and for individual resources. >> >> Henrik indicated that the use of unbound class variables was causing >> memory leaks so that's out and he recommended the use of a JSON file on the >> system that would be used as a temporary cache. >> >> While this would work, I'm really not thrilled about having anything that >> we read thousands of time from disk. >> >> I did some tinkering and managed to bind an additional variable set to >> the catalog (being the only globally persistent thing that I could find) >> but that's obviously also a Very Bad(tm) solution. >> >> So, a couple of questions: >> >> 1) Do any of the client internals gurus have a suggestion as to how to do >> this properly. >> >> 2) If there's no way to do it properly, whats the most system-efficient >> way to do it successfully, albeit improperly. >> >> I'm not going to post the catalog shim code unless pressed because I fear >> being stoned at the next PuppetConf :-D. >> >> Thanks, >> >> Trevor >> >> -- >> Trevor Vaughan >> Vice President, Onyx Point, Inc >> (410) 541-6699 >> [email protected] >> >> -- This account not approved for unencrypted proprietary information -- >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWGzUtfJW-oz62ByjPGFMye%2BCU8XbT2j3YRtoqgHRY-1A%40mail.gmail.com >> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWGzUtfJW-oz62ByjPGFMye%2BCU8XbT2j3YRtoqgHRY-1A%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CABy1mMLzyurYms0LLVqW0BiEup%3DrnMmv6FafWQom0-WQDL5%2BNg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
