First, a vote against reverting to explicit loading -- the module autoloader is *awesome* from a user perspective.
Second, to the idea of doing a better job of invalidating caches between runs, I don't think anyone (except perhaps Henrik, in the comment that I, like Luke, can't quite parse) has considered the impact of negative stat results. That is, if the parser caches an AST for a file found in the third entry in the MODULEPATH, and a new file is added in the first entry in the MODULEPATH that would have been loaded for that symbol, then the cache needs to be invalidated. Yuck. I like the idea of a single serialized AST, but it has the indicated downsides for 'puppet apply' users. I don't think implementing two resolvers -- one with current semantics and one that just de-serializes -- is a good idea unless the one with the current semantics is used to build the AST before serialization. In which case you've optimized manifest loading, but done nothing to fix the fragility that started this whole thread. Most scripting languages have similar problems. Perl and Ruby just try to open(1) a bajillion paths for every 'use' or 'require' directive. Python does something similar, but looks for .pyc/.pyo files too, which contain compiled bytecode. Python also has a mandatory mapping from filename to module name, more similar to Puppet than Perl or Ruby. In any of those cases, load order can cause funny behavior, and they don't deal cleanly with files changing while a program is running. That said, users are accustomed to the vagaries of the lazy loading these languages implement, and there are established ways to work around reloads or make loads faster. What those do *not* deal with is the need to re-read everything repeatedly in the same process. To put it another way: in the vast majority of cases, users expect a Python program to load all of its source files once, near the beginning of its execution, and never look again. But users expect their puppetmaster to parse each and every manifest file anew on every catalog request. But all of that is getting into questions of efficiency and response to changes. Neither of those were the topic of the original post. That post said that lazy loading is hard and ill-implemented, and that the proposed solution was to switch to eager loading. I would counter by saying that the lazy-loading paradigm is a good one, and the current implementation is pretty effective from the perspective of users. This is exactly the situation Python-2.x was in. The Python devs took the existing behavior, defined it carefully, and then rewrote the implementation to meet the same definition. It seems like this is the right solution for Puppet, as well. And if there are optimizations to be made in there, either invisible or user-initiated (e.g., `puppet compile`), then plan the new implementation to support those optimizations. Dustin (sorry if that seems to ramble...) -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-dev. For more options, visit https://groups.google.com/groups/opt_out.
