On Wed, Aug 27, 2014 at 8:10 AM, jcbollinger <john.bollin...@stjude.org> wrote:
> > > On Tuesday, August 26, 2014 6:24:57 PM UTC-5, Nigel Kersten wrote: >> >> >> >> >> On Tue, Aug 26, 2014 at 12:27 PM, Erik Dalén <erik.gus...@gmail.com> >> wrote: >> >> >>> >>> >>> On 26 August 2014 20:22, jcbollinger <john.bo...@stjude.org> wrote: >>> >>>> >>>> >>>> On Monday, August 25, 2014 11:13:40 AM UTC-5, Matt W wrote: >>>> >>>>> Comments inline >>>>> >>>>> Matt Wise >>>>> Sr. Systems Architect >>>>> Nextdoor.com >>>>> >>>>> >>>>> On Mon, Aug 25, 2014 at 6:55 AM, jcbollinger <john.bo...@stjude.org> >>>>> wrote: >>>>> >>>>>> >>>>>> > >> >>>> >>>>> Can you say a bit more about that? What do you see that suggests >>>>>> agents are pulling down "node information" other than their catalogs (and >>>>>> later, any 'source'd files)? >>>>>> >>>>>> >>>>> With nearly every puppet catalog compile, we also see GET requests >>>>> like this: >>>>> >>>>> 10.216.61.76 - XXX - puppet "GET /production/node/xyz? HTTP/1.1" 200 >>>>>> 13733 "-" "-" 0.021 >>>>> >>>>> >>>>> Where 10.216.61.76 is *not* the local IP of the puppet master... its >>>>> the remote IP of the ELB, which indicates that its remote traffic from our >>>>> puppet clients. >>>>> >>>>> >>>> >>>> That traffic might be coming from nodes, but all you know for sure is >>>> that it is traversing the ELB. Surely the master could send requests >>>> through the ELB that end up coming back to it. For all I know, the ELB >>>> might preferentially route such requests back to the originating host. >>>> >>>> From the perspective of the Puppet service lifecycle, the two most >>>> likely sources of such traffic are (1) an ENC retrieving node facts, and >>>> (2) the master determining nodes' environments. I don't know any reason >>>> why nodes would be requesting their own node information, and even if they >>>> did, I can't see how that would affect the catalog the master serves to >>>> them. >>>> >>> >>> The reason for them to do this is to be able to use the environment that >>> was configured on the master to fetch the plugins from. So first it tries >>> to fetch its node info from the master to see if the environment in that is >>> different than what it had configured locally. This is a new feature since >>> puppet 3.0. >>> >>> In puppet 2.7 it used the fact plugins from the agent configured >>> environment and the catalog from the master configured if I remember >>> correctly. >>> >> >> Yes, and also file resources came from the agent-configured environment >> even though the resource was from the master-configured environment, which >> resulted in much hilarity. >> >> >> > > I am well aware of all the old hilarity surrounding determining the > environment from which to serve various bits, but I was unaware that the > resolution involved agents requesting their environment from the master. > That implies that the master *still* relies on the agent to correctly > specify (echo back) the environment from which to serve those bits, else > why would the agent need to know? > > If that's really what's happening then it's a poor design (which I guess > is why I supposed it *wasn't* what was happening). If the master is > authoritative for a piece of information -- as it is for nodes' > environments -- then it should not rely on relaying that information back > to itself through an external actor -- that undermines its > authoritativeness for the information. Moreover, to the extent that the > master does have such a reliance, it leaves Puppet open to malicious > manipulation of the requested environment. > > So, um, are you sure? > Yes. The bit of info we haven't mentioned is that if the client and server environments don't match, and the server is set to be authoritative, then it triggers the client to do a new pluginsync and run with the server environment. Tracking back to older tickets, there's a succinct description here from Daniel Pittman: http://projects.puppetlabs.com/issues/16753 (which has related tickets for the rest of the change) "The reason this was removed was to support the changes that made the ENC authoritative over the agent environment. As part of that we had a bootstrapping problem: the agent had an idea of the environment to request, used that in pluginsync, and then as part of the request for the catalog. If that idea was wrong, the catalog would be returned from the correct, ENC specified environment, but it would have been generated with the wrong set of plugins – including custom facts. So, the agent would detect that, pluginsync to the new environment in the catalog, and compile a new catalog. That fixed the problem, but was inefficient – every agent run with an incorrect environment would mean two catalog compilations, and doubling master load in a common situation (ENC says !production, agent run from cron) was pretty unacceptable. So, instead, the agent was changed to query the master for node data about itself – and to use the environment that came back from that." -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAGUqgV8n_mon%2BzmoHJRPF0GptUvFcHNusoBuay88u9XLUGvLXA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
