On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar <r...@devco.net> wrote: > hello, > > Thanks for the replies, think I'll reply to both in one go > > ----- Original Message ----- > > From: "Luke Kanies" <l...@puppetlabs.com> > > To: "puppet-dev" <puppet-dev@googlegroups.com> > > Sent: Friday, February 5, 2016 6:15:28 AM > > Subject: Re: [Puppet-dev] metaparam question > > >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg <k...@puppetlabs.com> wrote: > >> > >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar <r...@devco.net> wrote: > >>> hello, > >>> > >>> I would like to add a metaparameter - which I think is easy now via > >>> Type.newmetaparam. > >> > >> We haven't been thinking of metaparameters as a general purpose > extension point. > >> This came up once before that I know of, about a year ago, and there's > a little > >> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281. > The > >> conclusion we reached at the time was, more or less, to explore whether > the > >> desired change could be accomplished with a puppet function and/or a > change to > >> core puppet. > > > > I don't remember my original logic for metaparams not being an extension > point. > > Given the system's support for easily loading this kind of plugin, it > should be > > easy to add. > > > > The discussion on the ticket is pretty limited. We've got multiple use > cases > > here, and I have seen use cases in the past brought up, but they were > mostly on > > lists or in person, so I don't think they made it into the ticketing > system. > > > > RI - you up for a pull request to add the autoloading? Have you thought > much > > about how this might go wrong? > > Unfortunately my time is pretty limited and I'll prefer to see what I can > do > with this information in the catalog rather than making a merge worthy > patch > at present. > > I'm happy to just patch my actual puppet code with no modules to get to > that > point, I'll have a look at the validate_cmd in file to see if there is a > way > in the provider base class to do it so all providers get it. >
I kept meaning to get back to this thread and it slipped my mind. Mea culpa on that. I'm curious where you got with this approach? E.g. did you end up patching an additional metaparameter and were there any lessons worth sharing in that? > > For me this being in every provider is a nice to have I guess - it would be > great to be able to improve puppets ability to check if something worked > in the broad sense, and it's going to apply to all resource types equally. > > I could take the route that https://github.com/nanliu/puppet-transport > does > and instead of making the meta param do something use it just to store this > information and then some other module do something with this information > and > that would let me play with what I want, but it seems a small step to > improve > puppet in general. > I'm intrigued by the idea of a generic 'validate' or 'health check'. It sounds like you're thinking of it as separate from the rest of the type/provider, i.e. an external command that could be run, but I also wonder (this is half-baked) if, rather than a separate metaparameter, it could use a standard provider method to do the validation. I.e. when puppet is applying the catalog it has to essentially ask the same question that a 'validate' method would to figure out if it needs to remediate, so can the same need be met with some refinement of the generic type/provider api? (And btw, thanks for the pointer to puppet-transport. I looked at that long ago and had forgotten (or more likely didn't internalize) the use there of a do-nothing metaparameter to store connection information.) > I've told many people this story so no doubt you remember the context Luke > but what I'd like to do is allow people to attach these checks to resources > and of course use them in catalog apply stage to get a deeper sense of > 'done' > in the same way that the File validate_cmd does but later I want to consume > the catalog in other tools and extract the same - or rather hopefully speak > to some API and get this. > > Imagine I had a bridge to Sensu where it loads the catalog and just checks > everything that has a validate attached for free with no configuration > required, > this is actually really easy with sensu as it doesnt require you to declare > the checks upfront on the server and all you have to do is write JSON to a > socket for every check, it'll promiscuously take them and show them and do > alerts etc. > > Once you have this information and can surface it additionally into the > Puppet > eco system you can gather the state regularly like a monitoring system does > and it becomes quite interesting later down the line when perhaps you can > use this to influence the node assignment logic in the orchastrator so that > a Web app will be configured to talk to a known good DB server - or that > you can detect that the web app should be reconfigured once it's chosen > DB isn't good anymore. That's quite far down the line though, I think just > being able to do the monitoring part into sensu would already be a huge win > for many. > I haven't heard this story before but it's super interesting. Since you mention the orchestrator and you're thinking about monitoring/validation to gate or to assign connections between nodes, have you seen http://docs.puppetlabs.com/pe/latest/app_orchestration_availability_tests.html#using-availability-test? Availability tests as described there are just a start but they seem like they're in the neighborhood of what you're describing above. The added idea here of integrating such a thing, be it availability test or validate metaparameter, with a monitoring system could be really powerful. Wrt catalog format (or perhaps report format?) it would be interesting to explore whether we could adapt the format to facilitate such integrations. Currently both catalog format and report format are more serializations of internal data structures than they are designed as external integration points. I'd have to noodle on this some more, but it's definitely a course worth exploring. Kylo > > > > > >>> The thing that I can't seem to find any example of though is how to > >>> make this metaparameter do something on the nodes for all providers > >>> or all types. > >>> > >>> Imagine there's a metaparam that might describe how to test a resource > >>> works, something like: > >>> > >>> service{"httpd": validate => "check_http --port 80 -H localhost"} > >>> > >>> I'd then want to have some code that would be run on the agent nodes > >>> for any resource that has this param set. > >> > >> I don't think something like this exists per se, but 'validate' might > be one > >> such example of something worth adding to core puppet. Fwiw, one > >> resource-specific example added not too long ago is the file type's > >> validate_cmd: > >> > >> > https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd > . > >> > >> Also, I'm guessing you're aware but another tool in the toolbox *might* > include > >> the postrun_command setting (which is *not* per-provider as you were > thinking): > >> > >> > https://docs.puppetlabs.com/puppet/latest/reference/configuration.html#postruncommand > > > > Ryan and others on his team have discussed ways of using the new > consume/produce > > system for doing service validation. I don't think the whole system is > quite in > > place, but there are some good ideas getting close. > > > > That doesn't settle the generalized plugin capability issue, though. > > > >> More broadly, there has been some discussion on a few threads and > tickets about > >> making the agent lifecycle provide some more useful hooks, so I'd love > to hear > >> more ideas about how to improve this. > > > >> > >> Thanks! > >> Kylo > >> > >>> > >>> Does anyone have any hints or know of an example of this? > >>> > >>> thanks > >>> > >>> --- > >>> R.I.Pienaar > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups > >>> "Puppet Developers" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > an email > >>> to puppet-dev+unsubscr...@googlegroups.com. > >>> To view this discussion on the web visit > >>> > https://groups.google.com/d/msgid/puppet-dev/70254077.345165.1454514435921.JavaMail.zimbra%40devco.net > . > >>> For more options, visit https://groups.google.com/d/optout. > >> > >> > >> > >> -- > >> Kylo Ginsberg | k...@puppetlabs.com | irc: kylo | twitter: @kylog > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Puppet Developers" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email > >> to puppet-dev+unsubscr...@googlegroups.com. > >> To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/puppet-dev/CALsUZFHniTYyYU0ThP-Jk37f6QXroJ0P0-K0PkbRPXMsdB6v0Q%40mail.gmail.com > . > >> For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email > > to puppet-dev+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/puppet-dev/FB2D2883-CE00-4D61-ACC2-0B59F807A4D5%40puppetlabs.com > . > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/1273177970.271.1454659320271.JavaMail.zimbra%40devco.net > . > For more options, visit https://groups.google.com/d/optout. > -- Kylo Ginsberg | k...@puppetlabs.com | irc: kylo | twitter: @kylog -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CALsUZFFNy%3Dt1LWphrTkgiVhtTAPR1%3DZOeaMjBzf2hZh4PHPVAQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
