Hi Simon,
On May 14, 2:55 am, Simon J Mudd <sjm...@pobox.com> wrote:
> Hi John,
>
> While you obviously disagree with me, thanks for taking the time to
> reply. I may be looking at the problem the wrong way which is why
> I'm trying to figure out if that's the case and why.
Fair enough.
> john.bollin...@stjude.org (jcbollinger) writes:
> > On May 13, 8:07 am, jcbollinger <john.bollin...@stjude.org> wrote:
> > On the other hand, extlookup() is easy to set up and use, and it is
> > powerful enough for most needs. If you need a more powerful data
> > lookup function then you would be lucky to find a canned one that
> > meets all your needs.
>
> Perhaps but I don't see it as being that bad, and it seems to be
> the closest thing to a solution I could find at the moment. My current
> recipes (perhaps incorrectly) are heavily customised in a way
> which often is not just "host specific" or "domain" specific. Hence
> the need to override the default settings. Other settings though
> may need this thus requiring this to be more dynamic.
Let's not cast things in terms of "correctness," except insomuch as
whether they reliably produce the desired effect on clients.
Depending on what you're trying to do with Puppet, however, it is
certainly true that some approaches to structuring your manifests will
make them easier to write and maintain than will others. Personally,
when writing specializations into my manifests, I find it useful to
keep in mind the question of *why* a particular host or group of hosts
is different from others. I have yet to run into an answer that
didn't fall into one of these categories, or a combination of them:
1) It has a functional role that some other systems do not
2) Its network environment requires differences from some other
systems.
3) It's just special
It has worked well for me so far to model roles via (unparameterized)
classes, assigned to nodes via their node declarations. That leaves
only one level between "common" and "node-specific" where I might need
to customize data. (That intermediate level could in principal be
parameterized by network domain, but in my case it is by subnet.)
Most of my nodes do not require per-node customization, so I don't end
up with many data files for extlookup.
> > Moreover, I disagree with several of the opinions and conclusions in
> > your post:
>
> > 1) You write: "The extlookup() functionality only allows [...
> > specifying implicitly ...] where to look for this value." That is
> > false. Extlookup does provide for configuration of a standard set of
> > CSV files to search (which can be parameterized by nodes' facts), but
> > the function also has an optional parameter to specify a specific file
> > to be searched first on any given invocation.
>
> Perhaps coming from a database background, I'd like to mirror what
> seems more _natural_ and having values spread around over potentially
> a large number of files seems non-intuitive.
If extlookup use would indeed require you to maintain a large number
of separate files then that might be a good reason to find something
better, but in all likelihood you can avoid that, or else structure it
in a sane way. Consider also:
When you work with a database, do you normally focus on how it maps
data to the host filesystem?
Given the diverse data parameterization you described, if you created
a database for your configuration data, would you really organize
everything into a single table? (And what would be its key?)
If you wanted to allow customization of your data with varying scopes
(e.g. supporting both per-node and per-network customization of the
same parameters) then how would you design the DB?
> > 2) You would prefer looking up data via a compound key (config_item,
> > lookup_value) rather than via a simple key (config_item).
> > [...] And even a DB performs multi-key queries
> > slower than single-key searches.
>
> Not if they are part of the primary key. That's part of the point.
Well that's a possible answer for you then. Extlookup performs
lookups based on a single key, and nothing prevents you from using
keys that allow you to flatten your data. For example, you can
structure your data so that instead of extlookup("interface"), you can
extlookup("interface-$subnet-$is_master"). In other words, the
distinction in your examples between "config_item" and
"lookup_value(s)" is artificial and unnecessary. There is no reason
why you could not combine them all into a single field in the CSV.
> > 3) You argue that your suggested formulation of extlookup would be
> > "clearer as the configuration is more explicit then the current
> > extlookup() definition." I think you're missing the point. It would
> > indeed be clearer from which file the data would come, but the
> > objective of extlookup is to separate the *definition* of the data as
> > much as possible from the *use* of the data. And I like that. I
> > prefer that my manifests _not_ specify a bunch of details to every
> > extlookup() call, because that would obscure the details that are
> > important at the manifest level.
>
> ok. This perhaps is a different point of view we have. extlookup()
> looks like a function. As such it's nice if the function given the same
> input provides the same output. Otherwise confusion is easy.
Personally, I don't find it at all surprising when the output of a
function depends on factors other than its arguments, as long as the
dependencies are well documented. That's common in the programming
and database worlds. YMMV, of course.
There are things you could do with extlookup() that would contribute
to confusion, such as overriding the lookup precedence variable below
global scope. You could also write put 'exec { "cd /$mydir; rm -f
*": }' into one of your manifests. Avoid these things (even if
you're *sure* $mydir will always be defined).
> As written it's clear if you do extlookup('dns_server') that you are
> getting a "dns server" result. In many of the examples provided
> the expected use case is: check first for the host, then the domain,
> then ....., but that's not specific, _and_ may be important.
When you say it's not "specific," do you mean that you can't tell from
the extlookup() call itself? The typical use of extlookup() involves
defining the lookup precedence once, globally. For the exceptional
case where you need to override the lookup order, you specify the
first-resort lookup file explicitly in the function call. If you do
that, then you always know how the lookup will proceed. To the extent
that your concern may be premised on the notion that you'll override
the lookup precedence via its controlling variable, I suggest you
construe your point as a good reason to not do that.
> > 4) You seem to want to use an arbitrary-length list of values as your
> > lookup key, but I don't see how that can work while retaining the
> > apparent objective of reducing the number of data files that must be
> > associated with your configuration and retaining the use of CSV.
>
> The .csv files are currently lookup_key,lookup_value. AFAIK there is
> no limit on either parameter's size. I'm only suggesting a third
> grouping/column item_type, and that is unlikely to change the width
> of the .csv files much. My suggestion is the structure would be:
>
> dns_server,host1.example.com,ns1.example.com
> dns_server,host2.example.com,ns1.example.com
> dns_server,all_domains,ns1.maindomain.com
Ah. I thought you were suggesting a variable number of additional
columns between lookup_key and lookup_value. Nevertheless, and
contrary to my own (4), I just offered a way to approach the problem
that works with extlookup() as currently written. That is, put all
key information into the lookup_key field, delimited by some character
other than a comma. Not only will that allow you to flatten the file
to an arbitrary degree, it will also allow (force) you to explicitly
include all the key parts in your extlookup() calls.
Cheers,
John
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
