Re: [Puppet Users] Puppet "autoconfigured" clients on AWS - classes from EC2 tags/userdata

2016-05-27 Thread Martin Langhoff
ask interesting questions ~ http://linkedin.com/in/martinlanghoff - don't be distracted~ http://github.com/martin-langhoff by shiny stuff -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this grou

[Puppet Users] Puppet "autoconfigured" clients on AWS - classes from EC2 tags/userdata

2016-05-26 Thread Martin Langhoff
Hi Puppeteers, folks are mapping "role" from EC2 tags or userdata into a `case` statement in site.pp to choose a pre-built configuration. I wonder whether there is a way to bring in a listing of classes, as can be done with hiera (`hiera_include("classes")`). thoughts? m -- martin.langh...@

Re: [Puppet Users] Script to track orphaned resources

2014-08-22 Thread Martin Langhoff
For context... Our puppet setup is complex, with many behaviors controlled by facter facts, in part controlled by a .INI file that support personnel can edit. We manage thousands of VMs. So unit tests are interesting but offer very limited coverage. Tracking orphans on live nodes is much more comp

[Puppet Users] Tracking orphaned resources

2014-05-29 Thread Martin Langhoff
As the complexity of our $workplace puppet configuration grows, I am increasingly worried that puppet gives us very limited visibility over resources it no longer manages. In practical terms: if I mess up my class include/require/inherit structure so that a node A no longer indirectly includes mod

Re: [Puppet Users] Assign group membership in a module or conditionally?

2014-03-31 Thread Martin Langhoff
On Mon, Mar 31, 2014 at 10:09 AM, Martin Langhoff wrote: > This is really awkward for what I see as a "natural" operation. Am I doing > something wrong in my setup? > > And also... this is funny, but I discovered a change including this syntax > had already been rolled

Re: [Puppet Users] Assign group membership in a module or conditionally?

2014-03-31 Thread Martin Langhoff
On Friday, March 28, 2014 3:48:47 PM UTC-4, jcbollinger wrote: > Puppet DSL provides no mechanism, however, for selecting resources via a > search expression without realizing all virtual resources among those > selected. > This is really awkward for what I see as a "natural" operation. Am I do

Re: [Puppet Users] Assign group membership in a module or conditionally?

2014-03-28 Thread Martin Langhoff
On Friday, March 28, 2014 3:08:00 PM UTC-4, Jose Luis Ledesma wrote: > Realize some of them > Right. That does not address my concern, which is the implicit realize. The rule for which users we realize on our infra are orthogonal to the modules that define functionality; and need to remain so...

Re: [Puppet Users] Assign group membership in a module or conditionally?

2014-03-28 Thread Martin Langhoff
On Friday, March 28, 2014 2:30:31 PM UTC-4, Jose Luis Ledesma wrote: > I would really like to know what other people may suggest, the only thing > that comes to my mind is to make use of tags for your users. > How would you do this with tags _without_ realizing virtual users? m -- You recei

[Puppet Users] Assign group membership in a module or conditionally?

2014-03-28 Thread Martin Langhoff
Scenario - we have modules: - our_users: defines all the (virtual) users (sysadmins, support, developers) as members of wheel - our_base: gets applied to all nodes - our_webserver: defines the apache group What I want to achieve is: on nodes that use our_webserver (where the apache group exists

[Puppet Users] filebucket --local should have default bucket matching "apply"

2014-01-14 Thread Martin Langhoff
Background I am using puppet via direct invokation of puppet apply path/to/manifest.pp -- no puppet server involved. Puppet is 3.1.1 on RHEL6.x and CentOS6.x boxes. I needed to study a file change, coming from a concat. Looking at the report, I had the hash. This did not work: # puppet filebucket

[Puppet Users] DR hot/warm spare node patterns / anti-patterns?

2013-12-10 Thread Martin Langhoff
With our without puppet. We are looking at setting up DR nodes with some draft ideas. We use puppet against a git repository (with ppg) instead of against a puppet server. Nodes are actually VMs. Configuration ideas: - a filesystem-level flag that indicates that the node is running as standby sp

Re: [Puppet Users] Actual diffs in puppetdb?

2013-11-19 Thread Martin Langhoff
On Tue, Nov 19, 2013 at 12:25 PM, Bruce wrote: > Maybe. But usually I don't know I want this information, until I need it. > So having to turn on some debugging ahead of time doesn't help. I am using etckeeper in conjunction with Puppet for exactly this use case. Additionally, my puppet configur

Re: [Puppet Users] Re: File() or Exec() temporary override?

2013-11-06 Thread Martin Langhoff
On Wed, Nov 6, 2013 at 9:55 AM, jcbollinger wrote: > . If that's not viable, then something close to the idea you proposed > should be possible: By "should be possible, do you mean that you know or think that Puppet supports it? > node 'fqdn' { # I work for RL :-) > include rl_users > inclu

Re: [Puppet Users] Re: Dashboard running in masterless puppet

2013-11-05 Thread Martin Langhoff
Hi Dan, there's quite a bit. Google for 'masterless puppet howto' to find a couple tutorials and discussions that are popular. Add 'git' to the search string for more quality options... The main benefits are - scalability - pull model -- see http://www.infrastructures.org/bootstrap/pushpull.

Re: [Puppet Users] Dashboard running in masterless puppet

2013-11-05 Thread Martin Langhoff
Hi Gonzalo, > I'm running Puppet in masterless mode and trying to make Puppet dashboard > play nice with it in this non-standard setup. some of the tricks that are useful in this kind of scenario are encapsulated in puppet-git / ppg, which I've written, and use at Remote Learner, where it is grad

[Puppet Users] File() or Exec() temporary override?

2013-11-05 Thread Martin Langhoff
Hi Puppeteers, When you have complex/rich classes, and large numbers of machines/VMs, sometimes there is a machine that needs a temporary override on a file. Is there a way to say something like... ? node 'fqdn' { # I work for RL :-) include rl_users include rl_base include r

Re: [Puppet Users] Best practices for infrastructure

2013-08-30 Thread Martin Langhoff
On Thu, Aug 29, 2013 at 9:02 AM, jcbollinger wrote: > The master will always choose the node block to use based on the client's > SSL certname (spelled $::clientcert in Puppet DSL). Oh, that is considerably safer than what I feared. Thanks for the clarification. My comments earlier in this threa

Re: [Puppet Users] Best practices for infrastructure

2013-08-28 Thread Martin Langhoff
On Wed, Aug 28, 2013 at 9:31 AM, jcbollinger wrote: > If the objective is to render it into a small number of words, Just to double-check my understanding is right. If the client-reported $::hostname does not match the certname, _and_ I am only using 'node "fqdn"' entries in my Puppet manifests,

Re: [Puppet Users] Best practices for infrastructure

2013-08-27 Thread Martin Langhoff
On Tue, Aug 27, 2013 at 2:41 PM, jcbollinger wrote: > The client can provide a $::hostname fact that is different from the > certname it presents, but that is perfectly valid and expected under some > circumstances. It is possible that a client doing so is thereby able to > exploit weaknesses in

Re: [Puppet Users] Best practices for infrastructure

2013-08-25 Thread Martin Langhoff
On Sat, Aug 24, 2013 at 5:18 PM, Jakov Sosic wrote: > Only if you use autosign option. After the certificate is signed, agents > report certname and not hostname. Well-behaved clients report certname. A malicious client could use one cert, but report a different name. AIUI the puppet master check

Re: [Puppet Users] Best practices for infrastructure

2013-08-24 Thread Martin Langhoff
On Sat, Aug 24, 2013 at 6:33 AM, Félix Barbeira wrote: > Speaking in security terms, could be masterless puppet configuration less > secure? I mean, the puppet code is in *all* the clients. On the other hand, > the puppet code is only in the master, which I think is more secure (you > can isolate

Re: [Puppet Users] Best practices for infrastructure

2013-08-23 Thread Martin Langhoff
On Fri, Aug 23, 2013 at 12:03 PM, Paul Archer wrote: > I'm thinking about setting up a master in the colo with a slaved master at > each site, > I would strongly recommend using "master-less" recipes, which are actually "a git repository as a master, and cronjobs running puppet apply as client".

Re: [Puppet Users] any elegant way to iterate/map over data types?

2013-06-11 Thread Martin Langhoff
On Tue, Jun 11, 2013 at 7:36 AM, Matthias Saou wrote: > When it comes to iterating with puppet, the usual way to get where you > want is to apply a definition to an array. From there, you need to avoid > the (also usual) duplicate declarations, by extending and abusing the > $title if needed in or

[Puppet Users] any elegant way to iterate/map over data types?

2013-06-10 Thread Martin Langhoff
Sysadmins have the (reasonable?) expectation of installing more than one ssh key. Relevant bits from my current config follows: class rl_users { define ssh_user($uid, $gid, $password, $akey, $ensure=present) { user{ $name : ensure => $ensure, mana

[Puppet Users] puppet apply -- override node name, module path

2013-05-24 Thread Martin Langhoff
Hi folks, testing puppet configs, I have /home/martin/mytestingpuppetconfigs/{manifests,modules} and while working in there, I would like to be able to say something along the lines of: puppet apply --noop --nodename=foo01 --modulepath=./modules manifests/site.pp which should tell me what p

Re: [Puppet Users] Very slow puppet agent runs on empty/noop manifests (v3.2.1)

2013-05-23 Thread Martin Langhoff
On Thu, May 23, 2013 at 12:25 PM, Keith Burdis wrote: > Are you running into the Socket.gethostbyname(Socket.gethostname) issue > pointed out by Wil Cooley a few days ago? > > https://groups.google.com/forum/?fromgroups#!topic/puppet-dev/z09Nkk18tRE > > If so there is potential /etc/hosts change

[Puppet Users] Very slow puppet agent runs on empty/noop manifests (v3.2.1)

2013-05-23 Thread Martin Langhoff
This is a VM with 1GB RAM and almost nothing running on it. It takes 10s to read a manifest that defines one node, one class, and checks whether puppet package is installed... Where is all the time going? Is something wrong? Details - # cat /etc/redhat-release CentOS release 6.4 (Final) #

Re: [Puppet Users] Puppet, git & security

2013-05-16 Thread Martin Langhoff
On Wed, May 15, 2013 at 2:44 AM, Stephen Gran wrote: > Your push server can run git update and then rsync to the masters. Why rsync if you have git? You have - the machine(s) where you edit and make commits on git, you then _git push_ to what I'll call a "gold" git server - the gold git serv

Re: [Puppet Users] Re: High Availability of Puppet server for separate geographical location

2013-05-14 Thread Martin Langhoff
On Tue, May 14, 2013 at 7:35 AM, Erik Dalén wrote: > We are using SRV records for running multiple puppetmasters and selecting a > site local but allowing fallback to others in case it is down. > We have 6 puppetmasters for the production environment running in this way > currently. Each normally

Re: [Puppet Users] Re: High Availability of Puppet server for separate geographical location

2013-05-10 Thread Martin Langhoff
On Fri, May 10, 2013 at 1:52 PM, Ramin K wrote: > reasonably resilient or at least able to localize failure. Certainly some > designs and technology are better than others, but implementation always > matters. Of course. I think we're saying the same thing, at the end of the day. > Mast

Re: [Puppet Users] Re: High Availability of Puppet server for separate geographical location

2013-05-09 Thread Martin Langhoff
On Thu, May 9, 2013 at 2:31 PM, Ramin K wrote: > Hubris, today thy name is Martin. :-) Fair enough. I am happy about the tool I am writing (almost finished!) but, as the followup post makes clear, it isn't about the designe of ppg. It is about the design of git. > I'd argue that people have stre

Re: [Puppet Users] Re: High Availability of Puppet server for separate geographical location

2013-05-09 Thread Martin Langhoff
On Thu, May 9, 2013 at 10:42 AM, Martin Langhoff wrote: > I am writing some tooling for git+puppet (search for ppg in recent > posts to this list), and it's trivial to add N-tiers of redundant > servers... Heh, so trivial in fact that you can use round-robin DNS and it'll just

Re: [Puppet Users] Re: High Availability of Puppet server for separate geographical location

2013-05-09 Thread Martin Langhoff
On Thu, May 9, 2013 at 12:06 AM, John Warburton wrote: > I suppose all HA solutions are difficult Nah. A service correctly designed to be resilient can be HA with trivial investment. DNS is a good example. It may have blemishes but nobody stresses about its availability. Setup as many tiers of r

Re: [Puppet Users] Re: Practices: what _not_ to manage with Puppet?

2013-05-06 Thread Martin Langhoff
On Mon, May 6, 2013 at 8:49 AM, Bernardo Costa wrote: > Well, I'll tell you that for now some kind of configuration is difficult to > be done with puppet. At least I couldn't find a way to do it. Ex: > controlling a /etc/passwd file but partially with a libnss compat syntax. > This means entries o

Re: [Puppet Users] Re: Practices: what _not_ to manage with Puppet?

2013-05-06 Thread Martin Langhoff
On Sun, May 5, 2013 at 2:11 PM, Larry Fast wrote: > What about the larger processes involved in incremental updates? Eg. > sequencing your updates so that the service keeps running. I'm considering > using Jenkins to orchestrate sequencial activity. Coming from an ISConf background, I'd do it

Re: [Puppet Users] Re: Practices: what _not_ to manage with Puppet?

2013-05-04 Thread Martin Langhoff
On Fri, May 3, 2013 at 4:43 PM, Schofield wrote: > Everything else is managed by puppet. Do you manage complex network setups (bonding, routing) via puppet? There is a certain degree of chicken-and-egg in that; how do you handle managing configuration without breaking the network that delivers th

Re: [Puppet Users] Practices: what _not_ to manage with Puppet?

2013-05-04 Thread Martin Langhoff
On Sat, May 4, 2013 at 7:33 AM, Erik Dalén wrote: > I'd definitely configure the servers and > clients for those protocols using Puppet. Is that what you do, or what you _would_ do? ;-) > Or using DHCP to configure networking instead of having puppet setting it > statically on your hosts. Do yo

[Puppet Users] Re: ppg: Scheduled rollouts and dashboard with git in decentralized setup

2013-05-03 Thread Martin Langhoff
On Tue, Apr 23, 2013 at 5:22 PM, Martin Langhoff wrote: > We will have a wrapper, "ppg" for puppetgit -- and avoiding confusion > with PostgreSQL tools. After some delays in getting started... http://repo.or.cz/w/puppet-git.git/ Still a work in progress, but if I can get two

[Puppet Users] Practices: what _not_ to manage with Puppet?

2013-05-03 Thread Martin Langhoff
While I prep my scripts and tool up for a large infra, I want to revisit a question that I ask myself regularly: what do people not manage with Puppet (or wish they weren't)? In my situation (a RH-style world), initial base system install, inc disk layout and initial networking is handled with kic

Re: [Puppet Users] High Availability of Puppet server for separate geographical location

2013-05-03 Thread Martin Langhoff
On Mon, Apr 29, 2013 at 9:55 AM, shyam sundar Keshari wrote: > I have to configure puppet server in Primary-Secondary mode for 2 > distributed location . I am in a similar situation. Not liking the options available, I am building "puppetgit" https://groups.google.com/forum/?fromgroups#!topic/pup

Re: [Puppet Users] ppg: Scheduled rollouts and dashboard with git in decentralized setup

2013-05-02 Thread Martin Langhoff
On Mon, Apr 29, 2013 at 6:19 PM, Felix Frank wrote: > Interesting. It seems nicely thought out, but I stumbled here, reading: thanks for reading! > On 04/23/2013 11:22 PM, Martin Langhoff wrote: >> I am less certain of this part, and input will be specially valuable here. >>

[Puppet Users] ppg: Scheduled rollouts and dashboard with git in decentralized setup

2013-04-23 Thread Martin Langhoff
For a "server-less" puppet setup using git for config distribution, I am drafting out some scaffolding... Some background in the message I just posted: https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/A5Ywi1V1OrA Plan is to have two branches in git: master and production. Commits

[Puppet Users] git-based workflow with local proxy, dashboard and scheduled/embargoed rollouts

2013-04-23 Thread Martin Langhoff
Hi Puppeteers, I am working with a team to manage a large, complex infra covering several thousand VMs plus specialized hardware boxes in several data centers. I have some backstory with Puppet (and isconf2, and infrastructures.org) and I authored several tools in the git-swiss-army-chainsaw. Our

Re: [Puppet Users] Do people walk the filebucket tree searching by path?

2011-02-17 Thread Martin Langhoff
On Thu, Feb 17, 2011 at 3:55 PM, Nigel Kersten wrote: > ok. So it's unacceptable for you to refer to logs or reports to get > the checksum for a given replacement and then restore the file that It's really damn fiddly :-) As a git guts hacker, I appreciate that puppet stores things in a content

Re: [Puppet Users] Re: best practice for removing a module & maintained resources from a system ?

2010-10-21 Thread Martin Langhoff
On Thu, Oct 21, 2010 at 4:56 AM, Felix Frank wrote: > I guess what you're getting at is this: No, puppet is not exactly good > at "uninstall this now and from then on, don't care about it anymore". > This is not what puppet has been conceived for, though. OK - but putting (config) files into plac

Re: [Puppet Users] Re: best practice for removing a module & maintained resources from a system ?

2010-10-20 Thread Martin Langhoff
On Wed, Oct 20, 2010 at 3:00 PM, Mohit Chawla wrote: >> Except that some definitions may be gone. That's what worries me. Sure >> I can read the pp files as they are today. > > That's probably true for any tool or method. Unless it was in version > control. Not true of packages under any modern p

Re: [Puppet Users] Re: best practice for removing a module & maintained resources from a system ?

2010-10-20 Thread Martin Langhoff
On Wed, Oct 20, 2010 at 12:03 PM, Patrick wrote: > I wouldn't call it unreasonable.  I would call it "lack of a really cool > feature". Pretty fundamental feature :-) I am not saying Puppet needs to magically know what I'd like to happen with the file. However, it should at least give us enough

Re: [Puppet Users] Re: best practice for removing a module & maintained resources from a system ?

2010-10-20 Thread Martin Langhoff
On Mon, Oct 18, 2010 at 9:13 AM, jcbollinger wrote: > I'm guessing you mean you have written sub-*classes* to do that job. > That is indeed the Puppet way to do it, and I don't find it at all > ridiculous. As a puppet newcomer, that is a bit surprising, and IMO unreasonable. Imagine you are join

Re: [Puppet Users] Re: Using puppet to update itself

2010-10-08 Thread Martin Langhoff
On Fri, Oct 8, 2010 at 4:56 PM, Forrie wrote: > That's more of what we're looking to do.  I think it would be a bad > idea to have puppet automatically updating clients.  This would need > to be a one-off, scheduled item you would trigger from the puppet > master server, under the default {} node,

Re: [Puppet Users] Using puppet to update itself

2010-10-08 Thread Martin Langhoff
On Fri, Oct 8, 2010 at 4:18 PM, Disconnect wrote: > We just use packages: >  package { "puppet": ensure => latest } If the rpm/deb script attempts to restart the service, that will stop puppet in the middle of the execution of the pkg manager itself. (Unless there's special handling of this case

Re: [Puppet Users] Re: Puppet use with OLPC's XS

2010-09-10 Thread Martin Langhoff
On Fri, Sep 10, 2010 at 9:56 AM, Luke Kanies wrote: > It looks like the list has you pretty well handled, so mostly I'm writing > back to say I'm glad to see ISconf users still around, and very glad to see > it looks like Puppet will work for you. Indeed, very well treated. Even my oddest quest

Re: [Puppet Users] Re: Puppet use with OLPC's XS

2010-09-08 Thread Martin Langhoff
On Wed, Sep 8, 2010 at 11:45 AM, alcy wrote: >> 2a - Is there an ISConf-like facility that says "run it until it >> succeeds once"? [ Happy to use Makefiles, but if there's a >> Puppet-supported elegant way of doing it... ] > > If the code of the shell script can be minimized to use as much as of

[Puppet Users] Puppet use with OLPC's XS

2010-09-08 Thread Martin Langhoff
Hi Puppet Users, Hi Luke, Happy to find myself here. I am an old ISConf user -- Luke might remember my random questions back in the infrastructures / isconf list -- and have been developing the School Server ("XS") for OLPC for a while (and lately doing a dozen more things at OLPC which has shrunk