Hi Derek, when testing puppet-cleaner I wrote puppet-diff[1], which compiles the catalogs for two given manifests (before and after changes) and compares their YAML representation, previously removing some irrelevant stuff.
That helped me test that some transformations, like whitespace changes or single/double [un]quoting some tokens did nothing, or just what they were supposed to change. It will help you notice if something was removed or added to the catalog, but it will be difficult to test more complex changes. I've read rspec-puppet code and my first impression is that it also compiles the catalog but instead of comparing it with another one, it tests for anything you tell it; then you have to practically rewrite your manifests in rspec DSL. In any case, we're just testing the manifest on an imaginary clean box. The real result, as you know for sure, heavily depends on the current state of the box. [1] https://github.com/santana/puppet-cleaner/blob/master/bin/puppet-diff El viernes, 26 de abril de 2013 12:03:47 UTC-5, Derek Olsen escribió: > > > We are in the process of evaluating our puppet related test and > release process and interested in knowing what other folks are doing. > > We are in a position that is not ideal but is not unique from what I > can tell. Our current testing process is basically the > responsibility of each person making a change. Small changes are > committed and pushed to dev/qa/prod in one swoop with the committer > spot checking the results manually. Larger changes are tested by > running a node against a puppet environment which is pointed to the > change branch and the desired behavior is manually verified. > > What we would like to do is start with implementing some basic control > points which require passing tests before the changes move along. > With the goal of being able to increase the test coverage over time to > protect ourselves from ourselves. > > One thought we had as an initial step is to just verify catalog > compilation for some number of nodes against the proposed changes and > block the changes if catalog compilation fails. This raises the next > question around tooling. We could script up a catalog compiler test > calling the the puppet binaries but should we use this as an > opportunity to get familiar with rspec-puppet? > > Are people using catalog diffs at all in their release process? It > would seem nice to provide an automated catalog diff for people making > 'small' changes so they can make sure their change didn't accidentally > drop or change a large number of resources. > > So please share what you find works or doesn't work at your shop. > > TIA> > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
