On 21 October 2015 at 23:28, Gareth Rushgrove <gar...@morethanseven.net> wrote:
> On 20 October 2015 at 08:49, Alex Harvey <alexharv...@gmail.com> wrote: > > Hi all, > > > > I am investigating whether or not I can use Beaker to do acceptance > testing > > on roles and profiles. > > > > I've had a look at Liam Bennett's excellent blog posts - > > > http://tech.opentable.co.uk/blog/2014/09/01/testing-puppet-with-beaker-pt-dot-3-testing-roles/ > > http://www.slideshare.net/liamjbennett/cfgmgmt2015-testing-with-beaker > > > > I need to handle a situation in my tests where, say, a role that I am > > testing will apply a base class which will cause the node, for instance, > to > > join a FreeIPA domain. But I don't want Beaker to actually build a > FreeIPA > > box. And I don't want my short-lived node to join a real FreeIPA domain. > > > > I would hope that Beaker could either build Mock Services > > https://en.wikipedia.org/wiki/Mock_object > > > > Or better still, tell Beaker to expect the base class to try to apply the > > FreeIPA class, and just pretend it succeeds. Just as you can stub out > > methods in rspec etc. > > > > Has anyone done anything like this before? > > > > You can actually do something like that :) > > Beaker is going to run you Puppet code on the node(s) it spins up, > probably using apply manifest like so: > > apply_manifest(pp, :catch_failures=>true) > > If this sees an error while doing so (because your FreeIP class > doesn't apply) then it will throw an error. However... > > If you check the API docs you can see: > > > http://www.rubydoc.info/github/puppetlabs/beaker/Beaker/DSL/Helpers/PuppetHelpers#apply_manifest_on-instance_method > > apply_manifest(pp, :accept_all_exit_codes => true) > > This will run your puppet code and not throw an error. > > Now, whether this works for you really depends on if any other classes > depend on the FreeIPA class. And you can't easily test for idempotent > runs easily with this. > > Another approach if you can change the puppet code is to introduce a > switch in the profile, for instance if a fact exists like MOCK_FREEIPA > you could not include the class. And then drop that fact into your > beaker test environment. > > Also see the other points about testing at different levels but you > can absolutely do it in your acceptance tests. > Thanks Gareth. Both of these solutions are a bit messy but it is good to know of all the options. One of the big issues I would encounter immediately I used :accept_all_exit_codes is that the FreeIPA code takes several minutes to time out if the server can't be contacted. Or if I moved all this logic associated with testing into the code, that would work, but it is not really where test logic belongs, and it would rather clutter my base classes. I feel that there is some space here for a new approach to acceptance testing. I am not sure that I agree that acceptance tests should be end-to-end tests. Consider: - spinning up a FreeIPA system - not to mention all the other external services that a base class might depend upon - takes a long time. The longer tests take to run, the less likely people will actually run them. - not all developers will have workstations that can run such resource-intensive tests. - I actually don't want to test that every single role can join the FreeIPA domain - I only want to test that once, and preferably in tests associated with my FreeIPA modules and classes. Another approach might be to could create a long-lived acceptance test environment that only contains a DNS, FreeIPA, etc, however: - this would introduce a management burden - sysadmins needed to keep them running, clean up junk related to broken tests etc. - It would also introduce a cost - we would have to pay Amazon to always have these instances running. I actually think deploying into a real UAT environment is the only real end-to-end test of the code that matters before it goes to production. Now, if I am testing a web server role, rspec-puppet allows me to do very useful testing on catalogs - but if I am testing a front end web server role, I only want to know that it builds a properly configured Apache. I suppose, therefore, that I could just test the front end web server profile; although it would be good to also test as much of the integration as is practical. What if Beaker was extended to provide a framework for provisioning fakes - if it accepted plugins somehow for a mocked up FreeIPA service etc. Would that be unrealistically complicated? In the development world, developers must be mocking up these sorts of fakes all the time. How do they do it - do they just continually reinvent the wheel or are there tools for this? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAF0Ep4U-sSEkYS2v_5e%2Be2SwZo8ZgGb86_iSoo%3D6UyNPKFrX%2Bg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
