Hi,
I'm to understand how to get a full coverage for puppet code with spec tests to be generated so I can get output like the example from simplecov at https://github.com/colszowka/simplecov#example-output. Ideally I would subsequently try to hook into the coverage repo and do some post processing looking at where most of the code churn has been in our puppet control code repo and target those areas as important for test coverage (and likely refactoring). However I've stuck on not being able to get output from simplecov that shows the manifests that were tested by the spec tests, although the summary triggered from running 'RSpec::Puppet::Coverage.report!' suggests that something was capable of tracking coverage correctly. Although we have a different layout than normal, my testing so far suggests that the reporting simply doesn't work between puppet and simplecov. Usually when something like this happens, I assume I'm doing something blindingly stupid, so I've put together some example code that shows my problem should anyone like to run through the spec tests from it. Doing some simple experiments I've pushed up a repository showing the issue https://github.com/electrofelix/puppet-simplecov_bug containing 3 commits - (HEAD commit): convert to match more closely skeleton layout from https://github.com/garethr/puppet-module-skeleton/blob/master/skeleton/Gemfile - (HEAD~1 commit): try to follow a more standard layout - (root commit): Use simple define taken from our codebase and matching the layout we have as closely as possible Note we're still stuck on Puppet 3.8 and using ruby 2.1.10 via rvm while I was doing the spec tests, plan to upgrade to 5 in the near future, so if this problem has been solved via something in the puppet engine (some suggestions that the multiple loading by puppet was wiping SimpleCov's tracking over coverage) and we just need to hurry up that's great. But no matter what layout/format I seem to try, the simplecov report prints nothing as having been covered, e.g.: *************************************************************************************************** Total resources: 9 Touched resources: 8 Resource coverage: 88.89% Untouched resources: Ssh::Config_entry[example custom-host] Finished in 2.01 seconds (files took 1.05 seconds to load) 12 examples, 0 failures Coverage report generated for RSpec to <path>/puppet-simplecov_bug/html. 0 / 67 LOC (0.0%) covered. COVERAGE: 0.00% -- 0/67 lines in 2 files +----------+-------------------------------+-------+--------+-----------------------------------------+ | coverage | file | lines | missed | missing | +----------+-------------------------------+-------+--------+-----------------------------------------+ | 0.00% | manifests/server/localuser.pp | 67 | 67 | 5-13, 15-19, 21-34, 36-44, 46-55, 57-76 | +----------+-------------------------------+-------+--------+-----------------------------------------+ 1 file(s) with 100% coverage not shown *************************************************************************************************** RSpec::Puppet::Coverage.report causes the first part to be printed and certainly it shows it's tracking coverage on the files touched, however the simplecov report at the end does not appear to be able to correlate the tests in 'spec/defines/server_localuser_spec.rb' as having tested 'manifests/server/localuser.pp'. Additionally what is missing from both is all files that are not touched at all so far by tests, though I do have some code that appears to be able to handle that by passing 'track_files("**/*.pp")' to the SimpleCov object configuration. Thinking maybe I started from the wrong place, I took https://github.com/garethr/puppet-module-skeleton and created a fresh module from that and then dropped in both the spec file (https://github.com/electrofelix/puppet-simplecov_bug/blob/master/spec/defines/server_localuser_spec.rb) and define (https://github.com/electrofelix/puppet-simplecov_bug/blob/master/manifests/server/localuser.pp) that were being tested and the result remained the same. Rspec::Puppet::Coverage.report! gave: Total resources: 15 Touched resources: 14 Resource coverage: 93.33% Untouched resources: Ssh::Config_entry[example custom-host] But SimpleCov gave: Coverage report generated for RSpec to <path>/test-coverage/coverage. 0.0 / 0.0 LOC (100.0%) covered. So even worse than before... Resources I've looked at: https://github.com/garethr/puppet-module-skeleton https://github.com/rodjek/rspec-puppet https://github.com/rodjek/rspec-puppet#producing-coverage-reports https://github.com/rodjek/rspec-puppet/issues/316 https://github.com/puppetlabs/puppetlabs-rook (seems to have the same issue) https://github.com/colszowka/simplecov#example-output https://github.com/colszowka/simplecov/issues/682 https://github.com/colszowka/simplecov/issues/389 https://github.com/colszowka/simplecov/issues/340 https://github.com/colszowka/simplecov/issues/77 my example of the issue: https://github.com/electrofelix/puppet-simplecov_bug Hoping that there is something relatively trivial here that I'm missing that allows the reports generated by simplecov to correctly reference spec tests with puppet code. Regards, Darragh -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/3afe6401-44d9-4bcb-8628-6bc76e4b3491%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
