Hello all, At the moment we have a lot of Marvin tests that follow a pattern that looks roughly like this:
1. Setup some resources (e.g. accounts, service offerings, VMs etc) 2. Add the resources to a list in the testcase (often called self.cleanup) 3. Do the test(s) 4. Call cleanup_resources with the list of resources from 2 (obviously in some cases resources get created/allocated during the actual test rather than in setup, but it's a similar principle) In theory this is fine, however there are a number of cases where resources are being created and then not added to the cleanup list, which results in things being 'left behind', potentially using up resources which may then affect future tests. For example I'm currently attempting to run various tests in parallel (to speed up execution), and I'm hitting some issues I believe to be caused by this. The thought that occurs to me here is do we actually need the testcase to have to manually add resources to a list to cleanup, with the inherent risk of resources getting missed etc - could we not make this something the framework does for us (at least by default, with the option to override the behaviour if needed). I've got some ideas as to how this could be done (one example that's a bit of a layer violation but might be acceptable would be to wrap/extend the apiClient to have a method that can be called on it from the various object create methods to add the resulting object for cleanup), but before I go ahead and start trying to prototype something I wanted to see if anybody had any reasons why this sort of automatic cleanup behaviour might be a bad idea or has investigated anything similar in the past? Cheers, Alex
