There's a spec in progress related to this, I'd love to see your comments in there:
https://review.openstack.org/#/c/94741/ Andrea Sent from my tiny device ---- Daryl Walleck wrote ---- I really like this option, especially if it leaves a generic hook available for validation. This could allow for different types of compute validators such as hypervisor specific or 3rd party compute validators to be implemented. From: Fei Long Wang <feil...@catalyst.net.nz<mailto:feil...@catalyst.net.nz>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Wednesday, June 25, 2014 at 5:06 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [QA] Questions about test policy for scenario test Good to know. I think it's a good idea to implement a common compute verifier after instances booted. Maybe we can define different checking levels so that it can be leveraged by different test cases. I will see what I can do. On 24/06/14 22:27, Sean Dague wrote: On 06/24/2014 01:29 AM, Fei Long Wang wrote: Greetings, We're leveraging the scenario test of Tempest to do the end-to-end functional test to make sure everything work great after upgrade, patching, etc. And We're happy to fill the gaps we found. However, I'm a little bit confused about the test policy from the scenario test perspective, especially comparing with the API test. IMHO, scenario test will cover some typical work flows of one specific service or mixed services, and it would be nice to make sure the function is really working instead of just checking the object status from OpenStack perspective. Is that correct? For example, live migration of Nova, it has been covered in API test of Tempest (see https://github.com/openstack/tempest/blob/master/tempest/api/compute/test_live_block_migration.py). But as you see, it just checks if the instance is Active or not instead of checking if the instance can be login/ssh successfully. Obviously, from an real world view, we'd like to check if it's working indeed. So the question is, should this be improved? If so, the enhanced code should be in API test, scenario test or any other places? Thanks you. The fact that computes aren't verified fully during the API testing is mostly historical. I think they should be. The run_ssh flag used to be used for this, however because of some long standing race conditions in the networking stack, that wasn't able to be turned on in upstream testing. My guess is that it's rotted now. We've had some conversations in the QA team about a compute verifier that would be run after any of the compute jobs to make sure they booted correctly, and more importantly, did a very consistent set of debug capture when they didn't. Would be great if that's something you'd like to help out with. -Sean _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Cheers & Best regards, Fei Long Wang (王飞龙) -------------------------------------------------------------------------- Senior Cloud Software Engineer Tel: +64-48032246 Email: flw...@catalyst.net.nz<mailto:flw...@catalyst.net.nz> Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington -------------------------------------------------------------------------- _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
