On Thu, 13 Mar 2014 06:40:58 +0000 Kenichi Oomichi <oomi...@mxs.nes.nec.co.jp> wrote: > > Hi Chris, > > Thank you for picking it up, > > > -----Original Message----- > > From: Christopher Yeoh [mailto:cbky...@gmail.com] > > Sent: Thursday, March 13, 2014 1:56 PM > > To: OpenStack Development Mailing List > > Subject: [openstack-dev] [qa][tempest] Where to do response body > > validation > > > > The new tempest body response validation is being added to > > individual testcases. See this as an example: > > > > https://review.openstack.org/#/c/78149 > > > > After having a look at https://review.openstack.org/#/c/80174/ > > I'm now thinking that perhaps we should be doing the response > > validation in the tempest/services/compute classes. And only check > > the response body if the status code is a success code (and then > > check that it is an appropriate success code). > > > > I think this will lead to fewer changes needed in the end as the > > response body checking will not needed to be added to individual > > tests. > > > > There may be some complications with handling extensions, but I > > think they are all implement backwards compatible behaviour so > > should be ok. > > > > Anyone have any thoughts about this alternative approach? > > I like the above idea that the response body validation will be > operated in REST client. Tempest will be able to check response body > anytime and reduce the test code. > > One concern is that Nova API returns different response body when > admin user or not. I'd like to check response body containing > attributes what admin user can get. For example, "get server info" > API returns a response including "OS-EXT-STS" and "OS-EXT-SRV-ATT" > attributes when an admin user.
Yes that's true, I think that's a case of extensions and policy changing what is returned. > > So how about operating the basic validation(without special > attributes) only in each REST client? > The the special validation(such as the above admin info) would be > operated in each test. The schema size for the special cases could be > reduced. That sounds reasonable to me. We'd just check for the additional parameters that we expect to be present. Chris _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
