Thanks for providing a bit more information on your goals, that helps shape my answers a bit more usefully. See within.
On Wed, 5 Jul 2017, Shewale, Bhagyashri wrote:
So I'd say the api is verified. What is missing, and could be useful, is using those tests to get accurate and up to date representations of the JSON request and response bodies. If that's something we'd like to pursue as I said in my other message the 'verbose' functionality that can be provided in gabbi->>based tests should be able to help.I have checked with the 'verbose' functionality, but that doesn't verify the entire response, it's simply prints header and response on console.
That's correct. My suggestion to use the 'verbose' functionality was if your main goal was to create samples to be used in documentation: a suite of gabbi tests can run which output those samples as needed, wherever they need to go.
On the current master code, few of the gabbi tests verify the entire response and the remaining verify’s only specific attributes from the response.
This is mostly intentional. It doesn't make sense to me that we would verify serialization code at the level of API tests. The difference you see in the extent to which each tests validates a response is the result of different people writing the tests.
So instead of verifying each and every key-value pair from the response object, it would nice if gabbit can add a support to accept response.json file and during execution of the test, it can verify whether all key-value pairs response.json file are matching with the actual response.
While I disagree with doing this (it makes the tests more fragile, requires more pieces to be changed when things are changed, and tests the serialization code in the api tests, rather than the API[1]) the functionality that you're describing can easily exist in gabbi so if you and others decide that it is useful it can be done, either in gabbi itself or in a custom content handler[2]. Is it your intent to compare the abstract nestable object represented by the JSON, or the string? If it's the latter there's existing code out there that does that[3] but that's probably not a good choice as it will break as soon as the format of the JSON changes for some reason. (I'm happy to write the code that does full object comparison if we decide it's desirable or help land a pull request.) [1] From https://gabbi.readthedocs.io/en/latest/jsonpath.html This is not a technique that should be used frequently as it can lead to difficult to read tests and it also indicates that your gabbi tests are being used to test your serializers and data models, not just your API interactions. [2] https://gabbi.readthedocs.io/en/latest/handlers.html And an example: [3] https://github.com/hogarthww/gabbi-tools/blob/master/src/gabbi_tools/response_handlers.py -- Chris Dent ┬──┬◡ﾉ(° -°ﾉ) https://anticdent.org/ freenode: cdent tw: @anticdent
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
