----- Original Message ----- > From: "Paul Michali (pcm)" <p...@cisco.com> > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > Sent: Wednesday, April 9, 2014 6:31:09 AM > Subject: Re: [openstack-dev] [infra]Requesting consideration of httmock > package for test-requirements in Juno > > On Apr 8, 2014, at 3:04 PM, Jamie Lennox < jamielen...@redhat.com > wrote: > > > > > > > ----- Original Message ----- > > > From: "Paul Michali (pcm)" < p...@cisco.com > > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org > > Cc: jamielen...@gmail.com > Sent: Wednesday, April 9, 2014 12:09:58 AM > Subject: [openstack-dev] [infra]Requesting consideration of httmock package > for test-requirements in Juno > > Reposting this, after discussing with Sean Dague… > > For background, I have developed a REST client lib to talk to a H/W device > with REST server for VPNaaS in Neutron. To support unit testing of this, I > created a UT module and a mock REST server module and used the httmock > package. I found it easy to use, and was able to easily create a sub-class > of my UT to run the same test cases with real H/W, instead of the mock REST > server. See the original email below, for links of the UT and REST mock to > see how I used it. > > > I created a bug under requirements, to propose adding httmock to the > test-requirements. Sean mentioned that there is an existing mock package, > called httpretty , which I found is used in keystone client UTs), and should > petition to see if httmock should replace httpretty, since the two appear to > overlap in functionality. > > I found this link, with a brief comparison of the two: > http://marekbrzoska.wordpress.com/2013/08/28/mocking-http-requests-in-python/ > > So… I’m wondering if the community is interested in adopting this package > (with the goal of deprecating the httpretty package). Otherwise, I will work > on reworking the UT code I have to try to use httpretty. > > Would be interested in peoples’ thoughts, especially those who have worked > with httpretty. > > Thanks in advance! > > So I introduced HTTPretty into the requirements and did the work around > keystoneclient and am well aware that it has a few warts. > > PCM: Great, I grabbed your name from keystone client logs and was hoping you > had some knowledge of httpretty. > > > > > > > At the time we were going through the changeover from httplib to requests and > httpretty gave a good way to change over the library and ensure that we > hadn't actually changed the issued requests at all. If we had already been > on requests i don't know if i'd have made the same choice. > > In general I am in favour of mocking the response layer rather than the > client layer - whether we do this with httpretty or httmock doesn't bother > me that much. Honestly I don't think a global patch of the requests Session > object is that much safer that a global patch of the socket interface, if > anything requests is under development and so this interface is less > defined. > > PCM: Not sure that httmock can be considered a global patch. It is a context > lib that intercepts the call through various decorators where the request > can be filtered/processed and if not, will fall through and call the actual > library. > > So, with the context lib, you can define several handlers for the request(s). > When the call is made, it will try each handler and if they all return None, > will call the original function, otherwise they return the value of the mock > routine. Here’s an example front he test cases I cerated: > > with httmock.HTTMock(csr_request.token, csr_request.put, > csr_request.normal_get): > keepalive_info = {'interval': 60, 'retry': 4} > self.csr.configure_ike_keepalive (keepalive_info) > self.assertEqual(requests.codes.NO_CONTENT, self.csr.status) > content = self.csr.get_request ('vpn-svc/ike/keepalive') > self.assertEqual(requests.codes.OK, self.csr.status) > expected = {'periodic': False} > expected.update(keepalive_info) > self.assertDictContainsSubset(expected, content) > > The client code (red) does a POST with authentication info to get token, does > a PUT with the setting, and then a GET to verify the value. The mock module > has these methods created: > > @httmock.urlmatch ( netloc =r'localhost') > def token( url , request): > if ' auth /token-services' in url.path: > return {'status_code': requests.codes.OK, > 'content': {'token-id': 'dummy-token'}} > > > @httmock.urlmatch ( netloc =r'localhost') > def normal_get( url , request): > if request.method != 'GET': > return > if not request.headers.get('X- auth -token', None): > return {'status_code': requests.codes.UNAUTHORIZED} > … > if 'vpn-svc/ike/keepalive' in url.path: > content = {u'interval': 60, > u'retry': 4, > u'periodic': True} > return httmock.response(requests.codes.OK, content=content) > > @httmock.urlmatch(netloc=r'localhost') > def put(url, request): > if request.method != 'PUT': > return > if not request.headers.get('X-auth-token', None): > return {'status_code': requests.codes.UNAUTHORIZED} > return {'status_code': requests.codes.NO_CONTENT} > > > Just a few notes…. > A) Could have created separate context lib for put vs get. > B) Could have a method for specific URI request matches (I do that in some > places, in others I catch all requests). > C) Can filter the requests based on request type or URI. > D) Can catch all URLs or filter. > E) Additional decorators can be created for these handlers. > F) There is lots of flexibility with manipulating the response data. > > For (C) I’ve done things like this: > > @filter_request(['post'], 'vpn-svc/site-to-site') > @httmock.urlmatch(netloc=r'localhost') > def post_missing_ipsec_policy(url, request): > if not request.headers.get('X-auth-token', None): > return {'status_code': requests.codes.UNAUTHORIZED} > return {'status_code': requests.codes.BAD_REQUEST} > > For (D), I have all my handlers set for localhost, and then I have another > test module that creates a subclass of the test class, sets the host to an > IP of a live system, and runs the same tests, only this time against a live > router, instead the mock module. This saves a lot of coding and allows me to > reuse the test code (and make sure it REALLY works with real hardware, in my > case). > > For (E), I did this to simulate a timeout in a request and then latter > success on retry: > > @filter_request (['get'], 'global/host-name') > @repeat(1) > @httmock.urlmatch ( netloc =r'localhost') > def expired_request( url , request): > """Simulate access denied failure on first request for this resource. > > Intent here is to simulate that the token has expired, by failing > the first request to the resource. Because of the repeat=1, this > will only be called once, and subsequent calls will not be handled > by this function, but instead will access the normal handler and > will pass. Currently configured for a GET request, but will work > with POST and PUT as well. For DELETE, would need to filter_request on a > different resource (e.g. 'global/local-users') > """ > > return {'status_code': requests.codes.UNAUTHORIZED} > > Ref: > https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/notest_cisco_csr_rest.py > Ref: > https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/cisco_csr_mock.py >
The reason that i call this a global mock is that behind the context manager it is still patching the class method not an individual instance of a requests.Session object. see: https://github.com/patrys/httmock/blob/master/httmock.py#L142 Everything you have specified is doable with httpretty. @httpretty.activate does the mocking of the library (socket rather than requests.Session) then you use httpretty.register_uri(httpretty.GET, 'http://testurl/testpath', body=json.dumps({'hello': 'world'}, status=200) You can do the same multiple queued responses, responses via callbck and everything else. Also the thing i find a use a lot that i don't see provided by httmock is things like: self.assertEqual(httpretty.last_request().headers['X-Auth-Token'], expected_token) In which you test that the request that you send is actually what you expect as well. Anyway there is a pretty long readme on the github page: https://github.com/gabrielfalcao/HTTPretty that explains these sort of features. > What i would like to see though is this mocking transferred into fixtures > like in https://review.openstack.org/#/c/77961/ and have the actual choice > in mock library hidden behind those fixtures. Is this a pattern that httmock > can handle? or something else again? > > PCM I see your point - it would be nice to be able to have the flexibility to > swap the underlying mock libraries. Would have to think about it more, to > see if a common fixture can be used for httpretty and httmock. I’m not sure > if they are different enough (one monkey patching, one using context lib) > that it would be possible (and I don’t have a strong understanding of > httpretty yet). > > Would love to hear peoples’ thoughts no this. Again, i'm not that strongly in favour of HTTPretty over httmock (though missing last_request will be a problem). My point is just that the two libraries can both do the same job and i'm wondering what the advantage of the new one is. As mentioned i would really like to see this moved into fixtures - httpretty will allow that somewhat, i don't see that httmock will, and if i had the time i think i would just rewrite a library like this making use of the tools we already have like mock, fixtures, and testresources. (My other thought is to just subclass a requests.Session object with an overridden request method -rather than a global patch- and allow that to be passed into clients but i don't think they all support that). Jamie > > PCM (Paul Michali) > > MAIL …..…. p...@cisco.com > IRC ……..… pcm_ ( irc.freenode.com ) > TW ………... @pmichali > GPG Key … 4525ECC253E31A83 > Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83 > > > > > > > > > Jamie > > > > > PCM (Paul Michali) > > MAIL …..…. p...@cisco.com > IRC ……..… pcm_ ( irc.freenode.com ) > TW ………... @pmichali > GPG Key … 4525ECC253E31A83 > Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83 > > > > On Apr 4, 2014, at 10:44 AM, Paul Michali (pcm) < p...@cisco.com > wrote: > > > > > I’d like to get this added to the test-requirements for Neutron. It is a very > flexible HTTP mock module that works with the Requests package. It is a > decorator that wraps the Request’s send() method and allows easy mocking of > responses, etc (w/o using a web server). > > The bug is: https://bugs.launchpad.net/neutron/+bug/1282855 > > Initially I had requested both httmock and newer requests, but was requested > to separate them, so this is to target httmock as it is more important (to > me :) to get approval, > > > The review request is: https://review.openstack.org/#/c/75296/ > > An example of code that would use this: > > https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/notest_cisco_csr_rest.py > https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/cisco_csr_mock.py > > Looking forward to hearing whether or not we can include this package into > Juno. > > Thanks in advance! > > > PCM (Paul Michali) > > MAIL …..…. p...@cisco.com > IRC ……..… pcm_ ( irc.freenode.com ) > TW ………... @pmichali > GPG Key … 4525ECC253E31A83 > Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83 > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
