On 01/07/2016 02:41 PM, Sean Dague wrote: > On 01/07/2016 02:09 PM, Jim Rollenhagen wrote: >> Hi all, >> >> A change to global-requirements[1] introduces mimic, which is an http >> server that can mock various APIs, including nova and ironic, including >> control of error codes and timeouts. The ironic team plans to use this >> for testing python-ironicclient without standing up a full ironic >> environment. >> >> Here's the catch - mimic is built on twisted. I know twisted was >> previously removed from OpenStack (or at least people said "pls no", I >> don't know the full history). We didn't intend to stealth-introduce >> twisted back into g-r, but it was pointed out to me that it may appear >> this way, so here I am letting everyone know. lifeless pointed out that >> when tests are failing, people may end up digging into mimic or twisted >> code, which most people in this community aren't familiar with AFAIK, >> which is a valid point though I hope it isn't required often. >> >> So, the primary question here is: do folks have a problem with adding >> twisted here? We're holding off on Ironic changes that depend on this >> until this discussion has happened, but aren't reverting the g-r change >> until we decide one way or another. >> >> // jim >> >> [1] https://review.openstack.org/#/c/220268/ > What is the advantage of running another server like this over using > requests-mock (which is used by other OpenStack projects for testing > today)? The only difference here seems to be that you actually execute > requests code in one case and not in the other. > > Requests-mock debugging when things go wrong seems a bit simpler. > > This is less about twisted and more about trying to not introduce yet > another way to mock code in the tree that people need to understand.
As an outsider and lurker...this is a fair point. I also maintain another project - StackInABox (and OpenStackInABox) - that resolves the issue in a slightly different way that utilizes requests-mock to do something similar, but closer to true unit tests. The main advantage of mimic (https://pypi.python.org/pypi/mimic) is that it can be independent of language framework (e.g you can use it for C++ or Java or Python2, Python3, or...) as its an independent process from your testing, and the tasks can be handled very asynchronously so it can mock out the responses for various tasks in a timely manner, and several parts of a system can interact together during a test. You're also utilizing a common mock of the entire system so multiple projects can benefit from the same code base. It's kind of integration-test but not quite - a good fit for projects where unit tests need multiple other components working together, e.g services that use taskflow would certainly benefit from using mimic. The disadvantage of mimic from what I can tell is how the tests are setup. But I'll leave that more to the mimic users and maintainers (OpenStack Poppy, Rackspace Autoscale, etc) to explain. For comparison: My StackInABox project provides a framework upon which OpenStackInABox builds to provide a common model so multiple projects benefit. I've used StackInABox on several of my own (non-OpenStack projects) and my unit tests (run by tox) have been able to be nearly complete integration tests. Aside from the slight use-case difference (as exemplified by Rackspace AutoScale), the main advantage of mimic over my OpenStackInABox is the momentum behind it due to its use by Rackspace AutoScale and OpenStack Poppy; I just don't have the resources to build out the models in OpenStackInABox efficiently or quickly (though I'd certainly welcome the help to do so). Advantage of the StackInABox approach is every unit test is self-contained and you don't have to change code to run the test in most cases (where you do, it's minimal) - just like with using requests-mock; but you can't have two services independently hit it - like with Rackspace AutoScale. HTH, Ben P.S here's my two projects: https://github.com/BenjamenMeyer/stackInABox ~ https://pypi.python.org/pypi/stackinabox https://github.com/BenjamenMeyer/openstackinabox ~ https://pypi.python.org/pypi/openstackinabox __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
