On 12/4/2013 10:16 AM, Nikola Đipanov wrote:
On 11/19/2013 05:52 PM, Peter Feiner wrote:
On Tue, Nov 19, 2013 at 11:19 AM, Chuck Short <chuck.sh...@canonical.com> wrote:
Hi
On Tue, Nov 19, 2013 at 10:43 AM, Peter Feiner <pe...@gridcentric.ca> wrote:
A substantive reason for switching from mox to mock is the derelict
state of mox releases. There hasn't been a release of mox in three
years: the latest, mox-0.5.3, was released in 2010 [1, 2]. Moreover,
in the past 3 years, substantial bugs have been fixed in upstream mox.
For example, with the year-old fix to
https://code.google.com/p/pymox/issues/detail?id=16, a very nasty bug
in nova would have been caught by an existing test [3].
Alternatively, a copy of the upstream mox code could be added in-tree.
Please no, I think we are in an agreement with mox3 and mock.
That's cool. As long as the mox* is phased out, the false-positive
test results will be fixed.
Of course, there's _another_ alternative, which is to retrofit mox3
with the upstream mox fixes (e.g., the bug I cited above exists in
mox3). However, the delta between mox3 and upstream mox is pretty huge
(I just checked), so effort is probably better spent switching to
mock. To that end, I plan on changing the tests I cited above.
Resurrecting this thread because of an interesting review that came up
yesterday [1].
It seems that our lack of a firm decision on what to do with the mocking
framework has left people confused. In hope to help - I'll give my view
of where things are now and what we should do going forward, and
hopefully we'll reach some consensus on this.
Here's the breakdown:
We should abandon mox:
* It has not had a release in over 3 years [2] and a patch upstream for 2
* There are bugs that are impacting the project with it (see above)
* It will not be ported to python 3
Proposed path forward options:
1) Port nova to mock now:
* Literally unmanageable - huge review overhead and regression risk
for not so much gain (maybe) [1]
2) Opportunistically port nova (write new tests using mock, when fixing
tests, move them to mock):
* Will take a really long time to move to mock, and is not really a
solution since we are stuck with mock for an undetermined period of time
- it's what we are doing now (kind of).
3) Same as 2) but move current codebase to mox3
* Buys us py3k compat, and fresher code
* Mox3 and mox have diverged and we would need to backport mox fixes
onto the mox3 three and become de-facto active maintainers (as per Peter
Feiner's last email - that may not be so easy).
I think we should follow path 3) if we can, but we need to:
1) Figure out what is the deal with mox3 and decide if owning it will
really be less trouble than porting nova. To be hones - I was unable to
even find the code repo for it, only [3]. If anyone has more info -
please weigh in. We'll also need volunteers
2) Make better testing guidelines when using mock, and maybe add some
testing helpers (like we do already have for mox) that will make porting
existing tests easier. mreidem already put this on this weeks nova
meeting agenda - so that might be a good place to discuss all the issues
mentioned here as well.
For anyone interested in seeing the nova meeting agenda notes before the
actual meeting (since it covers more than just mock/mox), here you go:
https://wiki.openstack.org/wiki/Meetings/Nova
Copying here for reference later if necessary:
- Testing Guides - we need some guide on using mox vs mock; previously
we were forcing all new tests to use mock but then mox3 was pointed out
and that seemed to go away, but is anyone using mox3 yet
(ceilomerclient, heatclient and ironicclient are all moving to mox3).
- ML on stubs vs mox vs mock:
http://lists.openstack.org/pipermail/openstack-dev/2013-November/018501.html
- nova/tests/README.rst is outdated; lots of TBD and last update was
on 2013/05/16:
https://github.com/openstack/nova/blob/master/nova/tests/README.rst
- Note keystone bug 1252454 where the keystone docs were
referencing the nova testing README
- Ultimately it seems the best guide is Horizon's:
http://docs.openstack.org/developer/horizon/topics/testing.html
- We should have something like the Horizon guide in the overall
OpenStack hacking guide so the projects can point to a single location,
http://docs.openstack.org/developer/hacking/, but even that is sparse on
details for writing unit tests (should almost merge or point to the
horizon guide), and then the single guide should have notes on stubs vs
mox vs mock with pros/cons and best practices / pitfalls.
===============
The net problem is we have a lot of various testing guides throughout
the projects, including hacking guides, readmes, wikis, and devref
pages, we should really consolidate that into the universal hacking
guide (IMO) and then integrate Horizon's excellent guide into that along
with whatever rules we have for mox/mox3/mock.
We should really take a stronger stance on this soon IMHO, as this comes
up with literally every commit.
Cheers,
Nikola
[1] https://review.openstack.org/#/c/59694/
[2] https://pypi.python.org/pypi/mox
[3] https://pypi.python.org/pypi/mox3/0.7.0
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Thanks,
Matt Riedemann
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
