On Sun, Oct 13, 2013 at 08:14:01AM +0530, Girish Shilamkar wrote: > Hello Prasanna, > > +1 to the new marvin architecture. > The proposed architecture will make marvin lot more stable, remove > defining data in code and will be easier to add testcases.
Great! thanks for reviewing. > > Can we have a run in jenkins.cloudstack.org to ensure that the > existing tests won't break with new marvin ? Absolutely. I will rebase the branch run the test suite on jenkins. > > How much effort would it be to convert existing tests to use new > marvin, sooner or later we might want to do it ? I think it would be a reasonably big task to take up and I wasn't sure if we were ever going to take that up. We would gradually have to write tests using the new way and with enough tests under our belt abandon the old tests. This is why I didn't break the compatibility or I would've refactored it slightly differently getting rid of the cloudstackAPI cmd objects altogether. > Going forward it will be critical to maintain these associations via > factory hierarchies else would beat the purpose. > And needs to be caught using reviews. Yes - we need a lot of care in defining our factories to make them reusable, well documented and carefully organized. > > In the branch what is the feature/ dir for ? I was experimenting with the behave framework for converting the new representation of cloudstack entities into a BDD based test. But this will be separate from the intermediary work of factories. > > Could you also put up a Todo list on the wiki ? I did not find one > in the branch. I will paste the TODO list in the branch on to the wiki. They're in errata.markdown in tools/marvin/doc > > Regards, > Girish > > > On 02-Oct-2013, at 10:12 PM, Prasanna Santhanam <t...@apache.org> wrote: > > > Once upon a time [1] I had propagated the idea of refactoring marvin to > > make test case writing simpler. At the time, there weren't enough > > people writing tests using marvin however. Now as focus on testing has > > become much more important for the stability of our releases I would > > like to bring back the discussion and to review the refactoring of > > marvin which I've been doing in the marvin_refactor branch. > > > > The key goal of this refactor was to simplify test case writing. In > > doing so I've transformed the library from its brittle hand-written > > nature to a completely auto-generated set of libraries. In that sense, > > marvin is much closer to cloudmonkey now. > > > > The two important changes in this refactor are: > > > > 1. data represented in an object-oriented fashion presented as factories > > 2. test case writing using entities and their operations rather than > > a sequence of disconnected API calls. > > > > To see the full nature of this proposal I've updated the spec I put up > > on the wiki: > > https://cwiki.apache.org/confluence/x/RI3lAQ > > > > For a quick comparison I wrote a test for the VPC vm's lifecycle in > > tools/marvin/marvin/test/test_vpc_life_cycle.py which one can compare > > with the existing tests for vpc under > > test/integration/component/test_vpc_vm_life_cycle.py > > > > These changes being 'architectural' so to speak and in a way > > disruptive even I would like to merge this at the beginning of the > > upcoming cloudstack release. > > > > This is only a small part of a larger change for marvin which will be > > moving to a more BDD like implementation [2] where tests are written > > using a gherkin-like language. But that will come later. > > > > I've also tried to disconnect marvin from depending on CloudStack's > > build and repo. This will help split marvin from CloudStack which I > > will discuss in a seperate thread. > > > > [1] http://markmail.org/message/4tsscn6zvtfsskuj > > [2] http://pythonhosted.org/behave/ > > > > -- > > Prasanna., > > > > ------------------------ > > Powered by BigRock.com > > > -- Prasanna., ------------------------ Powered by BigRock.com
