> -----Original Message----- > From: Pierre-Luc Dion [mailto:pd...@cloudops.com] > Sent: Thursday, August 21, 2014 11:21 AM > To: dev@cloudstack.apache.org > Cc: Will Stevens > Subject: Re: [DISCUSSION] CI > > On Thu, Aug 21, 2014 at 1:32 PM, Edison Su <edison...@citrix.com> wrote: > > > > > > > > -----Original Message----- > > > From: Pierre-Luc Dion [mailto:pd...@cloudops.com] > > > Sent: Thursday, August 21, 2014 5:06 AM > > > To: dev@cloudstack.apache.org > > > Cc: Will Stevens > > > Subject: Re: [DISCUSSION] CI > > > > > > Thanks for you responses, > > > > > > The revamp of marvin look promising, I'll have a closer look at it, > > > even > > if I'm > > > not Python fan ;). In our side, we have close to fully automated > > installation > > > of CloudStack, XenServer hypervisor and an initial zone creation. > > > Our > > missing > > > > > > Will it be open sourced?:) > > > Of course :-) for the Cloudstack configuration part. > https://github.com/cloudops/cookbook_cloudstack > > For the XenServer side it's a bit arch because we basically manage xenserver > kickstart with a Chef Cookbook that is quite ugly. but could be opensourced.
Internally in Citrix(every time I mentioned my company name in ML, I am so nerves.. ), we almost do The same thing, cobber/kickstart to install hypervisors, we support KVM/Vmware/Xenserver. Let me ask the team working it, when can we open source the code. > > Zone, service offerings, templates creation, I have a set of scripts not on > git > hub yet (still too ugly) but marven does it too. > > > > > > part is automated test cases to use it. So we could provide > > infrastructure and > > > run tests against it for one or more test scenarios during QA phase > > > of > > release. > > > Hopefully this could be integrated to the CI. > > Currently http://jenkins.buildacloud.org/, is CloudStack CI portal. > > If you can provide Infrastructure to integrate with it, that will be > > awesome! > > > > > > > > Would it make sense to have a sheet, maybe a wiki page that list > > > all infrastructure test scenarios and their result per version of ACS? > > > for: > > > xenserver, hyperv, vmware,kvm > > > advanced, simplenetworking, vlan/vxlan,gre isolated, networks > > s3,swift,nfs, > > > storage plugin > > > > > > Unless we have this kind of results somewhere, I don't feel we > > > really > > keep > > > track of what has been tested on each release. > > It's a good idea. Let me ask the team who wrote these test cases, > > maybe they have answers. > > > > > > > > > > > > > > > > > *Pierre-Luc DION* > > > Architecte de Solution Cloud | Cloud Solutions Architect t > > > 855.652.5683 > > > > > > *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* > > > tw @CloudOps_ > > > > > > > > > > > > On Thu, Aug 21, 2014 at 2:03 AM, Prasanna Santhanam <t...@qubole.com> > > > wrote: > > > > > > > I use rspec in Ruby these days and find it get complex to maintain > > > > pretty quickly. It's good for dealing with REST apis and > > > > middleware but with Marvin (and python in general) I find > > > > infrastructure handling easier. Stack traces of BDD code is also a > > > > little bit hard to troubleshoot and comprehend. I may be biased > > > > because my Python expertise is better, so you can take this with a pinch > of salt. > > > > > > > > Edison is right however, the setup in marvin is atrociously > > > > complex and needs to be a lot simpler and intuitive. In most cases > > > > I don't want to specify tons of args (service offering, disk > > > > offering, network > > > > type) to create a simple vM that is part of every test. To this > > > > end I refactored Marvin to use easily defined factories (in lines > > > > with Ruby's factory_girl) and also demo-ed some code in the > > > > branch. Haven't had time to maintain and revamp this honestly. So > > > > if anyone wants to pick up from there, I'm happy to help. > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+Refact > > > or > > > > > > > > > > > > > > > > On Thu, Aug 21, 2014 at 4:30 AM, Edison Su <edison...@citrix.com> > > wrote: > > > > > > > > > > > > > > >> -----Original Message----- > > > > >> From: Pierre-Luc Dion [mailto:pd...@cloudops.com] > > > > >> Sent: Wednesday, August 20, 2014 11:20 AM > > > > >> To: dev@cloudstack.apache.org; Will Stevens > > > > >> Subject: [DISCUSSION] CI > > > > >> > > > > >> I'd like to move this discussion out of the GIT workflow thread. > > > > >> > > > > >> Do we have any CI strategy in place or a starting point at least? > > > > > TravisCI will be a starting point for now. It tests against > > > > > simulator > > > > only. > > > > > I am working on a solution to test against KVM(that's most of > > > > > users are > > > > concerned about) in a machine I created in > > > > > Public cloud. It has big memory, super fast disk IO, should be a > > > > > good > > > > solution for KVM test. It may take few days to get it work. > > > > > They are both for simple test, and in small scale. > > > > >> > > > > >> I've never worked with marvin I from the small understanding > > > > >> that I have from it, it might not be the right too. I don't know. > > > > >> > > > > >> I've spend some times recently with test-kitchen and serverspec > > > > >> and > > > > found > > > > >> that quite promising as test platform for "real cloudstack > > > > >> deployment > > > > test > > > > >> scenarios". By using test framework like Rspec the test output > > > > >> is clean > > > > to > > > > >> read for human and test scenarios seams quite simple to write. > > > > >> > > > > >> I've started a rspec lib for cloudstack to enhance serverspec > > > > >> for > > > > CloudStack > > > > >> tests (cloudstack-rspec). I'm wondering if it would be usable > > > > >> in > > > > future CI > > > > >> system for CloudStack... > > > > > I am familiar with Specs, like Rspec, but in scala. I agree > > > > > with you > > > > that the BDD style test case > > > > > Is easier to read/write. It's good to have, but I think before > > > > > we move > > > > to new test framework, we should figure out the issues > > > > > We have today in Marvin, so that we can take effort to address > > > > > them, > > > > either fixing Marvin, or switch to a new test framework. > > > > > The issue I have in current Marvin, is that it still needs to > > > > > write a > > > > lot of setup code for each test case, which is complicated enough. > > > > Let's take an Marvin test case as an example: > > > > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f= > > > > test > > > > > > > > /integration/smoke/test_vm_life_cycle.py;h=57e47c24838ffc2ec5e9af4ad > > > 38 > > > > 72d5e27342d91;hb=HEAD > > > > > > > > > > Like this kind of code in test case setup code: > > > > > > > > > > # Get Zone, Domain and templates > > > > > cls.domain = get_domain(cls.apiclient) > > > > > cls.zone = get_zone(cls.apiclient, > > testClient.getZoneForTests()) > > > > > cls.services['mode'] = cls.zone.networktype > > > > > > > > > > #If local storage is enabled, alter the offerings to use > > > > localstorage > > > > > #this step is needed for devcloud > > > > > if cls.zone.localstorageenabled == True: > > > > > > > > > > cls.services["service_offerings"]["tiny"]["storagetype"] > > > > > = > > > > 'local' > > > > > > > > > > cls.services["service_offerings"]["small"]["storagetype"] = > > > > 'local' > > > > > > > > > > cls.services["service_offerings"]["medium"]["storagetype"] = > > > > 'local' > > > > > > > > > > this kind of information should be provided by test framework > > > > automatically, instead of explicitly set in every test case. > > > > > Do you have other concerns about current Marvin, other than test > > style? > > > > > > > > > > > > > > >> > > > > >> Also, having a CI in place for cloudstack is quite of a huge > > > > >> deal if we > > > > want to > > > > >> validate all kind of deployment scenario, networks, hypervisor > > > > >> version > > > > and > > > > >> guest OS version. > > > > > > > > > > > > > > > Can't agree with you more. > > > > > > > > > >> > > > > >> > > > > >> For those that have some knowledge on the CI subject in > > > > >> CloudStack > > > > please > > > > >> update the community. I don't feel alone being lost about this > > > > CloudStack > > > > >> topic. > > > > > > > > > > There is an update-to-date write up regarding to Marvin: > > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+- > > > +Testin > > > > g+with+Python > > > > > Marvin does two things, one is to call cloudstack mgt API to add > > > > resources into CloudStack mgt server, such as hypervisors, primary > > > > storages, networks, and secondary storages etc. > > > > > Another one is to run test cases under tools/integration/smoke > > > > > or any > > > > other python test cases. > > > > > It doesn't cover how to deploy hypervisors on real HW, how to > > > > > setup > > > > primary storages, secondary storage etc. That's something like > > > > test-kitchen and serverspec can help, I think. > > > > > > > > > >> > > > > >> Thanks, > > > > >> > > > > >> > > > > >> Pierre-Luc > > > > > > > > > > > > > > > > -- > > > > Prasanna., > > > > > >
