Just for anybody interest. They are working to give a result on a pull request on github before merging. Example: https://github.com/apache/cloudstack/pull/13
On 1 September 2014 15:13, Hugo Trippaers <h...@trippaers.nl> wrote: > Hey Ian, > > I know, and i really like the effort. I just doubt if spending effort on > two things to achieve the same goal is worthwhile. If we want to run the > simulator tests on Travis thats fine, but then don’t waste time and > resources on running them on the jenkins.b.o. as well. If we focus our > scarce resources in one direction we can do so much more i think. > > That said, it is a job well done to get our stuff running on Travis. :D > > Cheers, > > Hugo > > > > > On 1 sep. 2014, at 16:07, Ian Duffy <i...@ianduffy.ie> wrote: > > > Hi Hugo, > > > > Just to be clear this was just an investigation to see if people were > > interested. > > > > Travis will give all forks of Cloudstack CI for free which is why it was > > interesting. Just cause its enabled/there doesn't mean it has to be used > > but it does give us another option should we wish to explore it. > > > > I made a commit to master this morning that would enable build jobs to > > execute over there (all notification emails are disabled): > > > > https://travis-ci.org/apache/cloudstack/builds/34096969 > > > > The test results are mimicking that of jenkins, we get a failure on > > test_09_expunge_vm. > > > > > > On 1 September 2014 14:41, Hugo Trippaers <h...@trippaers.nl> wrote: > > > >> Sorry for the late reply, was away for some time.. > >> > >> > >> > >> On 21 aug. 2014, at 19:40, Edison Su <edison...@citrix.com> wrote: > >> > >>> > >>> > >>>> -----Original Message----- > >>>> From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers > >>>> Sent: Thursday, August 21, 2014 5:10 AM > >>>> To: dev@cloudstack.apache.org > >>>> Subject: Re: Integration Tests on TravisCI > >>>> > >>>> Hey guys, > >>>> > >>>> Just to make sure we are all on the same page on this. > >>>> > >>>> There are several initiatives to bolster our CI capabilities and some > >> of them > >>>> are overlapping. > >>>> * There are ongoing discussion to push for a CI environment at the ASF > >>>> * There are a number of companies donating hardware/cloud instances to > >>>> beef up jenkins.buildacloud.org > >>> Sorry for my ignorance, I don't know the situation yet. Can't wait for > >> it to happen. > >> > >> Jenkins.buildacloud.org is already being supported by multiple > companies > >> and individuals, like Citrix, Schuberg Philis and the company Erik > Weber is > >> working for. David donated some space in his own private cloud as well. > So > >> we have quite decent resources already. We just need some more > specialized > >> equipment like F5, netscaler etc. > >> > >> > >>> > >>>> * Edison is doing CI builds on Amazon i believe > >>>> * There is this TravisCI initiative > >>>> > >>>> While they are all very good, overlapping effort means we are wasting > >>>> somebodies time somewhere. With a delicate a subject as CI we should > try > >>>> to avoid that i think. > >>>> > >>>> I'll happily stop working on the CI stuff in jenkins.buildacloud.org > >> for example > >>>> and focus my efforts on Travis. So i just want to know where we want > to > >> put > >>>> our effort. > >>> I don't think jenkins.buildacloud.org will go away. IMHO, the free > >> TravisCI is good for simulator test, the hosted TravisCI > >>> Is expensive. Nevertheless, we need hosted CI to test against real > >> hardware, that's jenkins.buildacloud.orgcan help. > >>> > >>>> > >>>> Cheers, > >>>> > >>>> Hugo > >>>> > >>>> > >>>> > >>>> > >>>> On 21 aug. 2014, at 14:02, Ian Duffy <i...@ianduffy.ie> wrote: > >>>> > >>>>> Santosh, > >>>>> > >>>>> What command/flags did you use for the above test output? > >>>>> > >>>>> > >>>>> On 14 August 2014 17:22, Santhosh Edukulla > >>>>> <santhosh.eduku...@citrix.com> > >>>>> wrote: > >>>>> > >>>>>> Ian, > >>>>>> > >>>>>> I just installed latest marvin after cloudstack build on master, and > >>>>>> ran tests against the simulator from smoke folder, below are the > >>>>>> results. All passed, except one, i believe that as well could be > some > >>>>>> global param or some config issue. I hope you want to install, use > >>>>>> latest marvin and test suites from master. > >>>>>> > >>>>>> === TestName: test_DeployVmAntiAffinityGroup | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_deployvm_firstfit | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_deployvm_userconcentrated | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_deployvm_userdispersing | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_create_disk_offering | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_create_sparse_type_disk_offering | Status : > >>>>>> SUCCESS === > >>>>>> > >>>>>> === TestName: test_04_create_fat_type_disk_offering | Status : > >>>>>> SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_edit_disk_offering | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_03_delete_disk_offering | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_UpdateConfigParamWithScope | Status : SUCCESS > >>>> === > >>>>>> > >>>>>> === TestName: test_dedicateGuestVlanRange | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_create_iso | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_edit_iso | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_03_delete_iso | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_05_iso_permissions | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_nic_secondaryip_add_remove | Status : SUCCESS > >>>> === > >>>>>> > >>>>>> === TestName: test_delete_account | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_public_ip_admin_account | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_public_ip_user_account | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_releaseIP | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_extendPhysicalNetworkVlan | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_UpdateStorageOverProvisioningFactor | Status : > >>>>>> SUCCESS === > >>>>>> > >>>>>> === TestName: test_createPortablePublicIPAcquire | Status : SUCCESS > >>>>>> === > >>>>>> > >>>>>> === TestName: test_createPortablePublicIPRange | Status : SUCCESS > === > >>>>>> > >>>>>> === TestName: test_01_primary_storage_nfs | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_privategw_acl | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_dedicatePublicIpRange | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_create_pvlan_network | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_createRegion | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_updatevolumedetail | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_03_restart_network_cleanup | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_05_router_basic | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_06_router_advanced | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_07_stop_router | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_08_start_router | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_09_reboot_router | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_sys_vm_start | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_sys_template_ready | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_create_service_offering | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_edit_service_offering | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_03_delete_service_offering | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_list_sec_storage_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_list_cpvm_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_create_template | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_edit_template | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_03_delete_template | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_05_template_permissions | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_07_list_public_templates | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_08_list_system_templates | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_advZoneVirtualRouter | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_deploy_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_deploy_vm_multiple | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_01_stop_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_02_start_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_03_reboot_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_06_destroy_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_07_restore_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_08_migrate_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_09_expunge_vm | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_03_download_attached_volume | Status : SUCCESS > >>>> === > >>>>>> > >>>>>> === TestName: test_04_delete_attached_volume | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_05_detach_volume | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_09_delete_detached_volume | Status : SUCCESS > >>>> === > >>>>>> > >>>>>> === TestName: test_vpc_remote_access_vpn | Status : SUCCESS === > >>>>>> > >>>>>> === TestName: test_vpc_site2site_vpn | Status : SUCCESS === > >>>>>> > >>>>>> > >>>>>> Thanks! > >>>>>> Santhosh > >>>>>> ________________________________________ > >>>>>> From: Santhosh Edukulla > >>>>>> Sent: Thursday, August 14, 2014 11:23 AM > >>>>>> To: dev@cloudstack.apache.org > >>>>>> Subject: RE: Integration Tests on TravisCI > >>>>>> > >>>>>> Few test suites under smoke/misc folder say test_vm_ha,test_vm_sync, > >>>>>> test_deploy_vm have a dependency to run only on simulator and > >>>>>> sequentially with no entries available in CS, these are written > using > >>>>>> simulator mocks, so if you run along with other test suites they > will > >> fail. > >>>>>> > >>>>>> with -w switch, nose will run all along with other suites under > >>>>>> smoke, so, you can use noseexclude plugin and exclude them from > >>>>>> normal run, and later run them in serially with out any > dependencies. > >>>>>> To exclude add the switch --exclude-dir="../smoke/misc" to nose > >>>> command. > >>>>>> > >>>>>> Other should work. Where can we see the detailed logs for these > runs? > >>>>>> > >>>>>> Regards, > >>>>>> Santhosh > >>>>>> ________________________________________ > >>>>>> From: Ian Duffy [i...@ianduffy.ie] > >>>>>> Sent: Thursday, August 14, 2014 9:42 AM > >>>>>> To: CloudStack Dev > >>>>>> Subject: Integration Tests on TravisCI > >>>>>> > >>>>>> Hi All, > >>>>>> > >>>>>> TL;DR: See https://github.com/imduffy15/Cloudstack-Clone > specifically > >>>>>> .travis.yml and tools/travis/* > >>>>>> > >>>>>> Myself, Sebastien and Rohit have been looking at executing > >>>>>> Cloudstack's integration tests against the simulator using > >> http://travis- > >>>> ci.org. > >>>>>> > >>>>>> Doing this will enable us to execute the integration tests on all > >>>>>> branches for free. It also means all community forks could make use > >>>>>> of the travis-ci service too. > >>>>>> > >>>>>> > >>>>>> Currently all tests are executing, there appears to be some tests > >>>>>> failing on the simulator specifically: > >>>>>> > >>>>>> test_routers test_secondary_storage test_service_offerings test_ssvm > >>>>>> test_templates test_vm_ha test_vm_life_cycle test_vm_sync > >>>>>> test_volumes test_vpc_vpn > >>>>>> > >>>>>> Running these tests both locally and on travis-ci on the 4.4 branch > >>>>>> I'm seeing failures. Anybody know why? > >>>>>> > >>>>>> Full console output of test execution available at > >>>>>> https://travis-ci.org/imduffy15/Cloudstack-Clone/builds/32530691 > >>>>>> > >>>>>> For those interested in how all this is working: > >>>>>> > >>>>>> I have a fork of the Cloudstack source on my github: > >>>>>> https://github.com/imduffy15/Cloudstack-Clone > >>>>>> > >>>>>> In the root of this repository you will find a .travis.yml file. > This > >>>>>> file executes a few different stages: > >>>>>> > >>>>>> before install: Installs all dependencies and build tools. > >>>>>> install: compiles cloudstack > >>>>>> before_script: starts up cloudstack and deploys the > >>>>>> setup/dev/advanced.cfg > >>>>>> script: executes the nosetests > >>>>>> > >>>>>> One limitation with travis-ci is that a job is not allowed to run > for > >>>>>> longer than 50mins. > >>>>>> > >>>>>> In order to get around this 3 separate environments are provisioned. > >>>>>> They are described as below: > >>>>>> > >>>>>> Environment 1 runs test_affinity_groups test_deploy_vm > >>>>>> test_deploy_vms_with_varied_deploymentplanners > >>>>>> test_disk_offerings test_global_settings test_guest_vlan_range > >>>>>> test_iso test_multipleips_per_nic test_network > >>>>>> test_non_contigiousvlan > >>>>>> > >>>>>> Environment 2 runs: test_over_provisioning test_portable_publicip > >>>>>> test_primary_storage test_privategw_acl test_public_ip_range > >>>>>> test_pvlan test_regions test_reset_vm_on_reboot > test_resource_detail" > >>>>>> > >>>>>> Environment 3 runs: test_routers test_secondary_storage > >>>>>> test_service_offerings test_ssvm test_templates test_vm_ha > >>>>>> test_vm_life_cycle test_vm_sync test_volumes test_vpc_vpn > >>>>>> > >>>>>> Thanks! > >>>>>> > >>>>>> Ian > >> > >> > >
