Eli, I like this proposed approach. We did have a discussion with a few Stackers from openstack-infra in Tokyo to express our interest in using bare metal for gate testing. That’s still a way out, but that may be another way to speed this up further. A third idea would be to adjust the nova virt driver in our devstack image to use libvirt/lxc by default (instead of libvirt/kvm) which would allow for bays to be created more rapidly. This would potentially allow us to to perform repeated bay creations int he same pipeline in a reasonable timeframe.
Adrian On Nov 11, 2015, at 11:02 PM, Qiao,Liyong <liyong.q...@intel.com<mailto:liyong.q...@intel.com>> wrote: hello all: I will update some Magnum functional testing status, functional/integration testing is important to us, since we change/modify the Heat template rapidly, we need to verify the modification is correct, so we need to cover all templates Magnum has. and currently we only has k8s testing(only test with atomic image), we need to add more, like swarm(WIP), mesos(under plan), also , we may need to support COS image. lots of work need to be done. for the functional testing time costing, we discussed during the Tokyo summit, Adrian expected that we can reduce the timing cost to 20min. I did some analyses on the functional/integrated testing on gate pipeline. the stages will be follows: take k8s functional testing for example, we did follow testing case: 1) baymodel creation 2) bay(tls_disabled=True) creation/deletion 3) bay(tls_disabled=False) creation to testing k8s api and delete it after testing. for each stage, the time costing is follows: * devstack prepare: 5-6 mins * Running devstack: 15 mins(include downloading atomic image) * 1) and 2) 15 mins * 3) 15 +3 mins totally about 60 mins currently a example is 1h 05m 57s see http://logs.openstack.org/10/243910/1/check/gate-functional-dsvm-magnum-k8s/5e61039/console.html for all time stamps. I don't think it is possible to reduce time to 20 mins, since devstack setup will take 20 mins already. To reduce time, I suggest to only create 1 bay each pipeline and do vary kinds of testing on this bay, if want to test some specify bay (for example, network_driver etc), create a new pipeline . So, Ｉ think we can delete 2), since 3) will do similar things(create/delete), the different is 3) use tls_disabled=False. what do you think ? see https://review.openstack.org/244378 for the time costing, will reduce to 45 min (48m 50s in the example.) ================================= For other related functional testing works: I 'v done the split of functional testing per COE, we have pipeline as: * gate-functional-dsvm-magnum-api 30 mins * gate-functional-dsvm-magnum-k8s 60 mins And for swam pipeline, patches is done, under reviewing now(works fine on gate) https://review.openstack.org/244391 https://review.openstack.org/226125 ======================== -- BR, Eli(Li Yong)Qiao <liyong_qiao.vcf>__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
