Thanks for the confirmation, I agree use internal maintain CI whitelist is a good way, and I confirmed with our CI folks that we already did so and more can be removed per Eric's test result below, so we will compare and remove unnecessary cases to get more bandwidth to run 3rd party CI. thanks
Best Regards! Kevin (Chen) Ji 纪 晨 Engineer, zVM Development, CSTL Notes: Chen CH Ji/China/IBM@IBMCN Internet: [email protected] Phone: +86-10-82451493 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC From: Ghanshyam Mann <[email protected]> To: "OpenStack Development Mailing List \\" <[email protected]> Date: 09/07/2018 02:18 PM Subject: Re: [openstack-dev] [tempest][CI][nova compute] Skipping non-compute-driver tests ---- On Fri, 07 Sep 2018 04:41:32 +0900 Eric Fried <[email protected]> wrote ---- > Jichen- > > That patch is not ever intended to merge; hope that was clear from the > start :) It was just a proving ground to demonstrate which tests still > pass when there's effectively no compute driver in play. > > We haven't taken any action on this from our end, though we have done a > little brainstorming about how we would tool our CI to skip those tests > most (but not all) of the time. Happy to share our experiences with you > if/as we move forward with that. > > Regarding the tempest-level automation, I certainly had z in mind when > I was thinking about it. If you have the time and inclination to help > look into it, that would be most welcome. Sorry for late response, somehow i missed this thread. As you mentioned and noticed in your patch that there are ~700 tests which does not touch compute driver. Most of them are from neutron-tempest-plugins or other service tests. From Tempest compute tests, many of them are negative tests or DB layer tests [1]. neutron-tempest-plugin or other service test you can always avoid to run with regex. And i do not think compute negative or DB test will take much time to run. But still if you want to avoid to run then, I think it is easy to maintain a whitelist regex file on CI side which can run only compute driver tests(61 in this case). Tagging compute driver on tempest side is little hard to maintain i feel and it can goes out of date very easily. If you have any other idea on that, we can surly talk in PTG on this. [1] http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz > > Thanks, > efried > > On 09/06/2018 12:33 AM, Chen CH Ji wrote: > > I see the patch is still ongoing status and do you have a follow up > > plan/discussion for that? we are maintaining 2 CIs (z/VM and KVM on z) > > so skip non-compute related cases will be a good for 3rd part CI .. thanks > > > > Best Regards! > > > > Kevin (Chen) Ji 纪 晨 > > > > Engineer, zVM Development, CSTL > > Notes: Chen CH Ji/China/IBM@IBMCN Internet: [email protected] > > Phone: +86-10-82451493 > > Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian > > District, Beijing 100193, PRC > > > > Inactive hide details for Eric Fried ---09/04/2018 09:35:09 PM---Folks- > > The other day, I posted an experimental patch [1] withEric Fried > > ---09/04/2018 09:35:09 PM---Folks- The other day, I posted an > > experimental patch [1] with an effectively > > > > From: Eric Fried <[email protected]> > > To: "OpenStack Development Mailing List (not for usage questions)" > > <[email protected]> > > Date: 09/04/2018 09:35 PM > > Subject: [openstack-dev] [tempest][CI][nova compute] Skipping > > non-compute-driver tests > > > > ------------------------------------------------------------------------ > > > > > > > > Folks- > > > > The other day, I posted an experimental patch [1] with an effectively > > empty ComputeDriver (just enough to make n-cpu actually start) to see > > how much of our CI would pass. The theory being that any tests that > > still pass are tests that don't touch our compute driver, and are > > therefore not useful to run in our CI environment. Because anything that > > doesn't touch our code should already be well covered by generic > > dsvm-tempest CIs. The results [2] show that 707 tests still pass. > > > > So I'm wondering whether there might be a way to mark tests as being > > "compute driver-specific" such that we could switch off all the other > > ones [3] via a one-line conf setting. Because surely this has potential > > to save a lot of CI resource not just for us but for other driver > > vendors, in tree and out. > > > > Thanks, > > efried > > > > [1] https://review.openstack.org/#/c/599066/ > > [2] > > http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz > > [3] I get that there's still value in running all those tests. But it > > could be done like once every 10 or 50 or 100 runs instead of every time. > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: [email protected]?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: [email protected]?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
