Daily automated email with BVT test results would be helpful. -----Original Message----- From: Prasanna Santhanam [mailto:t...@apache.org] Sent: Friday, June 14, 2013 10:43 PM To: dev@cloudstack.apache.org Subject: Infra Issues from the IRC meeting (Wed, Jun 12)
Saw a few infra topics being discussed in the meeting and there was talk of bringing it to the list so I'm taking this opportunity to explain some things behind it. > 17:22:44 [Animesh]: I will send out my weekly reminder today on status > and include Sudha's test results > 17:23:28 [topcloud]: one thing that concerns me is that the bvt > continues to be at < 100% pass rate > 17:23:41 [topcloud]: is there anything we're doing about this? yes - I've fixed most tests. Some have existed because of bugs in packaging, systemvm templates that I see patches for now. > 17:25:20 [chipc]: topcloud: was BVT ever at 100% ? > 17:25:32 [chipc]: (real question, not sarcasm) It was - 100% - when the project was first proposed. But more tests have come in since then. > 17:26:41 [chipc]: once we get it back to 100%, I say we block all > changes when it drops to <100% > 17:26:49 [topcloud]: +1 +1 - this is what I've been driving towards but haven't announced some changes I've made in the past weeks because it's pointless to have tests fail soon as I announce we are 100%. We shouldn't wait for one run, but at least 10 to ensure that the bvt is indeed stable enough to be trusted as a 'gating' system for master stability. There's also a couple of issues here - 1. Does everyone know where the tests run? 2. Do people know how to spot the failures? 3. Do people know how to find the logs for the failures? If the answer is no to all this, I have more documentation on my hands. > 17:28:07 [Animesh]: agreed bvt also shows progress towards release > readines > 17:28:07 [chipc]: topcloud: +1 > 17:28:32 [chipc]: Animesh: BVT should show that master is stable, > regardless of release timeframes > 17:28:33 [chipc]: IMO that is > 17:28:44 [chipc]: master should only see good /tested code Which is why the BVT runs on master at all times on jenkins.buildacloud.org. There is also ability to run it against a feature branch but I would rather defer that to the release manager for now since it's tied with hardware resources and jenkins schedules. That feature should strictly be reserved for architecture changes in MERGE requests. > 17:43:27 [topcloud]: sorry...to bring back this topic but is bvt running on > apache infra? > 17:43:35 [chipc]: no > 17:43:57 [topcloud]: chipc: is there any talk about bringing it into apache > infra? It was brought up with the ASF infra back in January and the suggestion was to donate hardware to the ASF to manage. So if we're prepared to do that, great! But it certainly can't just be Citrix :) I'd prefer individual project related test hardware and resources to stay in the control of the project. Infrastructure is constantly changed to allow features and enhancements to be tested so it's best to have it in the core group. Which is why jenkins.buildacloud.org came to existence. This is similar to how cloudbees operates or (*gasp*) openstack-infra [1][2] operates. Ideally, I'd like those interested in infra activities to form a separate group for cloudstack-infra related topics. The group's focus will be to maintain, test and add to the infrastructure of this project. But that's in the future. Without such a group, building an IaaS cloud is not much fun :) > 17:44:17 [topcloud]: i can't imagine apache wanting bvt to only run inside > citrix all the time. It doesn't run within Citrix. It runs in a DC in Fremont. There are other environments within Citrix however that run their own tests for their needs - eg: object_store tests, cloudplatform tests, customer related issue tests etc. /me beginning to think more doc work is on my way :/ > 17:46:27 [chipc]: but generally, the ASF build infra is a bit > overloaded +1000 > 17:46:51 [jzb]: topcloud: when you say "in Citrix" - it's still visible > outside Citrix, yes? > 17:46:52 [chipc]: so frankly, CTXS donating an environment to run it, > publicly visible to everyone, is quite helpful > 17:46:58 [chipc]: jzb: it is We need more people to donate hardware/virtual resources for testing :) CTXS has been gracious to provide quite a few resources already IMO. > 17:47:18 [chipc]: actually, I think it is... > 17:47:34 [topcloud]: jzb: yeah it's still visible but it really should be > runnable by everyone. Not quite. It's a gating system. It runs automatically and shouldn't be runnable by everyone at will. I'm still waiting to implement devcloud tests based on that gerrit converstaion (which went nowhere) we had many months back. DevCloud stuff can be run at will. > 17:47:37 [jzb]: I'm all for building up Apache infra, but I also think > having vendors donate publicly visible resources that are usable by > the community is acceptable. +1 > 17:47:53 [jzb]: in fact, we probably ought to be hitting up some of > our ISP friends for more. +1 - who are our ISP friends? Would like to get help on this. > 17:49:49 [ke4qqq]: so tsp (along with abayer and roman) are working on > a publicly accessible jenkins instance in fremont This is basically to dogfood all instances via a hosted puppetmaster. I've just started so will take time to finish this. Thoughts are most welcome! [1] https://jenkins.openstack.org/ [2] http://ci.openstack.org/ -- Prasanna., ------------------------ Powered by BigRock.com
