Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-27 Thread Christopher Yeoh
On Wed, 26 Mar 2014 21:02:58 -0700 Maru Newby wrote: > > > > If an API is made and documented properly what domain expertise > > would be needed to use it? The opposite is true for tempest and the > > tests themselves. The tempest team focuses on just tests so they > > know how to write good test

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-27 Thread Joe Gordon
On Wed, Mar 26, 2014 at 9:02 PM, Maru Newby wrote: > > On Mar 26, 2014, at 12:59 PM, Joe Gordon wrote: > > > > > > > > > On Tue, Mar 25, 2014 at 1:49 AM, Maru Newby wrote: > > > > On Mar 21, 2014, at 9:01 AM, David Kranz wrote: > > > > > On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: >

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-27 Thread Malini Kamalambal
> >The beginning of this thread largely came from the fact that Marconi >clearly doing most of their QA not in an upstream way. To be integrated, >that needs to change. Marconi has very good test coverage within the project. These tests guarantee functionality at a project level (i.e the API wor

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-26 Thread Maru Newby
On Mar 26, 2014, at 1:52 PM, Sean Dague wrote: > On 03/25/2014 04:49 AM, Maru Newby wrote: >> >> On Mar 21, 2014, at 9:01 AM, David Kranz wrote: >> >>> On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: > -Original Message- > From: Malini Kamalambal [mailto:malini.ka

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-26 Thread Maru Newby
On Mar 26, 2014, at 12:59 PM, Joe Gordon wrote: > > > > On Tue, Mar 25, 2014 at 1:49 AM, Maru Newby wrote: > > On Mar 21, 2014, at 9:01 AM, David Kranz wrote: > > > On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: > >> > >>> -Original Message- > >>> From: Malini Kamalambal [

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-26 Thread Sean Dague
On 03/25/2014 04:49 AM, Maru Newby wrote: > > On Mar 21, 2014, at 9:01 AM, David Kranz wrote: > >> On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: >>> -Original Message- From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] Sent: Thursday, March 20, 2014

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-26 Thread Joe Gordon
On Tue, Mar 25, 2014 at 1:49 AM, Maru Newby wrote: > > On Mar 21, 2014, at 9:01 AM, David Kranz wrote: > > > On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: > >> > >>> -Original Message- > >>> From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] > >>> Sent: Thursday, M

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-26 Thread Maglana, Mark
This is really interesting discussion but was thrown off by the different use of ‘functional testing.’ I decided to reconcile it with my understanding and ended up with this two-pager. Sharing it in case it helps: https://docs.google.com/document/d/1ILxfoJlov9lBfuuZtvwW_7bmlGaYYw0UsE1R9lo6XwQ/ed

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-25 Thread Malini Kamalambal
We are talking about different levels of testing, 1. Unit tests - which everybody agrees should be in the individual project itself 2. System Tests - 'System' referring to (& limited to), all the components that make up the project. These are also the functional tests for the project. 3. Integr

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-25 Thread Maru Newby
On Mar 21, 2014, at 9:01 AM, David Kranz wrote: > On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: >> >>> -Original Message- >>> From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] >>> Sent: Thursday, March 20, 2014 12:13 PM >>> >>> 'project specific functional testi

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-24 Thread Joe Gordon
On Fri, Mar 21, 2014 at 9:38 AM, Malini Kamalambal < malini.kamalam...@rackspace.com> wrote: > > > On 3/21/14 12:01 PM, "David Kranz" wrote: > > >On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: > >> > >>> -Original Message- > >>> From: Malini Kamalambal [mailto:malini.kamalam...@ra

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-24 Thread Malini Kamalambal
On 3/21/14 3:49 PM, "Rochelle.RochelleGrober" wrote: > >> From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] > >> >> We are talking about different levels of testing, >> >> 1. Unit tests - which everybody agrees should be in the individual >> project >> itself >> 2. System Tests

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-21 Thread Rochelle.RochelleGrober
> From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] > > We are talking about different levels of testing, > > 1. Unit tests - which everybody agrees should be in the individual > project > itself > 2. System Tests - 'System' referring to (& limited to), all the > components > tha

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-21 Thread Malini Kamalambal
On 3/21/14 12:01 PM, "David Kranz" wrote: >On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: >> >>> -Original Message- >>> From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] >>> Sent: Thursday, March 20, 2014 12:13 PM >>> >>> 'project specific functional testing' in t

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-21 Thread David Kranz
On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: -Original Message- From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] Sent: Thursday, March 20, 2014 12:13 PM 'project specific functional testing' in the Marconi context is treating Marconi as a complete system, maki

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-21 Thread Malini Kamalambal
I have an etherpad started to document QA requirements https://etherpad.openstack.org/p/Tempest-Graduation-Criteria I hope Sean and the rest of QA team can add their thoughts here. I am also looking for inputs from the Sahara team, while the path to graduation is still fresh in their minds. We can

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-20 Thread Joe Gordon
On Thu, Mar 20, 2014 at 1:19 PM, Rochelle.RochelleGrober < rochelle.gro...@huawei.com> wrote: > > > > -Original Message- > > From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] > > Sent: Thursday, March 20, 2014 12:13 PM > > > > 'project specific functional testing' in the Mar

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-20 Thread Rochelle.RochelleGrober
> -Original Message- > From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] > Sent: Thursday, March 20, 2014 12:13 PM > > 'project specific functional testing' in the Marconi context is > treating > Marconi as a complete system, making Marconi API calls & verifying the > resp

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-20 Thread Sean Dague
I will agree that the TC language is not as strong as it should be (and really should be clarified, but I don't think that's going to happen until the release is looking solid). Honestly, though, I think Sahara is a good example here of the level of that we expect. They have actively engaged with u

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-20 Thread Malini Kamalambal
Thanks Matt for your response !! It has clarified some of the 'cloudy areas' ;) "So having only looked at the Marconi ML thread and not the actual TC meeting minutes I might be missing the whole picture. But, from what I saw when I looked at both a marconi commit and a tempest commit is that ther

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-20 Thread Matthew Treinish
On Thu, Mar 20, 2014 at 11:35:15AM +, Malini Kamalambal wrote: > Hello all, > > I have been working on adding tests in Tempest for Marconi, for the last few > months. > While there are many amazing people to work with, the process has been more > difficult than I expected. > > Couple of pai