Re: [Openstack-qa-team] [Openstack] [QA] Tempest Devstack/Jenkins gate job now passing

2012-05-04 Thread Doug Hellmann
+1000 On Fri, May 4, 2012 at 11:30 AM, Daryl Walleck wrote: > This is great news! Thanks Jay for all the hard work you've put into > getting this up and running. > > Daryl > > On May 4, 2012, at 8:34 AM, Jay Pipes wrote: > > > All, > > > > A momentous event has occurred. We now have all Tempest

Re: [Openstack-qa-team] [Openstack] [QA] Tempest Devstack/Jenkins gate job now passing

2012-05-04 Thread Wayne A. Walls
Awesome news, Jay! Big congrats to the entire QA team and others that have dedicated so much time and effort to this milestone! Cheers, Wayne Sent from my iPhone On May 4, 2012, at 8:34 AM, Jay Pipes wrote: > All, > > A momentous event has occurred. We now have all Tempest integration tes

Re: [Openstack-qa-team] [Openstack] [QA] Tempest Devstack/Jenkins gate job now passing

2012-05-04 Thread Venkatesan, Ravikumar
++1 Wow !. yes. It is fantastic. Regards, Ravi -Original Message- From: openstack-bounces+ravikumar.venkatesan=hp@lists.launchpad.net [mailto:openstack-bounces+ravikumar.venkatesan=hp@lists.launchpad.net] On Behalf Of Daryl Walleck Sent: Friday, May 04, 2012 8:31 AM To: Jay Pi

Re: [Openstack-qa-team] [Openstack] [QA] Tempest Devstack/Jenkins gate job now passing

2012-05-04 Thread Karajgi, Rohit
Great news indeed :) Cheers, Rohit -Original Message- From: openstack-bounces+rohit.karajgi=nttdata@lists.launchpad.net [mailto:openstack-bounces+rohit.karajgi=nttdata@lists.launchpad.net] On Behalf Of Daryl Walleck Sent: Friday, May 04, 2012 9:01 PM To: Jay Pipes Cc: ; openst..

[Openstack-qa-team] Spec for Swift container ACLs?

2012-05-04 Thread David Kranz
We were trying to use this feature and could not find such a thing. I filed a bug against the openstack storage manual for lack of documentation about this https://bugs.launchpad.net/openstack-manuals/+bug/994657. The new swift tempest tests must have to deal with this. How was that handled?

Re: [Openstack-qa-team] [Openstack] [QA] Tempest Devstack/Jenkins gate job now passing

2012-05-04 Thread Daryl Walleck
This is great news! Thanks Jay for all the hard work you've put into getting this up and running. Daryl On May 4, 2012, at 8:34 AM, Jay Pipes wrote: > All, > > A momentous event has occurred. We now have all Tempest integration tests > passing against the devstack-deployed test environment s

Re: [Openstack-qa-team] Policy for Tests linked to existing Bugs

2012-05-04 Thread David Kranz
Yes, I suppose it depends on how development teams think about tempest. In an ideal world, with complete test coverage of everything, any commit that fixes a bug would also have to unskip or change some test. But we can't expect developers to run tempest before a commit in the same way they wou

Re: [Openstack-qa-team] Policy for Tests linked to existing Bugs

2012-05-04 Thread Jay Pipes
On 05/04/2012 09:28 AM, David Kranz wrote: On 5/4/2012 9:08 AM, Jay Pipes wrote: On 05/04/2012 06:14 AM, Karajgi, Rohit wrote: Hi, What is the policy that we should or are following for test cases that fail due to an existing Open Bug in Launchpad? For eg: https://github.com/openstack/tempest

[Openstack-qa-team] [QA] Tempest Devstack/Jenkins gate job now passing

2012-05-04 Thread Jay Pipes
All, A momentous event has occurred. We now have all Tempest integration tests passing against the devstack-deployed test environment spun up in the dev-gate-tempest-devstack-vm: https://jenkins.openstack.org/job/dev-gate-tempest-devstack-vm/test/?width=800&height=600 Tempest is executing 15

Re: [Openstack-qa-team] Policy for Tests linked to existing Bugs

2012-05-04 Thread David Kranz
On 5/4/2012 9:08 AM, Jay Pipes wrote: On 05/04/2012 06:14 AM, Karajgi, Rohit wrote: Hi, What is the policy that we should or are following for test cases that fail due to an existing Open Bug in Launchpad? For eg: https://github.com/openstack/tempest/blob/master/tempest/tests/test_list_floatin

Re: [Openstack-qa-team] Policy for Tests linked to existing Bugs

2012-05-04 Thread Jay Pipes
On 05/04/2012 06:14 AM, Karajgi, Rohit wrote: Hi, What is the policy that we should or are following for test cases that fail due to an existing Open Bug in Launchpad? For eg: https://github.com/openstack/tempest/blob/master/tempest/tests/test_list_floating_ips.py#L64 skips the test and posts t

Re: [Openstack-qa-team] [Openstack] [QA] Aligning "smoke" / "acceptance" / "promotion" test efforts

2012-05-04 Thread Karajgi, Rohit
If the approach would be to use the client tools to "only smoke test" the Openstack API, I would somewhat agree. I think the smoke tests should be shallow but wide, covering every API in OpenStack but not ever 'scenario'. So to me a "subset of test cases" would be a subset of "test cases from eve

Re: [Openstack-qa-team] [Openstack] [QA] Aligning "smoke" / "acceptance" / "promotion" test efforts

2012-05-04 Thread Maru Newby
The rest api is the default interface, and the client tools target that interface. Since the clients are cli more than python api, they can be used by any language that can use a shell. What exactly does reimplementing the clients for the sake of testing accomplish? Double the maintenance eff

Re: [Openstack-qa-team] [Openstack] [QA] Aligning "smoke" / "acceptance" / "promotion" test efforts

2012-05-04 Thread John Garbutt
+1 to this plan > From the above, I would surmise that smoke tests should have all three of > the following characteristics: > > * Test basic operations of an API, usually in a specific order that makes > sense > as a bare-bones use case of the API > * Test only the correct action paths -- in o

[Openstack-qa-team] Policy for Tests linked to existing Bugs

2012-05-04 Thread Karajgi, Rohit
Hi, What is the policy that we should or are following for test cases that fail due to an existing Open Bug in Launchpad? For eg: https://github.com/openstack/tempest/blob/master/tempest/tests/test_list_floating_ips.py#L64 skips the test and posts the Bug ID in the message. Do we submit the te