Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so not sure of

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/11/2014 04:21 PM, Matthew Treinish wrote: I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond inline

[openstack-dev] Implications of moving functional tests to projects (was Re: Which program for Rally)

2014-08-12 Thread David Kranz
Changing subject line to continue thread about new $subj On 08/12/2014 08:56 AM, Doug Hellmann wrote: On Aug 11, 2014, at 12:00 PM, David Kranz <mailto:dkr...@redhat.com>> wrote: On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't k

Re: [openstack-dev] [all] The future of the integrated release

2014-08-13 Thread David Kranz
On 08/13/2014 08:07 AM, Daniel P. Berrange wrote: On Wed, Aug 13, 2014 at 12:55:48PM +0100, Steven Hardy wrote: On Wed, Aug 13, 2014 at 11:42:52AM +0100, Daniel P. Berrange wrote: On Thu, Aug 07, 2014 at 03:56:04AM -0700, Jay Pipes wrote: That said, I entirely agree with you and wish efforts t

Re: [openstack-dev] [nova][core] Expectations of core reviewers

2014-08-14 Thread David Kranz
On 08/14/2014 10:54 AM, Matt Riedemann wrote: On 8/14/2014 3:47 AM, Daniel P. Berrange wrote: On Thu, Aug 14, 2014 at 09:24:36AM +1000, Michael Still wrote: On Thu, Aug 14, 2014 at 3:09 AM, Dan Smith wrote: I'm not questioning the value of f2f - I'm questioning the idea of doing f2f meeting

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-20 Thread David Kranz
On 08/18/2014 04:57 PM, Matthew Treinish wrote: On Sat, Aug 16, 2014 at 06:27:19PM +0200, Marc Koderer wrote: Hi all, Am 15.08.2014 um 23:31 schrieb Jay Pipes : I suggest that "tempest" should be the name of the import'able library, and that the integration tests themselves should be what is

Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread David Kranz
On 08/21/2014 02:39 PM, gordon chung wrote: > The point I've been making is > that by the TC continuing to bless only the Ceilometer project as the > OpenStack Way of Metering, I think we do a disservice to our users by > picking a winner in a space that is clearly still unsettled. can we avoid

Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread David Kranz
On 08/21/2014 04:12 PM, Clint Byrum wrote: Excerpts from David Kranz's message of 2014-08-21 12:45:05 -0700: On 08/21/2014 02:39 PM, gordon chung wrote: The point I've been making is that by the TC continuing to bless only the Ceilometer project as the OpenStack Way of Metering, I think we do a

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-08-26 Thread David Kranz
On 08/26/2014 10:14 AM, Zane Bitter wrote: Steve Baker has started the process of moving Heat tests out of the Tempest repository and into the Heat repository, and we're looking for some guidance on how they should be packaged in a consistent way. Apparently there are a few projects already pac

Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-26 Thread David Kranz
On 08/26/2014 10:04 AM, Doug Hellmann wrote: On Aug 26, 2014, at 5:13 AM, Thierry Carrez wrote: OK, now that we have evacuated the terminology issue (we'll use liaison or janitor or secretary, not czar), and side-stepped the offtopic development (this is not about suppressing PTLs, just a fram

Re: [openstack-dev] [all] gate debugging

2014-08-27 Thread David Kranz
On 08/27/2014 02:54 PM, Sean Dague wrote: Note: thread intentionally broken, this is really a different topic. On 08/27/2014 02:30 PM, Doug Hellmann wrote:> On Aug 27, 2014, at 1:30 PM, Chris Dent wrote: On Wed, 27 Aug 2014, Doug Hellmann wrote: I have found it immensely helpful, for examp

Re: [openstack-dev] [all] gate debugging

2014-08-27 Thread David Kranz
On 08/27/2014 03:43 PM, Sean Dague wrote: On 08/27/2014 03:33 PM, David Kranz wrote: On 08/27/2014 02:54 PM, Sean Dague wrote: Note: thread intentionally broken, this is really a different topic. On 08/27/2014 02:30 PM, Doug Hellmann wrote:> On Aug 27, 2014, at 1:30 PM, Chris Dent wr

[openstack-dev] [qa] Lack of consistency in returning response from tempest clients

2014-08-29 Thread David Kranz
While reviewing patches for moving response checking to the clients, I noticed that there are places where client methods do not return any value. This is usually, but not always, a delete method. IMO, every rest client method should return at least the response. Some services return just the re

Re: [openstack-dev] [qa] Lack of consistency in returning response from tempest clients

2014-08-29 Thread David Kranz
On 08/29/2014 10:56 AM, Sean Dague wrote: On 08/29/2014 10:19 AM, David Kranz wrote: While reviewing patches for moving response checking to the clients, I noticed that there are places where client methods do not return any value. This is usually, but not always, a delete method. IMO, every

[openstack-dev] [qa] Tempest Bug Day: Tuesday September 9

2014-09-02 Thread David Kranz
It's been a while since we had a bug day. We now have 121 NEW bugs: https://bugs.launchpad.net/tempest/+bugs?field.searchtext=&field.status%3Alist=NEW&orderby=-importance The first order of business is to triage these bugs. This is a large enough number that I hesitate to mention anything else,

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-09-05 Thread David Kranz
On 09/05/2014 12:10 PM, Matthew Treinish wrote: On Fri, Sep 05, 2014 at 09:42:17AM +1200, Steve Baker wrote: On 05/09/14 04:51, Matthew Treinish wrote: On Thu, Sep 04, 2014 at 04:32:53PM +0100, Steven Hardy wrote: On Thu, Sep 04, 2014 at 10:45:59AM -0400, Jay Pipes wrote: On 08/29/2014 05:15

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-09-05 Thread David Kranz
On 09/05/2014 12:10 PM, Matthew Treinish wrote: On Fri, Sep 05, 2014 at 09:42:17AM +1200, Steve Baker wrote: On 05/09/14 04:51, Matthew Treinish wrote: On Thu, Sep 04, 2014 at 04:32:53PM +0100, Steven Hardy wrote: On Thu, Sep 04, 2014 at 10:45:59AM -0400, Jay Pipes wrote: On 08/29/2014 05:15

[openstack-dev] [qa] Reminder: Tempest Bug Day: Tuesday September 9 (tomorrow)

2014-09-08 Thread David Kranz
It's been a while since we had a bug day. We now have 121 (now 124) NEW bugs: https://bugs.launchpad.net/tempest/+bugs?field.searchtext=&field.status%3Alist=NEW&orderby=-importance The first order of business is to triage these bugs. This is a large enough number that I hesitate to mention a

Re: [openstack-dev] Kilo Cycle Goals Exercise

2014-09-11 Thread David Kranz
On 09/11/2014 07:32 AM, Eoghan Glynn wrote: As you all know, there has recently been several very active discussions around how to improve assorted aspects of our development process. One idea that was brought up is to come up with a list of cycle goals/project priorities for Kilo [0]. To that

[openstack-dev] [qa] Tempest Bug triage

2014-09-11 Thread David Kranz
So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to logs that no longer existed. This suggests that we really need to keep up with bug triage in real time. Since b

Re: [openstack-dev] [qa] Tempest Bug triage

2014-09-12 Thread David Kranz
On 09/12/2014 05:11 AM, Kashyap Chamarthy wrote: On Thu, Sep 11, 2014 at 03:52:56PM -0400, David Kranz wrote: So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to

Re: [openstack-dev] [Neutron][Tempest] Tempest API test for Neutron LBaaS

2013-11-20 Thread David Kranz
On 11/19/2013 09:40 PM, Sean Dague wrote: On 11/17/2013 09:55 PM, Eugene Nikanorov wrote: Hi folks, I'm working on major change to Neutron LBaaS API, obviously it will break existing tempest API tests for LBaaS. What would be the right process to deal with this? I guess I can't just push fixed

[openstack-dev] [all project] Time to fail tempest gate jobs when new log errors appear

2013-11-27 Thread David Kranz
tl;dr Soon, perhaps next week, tempest gate jobs will start failing if there are any ERROR lines in the logs that are not matched by an entry in https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. There is an exception for neutron because more work needs to be done there for th

Re: [openstack-dev] excessively difficult to support both iso8601 0.1.4 and 0.1.8 as deps

2013-11-27 Thread David Kranz
On 11/27/2013 12:46 PM, Alan Pevec wrote: 2013/11/27 Sean Dague : The problem is you can't really support both iso8601 was dormant for years, and the revived version isn't compatible with the old version. So supporting both means basically forking iso8601 and maintaining you own version of it mo

[openstack-dev] [ceilometer][qa] Punting ceilometer from whitelist

2013-11-29 Thread David Kranz
In preparing to fail builds with log errors I have been trying to make things easier for projects by maintaining a whitelist. But these bugs in ceilometer are coming in so fast that I can't keep up. So I am just putting ".*" in the white list for any cases I find before gate failing is turned

[openstack-dev] [qa] Pinging people for reviews in IRC

2013-11-29 Thread David Kranz
Folks, I understand that the review latency can be too long. We just added two core reviewers and I am sure we can do better still. But please, if you feel you must ping some one by name for a review, do so in #openstack-qa rather than pinging on a private channel. That way other people might s

[openstack-dev] [all-projects] PLEASE READ: Code to fail builds on new log errors is merging

2013-12-02 Thread David Kranz
As mentioned several times on this list, https://review.openstack.org/#/c/58848/ is in the process of merging. Once it does, builds will start failing if there are log ERRORs that are not filtered out by https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. There are too many is

[openstack-dev] [ceilometer][qa] log errors

2013-12-02 Thread David Kranz
So the last two bugs I filed were coming from check builds that were not merged. It was necessary to look at all builds to get to the point where gate failing on errors could be turned on. Here is the current whitelist. If you folks think the ".*' entries are no longer needed then please let me

Re: [openstack-dev] [ceilometer][qa] Punting ceilometer from whitelist

2013-12-02 Thread David Kranz
On 12/02/2013 10:24 AM, Julien Danjou wrote: On Fri, Nov 29 2013, David Kranz wrote: In preparing to fail builds with log errors I have been trying to make things easier for projects by maintaining a whitelist. But these bugs in ceilometer are coming in so fast that I can't keep up. So

Re: [openstack-dev] [ceilometer][qa] Punting ceilometer from whitelist

2013-12-02 Thread David Kranz
On 12/02/2013 10:24 AM, Julien Danjou wrote: On Fri, Nov 29 2013, David Kranz wrote: In preparing to fail builds with log errors I have been trying to make things easier for projects by maintaining a whitelist. But these bugs in ceilometer are coming in so fast that I can't keep up. So

Re: [openstack-dev] [ceilometer][qa] Punting ceilometer from whitelist

2013-12-03 Thread David Kranz
On 12/03/2013 09:30 AM, Eoghan Glynn wrote: - Original Message - On 12/02/2013 10:24 AM, Julien Danjou wrote: On Fri, Nov 29 2013, David Kranz wrote: In preparing to fail builds with log errors I have been trying to make things easier for projects by maintaining a whitelist. But

Re: [openstack-dev] Tool for detecting commonly misspelled words

2013-12-03 Thread David Kranz
On 12/03/2013 02:05 PM, John Griffith wrote: On Tue, Dec 3, 2013 at 11:54 AM, Nachi Ueno wrote: 2013/12/3 John Griffith : On Tue, Dec 3, 2013 at 11:38 AM, Russell Bryant wrote: On 12/03/2013 09:22 AM, Joe Gordon wrote: HI all, Recently I have seen a few patches fixing a few typos. I would

Re: [openstack-dev] [qa] Moving the QA meeting time

2013-12-05 Thread David Kranz
On 12/05/2013 07:16 AM, Sean Dague wrote: On 12/05/2013 02:37 AM, Koderer, Marc wrote: Hi all! -Ursprüngliche Nachricht- Von: Kenichi Oomichi [mailto:oomi...@mxs.nes.nec.co.jp] Gesendet: Donnerstag, 5. Dezember 2013 01:37 An: OpenStack Development Mailing List (not for usage questions)

Re: [openstack-dev] [qa][nova] The document for the changes from Nova v2 api to v3

2013-12-06 Thread David Kranz
On 11/13/2013 06:09 PM, Christopher Yeoh wrote: On Thu, Nov 14, 2013 at 7:52 AM, David Kranz <mailto:dkr...@redhat.com>> wrote: On 11/13/2013 08:30 AM, Alex Xu wrote: Hi, guys This is the document for the changes from Nova v2 api to v3: https://wiki.openstack

[openstack-dev] [qa] Changes slipping through the nova v2->v3 port

2013-12-06 Thread David Kranz
I have been trying to review all of the nova v3 changes. Many of these patches have been around for awhile and have not kept up with changes that were made to the v2 tests after a v2 test file was copied to v3. I think any one submitting a patch to the nova v2 test code needs to file a bug tick

Re: [openstack-dev] [qa][nova] The document for the changes from Nova v2 api to v3

2013-12-06 Thread David Kranz
, these schemas only have info for the json dict part, not any changes to the url suffix or return values. Second, there really needs to be a release note to help users upgrade their apps from v2 to v3. -David --- 2013/12/7 David Kranz : On 11/13/2013 06:09 PM, Christopher Yeoh wrote: On

[openstack-dev] [ironic][qa] How will ironic tests run in tempest?

2013-12-06 Thread David Kranz
It's great that tempest tests for ironic have been submitted! I was reviewing https://review.openstack.org/#/c/48109/ and noticed that the tests do not actually run. They are skipped because baremetal is not enabled. This is not terribly surprising but we have had a policy in tempest to only me

Re: [openstack-dev] [ironic][qa] How will ironic tests run in tempest?

2013-12-10 Thread David Kranz
On 12/09/2013 01:37 PM, Devananda van der Veen wrote: On Fri, Dec 6, 2013 at 2:13 PM, Clark Boylan <mailto:clark.boy...@gmail.com>> wrote: On Fri, Dec 6, 2013 at 1:53 PM, David Kranz mailto:dkr...@redhat.com>> wrote: > It's great that tempest tests for ironic hav

Re: [openstack-dev] [qa] [Solum] [tempest] Use of pecan test framework in functional tests

2013-12-10 Thread David Kranz
On 12/10/2013 04:12 PM, Russell Bryant wrote: On 12/10/2013 04:10 PM, Georgy Okrokvertskhov wrote: Hi, In Solum project we are currently creating tests environments for future test. We split unit tests and functional tests in order to use tempest framework from the beginning. Tempest framework

Re: [openstack-dev] [ironic][qa] How will ironic tests run in tempest?

2013-12-11 Thread David Kranz
On 12/10/2013 08:41 PM, Devananda van der Veen wrote: Tue, Dec 10, 2013 at 12:43 PM, David Kranz <mailto:dkr...@redhat.com>> wrote: On 12/09/2013 01:37 PM, Devananda van der Veen wrote: On Fri, Dec 6, 2013 at 2:13 PM, Clark Boylan mailto:clark.boy...@gmail.com>> wro

Re: [openstack-dev] [QA] Tempest blueprints status update and rationale, input demanded

2013-12-11 Thread David Kranz
one, shall we close it? (david kranz) We decided to leave it open until the whitelist is zeroed out. https://blueprints.launchpad.net/tempest/+spec/config-cleanup https://blueprints.launchpad.net/tempest/+spec/config-verification seems done, close? (mtreinish) These are both still in progress.

[openstack-dev] [cinder][qa]

2013-12-16 Thread David Kranz
Is there a document that describes the api changes from v1 to v2, similar to the one documenting nova v2 to v3? -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder][qa] release notes for cinder v1 to v2?

2013-12-16 Thread David Kranz
Sorry for lost subject in last message. Is there a document that describes the api changes from v1 to v2, similar to the one documenting nova v2 to v3? -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.o

[openstack-dev] [savanna][qa] How will tempest tests run?

2013-12-16 Thread David Kranz
So it's great to see a submission of savanna tests for tempest. We would like to see these tests run before reviewing them. Is the intent that savanna will be enabled by default in devstack? If not, then I guess there will need to be separate savanna jobs. I see that right now there are savanna

[openstack-dev] [qa][ceilometer] Who can be a asked to help review tempest tests?

2013-12-16 Thread David Kranz
Ceilometer team, we are reviewing tempest tests and hope to see more. The tempest review team is hoping to identify some ceilometer devs who could help answer questions or provide a review if needed for ceilometer patches. Since ceilometer is new we are not all familiar with many of the details

Re: [openstack-dev] [qa][ceilometer] Who can be a asked to help review tempest tests?

2013-12-16 Thread David Kranz
on I asked this was because I didn't know it was possible to ask all of you like that! -David On Mon, Dec 16, 2013 at 5:33 PM, David Kranz <mailto:dkr...@redhat.com>> wrote: Ceilometer team, we are reviewing tempest tests and hope to see more. The tempest review team is

[openstack-dev] [qa][neutron] Please remember to mark changed tests as 'smoke'

2013-12-23 Thread David Kranz
We like all code submitted to tempest to actually run. Since the neutron gate jobs are still running only smoke tests, please mark any test that is added or whose code has changed as smoke. Note that 'smoke' has no real other meaning now since it was applied haphazardly in the first place and h

Re: [openstack-dev] [Tempest][qa] Adding tags to commit messages

2013-12-24 Thread David Kranz
On 12/24/2013 06:32 AM, Sean Dague wrote: On 12/24/2013 01:47 AM, Yair Fried wrote: Hi, Suggestion: Please consider tagging your Tempest commit messages the same way you do your mails in the mailing list Explanation: Since tempest is a single project testing multiple Openstack project we have

Re: [openstack-dev] What should I do for resolving Tempest Failure in jenkins?

2013-12-26 Thread David Kranz
On 12/26/2013 02:09 PM, Clark Boylan wrote: On Thu, Dec 26, 2013 at 10:50 AM, Anita Kuno wrote: Unfortunately this approach leads to forcing in code that may increase the bug in question, which will affect all developers. I suggest you evaluate the Jenkins logs carefully, and link to the times

Re: [openstack-dev] [Ceilometer][Tempest]Tracking of blueprints and changes

2013-12-27 Thread David Kranz
On 12/27/2013 05:27 AM, Nadya Privalova wrote: Hello guys! I hope all of you are enjoying the holidays! But I'd like to raise a Tempest question. Again. I hope this email will not be lost after vacations :) After the summit we decided to track all tests that are being created for Ceilometer i

Re: [openstack-dev] [nova] minimum review period for functional changes that break backwards compatibility

2013-12-29 Thread David Kranz
On 12/28/2013 11:14 AM, Tim Bell wrote: I think there is a need for an incompatible change review process which includes more of the community than just those performing the code reviews. This kind of change can cause a lot of disruption for those of us running clouds so it is great to see tha

Re: [openstack-dev] [Tempest][qa] Adding tags to commit messages

2013-12-29 Thread David Kranz
On 12/24/2013 06:32 AM, Sean Dague wrote: On 12/24/2013 01:47 AM, Yair Fried wrote: Hi, Suggestion: Please consider tagging your Tempest commit messages the same way you do your mails in the mailing list Explanation: Since tempest is a single project testing multiple Openstack project we have

Re: [openstack-dev] [Tempest][qa] Adding tags to commit messages

2013-12-30 Thread David Kranz
On 12/29/2013 07:45 PM, Jeremy Stanley wrote: On 2013-12-29 15:09:24 -0500 (-0500), David Kranz wrote: [...] Looking at the docs I see the warning that you can't put this in the search field so I tried putting it directly in the url like the other parameters but it was ignored. Is there i

[openstack-dev] [all project] log files and logstash

2013-12-30 Thread David Kranz
In case any one other than me didn't know this, the log files that are indexed and searchable in logstash are not the same as the set of files that you see in the logs directory in jenkins, but only those that have an entry in http://git.openstack.org/cgit/openstack-infra/config/tree/modules/op

Re: [openstack-dev] [Ceilometer][Tempest]Tracking of blueprints and changes

2013-12-30 Thread David Kranz
On 12/27/2013 05:27 AM, Nadya Privalova wrote: Hello guys! I hope all of you are enjoying the holidays! But I'd like to raise a Tempest question. Again. I hope this email will not be lost after vacations :) After the summit we decided to track all tests that are being created for Ceilometer i

Re: [openstack-dev] [Tempest][qa] Adding tags to commit messages

2013-12-30 Thread David Kranz
On Dec 30, 2013, at 11:32 AM, Jeremy Stanley wrote: > On 2013-12-30 10:25:07 -0500 (-0500), David Kranz wrote: >> I get that but it seems like this is a single field per gerrit user. >> Is that not true? If true, it is not useful because the point is >> that we want

[openstack-dev] [qa] Negative test generation

2014-01-02 Thread David Kranz
I took a little time to think some more about this and pushed a little working prototype of some ideas I had https://review.openstack.org/#/c/64733/1. Comments about the approach are most welcome. Note that I minimized any refactoring of the existing hierarchy for this prorotype. -David

Re: [openstack-dev] [nova] minimum review period for functional changes that break backwards compatibility

2014-01-03 Thread David Kranz
On 01/03/2014 08:52 AM, Thierry Carrez wrote: Tim Bell wrote: Is there a mechanism to tag changes as being potentially more appropriate for the more ops related profiles ? I'm thinking more when someone proposes a change they suspect could have an operations impact, they could highlight this

Re: [openstack-dev] [qa] Have multiple config files

2014-01-06 Thread David Kranz
On 01/06/2014 05:14 AM, Shweta Jain wrote: Hello there , I am working on Tempest and is new to it . while I am using tempest.config but would like to know is there a way I can use multiple config files in tempest . My intention to use the multiple configs is purely for running the test in m

[openstack-dev] [qa] negative test generation and testscenarios

2014-01-06 Thread David Kranz
Thanks to all who looked at https://review.openstack.org/#/c/64733/. There were a few minor issues I will address but the biggest one was the suggestion to run each variation as a separate test case using testscenarios. After looking into that I see a problem with this use case. Many of these n

Re: [openstack-dev] [nova][neutron][qa] top gate bugs: a plea for help

2014-01-12 Thread David Kranz
On 01/11/2014 05:06 PM, Russell Bryant wrote: On 01/11/2014 11:38 AM, Sean Dague wrote: 3) (still testing) https://review.openstack.org/#/c/65805/ Right now when tempest runs in the devstack-gate jobs, it runs with concurrency=4 (run 4 tests at once). Unfortunately, it appears that this maxes

Re: [openstack-dev] [nova] [qa] api schema validation pattern changes

2014-01-12 Thread David Kranz
On 01/12/2014 10:14 PM, Matthew Treinish wrote: Last, is a question, is it possible to currently run the full API an build a json schema for it all? Or to query these validating schemas? We *really* want that over in tempest, so we can completely drop the manual creation of negative testing, an

[openstack-dev] [qa] [ceilometer] Can this be fixed

2014-01-13 Thread David Kranz
A bug was introduced before the holidays that made tempest stop failing successful builds that have new log ERRORs. We are ready to close that bug but unfortunately some new ones slipped through as indicated in https://bugs.launchpad.net/ceilometer/+bug/1268730 I just filed. It would be great i

Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely

2014-01-14 Thread David Kranz
On 01/14/2014 10:37 AM, Thomas Goirand wrote: On 01/13/2014 10:38 PM, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to

Re: [openstack-dev] [devstack] libvirt default log level

2014-01-15 Thread David Kranz
On 01/15/2014 11:56 AM, Steven Dake wrote: Hi, Ken'ichi Omichi submitted a change [1] in devstack to change the default log level to 1 for libvirt. This results in continual spam to /var/log/messages in my development system, even after exiting devstack. The spam looks like: " Jan 14 08:13

Re: [openstack-dev] [QA] The future of nosetests with Tempest

2014-01-17 Thread David Kranz
On 01/16/2014 10:56 PM, Matthew Treinish wrote: Hi everyone, With some recent changes made to Tempest compatibility with nosetests is going away. We've started using newer features that nose just doesn't support. One example of this is that we've started using testscenarios and we're planning to

Re: [openstack-dev] [QA] The future of nosetests with Tempest

2014-01-17 Thread David Kranz
On 01/17/2014 10:34 AM, Matthew Treinish wrote: On Fri, Jan 17, 2014 at 08:32:19AM -0500, David Kranz wrote: On 01/16/2014 10:56 PM, Matthew Treinish wrote: Hi everyone, With some recent changes made to Tempest compatibility with nosetests is going away. We've started using newer fea

Re: [openstack-dev] [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it

2014-01-17 Thread David Kranz
On 01/17/2014 09:06 AM, Koderer, Marc wrote: Hi Julien, most of the cases in tempest/stress are already covered by exiting tests in /api or /scenario. The only thing that is missing is the decorator on them. BTW here is the Etherpad from the summit talk that we had: https://etherpad.openstack.o

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-22 Thread David Kranz
On 01/22/2014 03:19 PM, Jay Pipes wrote: On Tue, 2014-01-21 at 01:15 -0500, Yair Fried wrote: I seem to be unable to convey my point using generalization, so I will give a specific example: I would like to have "update dns server" as an additional network scenario. Currently I could add it to

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-03 Thread David Kranz
On 02/03/2014 07:36 AM, Monty Taylor wrote: On 02/02/2014 08:33 PM, Jay Pipes wrote: On Sun, 2014-02-02 at 07:13 -0500, Sean Dague wrote: Just noticed this at the end of a successful run: http://logs.openstack.org/15/63215/13/check/check-tempest-dsvm-full/2636cae/console.html#_2014-02-02_12_02

Re: [openstack-dev] [qa][tempest][rally] Rally & Tempest integration: tempest.conf autogeneration

2014-02-12 Thread David Kranz
work with tempest. We added first patch that adds base functionality to Rally [3]: https://review.openstack.org/#/c/70131/ At QA meeting I discussed it with David Kranz, as a result we agree that part of this functionality (tempest.conf generator & cloud prepare), should be implemented inside

[openstack-dev] "bad" default values in conf files

2014-02-13 Thread David Kranz
I was recently bitten by a case where some defaults in keystone.conf were not appropriate for real deployment, and our puppet modules were not providing better values https://bugzilla.redhat.com/show_bug.cgi?id=1064061. Since there are hundreds (thousands?) of options across all the services. I

[openstack-dev] [qa] Cleaning OpenStack resources

2014-02-19 Thread David Kranz
I was looking at https://review.openstack.org/#/c/73274/1 which makes it configurable whether a brute-force cleanup of resources is done after success. This got my wondering how this should really be done. As admin, there are some resources that can be cleaned and some that I don't know how. F

[openstack-dev] [qa] Re: [openstack-qa] Not Able to Run Tempest API Tests

2014-02-20 Thread David Kranz
On 02/20/2014 05:58 AM, om prakash pandey wrote: I am not able to run Tempest API tests. The typical ERROR I am getting is "Connection Timed Out". When checking into the logs I found out that tempest is trying to access the admin URL which is a private IP for our deployment. Now, Tempest is d

[openstack-dev] [qa] Does scenario.test_minimum_basic need to upload ami images?

2014-02-20 Thread David Kranz
Running this test in tempest requires an ami image triple to be on the disk where tempest is running in order for the test to upload it. It would be a lot easier if this test could use a simple image file instead. That image file could even be obtained from the cloud being tested while configur

Re: [openstack-dev] [qa] Does scenario.test_minimum_basic need to upload ami images?

2014-02-20 Thread David Kranz
On 02/20/2014 04:53 PM, Sean Dague wrote: On 02/20/2014 04:31 PM, David Kranz wrote: Running this test in tempest requires an ami image triple to be on the disk where tempest is running in order for the test to upload it. It would be a lot easier if this test could use a simple image file

[openstack-dev] [qa] Can't interpret failure from XenServer CI

2014-02-23 Thread David Kranz
XenServer reported failure on https://review.openstack.org/#/c/73704. The pointer to the logs should look like a jenkins failure but does not in two ways. First, there is no console log, with a directory of other log files next to it. Second, the .gz log files are not set up to be downloaded as

Re: [openstack-dev] [qa] Can't interpret failure from XenServer CI

2014-02-23 Thread David Kranz
_______ From: David Kranz [dkr...@redhat.com] Sent: 23 February 2014 20:06 To: #OpenStack External Email Cc: OpenStack Development Mailing List Subject: [qa] Can't interpret failure from XenServer CI XenServer reported failure on https://review.openstack.org/#/c/7370

[openstack-dev] [qa][nova] Ownership and path to schema definitions

2014-03-04 Thread David Kranz
Given that 1. there is an ongoing api discussion in which using json schemas is an important part 2. tempest now has a schema-based auto-generate feature for negative tests I think it would be a good time to have at least an initial discussion about the requirements for theses schemas and whe

Re: [openstack-dev] [QA][Tempest] Proposing Ghanshyam Mann for Tempest Core

2014-11-21 Thread David Kranz
+1 On 11/21/2014 01:26 PM, Matthew Treinish wrote: Hi Everyone, I'd like to propose we add Ghanshyam Mann (gmann) to the tempest core team. Over the past couple of cycles Ghanshyam has been actively engaged in the Tempest community. Ghanshyam has had one of the highest review counts on Tempest

[openstack-dev] [qa] branchless tempest and the use of 'all' for extensions in tempest.conf

2014-12-03 Thread David Kranz
A recent proposed test to tempest was making explicit calls to the nova extension discovery api rather than using test.requires_ext. The reason was because we configure tempest.conf in the gate as 'all' for extensions, and the test involved an extension that was new in Juno. So the icehouse run

[openstack-dev] Reason for mem/vcpu ratio in default flavors

2014-12-11 Thread David Kranz
Perhaps this is a historical question, but I was wondering how the default OpenStack flavor size ratio of 2/1 was determined? According to http://aws.amazon.com/ec2/instance-types/, ec2 defines the flavors for General Purpose (M3) at about 3.7/1, with Compute Intensive (C3) at about 1.9/1 and M

[openstack-dev] [qa] Please do not merge neutron test changes until "client returns one value" is merged

2014-12-17 Thread David Kranz
This https://review.openstack.org/#/c/141152/ gets rid of the useless second return value from neutron client methods according to this spec: https://github.com/openstack/qa-specs/blob/master/specs/clients-return-one-value.rst. Because the client and test changes have to be in the same patch, th

[openstack-dev] [qa] neutron "client returns one value" has finally merged

2014-12-19 Thread David Kranz
Neutron patches can resume as normal. Thanks for the patience. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] tempest stable/icehouse builds are broken

2014-12-29 Thread David Kranz
Some kind of regression has caused stable/icehouse builds to fail, and hence prevents any code from merging in tempest. This is being tracked at https://bugs.launchpad.net/python-heatclient/+bug/1405579. Jeremy (fungi) provided a hacky work-around here https://review.openstack.org/#/c/144347/ w

[openstack-dev] [all] Proper use of 'git review -R'

2014-12-30 Thread David Kranz
Many times when I review a revision of an existing patch, I can't see just the change from the previous version due to other rebases. The git-review documentation mentions this issue and suggests using -R to make life easier for reviewers when submitting new revisions. Can some one explain when

Re: [openstack-dev] [all] Proper use of 'git review -R'

2014-12-30 Thread David Kranz
On 12/30/2014 11:37 AM, Jeremy Stanley wrote: On 2014-12-30 09:46:35 -0500 (-0500), David Kranz wrote: [...] Can some one explain when we should *not* use -R after doing 'git commit --amend'? [...] In the standard workflow this should never be necessary. The default behavior in git

[openstack-dev] [QA] Meeting Thursday January 8th at 22:00 UTC

2015-01-07 Thread David Kranz
time 22:00 UTC is in other timezones tomorrow's meeting will be at: 17:00 EST 07:00 JST 08:30 ACDT 23:00 CET 16:00 CST 14:00 PST -David Kranz ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/m

Re: [openstack-dev] [QA] Moving tempest clients to tempest-lib (was Meeting Thursday January 8th at 22:00 UTC)

2015-01-13 Thread David Kranz
used by neutron tests either through inheritance or delegation. Is that different than your vision? -David --- 2015-01-08 2:44 GMT+09:00 David Kranz : Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, January 8th at 22:00 UTC in

[openstack-dev] [QA] Meeting Thursday January 15th at 17:00 UTC

2015-01-14 Thread David Kranz
time 17:00 UTC is in other timezones tomorrow's meeting will be at: 12:00 EST 02:00 JST 03:30 ACDT 18:00 CET 11:00 CST 09:00 PST -David Kranz __ OpenStack Development Mailing List (not for usage questions) Unsubscribe

Re: [openstack-dev] [all] Branchless Tempest QA Spec - final draft

2014-04-28 Thread David Kranz
On 04/27/2014 10:02 PM, Matthew Treinish wrote: On Mon, Apr 28, 2014 at 01:01:00AM +, Kenichi Oomichi wrote: Hi, Sorry for my late response, but I'd like to discuss this again. Now we are working for adding Nova API responses checks to Tempest[1] to block backward incompatible changes. Wit

Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

2014-04-30 Thread David Kranz
On 04/30/2014 02:22 PM, Rochelle.RochelleGrober wrote: +1 but I don't get in until late Sunday :-( Any chance you could do this sometime Monday? I'd like to meet the people behind the IRC names and email addresses. --Rocky Same here. -Original Message- From: Ken'ichi Ohmichi [m

[openstack-dev] [qa] [cinder] Do we now require schema response validation in tempest clients?

2014-04-30 Thread David Kranz
There have been a lot of patches that add the validation of response dicts. We need a policy on whether this is required or not. For example, this patch https://review.openstack.org/#/c/87438/5 is for the equivalent of 'cinder service-list' and is a basically a copy of the nova test which now

Re: [openstack-dev] [all] Branchless Tempest QA Spec - final draft

2014-05-01 Thread David Kranz
On 05/01/2014 11:36 AM, Matthew Treinish wrote: On Thu, May 01, 2014 at 06:18:10PM +0900, Ken'ichi Ohmichi wrote: # Sorry for sending this again, previous mail was unreadable. 2014-04-28 11:54 GMT+09:00 Ken'ichi Ohmichi : This is also why there are a bunch of nova v2 extensions that just add p

Re: [openstack-dev] [Openstack-dev][qa] Running tempest tests against existing openstack installation

2014-05-05 Thread David Kranz
On 05/05/2014 02:26 AM, Swapnil Kulkarni wrote: Hello, I am trying to run tempest tests against an existing openstack deployment. I have configured tempest.conf for the environment details. But when I execute run_tempest.sh, it does not run any tests. Although when I run testr run, the tests

[openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-07 Thread David Kranz
I just looked at a patch https://review.openstack.org/#/c/90310/3 which was given a -1 due to not checking that every call to list_hosts returns 200. I realized that we don't have a shared understanding or policy about this. We need to make sure that each api is tested to return the right respo

Re: [openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-07 Thread David Kranz
On 05/07/2014 10:07 AM, Duncan Thomas wrote: On 7 May 2014 14:53, David Kranz wrote: I just looked at a patch https://review.openstack.org/#/c/90310/3 which was given a -1 due to not checking that every call to list_hosts returns 200. I realized that we don't have a shared understandi

Re: [openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-08 Thread David Kranz
On 05/07/2014 10:48 AM, Ken'ichi Ohmichi wrote: Hi Sean, 2014-05-07 23:28 GMT+09:00 Sean Dague : On 05/07/2014 10:23 AM, Ken'ichi Ohmichi wrote: Hi David, 2014-05-07 22:53 GMT+09:00 David Kranz : I just looked at a patch https://review.openstack.org/#/c/90310/3 which was given a

Re: [openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-09 Thread David Kranz
On 05/09/2014 11:29 AM, Matthew Treinish wrote: On Thu, May 08, 2014 at 09:50:03AM -0400, David Kranz wrote: On 05/07/2014 10:48 AM, Ken'ichi Ohmichi wrote: Hi Sean, 2014-05-07 23:28 GMT+09:00 Sean Dague : On 05/07/2014 10:23 AM, Ken'ichi Ohmichi wrote: Hi David, 2014-05-07 22:53

[openstack-dev] [qa][nova] Status of v3 tests in tempest

2014-05-19 Thread David Kranz
It seems the nova team decided in Atlanta that "v3" as currently understood is never going to exist: https://etherpad.openstack.org/p/juno-nova-v3-api. There are a number of patches in flight that tweak how we handle supporting both v2/v3 in tempest to reduce duplication. We need to decide what

  1   2   3   >