We haven't got much feedback from TripleO core reviewers, who are usually more involved on this topic. I'll give a chance to let them talk because we take some actions based on the feedback brought in this discussion.
Thanks, On Fri, Apr 7, 2017 at 4:26 AM, Ravi Sekhar Reddy Konda <ravisekhar.ko...@oracle.com> wrote: > Hi Diarmuid, > > Even in our setup Sravanthi by mistake has deleted all the ports > we are trying to bring up again, if it is done today wil ping you and > schedule again > > Otherwise i will schedule for Monday > > Thanks, > Ravi > ----- Original Message ----- > From: jtale...@redhat.com > To: openstack-dev@lists.openstack.org, jkilp...@redhat.com > Sent: Thursday, April 6, 2017 4:15:58 PM GMT +05:30 Chennai, Kolkata, Mumbai, > New Delhi > Subject: Re: [openstack-dev] [tripleo] pingtest vs tempest > > On Thu, Apr 6, 2017 at 5:32 AM, Sagi Shnaidman <sshna...@redhat.com> wrote: >> HI, >> >> I think Rally or Browbeat and other performance oriented solutions won't >> serve our needs, because we run TripleO CI on virtualized environment with >> very limited resources. Actually we are pretty close to full utilizing these >> resources when deploying openstack, so very little is available for test. >> It's not a problem to run tempest API tests because they are cheap - take >> little time, little resources, but also gives little coverage. Scenario test >> are more interesting and gives us more coverage, but also takes a lot of >> resources (which we don't have sometimes). > > Sagi, > In my original message I mentioned a "targeted" test, I should > explained that more. We could configure the specific scenario so that > the load on the virt overcloud would be minimal. Justin Kilpatrick > already have Browbeat integrated with TripleO Quickstart[1], so there > shouldn't be much work to try this proposed solution. > >> >> It may be useful to run a "limited edition" of API tests that maximize >> coverage and don't duplicate, for example just to check service working >> basically, without covering all its functionality. It will take very little >> time (i.e. 5 tests for each service) and will give a general picture of >> deployment success. It will cover fields that are not covered by pingtest as >> well. >> >> I think could be an option to develop a special scenario tempest tests for >> TripleO which would fit our needs. > > I haven't looked at Tempest in a long time, so maybe its functionality > has improved. I just saw the opportunity to integrate Browbeat/Rally > into CI to test the functionality of OpenStack services, while also > capturing performance metrics. > > Joe > > [1] > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_browbeat_tree_master_ci-2Dscripts&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=Z0jGFw40ezDmSb3F6ns5SXRvacH6AgU0TK5dKBSRgEs&e= > >> >> Thanks >> >> >> On Wed, Apr 5, 2017 at 11:49 PM, Emilien Macchi <emil...@redhat.com> wrote: >>> >>> Greetings dear owls, >>> >>> I would like to bring back an old topic: running tempest in the gate. >>> >>> == Context >>> >>> Right now, TripleO gate is running something called pingtest to >>> validate that the OpenStack cloud is working. It's an Heat stack, that >>> deploys a Nova server, some volumes, a glance image, a neutron network >>> and sometimes a little bit more. >>> To deploy the pingtest, you obviously need Heat deployed in your >>> overcloud. >>> >>> == Problems: >>> >>> Although pingtest has been very helpful over the last years: >>> - easy to understand, it's an Heat template, like an OpenStack user >>> would do to deploy their apps. >>> - fast: the stack takes a few minutes to be created and validated >>> >>> It has some limitations: >>> - Limitation to what Heat resources support (example: some OpenStack >>> resources can't be managed from Heat) >>> - Impossible to run a dynamic workflow (test a live migration for example) >>> >>> == Solutions >>> >>> 1) Switch pingtest to Tempest run on some specific tests, with feature >>> parity of what we had with pingtest. >>> For example, we could imagine to run the scenarios that deploys VM and >>> boot from volume. It would test the same thing as pingtest (details >>> can be discussed here). >>> Each scenario would run more tests depending on the service that they >>> run (scenario001 is telemetry, so it would run some tempest tests for >>> Ceilometer, Aodh, Gnocchi, etc). >>> We should work at making the tempest run as short as possible, and the >>> close as possible from what we have with a pingtest. >>> >>> 2) Run custom scripts in TripleO CI tooling, called from the pingtest >>> (heat template), that would run some validations commands (API calls, >>> etc). >>> It has been investigated in the past but never implemented AFIK. >>> >>> 3) ? >>> >>> I tried to make this text short and go straight to the point, please >>> bring feedback now. I hope we can make progress on $topic during Pike, >>> so we can increase our testing coverage and detect deployment issues >>> sooner. >>> >>> Thanks, >>> -- >>> Emilien Macchi >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e= >> >> >> >> >> -- >> Best regards >> Sagi Shnaidman >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e= >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e= > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
