I would say smoke tests, and at least the minimum scenario tests. Smoke tests takes 14 minutes (113 tests) to run, and I can check how long it takes the minimum scenario tests later. So it won't take a long time running.
Kind regards, Arx Cruz On Thu, Apr 6, 2017 at 2:44 PM, Andrea Frittoli <andrea.fritt...@gmail.com> wrote: > I don't really have much context in what the decision is going to be based > on here, > so I'll just add some random comments here and there. > > On Thu, Apr 6, 2017 at 12:48 PM Arx Cruz <arxc...@redhat.com> wrote: > >> Having tempest running will allow these jobs to appear in >> openstack-health system as well. >> > > I agree that's a plus. It's also rather easy to produce subunit from > whatever you > are using to run tests, and that's all you need in fact to get data into > open stack-health > without touching the existing infrastructure. So in case you decide not to > use Tempest, > open stack-health can still be on the list. > > >> >> On Thu, Apr 6, 2017 at 1:29 PM, Justin Kilpatrick <jkilp...@redhat.com> >> wrote: >> >> Maybe I'm getting a little off topic with this question, but why was >> Tempest removed last time? >> >> I'm not well versed in the history of this discussion, but from what I >> understand Tempest in the gate has >> been an off and on again thing for a while but I've never heard the >> story of why it got removed. >> >> On Thu, Apr 6, 2017 at 7:00 AM, Chris Dent <cdent...@anticdent.org> >> wrote: >> > On Thu, 6 Apr 2017, Sagi Shnaidman wrote: >> > >> >> It may be useful to run a "limited edition" of API tests that maximize >> >> coverage and don't duplicate, for example just to check service working >> >> basically, without covering all its functionality. It will take very >> >> little >> >> time (i.e. 5 tests for each service) and will give a general picture of >> >> deployment success. It will cover fields that are not covered by >> pingtest >> >> as well. >> >> > >> >> > We have a smoke attribute here an there, but it's not well curated at all, > so you're > probably better off maintaining your own list. > Since presumably you're more interested in verifying that a deployed cloud > is > functional - as opposed to verify specific APIs are working properly - you > may want > to look at scenario tests, where with a couple of test you can cover > already a lot of > basic stuff, e.g. if you can boot a server from a volume with an image > from glance, > and ssh into it, you have proven a lot already about the general health of > your cloud. > > >> > >> > It's sound like using some parts of tempest is perhaps the desired >> > thing here but in case a "limited edition" test against the APIs to >> > do what amounts to a smoke test is desired, it might be worthwhile >> > to investigate using gabbi[1] and its command line gabbi-run[2] tool for >> > some fairly simple and readable tests that can describe a sequence >> > of API interactions. There are lots of tools that can do the same >> > thing, so gabbi may not be the right choice but it's there as an >> > option. >> > >> > The telemetry group had (an may still have) some integration tests >> > that use gabbi files to integrate ceilometer, heat (starting some >> > vms), aodh and gnocchi and confirm that the expected flow happened. >> > Since the earlier raw scripts I think there's been some integration >> > with tempest, but gabbi files are still used[3]. >> > >> > If this might be useful and I can help out, please ask. >> > >> > [1] http://gabbi.readthedocs.io/ >> > [2] http://gabbi.readthedocs.io/en/latest/runner.html >> > [3] >> > https://github.com/openstack/ceilometer/tree/master/ >> ceilometer/tests/integration >> > >> > -- >> > Chris Dent ¯\_(ツ)_/¯ https://anticdent.org/ >> > freenode: cdent tw: @anticdent >> > ____________________________________________________________ >> ______________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: >> unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> >> ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: >> unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: >> unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
