On Tue, 28 Aug 2018, Matt Riedemann wrote:
Are people okay with that and willing to commit to being okay with that answer in reviews? To some extent we need to have some faith on the end result: the tests work. If people are not okay with that, we need the people who are not to determine and prove the alternate strategy. I've had this one work and work well.Seems reasonable to me. But to be clear, if there are 70 failed tests, are you going to have 70 separate patches? Or this is just one of those things where you start with 70, fix something, get down to 50 failed tests, and iterate until you're down to all passing. If so, I'm OK with that. It's hard to say without knowing how many patches get from 70 failures to 0 and what the size/complexity of those changes is, but without knowing I'd default to the incremental approach for ease of review.
It's lumpy. But at least at the begining it will be something like: 0 passing, stil 0 passing; still 0 passing; still 0 passing; 150 passing, 700 failing; 295 passing, X failing, etc. Because in the early stages, test discovery and listing doesn't work at all, for quite a few different reasons. Based on the discussion here, resolving those "different reasons" is things people want to see in different commits. One way to optimize this (if people preferred) would be to not use stestr as called by tox, with its built in test discovery, but instead run testtools or subunit in a non-parallel and failfast where not all tests need to be discovered first. That would provide a more visible sense of "it's getting better" to someone who is running the tests locally using that alternate method, but would not do much for the jobs run by zuul, so probably not all that useful. Thanks for the other info on the devstack and grenade stuff. If I read you right, from your perspective it's a case of "we'll see" and "we'll figure it out", which sounds good to me. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
