On 08/09/2016 11:25 AM, Matthew Thode wrote:
> On 08/09/2016 10:22 AM, Ian Cordasco wrote:
>> -----Original Message-----
>> From: Matthew Thode <prometheanf...@gentoo.org>
>> Reply: prometheanf...@gentoo.org <prometheanf...@gentoo.org>, OpenStack
>> Development Mailing List (not for usage questions)
>> <openstack-dev@lists.openstack.org>
>> Date: August 9, 2016 at 09:53:53
>> To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
>> Subject: Re: [openstack-dev] [requirements] History lesson please
>>
>>> One of the things on our todo list is to test the 'lower-constraints' to
>>> make sure they still work with the head of branch.
>>
>> That's not sufficient. You need to find versions in between the lowest
>> tested version and the current version to also make sure you don't end up
>> with breakage. You might have somepackage that has a lower version of 2.0.1
>> and a current constraint of 2.12.3. You might even have a blacklist of
>> versions in between those two versions, but you still need other versions to
>> ensure that things in between those continue to work.
>>
>> THe tiniest of accidental incompatibilities can cause some of the most
>> bizarre bugs.
>>
>> --
>> Ian Cordasco
>>
>
> I'm aware of this, but this would be a good start.
And, more importantly, assuming that testing is only valid if it covers
every scenario, sets the bar at entirely the wrong place.
A lower bound test would eliminate some of the worst fiction we've got.
Testing is never 100%. With a complex system like OpenStack, it's
probably not even 1% (of configs matrix for sure). But picking some
interesting representative scenarios and seeing that it's not completely
busted is worth while.
-Sean
--
Sean Dague
http://dague.net
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
