-----Original Message----- From: Sean Dague <s...@dague.net> Reply: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org> Date: August 9, 2016 at 11:21:47 To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org> Subject: Re: [openstack-dev] [requirements] History lesson please
> On 08/09/2016 11:25 AM, Matthew Thode wrote: > > On 08/09/2016 10:22 AM, Ian Cordasco wrote: > >> -----Original Message----- > >> From: Matthew Thode > >> Reply: prometheanf...@gentoo.org , OpenStack Development > Mailing List (not for usage questions) > >> Date: August 9, 2016 at 09:53:53 > >> To: openstack-dev@lists.openstack.org > >> Subject: Re: [openstack-dev] [requirements] History lesson please > >> > >>> One of the things on our todo list is to test the 'lower-constraints' to > >>> make sure they still work with the head of branch. > >> > >> That's not sufficient. You need to find versions in between the lowest > >> tested version > and the current version to also make sure you don't end up with breakage. You > might have > somepackage that has a lower version of 2.0.1 and a current constraint of > 2.12.3. You > might even have a blacklist of versions in between those two versions, but > you still need > other versions to ensure that things in between those continue to work. > >> > >> THe tiniest of accidental incompatibilities can cause some of the most > >> bizarre bugs. > >> > >> -- > >> Ian Cordasco > >> > > > > I'm aware of this, but this would be a good start. > > And, more importantly, assuming that testing is only valid if it covers > every scenario, sets the bar at entirely the wrong place. > > A lower bound test would eliminate some of the worst fiction we've got. > Testing is never 100%. With a complex system like OpenStack, it's > probably not even 1% (of configs matrix for sure). But picking some > interesting representative scenarios and seeing that it's not completely > busted is worth while. Right. I'm not advocating for testing every released version of a dependency. In general, it's good to test versions that have *triggered* changes though. If upgrading from 2.3.0 to 2.4.1 caused you to need to fix something, test something earlier than 2.4.1, and 2.4.1, and then something later. That's what I'm advocating for. -- Ian Cordasco __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
