On 10/15/2015 05:29 PM, Robert Collins wrote: > On 16 October 2015 at 11:21, Matthew Thode <prometheanf...@gentoo.org> wrote: >> On 10/15/2015 05:12 PM, Robert Collins wrote: >>> On 16 October 2015 at 08:10, Matthew Thode <prometheanf...@gentoo.org> >>> wrote: >>>> On 10/15/2015 02:04 PM, Robert Collins wrote: >>> ... >>>>>> Where are my caps? >>>>> >>>>> The known good versions of dependencies for liberty are >>>>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty >>>>> >>>> That's good, does that represent an upper constraint for the lower >>>> constraint imposed by by the package? Why is this kept separate? >>>> Keeping it separate means that it's not trivial to merge them with >>>> what's in each package's requirements. >>> >>> It represents *one* known good combination of packages. We know that >>> that combination passed CI, and we then do all our tests with it. To >>> change it, we run it past CI and only move onto using the new set when >>> its passed CI. >>> >>> If we merged it to each packages requirements, we'd reintroduce the >>> deadlock that caused so much grief only 6 months back - I don't see >>> any reason, desire, or tolerance for doing that upstream. >>> >>> Its kept separate because it requires 2N commits to shift known-good >>> caps around for N repos using per-repo rules. >>> >>> With hundreds of repositories, it takes us hundreds of commits in two >>> batches - and a round trip time of 2 hours per batch (check + gate) to >>> shift *a single* requirement. With hundreds of dependencies, thats an >>> intolerable amount of overhead. >>> >> >> ya, that is annoying, unfortunately packages don't need to know *ONE* >> good combination, they need to know them all, or at least the cap. What >> you are basically doing is shifting all of this extra work onto the >> packagers, in fact I wouldn't be surprised if they needed to start >> vendoring all of openstack in a virtualenv instead of doing actual packages. > > Here's an example of the havoc caps cause: > https://review.openstack.org/#/c/234862/ > > I don't understand the statement about shifting work. Previously the > situation was that you had to guess whether a given release of a > dependency (both internal and external) worked with e.g. liberty. > > Now there is an explicit list of what works. > > Isn't that *better* ? > Yes, it's good to know what works, does that show multiple versions of the same package when multiple are known to work? If so I can build a range from that. It's not better as it is because I still don't know where liberty ends and mitaka begins. Is there any place I can find that?
>> My question remains though, if someone pip installs nova liberty, will >> it pull in deps from mitaka? As it is now, without caps I cannot >> reliably package openstack, do you have a solution? I should probably >> start removing the liberty packages I did package since upstream seems >> so hostile... > > If you don't use the constraints file (which is pip consumable and > published on the web so it can be used with pip install trivially) > then yes, it will install the latest versions of all packages which > are presumed to be doing backwards compatible changes. Things that we > *know* are going to be broken still get caps - and we accept the cost > of the 2-step dance needed to raise them. > What about a daily or weekly check without the constraints file so you know when something breaks? This would allow you to at least know when you need to place caps and I could consume that. I'm fine with leaving caps off. If I can consume mitaka deps in liberty then that's great :D > The big question here is, I think, 'should we assume OpenStack > originated dependencies are better or worse than third party > dependencies?' Third party dependencies we presume are backwards > compatible (and will thus only break by mistake, which constraints > guards against) unless we have reason not to - and we have open ended > deps on them. This is where the spec about clients libraries is aimed. > > It makes me very sad to know that you consider what we've done as > hostile. We did it for a bunch of good reasons: > > - safer release process > - smoother updates for [apt, rpm at least] redistributors > - faster turnaround of dependency usage in CI > - step on the path to testing lower bounds > - step on the path to alignment with upstream packaging tooling > > -Rob > > It's a step backwards from my perspective, before I had a clear delineation where support of something stops. Now I don't know when it stops and any given update could break the system. I'm not sure how it could be smoother for other systems. So, does this mean that I can just leave the packages uncapped and know that they will work? Are there tests being run for this scenario? -- Matthew Thode (prometheanfire) __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
