On 18-03-15 19:29:37, Doug Hellmann wrote: > Excerpts from Matthew Thode's message of 2018-03-15 10:24:10 -0500: > > On 18-03-15 07:03:11, Doug Hellmann wrote: > > > What I Want to Do > > > ----------------- > > > > > > 1. Update the requirements-check test job to change the check for > > > an exact match to be a check for compatibility with the > > > upper-constraints.txt value. > > > > > > We would check the value for the dependency from upper-constraints.txt > > > against the range of allowed values in the project. If the > > > constraint version is compatible, the dependency range is OK. > > > > > > This rule means that in order to change the dependency settings > > > for a project in a way that are incompatible with the constraint, > > > the constraint (and probably the global requirements list) would > > > have to be changed first in openstack/requirements. However, if > > > the change to the dependency is still compatible with the > > > constraint, no change would be needed in openstack/requirements. > > > For example, if the global list constraints a library to X.Y.Z > > > and a project lists X.Y.Z-2 as the minimum version but then needs > > > to raise that because it needs a feature in X.Y.Z-1, it can do > > > that with a single patch in-tree. > > > > > > > I think what may be better is for global-requirements to become a > > gathering place for projects that requirements watches to have their > > smallest constrainted installable set defined in. > > > > Upper-constraints has a req of foo===2.0.3 > > Project A has a req of foo>=1.0.0,!=1.6.0 > > Project B has a req of foo>=1.4.0 > > Global reqs would be updated with foo>=1.4.0,!=1.6.0 > > Project C comes along and sets foo>=2.0.0 > > Global reqs would be updated with foo>=2.0.0 > > > > This would make global-reqs descriptive rather than prescriptive for > > versioning and would represent the 'true' version constraints of > > openstack. > > It sounds like you're suggesting syncing in the other direction, which > could be useful. I think we can proceed with what I've described and > consider the work to build what you describe as a separate project. >
Yes, this would be a follow-on thing. > > > > > We also need to change requirements-check to look at the exclusions > > > to ensure they all appear in the global-requirements.txt list > > > (the local list needs to be a subset of the global list, but > > > does not have to match it exactly). We can't have one project > > > excluding a version that others do not, because we could then > > > end up with a conflict with the upper constraints list that could > > > wedge the gate as we had happen in the past. > > > > > > > How would this happen when using constraints? A project is not allowed > > to have a requirement that masks a constriant (and would be verified via > > the requirements-check job). > > If project A excludes version X before the constraint list is updated to > use it, and then project B starts trying to depend on version X, they > become incompatible. > > We need to continue to manage our declarations of incompatible versions > to ensure that the constraints list is a good list of versions to test > everything under. > > > There's a failure mode not covered, a project could add a mask (!=) to > > their requirements before we update constraints. The project that was > > passing the requirements-check job would then become incompatable. This > > means that the requirements-check would need to be run for each > > changeset to catch this as soon as it happens, instead of running only > > on requirements changes. > > I'm not clear on what you're describing here, but it sounds like a > variation of the failure modes that would be prevented if we require > exclusions to exist in the global list before they could be added to the > local list. > Yes, that'd work (require exclusions to be global before local). > > > > > We also need to verify that projects do not cap dependencies for > > > the same reason. Caps prevent us from advancing to versions of > > > dependencies that are "too new" and possibly incompatible. We > > > can manage caps in the global requirements list, which would > > > cause that list to calculate the constraints correctly. > > > > > > This change would immediately allow all projects currently > > > following the global requirements lists to specify different > > > lower bounds from that global list, as long as those lower bounds > > > still allow the dependencies to be co-installable. (The upper > > > bounds, managed through the upper-constraints.txt list, would > > > still be built by selecting the newest compatible version because > > > that is how pip's dependency resolver works.) > > > > > > 2. We should stop syncing dependencies by turning off the > > > propose-update-requirements job entirely. > > > > > > Turning off the job will stop the bot from proposing more > > > dependency updates to projects. > > > > > > As part of deleting the job we can also remove the "requirements" > > > case from playbooks/proposal/propose_update.sh, since it won't > > > need that logic any more. We can also remove the update-requirements > > > command from the openstack/requirements repository, since that > > > is the tool that generates the updated list and it won't be > > > needed if we aren't proposing updates any more. > > > > > > 3. Remove the minimum specifications from the global requirements > > > list to make clear that the global list is no longer expressing > > > minimums. > > > > > > This clean-up step has been a bit more controversial among the > > > requirements team, but I think it is a key piece. As the minimum > > > versions of dependencies diverge within projects, there will no > > > longer *be* a real global set of minimum values. Tracking a list of > > > "highest minimums", would either require rebuilding the list from the > > > settings in all projects, or requiring two patches to change the > > > minimum version of a dependency within a project. > > > > > > Maintaining a global list of minimums also implies that we > > > consider it OK to run OpenStack as a whole with that list. This > > > message conflicts with the message we've been sending about the > > > upper constraints list since that was established, which is that > > > we have a known good list of versions and deploying all of > > > OpenStack with different versions of those dependencies is > > > untested. > > > > > > > As noted above I think that gathering the min versions/maskings from > > openstack projects to be valuable (especially to packagers who already > > use our likely invalid values already). > > OK. I don't feel that strongly about the cleanup work, so if we want to > keep the lower bounds in place I think that's OK. > > > > > > After these 3 steps are done, the requirements team will continue > > > to maintain the global-requirements.txt and upper-constraints.txt > > > files, as before. Adding a new dependency to a project will still > > > involve a review step to add it to the global list so we can monitor > > > licensing, duplication, python 3 support, etc. But adjusting the > > > version numbers once that dependency is in the global list will be > > > easier. > > > > > > > Thanks for writing this up, I think it looks good in general, but like > > you mentioned before, there is some discussion to be had about gathering > > and creating a versionspec from all of openstack for requirements. > > > -- Matthew Thode (prometheanfire)
signature.asc
Description: PGP signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
