On Mon, Apr 13, 2015 at 9:12 AM, Monty Taylor <mord...@inaugust.com> wrote:
> On 04/12/2015 06:43 PM, Robert Collins wrote: > > Right now we do something that upstream pip considers wrong: we make > > our requirements.txt be our install_requires. > > > > Upstream there are two separate concepts. > > > > install_requirements, which are meant to document what *must* be > > installed to import the package, and should encode any mandatory > > version constraints while being as loose as otherwise possible. E.g. > > if package A depends on package B version 1.5 or above, it should say > > B>=1.5 in A's install_requires. They should not specify maximum > > versions except when that is known to be a problem: they shouldn't > > borrow trouble. > > > > deploy requirements - requirements.txt - which are meant to be *local > > to a deployment*, and are commonly expected to specify very narrow (or > > even exact fit) versions. > That sounds, to me, very similar to a discussion we had a few weeks ago in the context of our stable branches. In that context, we have two competing requirements. One requirement is that our CI system wants a very tightly pinned requirements, as do downstream CI systems and deployers that want to test and deploy exact known-tested versions of things. On the other hand, downstream distributors (including OS packagers) need to balance OpenStack's version requirements with version requirements from all the other packages in their distribution; the tighter the requirements we list are, the harder it is for the requirements to work with the requirements of other packages in the distribution. > tl;dr - I'm mostly in support of what you're saying - but I'm going to > bludgeon it some. > > To be either fair or unfair, depending on how you look at it - some > people upstream consider those two to be a pattern, but it is not > encoded anywhere except in hidden lore that is shared between secret > people. Upstream's tools have bumpkiss for support for this, and if we > hadn't drawn a line in the sand encoding SOME behavior there would still > be nothing. > > Nor, btw, is it the right split. It optimizes for the wrong thing. > > rust gets it wright. There is a Cargo.toml and a Cargo.lock, which are > understood by the tooling in a manner similar to what you have > described, and it is not just understood but DOCUMENTED that an > _application_ should ship with a Cargo.lock and a _library_ should not. > This sounds similar to a solution that was proposed for the stable branches: a requirements.in with mandatory version constraints while being as loose as otherwise possible, which is used to generate a requirements.txt which has the "local to deployment" exact versions that are used in our CI. The details of the proposal are in https://review.openstack.org/#/c/161047/ > Without the library/application distinction, the effort in > differentiating is misplaced, I believe - because it's libraries that > need flexible depends - and applications where the specific set of > depends that were tested in CI become important to consumers. > > > What pbr, which nearly if not all OpenStack projects use, does, is to > > map the contents of requirements.txt into install_requires. And then > > we use the same requirements.txt in our CI to control whats deployed > > in our test environment[*]. and there we often have tight constraints > > like seen here - > > > http://git.openstack.org/cgit/openstack/requirements/tree/global-requirements.txt#n63 > > That is, btw, because that's what the overwhelming majority of consumers > assume those files mean. I take "overwhelming majority" from the days > when we had files but did not process them automatically and everyone > was confused. > > > I'd like to align our patterns with those of upstream, so that we're > > not fighting our tooling so much. > > Ok. I mean, they don't have a better answer, but if it makes "python" > hate us less, sweet. > > > Concretely, I think we need to: > > - teach pbr to read in install_requires from setup.cfg, not > requirements.txt > > - when there are requirements in setup.cfg, stop reading > requirements.txt > > - separate out the global intall_requirements from the global CI > > requirements, and update our syncing code to be aware of this > > > > Then, setup.cfg contains more open requirements suitable for being on > > PyPI, requirements.txt is the local CI set we know works - and can be > > much more restrictive as needed. > > > > Thoughts? If there's broad apathy-or-agreement I can turn this into a > > spec for fine coverage of ramifications and corner cases. > > I'm concerned that it adds a layer of difference that is confusing to > people for the sole benefit of pleasing someone else's pedantic worldview. > > I'm also concerned that dstufft is actively wanting to move towards a > world where the build tooling is not needed or used as part of the > install pipeline (metadata 2.0) -- so I'm concerned that we're aligning > with a pattern that isn't very robust and isn't supported by tooling > directly and that we're going to need to change understood usage > patterns across a large developer based to chase something that _still_ > isn't going to be "how people do it" > > I'm concerned that "how people do it" is a myth not worth chasing. > > I'm not _opposed_ to making this richer and more useful for people. I > just don't know what's broken currently for us. > To be clear, I don't mean to suggest that the solution proposed in https://review.openstack.org/#/c/161047/ is necessarily the correct solution to this problem - but I do think that it is illustrative of at last one thing that's currently broken for us. > Monty > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
