Excerpts from Monty Taylor's message of 2017-10-05 08:45:52 -0500: > On 09/30/2017 10:40 AM, Doug Hellmann wrote: > > Excerpts from Monty Taylor's message of 2017-09-30 10:20:08 -0500: > >> Hey everybody, > >> > >> Oh goodie, I can hear you say, let's definitely spend some time > >> bikeshedding about specific command invocations related to building docs > >> and tarballs!!! > >> > >> tl;dr I want to change the PTI for docs and tarball building to be less > >> OpenStack-specific > >> > >> The Problem > >> =========== > >> > >> As we work on Zuul v3, there are a set of job definitions that are > >> rather fundamental that can totally be directly shared between Zuul > >> installations whether those Zuuls are working with OpenStack content or > >> not. As an example "tox -epy27" is a fairly standard thing, so a Zuul > >> job called "tox-py27" has no qualities specific to OpenStack and could > >> realistically be used by anyone who uses tox to manage their project. > >> > >> Docs and Tarballs builds for us, however, are the following: > >> > >> tox -evenv -- python setup.py sdist > >> tox -evenv -- python setup.py build_sphinx > >> > >> Neither of those are things that are likely to work outside of > >> OpenStack. (The 'venv' tox environment is not a default tox thing) > >> > >> I'm going to argue neither of them are actually providing us with much > >> value. > >> > >> Tarball Creation > >> ================ > >> > >> Tarball creation is super simple. setup_requires is already handled out > >> of band of everything else. Go clone nova into a completely empty system > >> and run python setup.py sdist ... and it works. (actually, nova is big. > >> use something smaller like gertty ...) > >> > >> docker run -it --rm python bash -c 'git clone \ > >> https://git.openstack.org/openstack/gertty && cd gertty \ > >> && python setup.py sdist' > >> > >> There is not much value in that tox wrapper - and it's not like it's > >> making it EASIER to run the command. In fact, it's more typing. > >> > >> I propose we change the PTI from: > >> > >> tox -evenv python setup.py sdist > >> > >> to: > >> > >> python setup.py sdist > >> > >> and then change the gate jobs to use the non-tox form of the command. > >> > >> I'd also like to further change it to be explicit that we also build > >> wheels. So the ACTUAL commands that the project should support are: > >> > >> python setup.py sdist > >> python setup.py bdist_wheel > >> > >> All of our projects support this already, so this should be a no-op. > >> > >> Notes: > >> > >> * Python projects that need to build C extensions might need their pip > >> requirements (and bindep requirements) installed in order to run > >> bdist_wheel. We do not support that broadly at the moment ANYWAY - so > >> I'd like to leave that as an outlier and handle it when we need to > >> handle it. > >> > >> * It's *possible* that somewhere we have a repo that has somehow done > >> something that would cause python setup.py sdist or python setup.py > >> bdist_wheel to not work without pip requirements installed. I believe we > >> should consider that a bug and fix it in the project if we find such a > >> thing - but since we use pbr in all of the OpenStack projects, I find it > >> extremely unlikely. > >> > >> Governance patch submitted: https://review.openstack.org/508693 > >> > >> Sphinx Documentation > >> ==================== > >> > >> Doc builds are more complex - but I think there is a high amount of > >> value in changing how we invoke them for a few reasons. > >> > >> a) nobody uses 'tox -evenv -- python setup.py build_sphinx' but us > >> b) we decided to use sphinx for go and javascript - but we invoke sphinx > >> differently for each of those (since they naturally don't have tox), > >> meaning we can't just have a "build-sphinx-docs" job and even share it > >> with ourselves. > >> c) readthedocs.org is an excellent Open Source site that builds and > >> hosts sphinx docs for projects. They have an interface for docs > >> requirements documented and defined that we can align. By aligning, > >> projects can use migrate between docs.o.o and readthedocs.org and still > >> have a consistent experience. > >> > >> The PTI I'd like to propose for this is more complex, so I'd like to > >> describe it in terms of: > >> > >> - OpenStack organizational requirements > >> - helper sugar for developers with per-language recommendations > >> > >> I believe the result can be a single in-tree doc PTI that applies to > >> python, go and javascript - and a single Zuul job that applies to all of > >> our projects AND non-OpenStack projects as well. > >> > >> Organizational Requirements > >> --------------------------- > >> > >> Following readthedocs.org logic we can actually support a wider range of > >> schemes technically, but there is human value in having consistency on > >> these topics across our OpenStack repos. > >> > >> * docs live in doc/source > >> * Python requirements needed by Sphinx to build the docs live in > >> doc/requirements.txt > >> > >> If the project is python: > >> > >> * doc/requirements.txt can assume the project will have been installed > >> * The following should be set in setup.cfg: > >> > >> [build_sphinx] > >> source-dir = doc/source > >> build-dir = doc/build > >> > >> Doing the above allows the following commands to work cleanly in > >> automation no matter what the language is: > >> > >> [ -f doc/requirements.txt ] && pip install -rdoc/requirements.txt > >> sphinx-build -b doc/source doc/build > > > > I suspect you mean "sphinx-build -b html doc/source doc/build" (the > > "html" arg was missing). > > Yes! Please amend all code examples with the things to make them > actually work. :) > > > We should also include a PDF build, since it is useful for > > users who do not have good access to docs.o.o (especially when > > governments block the site). > > YES > > > We should also formalize how we are going to handle translations > > for both HTML and PDF. I think some of that is implemented, but I'm > > not sure if it's actually documented anywhere. > > We've got translation handling connected to sphinx builds integrated > into the releasenotes job. I was just saying to AJaeger that I would > love to see that extracted into a Sphinx plugin perhaps - but whatever > it is, I could not agree more. > > It also occurs to me (just was poking at the v3 releasenotes job updates > mnaser did) - that we should perhaps: > > a) add reno releasenotes to the PTI for all languages > b) similar to doc/requirements.txt add support for > releasenotes/requirements.txt - so that we could share much of the same > logic between releasenotes and docs jobs. > > and ... > > c) probably add 'use openstackdocstheme" to the docs PTI.
Those additions all make sense. Reno is most useful where we have stable branches, but it does address merge conflict issues on a single branch, too. And building the release notes with sphinx would work the same way as building the docs, just using different paths as input arguments. > > > Ian started a separate thread about this [1] that didn't see much > > in the way of response because everyone was involved in the zuul > > migration work. > > > > [1] > > http://lists.openstack.org/pipermail/openstack-dev/2017-September/122461.html > > Awesome. I'll respond over there too. > >> > >> No additional commands should be required. > >> > >> The setup.cfg stanza allows: > >> > >> python setup.py build_sphinx > >> > >> to continue to work. (also, everyone already has one) > >> > >> Helper sugar for developers > >> --------------------------- > >> > >> The following are recommended ways to provide in-tree developer > >> convenience: > >> > >> * tox -edocs # recommended for Python projects > >> * make docs # recommended for golang projects > >> * npm run document # recommended for javascript projects > >> > >> These will not be used by Zuul - they are there purely for local > >> developer convenience. > >> > >> Rolling it Out > >> -------------- > >> > >> First, the governance patch: > >> > >> https://review.openstack.org/508694 > >> > >> I've also written a patch for pbr to add support for finding a > >> doc/requirements.txt file, if one exists, and adding the contents to a > >> "docs" extra: > >> > >> https://review.openstack.org/#/c/492620 > >> > >> That will let people do this in their tox.ini files: > >> > >> [testenv:docs] > >> extras = docs > >> commands= > >> python setup.py build_sphinx > >> > >> And a WIP job for Zuul v3 written to provide a non-tox based build: > >> > >> https://review.openstack.org/#/c/492709/ > >> > >> (that job can be improved a little if we land the pbr docs extras patch) > >> > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
