This series switches all our QAPI doc comments over from
texinfo format to rST.
The basic approach is somewhat similar to how we deal with kerneldoc
and hxtool: we have a custom Sphinx extension which is passed a
filename which is the json file it should run the QAPI parser over and
generate documentation for. Unlike 'kerneldoc' but somewhat like
hxtool, I have chosed to generate documentation by generating a tree
of docutils nodes, rather than by generating rST source that is then
fed to the rST parser to generate docutils nodes. Individual lumps of
doc comment go to the rST parser, but the structured parts we render
directly. This makes it easier to get the structure and heading level
nesting correct.
Rather than trying to exactly handle all the existing comments I have
opted (as Markus suggested) to tweak them where this seemed more
sensible than contorting the rST generator to deal with
weirdnesses. The principal changes are:
* whitespace is now significant, and multiline definitions must have
their second and subsequent lines indented to match the first line
* general rST format markup is permitted, not just the small set of
markup the old texinfo generator handled. For most things (notably
bulleted and itemized lists) the old format is the same as rST was.
* Specific things that might trip people up:
- instead of *bold* and _italic_ rST has **bold** and *italic*
- lists need a preceding and following blank line
- a lone literal '*' will need to be backslash-escaped to
avoid a rST syntax error
* the old leading '|' for example (literal text) blocks is replaced
by the standard rST '::' literal block.
* headings and subheadings must now be in a freeform documentation
comment of their own
* we support arbitrary levels of sub- and sub-sub-heading, not just a
main and sub-heading like the old texinfo generator
* as a special case, @foo is retained and is equivalent to ``foo``
This policy means that most of the patch series is cleanups to the doc
comments:
- doc comments missing the ':' after @argument (misrenders
in the texi, is a syntax error for rST)
- doc comments with indent that doesn't match the tightened
requirements (no effect on the texi, but rST cares)
- some lists needed leading/trailing blank lines adding
- a graph intended to be rendered as ascii-art wasn't correctly
marked up as a literal block (misrenders in the texi, syntax error
for rST)
- some stray hardcoded tab characters needed changing to spaces
- in a few places parts of a doc comment were in the wrong order
resulting in it being rendered into the wrong section by mistake
- a few instances of texinfo `quotes' needed to be changed to rST
'quotes'; similarly _this_ isn't valid rST markup, and in one place
a literal * needed backslash-escaping
- numerous places were trying to have lists without using the list
markup, which renders as weird run-on sentences
- the two places which define headings in the middle of a doc comment
are changed to put the heading definition in the
standalone-comment-block that the new generator wants (no change to
the texinfo output)
Patch 9 changes 663 lines in 20 files but it is purely a whitespace
change (verifiable with 'git show -w'). The other patches are
generally smaller and straightforward.
All of these patches except for the "escape a literal '*'" one at the
end have either no/minimal effect on the generated texinfo or fix
misrenderings.
Moving on to the actual code changes:
* we start by changing the existing parser code to be more careful
with leading whitespace: instead of stripping it all, it strips
only the amount required for indented multiline definitions, and
complains if it finds an unexpected de-indent. The texinfo
generator code is updated to do whitespace stripping itself, so
there is no change to the generated texi source.
* then we add the new qapidoc Sphinx extension, which is not yet used
by anything. This is a 500 line script, all added in one patch. I
can split it if people think that would help, but I'm not sure I
see a good split point.
* then we can convert the two generated reference documents, one at a
time. This is mostly just updating makefile rules and the like.
* after that we can do some minor tweaks to doc comments that would
have confused the texinfo parser: changing our two instances of
'|'-markup literal blocks to rST '::' literal blocks, and adding
some headings to the GA reference so the rST interop manual ToC
looks better.
* finally, we can delete the old texinfo machinery and update the
markup docs in docs/devel/qapi-code-gen.txt
There are a few things I have left out of this initial series:
* unlike the texinfo, there is no generation of index entries
or an index in the HTML docs
* although there are HTML anchors on all the command/object/etc
headings, they are not stable but just serial-number based
tags like '#qapidoc-35', so not suitable for trying to link
to from other parts of the docs
My view is that we can add niceties like this later; the series
already seems big enough to me.
You can find the HTML rendered version of the results
of this series at:
http://people.linaro.org/~peter.maydell/qdoc-snapshot/interop/qemu-ga-ref.html
http://people.linaro.org/~peter.maydell/qdoc-snapshot/interop/qemu-qmp-ref.html
(look also at
http://people.linaro.org/~peter.maydell/qdoc-snapshot/interop/index.html
if you want to see how the ToC for the interop manual comes out)
The manpages are
http://people.linaro.org/~peter.maydell/qemu-ga-ref.7
http://people.linaro.org/~peter.maydell/qemu-qmp-ref.7
(download and render with 'man -l path/to/foo.7')
For comparison, the old texinfo-to-HTML versions of the docs are:
https://www.qemu.org/docs/master/qemu-ga-ref.html
https://www.qemu.org/docs/master/qemu-qmp-ref.html
The first four patches have already been posted separately and
reviewed.
I did at some point while working on this eyeball all the generated
documentation against the old versions, but there's an awful lot of
it, so I might have missed some minor stuff, and I didn't try to redo
that full-eyeball-diff with the absolute final version of the code. So
it's possible there are some minor misrenderings lurking, but I don't
think so.
thanks
-- PMM
Peter Maydell (29):
configure: Allow user to specify sphinx-build binary
configure: Check that sphinx-build is using Python 3
Makefile: Fix typo in dependency list for interop manpages
qga/qapi-schema.json: Fix missing '-' in GuestDiskBusType doc comment
qga/qapi-schema.json: Fix indent level on doc comments
qga/qapi-schema.json: minor format fixups for rST
qapi/block-core.json: Use literal block for ascii art
qapi: Use ':' after @argument in doc comments
qapi: Fix indent level on doc comments in json files
qapi: Remove hardcoded tabs
qapi/ui.json: Put input-send-event body text in the right place
qapi: Explicitly put "foo: dropped in n.n" notes into Notes section
qapi/ui.json: Avoid `...' texinfo style quoting
qapi/block-core.json: Use explicit bulleted lists
qapi/ui.json: Use explicit bulleted lists
qapi/{block,misc,tmp}.json: Use explicit bulleted lists
qapi: Add blank lines before bulleted lists
qapi/migration.json: Replace _this_ with *this*
qapi/qapi-schema.json: Put headers in their own doc-comment blocks
qapi/machine.json: Escape a literal '*' in doc comment
scripts/qapi: Move doc-comment whitespace stripping to doc.py
scripts/qapi/parser.py: improve doc comment indent handling
docs/sphinx: Add new qapi-doc Sphinx extension
docs/interop: Convert qemu-ga-ref to rST
docs/interop: Convert qemu-qmp-ref to rST
qapi: Use rST markup for literal blocks
qga/qapi-schema.json: Add some headings
scripts/qapi: Remove texinfo generation support
docs/devel/qapi-code-gen.txt: Update to new rST backend conventions
docs/devel/qapi-code-gen.txt | 90 ++-
configure | 22 +-
Makefile | 58 +-
tests/Makefile.include | 15 +-
qapi/block-core.json | 1127 ++++++++++++++++----------------
qapi/block.json | 47 +-
qapi/char.json | 10 +-
qapi/dump.json | 4 +-
qapi/introspect.json | 12 +-
qapi/job.json | 32 +-
qapi/machine-target.json | 18 +-
qapi/machine.json | 16 +-
qapi/migration.json | 206 +++---
qapi/misc-target.json | 8 +-
qapi/misc.json | 138 ++--
qapi/net.json | 26 +-
qapi/qapi-schema.json | 18 +-
qapi/qdev.json | 10 +-
qapi/qom.json | 4 +-
qapi/rocker.json | 12 +-
qapi/run-state.json | 34 +-
qapi/sockets.json | 8 +-
qapi/tpm.json | 4 +-
qapi/trace.json | 15 +-
qapi/transaction.json | 4 +-
qapi/ui.json | 119 ++--
qga/qapi-schema.json | 160 ++---
MAINTAINERS | 3 +-
docs/conf.py | 16 +-
docs/index.html.in | 2 -
docs/interop/conf.py | 4 +
docs/interop/index.rst | 2 +
docs/interop/qemu-ga-ref.rst | 4 +
docs/interop/qemu-ga-ref.texi | 80 ---
docs/interop/qemu-qmp-ref.rst | 4 +
docs/interop/qemu-qmp-ref.texi | 80 ---
docs/sphinx/qapidoc.py | 504 ++++++++++++++
scripts/qapi-gen.py | 2 -
scripts/qapi/doc.py | 302 ---------
scripts/qapi/gen.py | 7 -
scripts/qapi/parser.py | 94 ++-
41 files changed, 1726 insertions(+), 1595 deletions(-)
create mode 100644 docs/interop/qemu-ga-ref.rst
delete mode 100644 docs/interop/qemu-ga-ref.texi
create mode 100644 docs/interop/qemu-qmp-ref.rst
delete mode 100644 docs/interop/qemu-qmp-ref.texi
create mode 100644 docs/sphinx/qapidoc.py
delete mode 100644 scripts/qapi/doc.py
--
2.20.1
