Peter Maydell <peter.mayd...@linaro.org> writes:
> On Fri, 4 Sep 2020 at 14:37, Markus Armbruster <arm...@redhat.com> wrote:
>> Peter Maydell <peter.mayd...@linaro.org> writes:
>> > -.PHONY: check-tests/qapi-schema/doc-good.texi
>> > -check-tests/qapi-schema/doc-good.texi:
>> > tests/qapi-schema/doc-good.test.texi
>> > - @diff -u $(SRC_PATH)/tests/qapi-schema/doc-good.texi $<
>
>> We shouldn't just delete this test.
>>
>> It is for checking the doc generator does what it should for "good"
>> input. "Bad" input is coverd by the other doc-*.json.
>>
>> With the old doc generation system, the testing "good" input is
>> straightforward: generate Texinfo, diff to expected Texinfo, which is
>> committed to git.
>>
>> This test has been invaliable when maintaining and extending doc.py.
>>
>> With the new system, there is no ouput suitable for diffing, as the
>> various outputs all depend on the version of Sphinx.
>>
>> Or is there? Is there a way to have Sphinx "unparse" its internal
>> representation of the input?
>
> There is no built-in "unparse the internal representation" option.
> We could add one as a Sphinx extension (basically defining a new
> output format that was "print what you get"). This too is at
> least potentially liable to breakage with future versions, both
> if the Sphinx APIs for output-format extensions and change and
> if core Sphinx gets changes that mean input rST is parsed into
> a different-but-equivalent internal-tree-of-nodes representation.
Yes. We could update the test for current Sphinx then, and disable it
for old Sphinx. Not ideal, but good enough, I think.
> The HTML output definitely depends on the Sphinx version.
> The Texinfo output doesn't differ much, but it does differ in
> a couple of places (firstly it has the Sphinx version number
> baked into, and secondly what looks like a null-effect change
> in ordoring of @anchor{} nodes).
> The plain-text output is identical between Sphinx 1.6 and 3.0.
> (I think this is mostly because nobody really cares about it
> as an output generator, so it hasn't had any changes made to
> it other than general whole-tree cleanup type stuff...)
>
> So we could go for a simple comparison of the plaintext, and
> hope future Sphinx versions don't break it. (If they did we'd
> need to put together something like the iotests handling of
> "these parts need to match and these might be anything" in
> the golden-reference).
Again, we could also update the test for current Sphinx then, and
disable it for old Sphinx.
Diffing plain text output is a weaker test than diffing the intermediate
Texinfo or a Sphinx unparse. Still better than nothing.
Blocking this series on a a yet-to-be-written unparse extension would be
a bad idea. But thinking things through is not :)
