On Mon, May 20, 2019 at 05:25:28PM -0400, John Snow wrote:
>
>
> On 5/20/19 12:37 PM, John Snow wrote:
> >
> >
> > On 5/20/19 7:30 AM, Aarushi Mehta wrote:
> >> https://paste.fedoraproject.org/paste/kOPx4jhtUli---TmxSLrlw
> >> running python3-sphinx-2.0.1-1.fc31.noarch on Fedora release 31
> >> (Rawhide)
> >>
> >> uname - a
> >> Linux iouring 5.1.0-0.rc6.git3.1.fc31.x86_64 #1 SMP Thu Apr 25 14:25:32
> >> UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> >>
> >> Reverting commmit 90edef80a0852cf8a3d2668898ee40e8970e431
> >> allows for the build to occur
> >>
> >> Regards
> >> Aarushi Mehta
> >>
> >>
> >
> > Ah, dang. The blocks aren't strictly conforming json, but the version I
> > tested this under didn't seem to care. Your version is much newer. (I
> > was using 1.7 as provided by Fedora 29.)
> >
> > For now, try reverting 9e5b6cb87db66dfb606604fe6cf40e5ddf1ef0e7 instead,
> > which should at least turn off the "warnings as errors" option, but I
> > don't think that reverting -n will turn off this warning.
> >
> > I'll try to get ahold of this newer version and see if I can't fix it
> > more appropriately.
> >
> > --js
> >
>
> ...Sigh, okay.
>
> So, I am still not actually sure what changed from pygments 2.2 and
> sphinx 1.7 to pygments 2.4 and sphinx 2.0.1, but it appears as if Sphinx
> by default always tries to do add a filter to the pygments lexer that
> raises an error on highlighting failure, instead of the default behavior
> which is to just highlight those errors in the output. There is no
> option to Sphinx that I am aware of to retain this lexing behavior.
> (Effectively, it's strict or nothing.)
>
> This approach, apparently, is broken in Sphinx 1.7/Pygments 2.2, so the
> build works with our malformed json.
>
> There are a few options:
>
> 1. Update conf.py to ignore these warnings (and all future lexing
> errors), and settle for the fact that there will be no QMP highlighting
> wherever we use the directionality indicators ('->', '<-').
>
> 2. Update bitmaps.rst to remove the directionality indicators.
>
> 3. Update bitmaps.rst to format the QMP blocks as raw text instead of JSON.
>
> 4. Update bitmaps.rst to remove the "json" specification from the code
> block. This will cause sphinx to "guess" the formatting, and the
> pygments guesser will decide it's Python3.
>
> This will parse well enough, but will mis-highlight 'true' and 'false'
> which are not python keywords. This approach may break in the future if
> the Python3 lexer is upgraded to be stricter (because '->' and '<-' are
> still invalid), and leaves us at the mercy of both the guesser and the
> lexer.
>
> I'm not actually sure what I dislike the least; I think I dislike #1 the
> most. #4 gets us most of what we want but is perhaps porcelain.
>
> I suspect if we attempt to move more of our documentation to ReST and
> Sphinx that we will need to answer for ourselves how we intend to
> document QMP code flow examples.
Writing a custom lexer that handles "<-" and "->" was simple (see below).
Now, is it possible to convince Sphinx to register and use a custom lexer?
$ cat > /tmp/lexer.py <<EOF
from pygments.lexer import RegexLexer, DelegatingLexer
from pygments.lexers.data import JsonLexer
import re
from pygments.token import *
class QMPExampleMarkersLexer(RegexLexer):
tokens = {
'root': [
(r' *-> *', Generic.Prompt),
(r' *<- *', Generic.Output),
]
}
class QMPExampleLexer(DelegatingLexer):
def __init__(self, **options):
super(QMPExampleLexer, self).__init__(JsonLexer,
QMPExampleMarkersLexer, Error, **options)
EOF
$ pygmentize -l /tmp/lexer.py:QMPExampleLexer -x -f html <<EOF
-> {
"execute": "drive-backup",
"arguments": {
"device": "drive0",
"bitmap": "bitmap0",
"target": "drive0.inc0.qcow2",
"format": "qcow2",
"sync": "incremental",
"mode": "existing"
}
}
<- { "return": {} }
EOF
<div class="highlight"><pre><span></span><span class="gp"> ->
</span><span class="p">{</span>
<span class="nt">"execute"</span><span class="p">:</span>
<span class="s2">"drive-backup"</span><span class="p">,</span>
<span class="nt">"arguments"</span><span class="p">:</span>
<span class="p">{</span>
<span class="nt">"device"</span><span class="p">:</span>
<span class="s2">"drive0"</span><span class="p">,</span>
<span class="nt">"bitmap"</span><span class="p">:</span>
<span class="s2">"bitmap0"</span><span class="p">,</span>
<span class="nt">"target"</span><span class="p">:</span>
<span class="s2">"drive0.inc0.qcow2"</span><span class="p">,</span>
<span class="nt">"format"</span><span class="p">:</span>
<span class="s2">"qcow2"</span><span class="p">,</span>
<span class="nt">"sync"</span><span class="p">:</span>
<span class="s2">"incremental"</span><span class="p">,</span>
<span class="nt">"mode"</span><span class="p">:</span>
<span class="s2">"existing"</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="go"> <- </span><span class="p">{</span> <span
class="nt">"return"</span><span class="p">:</span> <span
class="p">{}</span> <span class="p">}</span>
</pre></div>
$
--
Eduardo
