On Apr 4, 4:38 pm, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > George Sakkis wrote: > >> If it was a bug it has to violate a functional requirement. I can't > >> see which one. > > > Perhaps it's not a functional requirement but it came up as a real > > problem on a source colorizer I use. I count on newlines generating > > token.NEWLINE or tokenize.NL tokens in order to produce <br> tags. It > > took me some time and head scratching to find out why some comments > > were joined together with the following line. Now I have to check > > whether a comment ends in new line and if it does output an extra <br> > > tag.. it works but it's a kludge. > > well, the real kludge here is of course that you're writing your own > colorizer, when you can just go and grab Pygments: > > http://pygments.org/ > > or, if you prefer something tiny and self-contained, something like the > colorizer module in this directory: > > http://svn.effbot.org/public/stuff/sandbox/pythondoc/ > > (the element_colorizer module in the same directory gives you XHTML in > an ElementTree instead of raw HTML, if you want to postprocess things) > > </F>
First off, I didn't write it from scratch, I just tweaked a single module colorizer I had found online. Second, whether I or someone else had to deal with it is irrelevant; the point is that generate_tokens() is not consistent with respect to new lines after comments. George -- http://mail.python.org/mailman/listinfo/python-list
