On Wed, Dec 14, 2016 at 7:17 PM, Jani Nikula <jani.nik...@linux.intel.com> wrote: > On Wed, 14 Dec 2016, Robert Bragg <rob...@sixbynine.org> wrote: >> On Tue, Dec 13, 2016 at 11:08 PM, Rob Clark <robdcl...@gmail.com> wrote: >>> On Tue, Dec 13, 2016 at 5:47 PM, Eric Anholt <e...@anholt.net> wrote: >>>> Jason Ekstrand <ja...@jlekstrand.net> writes: >>>> >>>>> Hey All, >>>>> I don't figure this will be terribly controversial (I'm about to be wrong, >>>>> aren't I?) but how do people feel about switching our "primary" >>>>> documentation focus, as far as we have one, to sphinx? Right now, Gallium >>>>> uses sphinx for documenting all of it's "public" interfaces. A while ago, >>>>> Connor wrote some nice NIR documentation using sphinx that never got >>>>> merged. I've also spent some time this year and written some additional >>>>> documentation about Intel surface layout that I'd like to have a place to >>>>> put. The thing that's in common with each of these is that they contain >>>>> quite a bit of prose about the general theory of how things work and not >>>>> just code comments so sphinx seems like a fairly good fit. There are also >>>>> some sort-of "architectural" things in the Vulkan driver that I would like >>>>> to document better and we don't have a good place for that right now. >>>>> >>>>> Here's what I have in mind: >>>>> >>>>> Each component that we care to document (this may not be everything!) >>>>> would >>>>> have a docs/ or sphinx/ folder. In that folder would be any side-band >>>>> (not >>>>> in code comments) documentation. We would also set up some sort of >>>>> source-scraping tool to turn inline comments into sphinx documentation. >>>>> At >>>>> the component maintainer's discretion, that code could either be imported >>>>> into sphinx whole-sale or curated by importing bits into other >>>>> documentation files in a more carefully curated manner. Of course, a >>>>> maintainer could also choose to do all of their documentation side-band >>>>> (sphinx allows for this) or to not document at all. :-) >>> >>> So, +100 to sphinx... I've been quite happy with it (and would be nice >>> to merge the NIR docs too while where at it). Maybe someday we'll >>> have a "mesa book" after all ;-) >>> >>> >>>> I've written doxygen-style comments in my code for a long time, but >>>> never really looked at the doxygen output, so I wouldn't say I'm wedded >>>> to doxygen. I think some side-band docs for architectural descriptions >>>> (like NIR needs) would be great, and if sphinx would help with >>>> encouraging those to be written and merged, then that's pretty good with >>>> me. >>>> >>>> I'm a little skeptical of sphinx due to not being able to extract docs >>>> From C comments in code in the base tool. Apparently Breathe can chain >>>> doxygen into sphinx, so hopefully a "let's get excited about sphinx" >>>> patch series would convert our doxygen-based docs generation to using >>>> that. >>> >>> So as far as I understand on kernel side we are *somehow* extracting >>> out C comment docs and combining w/ sphinx stuff.. maybe there is >>> something to re-use from that. >> >> The actual parsing, extraction and generation of reStructureText for >> the kernel is handled by scripts/kernel-doc, which is a perl script >> based around regex parsing of C code. Jani Nikula has also written a >> sphinx extension for using this script which can be seen in >> Documentation/sphinx/. As you can probably guess kernel-doc is pretty >> linux kernel specific. The script existed before the recent adoption >> of sphinx and has support for docbook and man page generation. Being >> based on regex parsing it has a somewhat limited understanding of the >> underlying types of symbols being documented. >> >> I recently raised the idea of using clang's python bindings to extract >> kerneldoc on dri-devel due to some of the limitations of kernel-doc I >> was hitting and pointed at an experimental tool I wrote last year for >> generating reStructureText based on gtk-doc comments: >> https://github.com/rib/clib/blob/master/site/rst-from-c.py Probably >> not developed enough to be very useful here, but provides one basic >> example of generating reStructuredText from structured comments. >> >> Jani mentioned he had a bit of a side project going to use the same >> clang bindings to support extracting doxygen comments, so maybe what >> he's been working on could be useful for Mesa too, but I think maybe >> that was for doxygen itself so maybe nothing to do with generating >> reStructureText. > > I'm afraid what I have is really not ready for consumption by > anyone. The idea is to basically have a Sphinx extension to extract > documentation comments from source using python-clang. The goal is to > keep it *very* simple, and *not* do any of the bloat Doxygen has. If you > want to bolt stuff into Sphinx, why have the complexity of Doxygen, and > the format conversion with inevitable hickups, when the documentation > comments could just be reStructureText to begin with? > > I haven't pursued this for the kernel, because I doubt adding a clang > dependency on the documentation build would go very well. Plus you'd > have to emulate everything kernel-doc the perl script monster does, and > I kind of want to do something that sucks less. > >> In the specific case of doxygen format comments, I have a vague >> recollection that clang itself can actually parse and provide some >> kind of AST for your comments, so maybe that would be convenient. > > The AST does include comments right above each cursor, but it falls > short for comments not attached to cursors. You have to walk the tokens > for that. I think I have most of this solved, though it hasn't seen much > testing.
Right, what I was thinking of though is that clang itself can actually parse doxygen style comment sections like \brief, \param and \returns: http://clang.llvm.org/doxygen/classclang_1_1comments_1_1Parser.html Seems a bit mad, but this e.g. helps enable a -Wdocumentation warning to check \params match actual function arguments I was wondering if that can somehow be exposed via the AST accessible through the python bindings. Regards, - Robert > > No matter what, I think you should pursue documentation comment > extraction. Manually documenting C APIs and stuff in Sphinx is not > fun. If you want to keep compatibility with Doxygen, you should probably > look into Doxygen+Breathe, but be wary of the format > conversions. Otherwise, I think the options are to reuse what kernel has > (a perl script nobody in their right mind admits to understanding) or > write a Sphinx extension, likely based on python-clang (which is not > going to be very feature rich or Doxygen compatible at the beginning, > even if I polish what I have to give you a tiny jump start). > > I'm not subscribed on mesa-dev, but send mail my way if you want my > opinions on whatever you're planning. > > BR, > Jani. > > > -- > Jani Nikula, Intel Open Source Technology Center _______________________________________________ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
