[PATCH 00/10] Documentation/Sphinx
Hi Jon, all -
I've had a few moments of spare time to look into Sphinx. This is a sort
of status report on the progress.
I suppose we all thought it would be easiest to use docproc for the
Sphinx toolchain, for starters. I tried it, hard, and even sent a bunch
of docproc prep patches. However I ran into some problems. Out-of-tree
builds were a nightmare, as Sphinx has trouble to pick up some of the
files from $(srctree) and some from $(objtree). Having dependencies on
the source files work without rebuilding everything was getting tricky
too.
I decided to look into writing a Sphinx extension for kernel-doc, and it
turns out to be a really nice solution. We can keep all the .rst files
in $(srctree), we don't have to explicitly specify the .rst files to
process, there are no intermediate files, and Sphinx runs the kernel-doc
script using the extension based on the directives in the .rst
files. The extension tells Sphinx about the dependencies on the source
files, and Sphinx handles rebuilding as needed. Out-of-tree builds just
work. Sites like https://readthedocs.org/ can build the documentation,
including kernel-doc, without extra tweaks. As a whole, the build
becomes much simpler.
There are a few tradeoffs, of course. First, this requires that the
EXPORT_SYMBOL markers are placed immediately after the function being
exported, as kernel-doc will only look at one file at a time. This is
the recommendation anyway. See the corresponding patch for further
details ("kernel-doc: support printing exported and non-exported
symbols"). Second, we lose support for the !C docproc directive to check
that all kernel-doc comments in a file are used. This is probably
something we'd like to have back in the future, but at this time I think
it's an acceptable tradeoff wrt the gains.
With this, we can put any .rst files (including ones that have
kernel-doc directives) anywhere under Documentation, add a link to them
in Documentation/index.rst table of contents, and it will just work. It
can't get much simpler than that.
At this time I've put most effort into the configuration and build side
of things, solving the problems described above, and handling missing
tools and packages gracefully. There are still issues to be ironed out
in a) the kernel-doc script rst output and b) the xml template to rst
conversion. These are somewhat orthogonal from each other and the build,
and I expect some hand-editing will be required in the end.
The patches are available in the "sphinx" branch of [1], and I've set up
a project at Read the Docs to build that into documentation [2] (mostly
to test this approach also works and so I don't have to host this
anywhere).
Any comments are welcome, but please do remember that I've focused on
polishing the toolchain and build, not the output quality, with release
early, release often in mind.
BR,
Jani.
[1] git://people.freedesktop.org/~jani/drm
[2] https://kernel.readthedocs.io/
Jani Nikula (9):
kernel-doc: fix use of uninitialized value
kernel-doc: support printing exported and non-exported symbols
Documentation/sphinx: add basic working Sphinx configuration and build
Documentation: add .gitignore
Documentation/sphinx: add Sphinx kernel-doc directive extension
Documentation/sphinx: configure the kernel-doc extension
Documentation: add kernel hacking rst
Documentation: add kernel api rst
Documentation: moar files
Jonathan Corbet (1):
sphinx: cheesy script to convert .tmpl files
Documentation/.gitignore |1 +
Documentation/DocBook/Makefile|7 +-
Documentation/Makefile.sphinx | 63 +
Documentation/conf.py | 384 +
Documentation/crypto-API.rst | 1870 +
Documentation/filesystems.rst | 314
Documentation/gpu.rst | 2556 +
Documentation/index.rst | 30 +
Documentation/kernel-api.rst | 419 +
Documentation/kernel-hacking.rst | 795 +
Documentation/sphinx/convert_template.sed | 14 +
Documentation/sphinx/kernel-doc.py| 99 ++
Documentation/sphinx/post_convert.sed | 19 +
Documentation/sphinx/tmplcvt | 19 +
Makefile |5 +-
scripts/kernel-doc| 32 +-
16 files changed, 6619 insertions(+), 8 deletions(-)
create mode 100644 Documentation/.gitignore
create mode 100644 Documentation/Makefile.sphinx
create mode 100644 Documentation/conf.py
create mode 100644 Documentation/crypto-API.rst
create mode 100644 Documentation/filesystems.rst
create mode 100644 Documentation/gpu.rst
create mode 100644 Documentation/index.rst
create mode 100644 Documentation/kernel-api.rst
create mode 100644 Documentation/kernel-hacking.rst
create mode 100644 Documentation/sphinx/convert_template.sed
create mode 100644 Documentation/sphinx/kernel-doc.py
create mode 100644 Do
[PATCH 09/10] Documentation: add kernel api rst
Crude conversion. This one uses kernel-doc. Signed-off-by: Jani Nikula --- Documentation/index.rst | 1 + Documentation/kernel-api.rst | 419 +++ 2 files changed, 420 insertions(+) create mode 100644 Documentation/kernel-api.rst diff --git a/Documentation/index.rst b/Documentation/index.rst index 1ed1c4e91c27..6ca68ef5c89a 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -16,6 +16,7 @@ Contents: :maxdepth: 2 kernel-hacking + kernel-api Indices and tables == diff --git a/Documentation/kernel-api.rst b/Documentation/kernel-api.rst new file mode 100644 index ..ac24d8c2461f --- /dev/null +++ b/Documentation/kernel-api.rst @@ -0,0 +1,419 @@ + +The Linux Kernel API + + +Data Types +== + +Doubly Linked Lists +--- + +.. kernel-doc:: include/linux/list.h + :internal: + +Basic C Library Functions += + +When writing drivers, you cannot in general use routines which are from +the C Library. Some of the functions have been found generally useful +and they are listed below. The behaviour of these functions may vary +slightly from those defined by ANSI, and these deviations are noted in +the text. + +String Conversions +-- + +.. kernel-doc:: lib/vsprintf.c + :export: + +.. kernel-doc:: include/linux/kernel.h + :functions: kstrtol + +.. kernel-doc:: include/linux/kernel.h + :functions: kstrtoul + +.. kernel-doc:: lib/kstrtox.c + :export: + +String Manipulation +--- + +.. kernel-doc:: lib/string.c + :export: + +Bit Operations +-- + +.. kernel-doc:: arch/x86/include/asm/bitops.h + :internal: + +Basic Kernel Library Functions +== + +The Linux kernel provides more basic utility functions. + +Bitmap Operations +- + +.. kernel-doc:: lib/bitmap.c + :export: + +.. kernel-doc:: lib/bitmap.c + :internal: + +Command-line Parsing + + +.. kernel-doc:: lib/cmdline.c + :export: + +CRC Functions +- + +.. kernel-doc:: lib/crc7.c + :export: + +.. kernel-doc:: lib/crc16.c + :export: + +.. kernel-doc:: lib/crc-itu-t.c + :export: + +.. kernel-doc:: lib/crc32.c + :export: + +.. kernel-doc:: lib/crc-ccitt.c + :export: + +idr/ida Functions +- + +.. kernel-doc:: include/linux/idr.h + :doc: idr sync + +.. kernel-doc:: lib/idr.c + :doc: IDA description + +.. kernel-doc:: lib/idr.c + :export: + +Memory Management in Linux +== + +The Slab Cache +-- + +.. kernel-doc:: include/linux/slab.h + :internal: + +.. kernel-doc:: mm/slab.c + :export: + +.. kernel-doc:: mm/util.c + :export: + +User Space Memory Access + + +.. kernel-doc:: arch/x86/include/asm/uaccess_32.h + :internal: + +.. kernel-doc:: arch/x86/lib/usercopy_32.c + :export: + +More Memory Management Functions + + +.. kernel-doc:: mm/readahead.c + :export: + +.. kernel-doc:: mm/filemap.c + :export: + +.. kernel-doc:: mm/memory.c + :export: + +.. kernel-doc:: mm/vmalloc.c + :export: + +.. kernel-doc:: mm/page_alloc.c + :internal: + +.. kernel-doc:: mm/mempool.c + :export: + +.. kernel-doc:: mm/dmapool.c + :export: + +.. kernel-doc:: mm/page-writeback.c + :export: + +.. kernel-doc:: mm/truncate.c + :export: + +Kernel IPC facilities += + +IPC utilities +- + +.. kernel-doc:: ipc/util.c + :internal: + +FIFO Buffer +=== + +kfifo interface +--- + +.. kernel-doc:: include/linux/kfifo.h + :internal: + +relay interface support +=== + +Relay interface support is designed to provide an efficient mechanism +for tools and facilities to relay large amounts of data from kernel +space to user space. + +relay interface +--- + +.. kernel-doc:: kernel/relay.c + :export: + +.. kernel-doc:: kernel/relay.c + :internal: + +Module Support +== + +Module Loading +-- + +.. kernel-doc:: kernel/kmod.c + :export: + +Inter Module support + + +Refer to the file kernel/module.c for more information. + +Hardware Interfaces +=== + +Interrupt Handling +-- + +.. kernel-doc:: kernel/irq/manage.c + :export: + +DMA Channels + + +.. kernel-doc:: kernel/dma.c + :export: + +Resources Management + + +.. kernel-doc:: kernel/resource.c + :internal: + +.. kernel-doc:: kernel/resource.c + :export: + +MTRR Handling +- + +.. kernel-doc:: arch/x86/kernel/cpu/mtrr/main.c + :export: + +PCI Support Library +--- + +.. kernel-doc:: drivers/pci/pci.c + :export: + +.. kernel-doc:: drivers/pci/pci-driver.c + :export: + +.. kernel-doc:: drivers/pci/remove.c + :export: + +.. kernel-doc:: drivers/pci/search.c + :export: + +.. k
[PATCH 04/10] Documentation: add .gitignore
The Sphinx output directory is generated. Signed-off-by: Jani Nikula --- Documentation/.gitignore | 1 + 1 file changed, 1 insertion(+) create mode 100644 Documentation/.gitignore diff --git a/Documentation/.gitignore b/Documentation/.gitignore new file mode 100644 index ..53752db253e3 --- /dev/null +++ b/Documentation/.gitignore @@ -0,0 +1 @@ +output -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 06/10] Documentation/sphinx: configure the kernel-doc extension
Tell Sphinx where to find the extension, and pass on the kernel src tree
and kernel-doc paths to the extension.
With this, any .rst files under Documentation may contain the kernel-doc
rst directive to include kernel-doc documentation from any source file.
While building, it may be handy to pass kernel-doc extension
configuration on the command line. For example, 'make SPHINXOPTS="-D
kerneldoc_verbosity=0" htmldocs' silences all stderr output from
kernel-doc when the kernel-doc exit code is 0. (The stderr will be
logged unconditionally when the exit code is non-zero.)
Signed-off-by: Jani Nikula
---
Documentation/Makefile.sphinx | 4 +++-
Documentation/conf.py | 10 --
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
index 25a7970bfe5c..8a662be9bd87 100644
--- a/Documentation/Makefile.sphinx
+++ b/Documentation/Makefile.sphinx
@@ -25,7 +25,9 @@ HAVE_RST2PDF := $(shell if python -c "import rst2pdf"
>/dev/null 2>&1; then echo
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
-ALLSPHINXOPTS = -d $(BUILDDIR)/.doctrees $(PAPEROPT_$(PAPER)) -c
$(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
+KERNELDOC = $(srctree)/scripts/kernel-doc
+KERNELDOC_CONF = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
+ALLSPHINXOPTS = -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF)
$(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
diff --git a/Documentation/conf.py b/Documentation/conf.py
index ff86f5e88049..0d33cb157716 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -18,7 +18,7 @@ import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
-#sys.path.insert(0, os.path.abspath('.'))
+sys.path.insert(0, os.path.abspath('sphinx'))
# -- General configuration
@@ -28,7 +28,7 @@ import os
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = []
+extensions = ['kernel-doc']
# Gracefully handle missing rst2pdf.
try:
@@ -376,3 +376,9 @@ epub_exclude_files = ['search.html']
pdf_documents = [
('index', u'Kernel', u'Kernel', u'J. Random Bozo'),
]
+
+# kernel-doc extension configuration for running Sphinx directly (e.g. by Read
+# the Docs). In a normal build, these are supplied from the Makefile via
command
+# line arguments.
+kerneldoc_bin = '../scripts/kernel-doc'
+kerneldoc_srctree = '..'
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 03/10] Documentation/sphinx: add basic working Sphinx configuration and build
Add basic configuration and makefile to build documentation from any .rst files under Documentation using Sphinx. For starters, there's just the placeholder index.rst. At the top level Makefile, hook Sphinx documentation targets alongside (but independent of) the DocBook toolchain, having both be run on the various 'make *docs' targets. All Sphinx processing is placed into Documentation/Makefile.sphinx. Both that and the Documentation/DocBook/Makefile are now expected to handle all the documentation targets, explicitly ignoring them if they're not relevant for that particular toolchain. The changes to the existing DocBook Makefile are kept minimal. There is graceful handling of missing Sphinx and rst2pdf (which is needed for pdf output) by checking for the tool and python module, respectively, with informative messages to the user. If the Read the Docs theme (sphinx_rtd_theme) is available, use it, but otherwise gracefully fall back to the Sphinx default theme, with an informative message to the user, and slightly less pretty HTML output. Sphinx can now handle htmldocs, pdfdocs (if rst2pdf is available), epubdocs and xmldocs targets. The output documents are written into per output type subdirectories under Documentation/output. Finally, you can pass options to sphinx-build using the SPHINXBUILD make variable. For example, 'make SPHINXOPTS=-v htmldocs' for more verbose output from Sphinx. This is based on the original work by Jonathan Corbet, but he probably wouldn't recognize this as his own anymore. Signed-off-by: Jani Nikula --- Documentation/DocBook/Makefile | 7 +- Documentation/Makefile.sphinx | 61 +++ Documentation/conf.py | 378 + Documentation/index.rst| 23 +++ Makefile | 5 +- 5 files changed, 469 insertions(+), 5 deletions(-) create mode 100644 Documentation/Makefile.sphinx create mode 100644 Documentation/conf.py create mode 100644 Documentation/index.rst diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile index d70f9b68174e..e0c7e1e0590b 100644 --- a/Documentation/DocBook/Makefile +++ b/Documentation/DocBook/Makefile @@ -33,10 +33,6 @@ PDF_METHOD = $(prefer-db2x) PS_METHOD = $(prefer-db2x) -### -# The targets that may be used. -PHONY += xmldocs sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs cleandocs - targets += $(DOCBOOKS) BOOKS := $(addprefix $(obj)/,$(DOCBOOKS)) xmldocs: $(BOOKS) @@ -63,6 +59,9 @@ installmandocs: mandocs sort -k 2 -k 1 | uniq -f 1 | sed -e 's: :/:' | \ xargs install -m 644 -t /usr/local/man/man9/ +# no-op for the DocBook toolchain +epubdocs: + ### #External programs used KERNELDOCXMLREF = $(srctree)/scripts/kernel-doc-xml-ref diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx new file mode 100644 index ..25a7970bfe5c --- /dev/null +++ b/Documentation/Makefile.sphinx @@ -0,0 +1,61 @@ +# -*- makefile -*- +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXBUILD = sphinx-build +SPHINXOPTS= +PAPER = +BUILDDIR = $(obj)/output + +# User-friendly check for sphinx-build +HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi) + +ifeq ($(HAVE_SPHINX),0) + +.DEFAULT: + $(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.) + @echo " SKIPSphinx $@ target." + +else # HAVE_SPHINX + +# User-friendly check for rst2pdf +HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo 1; else echo 0; fi) + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/.doctrees $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src) +# the i18n builder cannot share the environment and doctrees with the others +I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . + +quiet_cmd_sphinx = SPHINX $@ + cmd_sphinx = $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2 + +htmldocs: + $(call cmd,sphinx,html) + +pdfdocs: +ifeq ($(HAVE_RST2PDF),0) + $(warning The Python 'rst2pdf' module was not found. Make sure you have the module installed to produce PDF output.) + @echo " SKIPSphinx $@ target." +else # HAVE_RST2PDF + $(call cmd,sphinx,pdf) +endif # HAVE_RST2PDF + +epubdocs: + $(call cmd,sphinx,epub) + +xmldocs: + $(call cmd,sphinx,xml) + +# no-ops for the Sphinx toolchain +sgmldocs: +psdocs: +mandocs: +installmandocs: + +cleandocs: + $(Q)rm -rf $(BUILDDIR) + +endif # HAVE_SPHINX diff --git a/Documentation/conf.py b/Documentation/conf.py new file mode 100644 index ..ff86f5e88049 --- /dev/null +++ b/Documentation/conf.py @@
[PATCH 02/10] kernel-doc: support printing exported and non-exported symbols
Currently we use docproc to figure out which symbols are exported, and
then docproc calls kernel-doc on specific functions, to get
documentation on exported functions. According to git blame and docproc
comments, this is due to historical reasons, as functions and their
corresponding EXPORT_SYMBOL* may have been in different files. However
for more than ten years the recommendation in CodingStyle has been to
place the EXPORT_SYMBOL* immediately after the closing function brace
line.
Additionally, the kernel-doc comments for functions are generally placed
above the function definition in the .c files (i.e. where the
EXPORT_SYMBOL* is) rather than above the declaration in the .h
files. There are some exceptions to this, but AFAICT none of these are
included in DocBook documentation using the "!E" docproc directive.
Therefore, assuming the EXPORT_SYMBOL* and kernel-doc are with the
function definition, kernel-doc can extract the exported vs. not
information by making two passes on the input file. Add support for that
via the new -export and -internal parameters.
Signed-off-by: Jani Nikula
---
scripts/kernel-doc | 29 +++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index babb374c043d..0e4109bcaa41 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -59,6 +59,12 @@ Output format selection (mutually exclusive):
-textOutput plain text format.
Output selection (mutually exclusive):
+ -export Only output documentation for symbols that have been
+ exported using EXPORT_SYMBOL() or EXPORT_SYMBOL_GPL()
+ in the same FILE.
+ -internalOnly output documentation for symbols that have NOT been
+ exported using EXPORT_SYMBOL() or EXPORT_SYMBOL_GPL()
+ in the same FILE.
-function NAME Only output documentation for the given function(s)
or DOC: section title(s). All other functions and DOC:
sections are ignored. May be specified multiple times.
@@ -380,6 +386,7 @@ my $doc_block = $doc_com . 'DOC:\s*(.*)?';
my $doc_split_start = '^\s*/\*\*\s*$';
my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
my $doc_split_end = '^\s*\*/\s*$';
+my $export_symbol = '^\s*EXPORT_SYMBOL(_GPL)?\s*\(\s*(\w+)\s*\)\s*;';
my %constants;
my %parameterdescs;
@@ -444,6 +451,12 @@ while ($ARGV[0] =~ m/^-(.*)/) {
$function_only = 2;
$function = shift @ARGV;
$function_table{$function} = 1;
+} elsif ($cmd eq "-export") { # only exported symbols
+ $function_only = 3;
+ %function_table = ()
+} elsif ($cmd eq "-internal") { # only non-exported symbols
+ $function_only = 4;
+ %function_table = ()
} elsif ($cmd eq "-v") {
$verbose = 1;
} elsif (($cmd eq "-h") || ($cmd eq "--help")) {
@@ -1971,8 +1984,10 @@ sub output_declaration {
my $functype = shift;
my $func = "output_${functype}_$output_mode";
if (($function_only==0) ||
- ( $function_only == 1 && defined($function_table{$name})) ||
- ( $function_only == 2 && !($functype eq "function" &&
defined($function_table{$name}
+ ( ($function_only == 1 || $function_only == 3) &&
+ defined($function_table{$name})) ||
+ ( ($function_only == 2 || $function_only == 4) &&
+ !($functype eq "function" && defined($function_table{$name}
{
&$func(@_);
$section_counter++;
@@ -2677,6 +2692,16 @@ sub process_file($) {
$. = 1;
+# two passes for -export and -internal
+if ($function_only == 3 || $function_only == 4) {
+ while () {
+ if (/$export_symbol/o) {
+ $function_table{$2} = 1;
+ }
+ }
+ seek(IN, 0, 0);
+}
+
$section_counter = 0;
while () {
while (s/\\\s*$//) {
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 08/10] Documentation: add kernel hacking rst
Crude conversion. This one doesn't use kernel-doc. Signed-off-by: Jani Nikula --- Documentation/index.rst | 5 +- Documentation/kernel-hacking.rst | 795 +++ 2 files changed, 799 insertions(+), 1 deletion(-) create mode 100644 Documentation/kernel-hacking.rst diff --git a/Documentation/index.rst b/Documentation/index.rst index 71a276f34c7f..1ed1c4e91c27 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -6,13 +6,16 @@ Welcome to The Linux Kernel's documentation! -Nothing for you to see here *yet*. Please move along. +This is a hacked together set of pages generated from the kernel .tmpl files +using sphinx. There's lots of glitches and such still - lots of details to work +out, but at least it's a start. Contents: .. toctree:: :maxdepth: 2 + kernel-hacking Indices and tables == diff --git a/Documentation/kernel-hacking.rst b/Documentation/kernel-hacking.rst new file mode 100644 index ..1b539677db20 --- /dev/null +++ b/Documentation/kernel-hacking.rst @@ -0,0 +1,795 @@ + +Unreliable Guide To Hacking The Linux Kernel + + +:Author: Rusty Russell + +Introduction + + +Welcome, gentle reader, to Rusty's Remarkably Unreliable Guide to Linux +Kernel Hacking. This document describes the common routines and general +requirements for kernel code: its goal is to serve as a primer for Linux +kernel development for experienced C programmers. I avoid implementation +details: that's what the code is for, and I ignore whole tracts of +useful routines. + +Before you read this, please understand that I never wanted to write +this document, being grossly under-qualified, but I always wanted to +read it, and this was the only way. I hope it will grow into a +compendium of best practice, common starting points and random +information. + +The Players +=== + +At any time each of the CPUs in a system can be: + +- not associated with any process, serving a hardware interrupt; + +- not associated with any process, serving a softirq or tasklet; + +- running in kernel space, associated with a process (user context); + +- running a process in user space. + +There is an ordering between these. The bottom two can preempt each +other, but above that is a strict hierarchy: each can only be preempted +by the ones above it. For example, while a softirq is running on a CPU, +no other softirq will preempt it, but a hardware interrupt can. However, +any other CPUs in the system execute independently. + +We'll see a number of ways that the user context can block interrupts, +to become truly non-preemptable. + +User Context + + +User context is when you are coming in from a system call or other trap: +like userspace, you can be preempted by more important tasks and by +interrupts. You can sleep, by calling :c:func:`schedule`. + +**Note** + +You are always in user context on module load and unload, and on +operations on the block device layer. + +In user context, the ``current`` pointer (indicating the task we are +currently executing) is valid, and :c:func:`in_interrupt` +(``include/linux/interrupt.h``) is false. + +**Caution** + +Beware that if you have preemption or softirqs disabled (see below), +:c:func:`in_interrupt` will return a false positive. + +Hardware Interrupts (Hard IRQs) +--- + +Timer ticks, network cards and keyboard are examples of real hardware +which produce interrupts at any time. The kernel runs interrupt +handlers, which services the hardware. The kernel guarantees that this +handler is never re-entered: if the same interrupt arrives, it is queued +(or dropped). Because it disables interrupts, this handler has to be +fast: frequently it simply acknowledges the interrupt, marks a 'software +interrupt' for execution and exits. + +You can tell you are in a hardware interrupt, because +:c:func:`in_irq` returns true. + +**Caution** + +Beware that this will return a false positive if interrupts are +disabled (see below). + +Software Interrupt Context: Softirqs and Tasklets +- + +Whenever a system call is about to return to userspace, or a hardware +interrupt handler exits, any 'software interrupts' which are marked +pending (usually by hardware interrupts) are run (``kernel/softirq.c``). + +Much of the real interrupt handling work is done here. Early in the +transition to SMP, there were only 'bottom halves' (BHs), which didn't +take advantage of multiple CPUs. Shortly after we switched from wind-up +computers made of match-sticks and snot, we abandoned this limitation +and switched to 'softirqs'. + +``include/linux/interrupt.h`` lists the different softirqs. A very +important softirq is the timer softirq (``include/linux/timer.h``):
[PATCH 05/10] Documentation/sphinx: add Sphinx kernel-doc directive extension
Add an extension to handle kernel-doc directives, to call kernel-doc
according to the arguments and parameters given to the reStructuredText
directive.
The syntax for the kernel-doc directive is:
.. kernel-doc:: FILENAME
:export:
:internal:
:functions: FUNCTION [FUNCTION ...]
:doc: SECTION TITLE
Of the directive options export, internal, functions, and doc, currently
only one option may be given at a time.
The FILENAME is relative from the kernel source tree root.
The extension notifies Sphinx about the document dependency on FILENAME,
causing the document to be rebuilt when the file has been changed.
Signed-off-by: Jani Nikula
---
Documentation/sphinx/kernel-doc.py | 99 ++
1 file changed, 99 insertions(+)
create mode 100644 Documentation/sphinx/kernel-doc.py
diff --git a/Documentation/sphinx/kernel-doc.py
b/Documentation/sphinx/kernel-doc.py
new file mode 100644
index ..21f61373daf0
--- /dev/null
+++ b/Documentation/sphinx/kernel-doc.py
@@ -0,0 +1,99 @@
+# coding=utf-8
+#
+# Copyright © 2016 Intel Corporation
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+# Authors:
+#Jani Nikula
+
+import os
+import subprocess
+import sys
+
+from docutils import nodes, statemachine
+from docutils.parsers.rst import directives
+from sphinx.util.compat import Directive
+
+class KernelDocDirective(Directive):
+"""Extract kernel-doc comments from the specified file"""
+required_argument = 1
+optional_arguments = 4
+option_spec = {
+'doc': directives.unchanged_required,
+'functions': directives.unchanged_required,
+'export': directives.flag,
+'internal': directives.flag,
+}
+has_content = False
+
+def run(self):
+env = self.state.document.settings.env
+cmd = [env.config.kerneldoc_bin, '-rst']
+
+filename = env.config.kerneldoc_srctree + '/' + self.arguments[0]
+
+# Tell sphinx of the dependency
+env.note_dependency(os.path.abspath(filename))
+
+tab_width = self.options.get('tab-width',
self.state.document.settings.tab_width)
+source = self.state_machine.input_lines.source(self.lineno -
self.state_machine.input_offset - 1)
+
+# FIXME: make this nicer and more robust against errors
+if 'export' in self.options:
+cmd += ['-export']
+elif 'internal' in self.options:
+cmd += ['-internal']
+elif 'doc' in self.options:
+cmd += ['-function', str(self.options.get('doc'))]
+elif 'functions' in self.options:
+for f in str(self.options.get('functions')).split(' '):
+cmd += ['-function', f]
+
+cmd += [filename]
+
+try:
+env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
+
+p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
+out, err = p.communicate()
+
+if p.returncode != 0:
+sys.stderr.write(err)
+
+env.app.warn('kernel-doc \'%s\' failed with return code %d' %
(" ".join(cmd), p.returncode))
+return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
+elif env.config.kerneldoc_verbosity > 0:
+sys.stderr.write(err)
+
+lines = statemachine.string2lines(out, tab_width,
convert_whitespace=True)
+self.state_machine.insert_input(lines, source)
+return []
+except Exception as e:
+env.app.warn('kernel-doc \'%s\' processing failed with: %s' %
+ (" ".join(cmd), str(e)))
+return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
+
+def setup(app):
+app.add_config_value('kerneldoc_bin', None, 'env')
+app.add_config_value('kerneldoc_srctree', None, 'env')
+app.add_config_value('kerneldoc_verbosity', 1, 'e
[PATCH 01/10] kernel-doc: fix use of uninitialized value
I'm not quite sure why the errors below are happening, but this fixes
them.
Use of uninitialized value in string ne at ./scripts/kernel-doc line 1819,
line 6494.
Use of uninitialized value $_[0] in join or string at ./scripts/kernel-doc line
1759, line 6494.
Signed-off-by: Jani Nikula
---
scripts/kernel-doc | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 2fc8fad5195e..babb374c043d 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1803,7 +1803,8 @@ sub output_function_rst(%) {
} else {
print " ``$parameter``\n";
}
- if ($args{'parameterdescs'}{$parameter_name} ne $undescribed) {
+ if (defined($args{'parameterdescs'}{$parameter_name}) &&
+ $args{'parameterdescs'}{$parameter_name} ne $undescribed) {
my $oldprefix = $lineprefix;
$lineprefix = "";
output_highlight_rst($args{'parameterdescs'}{$parameter_name});
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 07/10] sphinx: cheesy script to convert .tmpl files
From: Jonathan Corbet This script uses pandoc to convert existing DocBook template files to RST templates. A couple of sed scripts are need to massage things both before and after the conversion, but the result is then usable with no hand editing. [Jani: Change usage to tmplcvt . Fix escaping for docproc directives. Add support the new kernel-doc extension.] Signed-off-by: Jonathan Corbet Signed-off-by: Jani Nikula --- Documentation/sphinx/convert_template.sed | 14 ++ Documentation/sphinx/post_convert.sed | 19 +++ Documentation/sphinx/tmplcvt | 19 +++ 3 files changed, 52 insertions(+) create mode 100644 Documentation/sphinx/convert_template.sed create mode 100644 Documentation/sphinx/post_convert.sed create mode 100755 Documentation/sphinx/tmplcvt diff --git a/Documentation/sphinx/convert_template.sed b/Documentation/sphinx/convert_template.sed new file mode 100644 index ..d53bb8220a26 --- /dev/null +++ b/Documentation/sphinx/convert_template.sed @@ -0,0 +1,14 @@ +# +# Pandoc doesn't grok or , so convert them +# ahead of time. +# +# Use "$bq" instead of "`" so that pandoc won't mess with it. +# +s%\([^<(]\+\)()%:c:func:$bq\1$bq%g +s%\([^<(]\+\)%:c:func:$bq\1$bq%g +s%struct *\([^<]\+\)%:ref:$bqstruct \1$bq%g +s%\([^<]\+\)%:ref:$bqstruct \1$bq%g +# +# Wrap docproc directives in para and code blocks. +# +s%^\(!.*\)$%DOCPROC: \1% diff --git a/Documentation/sphinx/post_convert.sed b/Documentation/sphinx/post_convert.sed new file mode 100644 index ..ce17049f6a8e --- /dev/null +++ b/Documentation/sphinx/post_convert.sed @@ -0,0 +1,19 @@ +# +# pandoc thinks that both "_" needs to be escaped. Remove the extra +# backslashes. Also put in proper backquotes now that pandoc won't quote +# them. +# +s/$bq/`/g +s/\\_/_/g +# +# Unwrap docproc directives. +# +s/^``DOCPROC: !E\(.*\)``$/.. kernel-doc:: \1\n :export:/ +s/^``DOCPROC: !I\(.*\)``$/.. kernel-doc:: \1\n :internal:/ +s/^``DOCPROC: !F\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n :functions: \2/ +s/^``DOCPROC: !P\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n :doc: \2/ +s/^``DOCPROC: \(!.*\)``$/.. WARNING: DOCPROC directive not supported: \1/ +# +# Trim trailing whitespace. +# +s/[[:space:]]*$// diff --git a/Documentation/sphinx/tmplcvt b/Documentation/sphinx/tmplcvt new file mode 100755 index ..909a73065e0a --- /dev/null +++ b/Documentation/sphinx/tmplcvt @@ -0,0 +1,19 @@ +#!/bin/bash +# +# Convert a template file into something like RST +# +# fix +# feed to pandoc +# fix \_ +# title line? +# + +in=$1 +rst=$2 +tmp=$rst.tmp + +cp $in $tmp +sed --in-place -f convert_template.sed $tmp +pandoc -s -S -f docbook -t rst -o $rst $tmp +sed --in-place -f post_convert.sed $rst +rm $tmp -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 1/2] Input: rotary-encoder- Add support for absolute encoder
On Thu, May 19, 2016 at 02:34:00PM +0530, Vignesh R wrote:
> There are rotary-encoders where GPIO lines reflect the actual position
> of the rotary encoder dial. For example, if dial points to 9, then four
> GPIO lines connected to the rotary encoder will read HLLH(1001b = 9).
> Add support for such rotary-encoder.
> The driver relies on rotary-encoder,absolute-encoder DT property to
> detect such encoders.
> Since, GPIO IRQs are not necessary to work with
> such encoders, optional polling mode support is added using
> input_poll_dev skeleton. This is can be used by enabling
> CONFIG_INPUT_GPIO_ROTARY_ENCODER_POLL_MODE_SUPPORT.
Does this really belong to a rotary encoder and not a new driver that
simply translates gpio-encoded value into ABS* event?
>
> Signed-off-by: Vignesh R
> ---
> .../devicetree/bindings/input/rotary-encoder.txt | 4 +
> Documentation/input/rotary-encoder.txt | 9 ++
> drivers/input/misc/Kconfig | 11 ++
> drivers/input/misc/rotary_encoder.c| 165
> -
> 4 files changed, 155 insertions(+), 34 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/input/rotary-encoder.txt
> b/Documentation/devicetree/bindings/input/rotary-encoder.txt
> index 6c9f0c8a846c..9c928dbd1500 100644
> --- a/Documentation/devicetree/bindings/input/rotary-encoder.txt
> +++ b/Documentation/devicetree/bindings/input/rotary-encoder.txt
> @@ -12,6 +12,10 @@ Optional properties:
> - rotary-encoder,relative-axis: register a relative axis rather than an
>absolute one. Relative axis will only generate +1/-1 events on the input
>device, hence no steps need to be passed.
> +- rotary-encoder,absolute-encoder: support encoders where GPIO lines
> + reflect the actual position of the rotary encoder dial. For example,
> + if dial points to 9, then four GPIO lines read HLLH(1001b = 9).
> + In this case, rotary-encoder,steps-per-period needed not be defined.
> - rotary-encoder,rollover: Automatic rollove when the rotary value becomes
>greater than the specified steps or smaller than 0. For absolute axis only.
> - rotary-encoder,steps-per-period: Number of steps (stable states) per
> period.
> diff --git a/Documentation/input/rotary-encoder.txt
> b/Documentation/input/rotary-encoder.txt
> index 46a74f0c551a..dc76092f5618 100644
> --- a/Documentation/input/rotary-encoder.txt
> +++ b/Documentation/input/rotary-encoder.txt
> @@ -36,6 +36,15 @@ The phase diagram of these two outputs look like this:
> |<>|
> one step (quarter-period mode)
>
> +
> +In some encoders, the value pointed by rotary encoders switch is directly
> +reflected by status of the GPIOs connecting rotary encoder to CPU. For
> +example, if rotary encoder dial points to 9, then the four GPIOs
> +conncting rotary encoder will read HLLH(1001b = 9). Such encoders are
> +supported by defining devictree binding "rotary-encoder,absolute-encoder".
> +These encoders are also supported without need for event(IRQ) generation
> +using Input subsytem's polling mode support.
> +
> For more information, please see
> https://en.wikipedia.org/wiki/Rotary_encoder
>
> diff --git a/drivers/input/misc/Kconfig b/drivers/input/misc/Kconfig
> index 1f2337abcf2f..81805f9a534c 100644
> --- a/drivers/input/misc/Kconfig
> +++ b/drivers/input/misc/Kconfig
> @@ -566,6 +566,17 @@ config INPUT_GPIO_ROTARY_ENCODER
> To compile this driver as a module, choose M here: the
> module will be called rotary_encoder.
>
> +config INPUT_GPIO_ROTARY_ENCODER_POLL_MODE_SUPPORT
> + bool "Polling mode support for rotary encoder"
> + depends on INPUT_GPIO_ROTARY_ENCODER
> + select INPUT_POLLDEV
> + help
> + Say Y here to enable polling mode support for rotary encoders
> + connected to GPIO lines that do have interrupt capability and
> + report rotary encoder position as absolute value over GPIO
> + lines. Check file: Documentation/input/rotary-encoder.txt for
> + more information.
> +
> config INPUT_RB532_BUTTON
> tristate "Mikrotik Routerboard 532 button interface"
> depends on MIKROTIK_RB532
> diff --git a/drivers/input/misc/rotary_encoder.c
> b/drivers/input/misc/rotary_encoder.c
> index c7fc8d4fb080..8f60289fb6cd 100644
> --- a/drivers/input/misc/rotary_encoder.c
> +++ b/drivers/input/misc/rotary_encoder.c
> @@ -25,24 +25,29 @@
> #include
> #include
> #include
> +#include
>
> #define DRV_NAME "rotary-encoder"
>
> struct rotary_encoder {
> struct input_dev *input;
> -
> +#ifdef CONFIG_INPUT_GPIO_ROTARY_ENCODER_POLL_MODE_SUPPORT
> + struct input_polled_dev *poll_dev;
> +#endif
> struct mutex access_mutex;
>
> u32 steps;
> u32 axis;
> bool relative_axis;
> bool rollover;
> + bool absolute_encoder;
>
> unsigned int pos;
>
> struct gpio_descs *gpios;
> + struct device *dev;
>
> - unsigned int *irq;
>
Re: [RFC PATCH 1/2] Input: rotary-encoder- Add support for absolute encoder
On Thu, May 19, 2016 at 02:34:00PM +0530, Vignesh R wrote: > There are rotary-encoders where GPIO lines reflect the actual position > of the rotary encoder dial. For example, if dial points to 9, then four > GPIO lines connected to the rotary encoder will read HLLH(1001b = 9). > Add support for such rotary-encoder. > The driver relies on rotary-encoder,absolute-encoder DT property to > detect such encoders. > Since, GPIO IRQs are not necessary to work with > such encoders, optional polling mode support is added using > input_poll_dev skeleton. This is can be used by enabling > CONFIG_INPUT_GPIO_ROTARY_ENCODER_POLL_MODE_SUPPORT. > > Signed-off-by: Vignesh R > --- > .../devicetree/bindings/input/rotary-encoder.txt | 4 + > Documentation/input/rotary-encoder.txt | 9 ++ > drivers/input/misc/Kconfig | 11 ++ > drivers/input/misc/rotary_encoder.c| 165 > - > 4 files changed, 155 insertions(+), 34 deletions(-) > > diff --git a/Documentation/devicetree/bindings/input/rotary-encoder.txt > b/Documentation/devicetree/bindings/input/rotary-encoder.txt > index 6c9f0c8a846c..9c928dbd1500 100644 > --- a/Documentation/devicetree/bindings/input/rotary-encoder.txt > +++ b/Documentation/devicetree/bindings/input/rotary-encoder.txt > @@ -12,6 +12,10 @@ Optional properties: > - rotary-encoder,relative-axis: register a relative axis rather than an >absolute one. Relative axis will only generate +1/-1 events on the input >device, hence no steps need to be passed. > +- rotary-encoder,absolute-encoder: support encoders where GPIO lines > + reflect the actual position of the rotary encoder dial. For example, > + if dial points to 9, then four GPIO lines read HLLH(1001b = 9). > + In this case, rotary-encoder,steps-per-period needed not be defined. I think this should be done as a new compatible string rather than a property. It seems like fundamentally different h/w. Rob -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
