Re: [PATCH v2 22/29] docs: iio: convert to ReST
On Tue, 18 Jun 2019 17:53:40 -0300
Mauro Carvalho Chehab wrote:
> Rename the iio documentation files to ReST, add an
> index for them and adjust in order to produce a nice html
> output via the Sphinx build system.
>
> At its new index.rst, let's add a :orphan: while this is not linked to
> the main index.rst file, in order to avoid build warnings.
>
> Signed-off-by: Mauro Carvalho Chehab
> Acked-by: Jonathan Cameron
I'll take this via IIO.
Applied to the togreg branch of iio.git and pushed out as testing.
I'll probably do a pull request to Greg tomorrow and these will hopefully
turn up in linux-next not long after that.
Thanks Mauro!
Jonathan
> ---
> .../iio/{ep93xx_adc.txt => ep93xx_adc.rst}| 15 +-
> .../{iio_configfs.txt => iio_configfs.rst}| 52 +++
> Documentation/iio/index.rst | 12 +
> drivers/iio/Kconfig | 2 +-
> 4 files changed, 56 insertions(+), 25 deletions(-)
> rename Documentation/iio/{ep93xx_adc.txt => ep93xx_adc.rst} (71%)
> rename Documentation/iio/{iio_configfs.txt => iio_configfs.rst} (73%)
> create mode 100644 Documentation/iio/index.rst
>
> diff --git a/Documentation/iio/ep93xx_adc.txt
> b/Documentation/iio/ep93xx_adc.rst
> similarity index 71%
> rename from Documentation/iio/ep93xx_adc.txt
> rename to Documentation/iio/ep93xx_adc.rst
> index 23053e7817bd..4fd8dea3f6b8 100644
> --- a/Documentation/iio/ep93xx_adc.txt
> +++ b/Documentation/iio/ep93xx_adc.rst
> @@ -1,12 +1,16 @@
> -Cirrus Logic EP93xx ADC driver.
> +==
> +Cirrus Logic EP93xx ADC driver
> +==
>
> 1. Overview
> +===
>
> The driver is intended to work on both low-end (EP9301, EP9302) devices with
> 5-channel ADC and high-end (EP9307, EP9312, EP9315) devices with 10-channel
> touchscreen/ADC module.
>
> 2. Channel numbering
> +
>
> Numbering scheme for channels 0..4 is defined in EP9301 and EP9302
> datasheets.
> EP9307, EP9312 and EP9312 have 3 channels more (total 8), but the numbering
> is
> @@ -17,13 +21,20 @@ Assuming ep93xx_adc is IIO device0, you'd find the
> following entries under
>
>+-+---+
>| sysfs entry | ball/pin name |
> - +-+---+
> + +=+===+
>| in_voltage0_raw | YM|
> + +-+---+
>| in_voltage1_raw | SXP |
> + +-+---+
>| in_voltage2_raw | SXM |
> + +-+---+
>| in_voltage3_raw | SYP |
> + +-+---+
>| in_voltage4_raw | SYM |
> + +-+---+
>| in_voltage5_raw | XP|
> + +-+---+
>| in_voltage6_raw | XM|
> + +-+---+
>| in_voltage7_raw | YP|
>+-+---+
> diff --git a/Documentation/iio/iio_configfs.txt
> b/Documentation/iio/iio_configfs.rst
> similarity index 73%
> rename from Documentation/iio/iio_configfs.txt
> rename to Documentation/iio/iio_configfs.rst
> index 4e5f101837a8..ecbfdb3afef7 100644
> --- a/Documentation/iio/iio_configfs.txt
> +++ b/Documentation/iio/iio_configfs.rst
> @@ -1,6 +1,9 @@
> +===
> Industrial IIO configfs support
> +===
>
> 1. Overview
> +===
>
> Configfs is a filesystem-based manager of kernel objects. IIO uses some
> objects that could be easily configured using configfs (e.g.: devices,
> @@ -10,20 +13,22 @@ See Documentation/filesystems/configfs/configfs.txt for
> more information
> about how configfs works.
>
> 2. Usage
> +
>
> In order to use configfs support in IIO we need to select it at compile
> time via CONFIG_IIO_CONFIGFS config option.
>
> -Then, mount the configfs filesystem (usually under /config directory):
> +Then, mount the configfs filesystem (usually under /config directory)::
>
> -$ mkdir /config
> -$ mount -t configfs none /config
> + $ mkdir /config
> + $ mount -t configfs none /config
>
> At this point, all default IIO groups will be created and can be accessed
> under /config/iio. Next chapters will describe available IIO configuration
> objects.
>
> 3. Software triggers
> +
>
> One of the IIO default configfs groups is the "triggers" group. It is
> automagically accessible when the configfs is mounted and can be found
> @@ -31,40 +36,40 @@ under /config/iio/triggers.
>
> IIO software triggers implementation offers support for creating multiple
> trigger types. A new trigger type is usually implemented as a separate
> -kernel module following the interface in include/linux/iio/sw_trigger.h:
> +kernel module following the interface in include/linux/iio/sw_trigger.h::
>
> -/*
> - * drivers/iio/trigger/iio-tri
Re: [PATCH v12 04/13] mfd: Add Ingenic TCU driver
Hi,
I'll make a V13 of this patchset on top of -rc6, any feedback
for this MFD driver? It's been already one month.
Thanks,
-Paul
Le mar. 21 mai 2019 à 16:51, Paul Cercueil a
écrit :
This driver will provide a regmap that can be retrieved very early in
the boot process through the API function ingenic_tcu_get_regmap().
Additionally, it will call devm_of_platform_populate() so that all the
children devices will be probed.
Signed-off-by: Paul Cercueil
---
Notes:
v12: New patch
drivers/mfd/Kconfig | 8 +++
drivers/mfd/Makefile| 1 +
drivers/mfd/ingenic-tcu.c | 113
include/linux/mfd/ingenic-tcu.h | 8 +++
4 files changed, 130 insertions(+)
create mode 100644 drivers/mfd/ingenic-tcu.c
diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig
index 294d9567cc71..a13544474e05 100644
--- a/drivers/mfd/Kconfig
+++ b/drivers/mfd/Kconfig
@@ -494,6 +494,14 @@ config HTC_I2CPLD
This device provides input and output GPIOs through an I2C
interface to one or more sub-chips.
+config INGENIC_TCU
+ bool "Ingenic Timer/Counter Unit (TCU) support"
+ depends on MIPS || COMPILE_TEST
+ select REGMAP_MMIO
+ help
+ Say yes here to support the Timer/Counter Unit (TCU) IP present
+ in the JZ47xx SoCs from Ingenic.
+
config MFD_INTEL_QUARK_I2C_GPIO
tristate "Intel Quark MFD I2C GPIO"
depends on PCI
diff --git a/drivers/mfd/Makefile b/drivers/mfd/Makefile
index 52b1a90ff515..fb89e131ae98 100644
--- a/drivers/mfd/Makefile
+++ b/drivers/mfd/Makefile
@@ -180,6 +180,7 @@ obj-$(CONFIG_AB8500_CORE) += ab8500-core.o
ab8500-sysctrl.o
obj-$(CONFIG_MFD_TIMBERDALE)+= timberdale.o
obj-$(CONFIG_PMIC_ADP5520) += adp5520.o
obj-$(CONFIG_MFD_KEMPLD) += kempld-core.o
+obj-$(CONFIG_INGENIC_TCU) += ingenic-tcu.o
obj-$(CONFIG_MFD_INTEL_QUARK_I2C_GPIO) += intel_quark_i2c_gpio.o
obj-$(CONFIG_LPC_SCH) += lpc_sch.o
obj-$(CONFIG_LPC_ICH) += lpc_ich.o
diff --git a/drivers/mfd/ingenic-tcu.c b/drivers/mfd/ingenic-tcu.c
new file mode 100644
index ..6c1d5e4310c1
--- /dev/null
+++ b/drivers/mfd/ingenic-tcu.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * JZ47xx SoCs TCU MFD driver
+ * Copyright (C) 2019 Paul Cercueil
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+struct ingenic_soc_info {
+ unsigned int num_channels;
+};
+
+static struct regmap *tcu_regmap __initdata;
+
+static const struct regmap_config ingenic_tcu_regmap_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+ .max_register = TCU_REG_OST_CNTHBUF,
+};
+
+static const struct ingenic_soc_info jz4740_soc_info = {
+ .num_channels = 8,
+};
+
+static const struct ingenic_soc_info jz4725b_soc_info = {
+ .num_channels = 6,
+};
+
+static const struct of_device_id ingenic_tcu_of_match[] = {
+ { .compatible = "ingenic,jz4740-tcu", .data = &jz4740_soc_info, },
+ { .compatible = "ingenic,jz4725b-tcu", .data = &jz4725b_soc_info, },
+ { .compatible = "ingenic,jz4770-tcu", .data = &jz4740_soc_info, },
+ { }
+};
+
+static struct regmap * __init ingenic_tcu_create_regmap(struct
device_node *np)
+{
+ struct resource res;
+ void __iomem *base;
+ struct regmap *map;
+
+ if (!of_match_node(ingenic_tcu_of_match, np))
+ return ERR_PTR(-EINVAL);
+
+ base = of_io_request_and_map(np, 0, "TCU");
+ if (IS_ERR(base))
+ return ERR_PTR(PTR_ERR(base));
+
+ map = regmap_init_mmio(NULL, base, &ingenic_tcu_regmap_config);
+ if (IS_ERR(map))
+ goto err_iounmap;
+
+ return map;
+
+err_iounmap:
+ iounmap(base);
+ of_address_to_resource(np, 0, &res);
+ release_mem_region(res.start, resource_size(&res));
+
+ return map;
+}
+
+static int __init ingenic_tcu_probe(struct platform_device *pdev)
+{
+ struct regmap *map = ingenic_tcu_get_regmap(pdev->dev.of_node);
+
+ platform_set_drvdata(pdev, map);
+
+ regmap_attach_dev(&pdev->dev, map, &ingenic_tcu_regmap_config);
+
+ return devm_of_platform_populate(&pdev->dev);
+}
+
+static struct platform_driver ingenic_tcu_driver = {
+ .driver = {
+ .name = "ingenic-tcu",
+ .of_match_table = ingenic_tcu_of_match,
+ },
+};
+
+static int __init ingenic_tcu_platform_init(void)
+{
+ return platform_driver_probe(&ingenic_tcu_driver,
+ingenic_tcu_probe);
+}
+subsys_initcall(ingenic_tcu_platform_init);
+
+struct regmap * __init ingenic_tcu_get_regmap(struct device_node *np)
+{
+ if (!tcu_regmap)
+ tcu_regmap = ingenic_tcu_create_regmap(np);
+
+ return tcu_regmap;
+}
+
+bool ingenic_tcu_pwm_can_use_chn(struct device *dev, unsigned int
channel)
+{
+ const struct ingenic_soc_info *soc =
device_get_match_data(dev->parent);
+
+ /* En
Re: [PATCH 1/3] Docs: An initial automarkup extension for sphinx
On Fri, 21 Jun 2019 22:00:46 -0300 Mauro Carvalho Chehab wrote: > > +# > > +# The DVB docs create references for these basic system calls, leading > > +# to lots of confusing links. So just don't link them. > > +# > > +Skipfuncs = [ 'open', 'close', 'write' ] > > and yeah, of course, if there's something weird, it has to be at > the media docs :-) > > Btw, if I'm not mistaken, we do the same for ioctl. So that's actually interesting. In, for example, Documentation/media/uapi/v4l/func-ioctl.rst, you see something that looks like this: > .. c:function:: int ioctl( int fd, int request, void *argp ) > :name: v4l2-ioctl Some digging around didn't turn up any documentation for :name:, but it seems to prevent ioctl() from going into the list of functions that can be cross-referenced. I wonder if the same should be done for the others? I think that would be better than putting a special-case hack into the toolchain. > I'm wandering if this could also handle the Documentation/* auto-replace. I think it's the obvious place for it, yes. Let's make sure I haven't badly broken anything with the existing change first, though :) Thanks, jon
Re: [PATCH v12 07/13] clocksource: Add a new timer-ingenic driver
Hi, I'll make a V13 of this patchset on top of -rc6, any feedback for this clocksource driver? It's been already one month. Thanks, -Paul Le mar. 21 mai 2019 à 16:51, Paul Cercueil a écrit : This driver handles the TCU (Timer Counter Unit) present on the Ingenic JZ47xx SoCs, and provides the kernel with a system timer, a clocksource and a sched_clock. Signed-off-by: Paul Cercueil --- Notes: v2: Use SPDX identifier for the license v3: - Move documentation to its own patch - Search the devicetree for PWM clients, and use all the TCU channels that won't be used for PWM v4: - Add documentation about why we search for PWM clients - Verify that the PWM clients are for the TCU PWM driver v5: Major overhaul. Too many changes to list. Consider it's a new patch. v6: - Add two API functions ingenic_tcu_request_channel and ingenic_tcu_release_channel. To be used by the PWM driver to request the use of a TCU channel. The driver will now dynamically move away the system timer or clocksource to a new TCU channel. - The system timer now defaults to channel 0, the clocksource now defaults to channel 1 and is no more optional. The ingenic,timer-channel and ingenic,clocksource-channel devicetree properties are now gone. - Fix round_rate / set_rate not calculating the prescale divider the same way. This caused problems when (parent_rate / div) would give a non-integer result. The behaviour is correct now. - The clocksource clock is turned off on suspend now. v7: Fix section mismatch by using builtin_platform_driver_probe() v8: - Removed ingenic_tcu_[request,release]_channel, and the mechanism to dynamically change the TCU channel of the system timer or the clocksource. - The driver's devicetree node can now have two more children nodes, that correspond to the system timer and clocksource. For these two, the driver will use the TCU timer that correspond to the memory resource supplied in their respective node. v9: - Removed support for clocksource / timer children devicetree nodes. Now, we use a property "ingenic,pwm-channels-mask" to know which PWM channels are reserved for PWM use and should not be used as OS timers. v10: - Use CLK_SET_RATE_UNGATE instead of CLK_SET_RATE_GATE + manually un-gating the clock before changing rate. Same for re-parenting. - Unconditionally create the clocksource and sched_clock even if the SoC possesses a OS Timer. That gives the choice back to the user which clocksource should be selected. - Use subsys_initcall() instead of builtin_platform_driver_probe(). The OS Timer driver calls builtin_platform_driver_probe, which requires the device to be created before that. - Cosmetic cleanups v11: - Change prototype of exported function ingenic_tcu_pwm_can_use_chn(), use a struct device * as first argument. - Read clocksource using the regmap instead of bypassing it. Bypassing the regmap makes sense only for the sched_clock where the read operation must be as fast as possible. - Fix incorrect format in pr_crit() macro v12: - Clock handling and IRQ handling are gone, and are now handled in their own driver. - Obtain regmap from the ingenic-tcu MFD driver. As a result, we cannot bypass the regmap anymore for the sched_clock. drivers/clocksource/Kconfig | 11 + drivers/clocksource/Makefile| 1 + drivers/clocksource/ingenic-timer.c | 357 3 files changed, 369 insertions(+) create mode 100644 drivers/clocksource/ingenic-timer.c diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig index 6bcaa4e2e72c..bb5d5c044341 100644 --- a/drivers/clocksource/Kconfig +++ b/drivers/clocksource/Kconfig @@ -672,4 +672,15 @@ config MILBEAUT_TIMER help Enables the support for Milbeaut timer driver. +config INGENIC_TIMER + bool "Clocksource/timer using the TCU in Ingenic JZ SoCs" + default MACH_INGENIC + depends on MIPS || COMPILE_TEST + depends on COMMON_CLK + select INGENIC_TCU + select TIMER_OF + select IRQ_DOMAIN + help + Support for the timer/counter unit of the Ingenic JZ SoCs. + endmenu diff --git a/drivers/clocksource/Makefile b/drivers/clocksource/Makefile index 236858fa7fbf..553f3c61717a 100644 --- a/drivers/clocksource/Makefile +++ b/drivers/clocksource/Makefile @@ -78,6 +78,7 @@ obj-$(CONFIG_ASM9260_TIMER) += asm9260_timer.o obj-$(CONFIG_H8300_TMR8) += h8300_timer8.o obj-$(CONFIG_H8300_TMR16) += h8300_timer16.o obj-$(CONFIG_H8300_TPU)+= h83
Re: [PATCH v12 06/13] irqchip: Add irq-ingenic-tcu driver
Hi,
I'll make a V13 of this patchset on top of -rc6, any feedback
for this IRQ driver? It's been already one month.
Thanks,
-Paul
Le mar. 21 mai 2019 à 16:51, Paul Cercueil a
écrit :
This driver handles the interrupt controller built in the
Timer/Counter
Unit (TCU) of the JZ47xx SoCs from Ingenic.
Signed-off-by: Paul Cercueil
---
drivers/irqchip/Kconfig | 11 ++
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-ingenic-tcu.c | 182
++
3 files changed, 194 insertions(+)
create mode 100644 drivers/irqchip/irq-ingenic-tcu.c
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 1c1f3f66dfd3..2d0700e52db7 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -293,6 +293,17 @@ config INGENIC_IRQ
depends on MACH_INGENIC
default y
+config INGENIC_TCU_IRQ
+ bool "Ingenic JZ47xx TCU interrupt controller"
+ default MACH_INGENIC
+ depends on MIPS || COMPILE_TEST
+ select INGENIC_TCU
+ help
+ Support for interrupts in the Timer/Counter Unit (TCU) of the
Ingenic
+ JZ47xx SoCs.
+
+ If unsure, say N.
+
config RENESAS_H8300H_INTC
bool
select IRQ_DOMAIN
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 606a003a..f403b2c221e4 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -73,6 +73,7 @@ obj-$(CONFIG_RENESAS_H8300H_INTC) +=
irq-renesas-h8300h.o
obj-$(CONFIG_RENESAS_H8S_INTC) += irq-renesas-h8s.o
obj-$(CONFIG_ARCH_SA1100) += irq-sa11x0.o
obj-$(CONFIG_INGENIC_IRQ) += irq-ingenic.o
+obj-$(CONFIG_INGENIC_TCU_IRQ) += irq-ingenic-tcu.o
obj-$(CONFIG_IMX_GPCV2)+= irq-imx-gpcv2.o
obj-$(CONFIG_PIC32_EVIC) += irq-pic32-evic.o
obj-$(CONFIG_MSCC_OCELOT_IRQ) += irq-mscc-ocelot.o
diff --git a/drivers/irqchip/irq-ingenic-tcu.c
b/drivers/irqchip/irq-ingenic-tcu.c
new file mode 100644
index ..738ed06c983f
--- /dev/null
+++ b/drivers/irqchip/irq-ingenic-tcu.c
@@ -0,0 +1,182 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * JZ47xx SoCs TCU IRQ driver
+ * Copyright (C) 2019 Paul Cercueil
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+struct ingenic_tcu {
+ struct regmap *map;
+ struct clk *clk;
+
+ struct irq_domain *domain;
+ unsigned int nb_parent_irqs;
+ u32 parent_irqs[3];
+};
+
+static void ingenic_tcu_intc_cascade(struct irq_desc *desc)
+{
+ struct irq_chip *irq_chip = irq_data_get_irq_chip(&desc->irq_data);
+ struct irq_domain *domain = irq_desc_get_handler_data(desc);
+ struct irq_chip_generic *gc = irq_get_domain_generic_chip(domain,
0);
+ struct regmap *map = gc->private;
+ uint32_t irq_reg, irq_mask;
+ unsigned int i;
+
+ regmap_read(map, TCU_REG_TFR, &irq_reg);
+ regmap_read(map, TCU_REG_TMR, &irq_mask);
+
+ chained_irq_enter(irq_chip, desc);
+
+ irq_reg &= ~irq_mask;
+
+ for_each_set_bit(i, (unsigned long *)&irq_reg, 32)
+ generic_handle_irq(irq_linear_revmap(domain, i));
+
+ chained_irq_exit(irq_chip, desc);
+}
+
+static void ingenic_tcu_gc_unmask_enable_reg(struct irq_data *d)
+{
+ struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
+ struct irq_chip_type *ct = irq_data_get_chip_type(d);
+ struct regmap *map = gc->private;
+ u32 mask = d->mask;
+
+ irq_gc_lock(gc);
+ regmap_write(map, ct->regs.ack, mask);
+ regmap_write(map, ct->regs.enable, mask);
+ *ct->mask_cache |= mask;
+ irq_gc_unlock(gc);
+}
+
+static void ingenic_tcu_gc_mask_disable_reg(struct irq_data *d)
+{
+ struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
+ struct irq_chip_type *ct = irq_data_get_chip_type(d);
+ struct regmap *map = gc->private;
+ u32 mask = d->mask;
+
+ irq_gc_lock(gc);
+ regmap_write(map, ct->regs.disable, mask);
+ *ct->mask_cache &= ~mask;
+ irq_gc_unlock(gc);
+}
+
+static void ingenic_tcu_gc_mask_disable_reg_and_ack(struct irq_data
*d)
+{
+ struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
+ struct irq_chip_type *ct = irq_data_get_chip_type(d);
+ struct regmap *map = gc->private;
+ u32 mask = d->mask;
+
+ irq_gc_lock(gc);
+ regmap_write(map, ct->regs.ack, mask);
+ regmap_write(map, ct->regs.disable, mask);
+ irq_gc_unlock(gc);
+}
+
+static int __init ingenic_tcu_irq_init(struct device_node *np,
+ struct device_node *parent)
+{
+ struct irq_chip_generic *gc;
+ struct irq_chip_type *ct;
+ struct ingenic_tcu *tcu;
+ struct regmap *map;
+ unsigned int i;
+ int ret, irqs;
+
+ map = ingenic_tcu_get_regmap(np);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ tcu = kzalloc(sizeof(*tcu),
[PATCH 00/12] Add the ABI documentation to the admin guide
Hi Greg/Jon, That's the rebased patches with add the ABI documentation to the admin-guide. It is rebased on the top of driver-core/driver-core-next, and it can work both on the "old" mode, where the scripts will avoid codes that would cause troubles for ReST, or at the new mode, with parses the description to the ReST output untouched. Patches 1 to 3 improves the get_abi.pl for it to parse ReST format, and to report where the errors/warnings occur at the ABI file. Patch 4 is implements support for a Sphinx log facility (info). Patches 5 to 10 adds the new kernel ABI extension, making it ready to parse files in ether "normal" or "rst-source" mode. Patch 11 addresses a Sphinx issue, with is there since old vesions, but it gets worse on Sphins 2.0: basically, the ReST parser is lazy: if it receives too much stuff, it silently ignores the end of the content. So, the extension needs to split the content on small chunks. IMO, it is safe to apply patches 1 to 11 via Greg's tree. The only potential conflict would be at patch 4, but looking at docs-next, I don't see any changes at Documentation/scripts/kernelog.py. Patch 12 actually enables everything. If Patch 12 gets merged via Greg's tree, it may rise a single-line trivial conflict with docs-next, as it will be adding kernel_abi to the list of extensions at Documentation/conf.py, and Jon is likely adding another extension. too. So, it could be wise postpone patch 12 to be merged after both trees got merged upstream. Patch 12 adds ABI with ReST-scaping code. I'll submit soon a second version for the RFC patches with turns it to "clean" mode. The entire ABI series (plus Documentation/features) is at: https://git.linuxtv.org/mchehab/experimental.git/log/?h=abi_patches_v4.2 And the ReST output (with Sphinx 2.0 - with IMO provides a nicer output) is at: https://www.infradead.org/~mchehab/rst_features/admin-guide/abi.html Mauro Carvalho Chehab (12): scripts/get_abi.pl: change script to allow parsing in ReST mode scripts/get_abi.pl: fix parsing on ReST mode scripts/get_abi.pl: Allow optionally record from where a line came from docs: kernellog.py: add support for info() docs: kernel_abi.py: add a script to parse ABI documentation docs: kernel_abi.py: fix UTF-8 support docs: kernel_abi.py: make it compatible with Sphinx 1.7+ docs: kernel_abi.py: Update copyrights docs: kernel_abi.py: add a SPDX header file docs: kernel_abi.py: use --enable-lineno for get_abi.pl docs: kernel_abi.py: Handle with a lazy Sphinx parser docs: add ABI documentation to the admin-guide book Documentation/admin-guide/abi-obsolete.rst | 10 ++ Documentation/admin-guide/abi-removed.rst | 4 + Documentation/admin-guide/abi-stable.rst | 13 ++ Documentation/admin-guide/abi-testing.rst | 19 +++ Documentation/admin-guide/abi.rst | 11 ++ Documentation/admin-guide/index.rst| 1 + Documentation/conf.py | 2 +- Documentation/sphinx/kernel_abi.py | 189 + Documentation/sphinx/kernellog.py | 6 +- scripts/get_abi.pl | 104 10 files changed, 325 insertions(+), 34 deletions(-) create mode 100644 Documentation/admin-guide/abi-obsolete.rst create mode 100644 Documentation/admin-guide/abi-removed.rst create mode 100644 Documentation/admin-guide/abi-stable.rst create mode 100644 Documentation/admin-guide/abi-testing.rst create mode 100644 Documentation/admin-guide/abi.rst create mode 100644 Documentation/sphinx/kernel_abi.py -- 2.21.0
[PATCH 10/12] docs: kernel_abi.py: use --enable-lineno for get_abi.pl
Just like kernel-doc extension, we need to be able to identify
what part of an imported document has issues, as reporting them
as:
get_abi.pl rest --dir $srctree/Documentation/ABI/obsolete
--rst-source:1689: ERROR: Unexpected indentation.
Makes a lot harder for someone to fix.
It should be noticed that it the line which will be reported is
the line where the "What:" definition is, and not the line
with actually has an error.
Signed-off-by: Mauro Carvalho Chehab
---
Documentation/sphinx/kernel_abi.py | 18 ++
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py
b/Documentation/sphinx/kernel_abi.py
index 5d43cac73d0a..efa338e18764 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -35,6 +35,7 @@ import codecs
import os
import subprocess
import sys
+import re
from os import path
@@ -92,7 +93,7 @@ class KernelCmd(Directive):
env = doc.settings.env
cwd = path.dirname(doc.current_source)
-cmd = "get_abi.pl rest --dir "
+cmd = "get_abi.pl rest --enable-lineno --dir "
cmd += self.arguments[0]
srctree = path.abspath(os.environ["srctree"])
@@ -136,7 +137,7 @@ class KernelCmd(Directive):
% (self.name, ErrorString(exc)))
return out
-def nestedParse(self, lines, fname):
+def nestedParse(self, lines, f):
content = ViewList()
node= nodes.section()
@@ -146,8 +147,17 @@ class KernelCmd(Directive):
code_block += "\n" + l
lines = code_block + "\n\n"
-for c, l in enumerate(lines.split("\n")):
-content.append(l, fname, c)
+line_regex = re.compile("^#define LINENO (\S+)\#([0-9]+)$")
+ln = 0
+
+for line in lines.split("\n"):
+match = line_regex.search(line)
+if match:
+f = match.group(1)
+# sphinx counts lines from 0
+ln = int(match.group(2)) - 1
+else:
+content.append(line, f, ln)
buf = self.state.memo.title_styles, self.state.memo.section_level,
self.state.memo.reporter
--
2.21.0
[PATCH 08/12] docs: kernel_abi.py: Update copyrights
As Markus doesn't want to maintain ths file, update it to put me as its maintainer. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_abi.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index 2d5d582207f7..ef91b1e1ff4b 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -7,7 +7,8 @@ u""" Implementation of the ``kernel-abi`` reST-directive. :copyright: Copyright (C) 2016 Markus Heiser -:copyright: Copyright (C) 2016 Mauro Carvalho Chehab +:copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab +:maintained-by: Mauro Carvalho Chehab :license:GPL Version 2, June 1991 see Linux/COPYING for details. The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the -- 2.21.0
[PATCH 02/12] scripts/get_abi.pl: fix parsing on ReST mode
When the source ABI file is using ReST notation, the script
should handle whitespaces and lines with care, as otherwise
the file won't be properly recognized.
Address the bugs that are on such part of the script.
Signed-off-by: Mauro Carvalho Chehab
---
scripts/get_abi.pl | 30 ++
1 file changed, 14 insertions(+), 16 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 107672cdacb3..0c403af86fd5 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -151,7 +151,8 @@ sub parse_abi {
$content = $2;
}
while ($space =~ s/\t+/' ' x
(length($&) * 8 - length($`) % 8)/e) {}
- $data{$what}->{$tag} .= $content;
+
+ $data{$what}->{$tag} .= "$content\n" if
($content);
} else {
$data{$what}->{$tag} = $content;
}
@@ -166,31 +167,28 @@ sub parse_abi {
}
if ($tag eq "description") {
+ my $content = $_;
+ while ($content =~ s/\t+/' ' x (length($&) * 8 -
length($`) % 8)/e) {}
if (!$data{$what}->{description}) {
- s/^($space)//;
- if (m/^(\s*)(.*)/) {
- my $sp = $1;
- while ($sp =~ s/\t+/' ' x (length($&) *
8 - length($`) % 8)/e) {}
- my $content = "$sp$2";
-
- $content =~ s/^($space)//;
-
- $data{$what}->{$tag} .= "$content";
+ # Preserve initial spaces for the first line
+ if ($content =~ m/^(\s*)(.*)$/) {
+ $space = $1;
+ $content = $2;
}
+
+ $data{$what}->{$tag} .= "$content\n" if
($content);
} else {
- my $content = $_;
if (m/^\s*\n/) {
$data{$what}->{$tag} .= $content;
next;
}
- while ($content =~ s/\t+/' ' x (length($&) * 8
- length($`) % 8)/e) {}
$space = "" if (!($content =~ s/^($space)//));
- # Compress spaces with tabs
- $content =~ s<^ {8}> <\t>;
- $content =~ s<^ {1,7}\t> <\t>;
- $content =~ s< {1,7}\t> <\t>;
+# # Compress spaces with tabs
+# $content =~ s<^ {8}> <\t>;
+# $content =~ s<^ {1,7}\t> <\t>;
+# $content =~ s< {1,7}\t> <\t>;
$data{$what}->{$tag} .= $content;
}
next;
--
2.21.0
[PATCH 07/12] docs: kernel_abi.py: make it compatible with Sphinx 1.7+
The same way kerneldoc.py needed changes to work with newer Sphinx, this script needs the same changes. While here, reorganize the include order to match kerneldoc.py. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_abi.py | 39 +- 1 file changed, 27 insertions(+), 12 deletions(-) diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index 0f3e51e67e8d..2d5d582207f7 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -30,18 +30,27 @@ u""" """ import codecs -import sys import os -from os import path import subprocess +import sys -from sphinx.ext.autodoc import AutodocReporter +from os import path -from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils import nodes, statemachine from docutils.statemachine import ViewList +from docutils.parsers.rst import directives, Directive from docutils.utils.error_reporting import ErrorString +# +# AutodocReporter is only good up to Sphinx 1.7 +# +import sphinx + +Use_SSI = sphinx.__version__[:3] >= '1.7' +if Use_SSI: +from sphinx.util.docutils import switch_source_input +else: +from sphinx.ext.autodoc import AutodocReporter __version__ = '1.0' @@ -139,11 +148,17 @@ class KernelCmd(Directive): content.append(l, fname, c) buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter -self.state.memo.title_styles = [] -self.state.memo.section_level = 0 -self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter) -try: -self.state.nested_parse(content, 0, node, match_titles=1) -finally: -self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf + +if Use_SSI: +with switch_source_input(self.state, content): +self.state.nested_parse(content, 0, node, match_titles=1) +else: +self.state.memo.title_styles = [] +self.state.memo.section_level = 0 +self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter) +try: +self.state.nested_parse(content, 0, node, match_titles=1) +finally: +self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf + return node.children -- 2.21.0
[PATCH 11/12] docs: kernel_abi.py: Handle with a lazy Sphinx parser
The Sphinx docutils parser is lazy: if the content is bigger than
a certain number of lines, it silenlty stops parsing it,
producing an incomplete content. This seems to be worse on newer
Sphinx versions, like 2.0.
So, change the logic to parse the contents per input file.
Signed-off-by: Mauro Carvalho Chehab
---
Documentation/sphinx/kernel_abi.py | 39 --
1 file changed, 26 insertions(+), 13 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py
b/Documentation/sphinx/kernel_abi.py
index efa338e18764..a417026ed690 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -36,6 +36,7 @@ import os
import subprocess
import sys
import re
+import kernellog
from os import path
@@ -79,12 +80,6 @@ class KernelCmd(Directive):
"debug" : directives.flag
}
-def warn(self, message, **replace):
-replace["fname"] = self.state.document.current_source
-replace["line_no"] = replace.get("line_no", self.lineno)
-message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) %
replace
-self.state.document.settings.env.app.warn(message, prefix="")
-
def run(self):
doc = self.state.document
@@ -110,7 +105,7 @@ class KernelCmd(Directive):
shell_env["srctree"] = srctree
lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
-nodeList = self.nestedParse(lines, fname)
+nodeList = self.nestedParse(lines, self.arguments[0])
return nodeList
def runCmd(self, cmd, **kwargs):
@@ -137,9 +132,9 @@ class KernelCmd(Directive):
% (self.name, ErrorString(exc)))
return out
-def nestedParse(self, lines, f):
+def nestedParse(self, lines, fname):
content = ViewList()
-node= nodes.section()
+node = nodes.section()
if "debug" in self.options:
code_block = "\n\n.. code-block:: rst\n:linenos:\n"
@@ -149,22 +144,42 @@ class KernelCmd(Directive):
line_regex = re.compile("^#define LINENO (\S+)\#([0-9]+)$")
ln = 0
+n = 0
+f = fname
for line in lines.split("\n"):
+n = n + 1
match = line_regex.search(line)
if match:
-f = match.group(1)
+new_f = match.group(1)
+
+# Sphinx parser is lazy: it stops parsing contents in the
+# middle, if it is too big. So, handle it per input file
+if new_f != f and content:
+self.do_parse(content, node)
+content = ViewList()
+
+f = new_f
+
# sphinx counts lines from 0
ln = int(match.group(2)) - 1
else:
content.append(line, f, ln)
-buf = self.state.memo.title_styles, self.state.memo.section_level,
self.state.memo.reporter
+kernellog.info(self.state.document.settings.env.app, "%s: parsed %i
lines" % (fname, n))
+if content:
+self.do_parse(content, node)
+
+return node.children
+
+def do_parse(self, content, node):
if Use_SSI:
with switch_source_input(self.state, content):
self.state.nested_parse(content, 0, node, match_titles=1)
else:
+buf = self.state.memo.title_styles,
self.state.memo.section_level, self.state.memo.reporter
+
self.state.memo.title_styles = []
self.state.memo.section_level = 0
self.state.memo.reporter = AutodocReporter(content,
self.state.memo.reporter)
@@ -172,5 +187,3 @@ class KernelCmd(Directive):
self.state.nested_parse(content, 0, node, match_titles=1)
finally:
self.state.memo.title_styles, self.state.memo.section_level,
self.state.memo.reporter = buf
-
-return node.children
--
2.21.0
[PATCH 09/12] docs: kernel_abi.py: add a SPDX header file
This file is released under GPL v2. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_abi.py | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index ef91b1e1ff4b..5d43cac73d0a 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -1,4 +1,5 @@ # coding=utf-8 +# SPDX-License-Identifier: GPL-2.0 # u""" kernel-abi -- 2.21.0
[PATCH 03/12] scripts/get_abi.pl: Allow optionally record from where a line came from
The get_abi.pl reads a lot of files and can join them on a
single output file. Store where each "What:" output came from,
in order to be able to optionally display it.
This is useful for the Sphinx extension, with can now be
able to blame what ABI file has issues, and on what line
the What: description with problems begin.
Signed-off-by: Mauro Carvalho Chehab
---
scripts/get_abi.pl | 22 +-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 0c403af86fd5..6a4d387ebf3b 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -10,6 +10,7 @@ use Fcntl ':mode';
my $help;
my $man;
my $debug;
+my $enable_lineno;
my $prefix="Documentation/ABI";
#
@@ -19,6 +20,7 @@ my $description_is_rst = 0;
GetOptions(
"debug|d+" => \$debug,
+ "enable-lineno" => \$enable_lineno,
"rst-source!" => \$description_is_rst,
"dir=s" => \$prefix,
'help|?' => \$help,
@@ -67,6 +69,7 @@ sub parse_abi {
$data{$nametag}->{file} = $name;
$data{$nametag}->{filepath} = $file;
$data{$nametag}->{is_file} = 1;
+ $data{$nametag}->{line_no} = 1;
my $type = $file;
$type =~ s,.*/(.*)/.*,$1,;
@@ -126,6 +129,8 @@ sub parse_abi {
if ($tag ne "" && $new_tag) {
$tag = $new_tag;
+ $data{$what}->{line_no} = $ln;
+
if ($new_what) {
@{$data{$what}->{label}} = @labels if
($data{$nametag}->{what});
@labels = ();
@@ -221,6 +226,12 @@ sub output_rest {
my $file = $data{$what}->{file};
my $filepath = $data{$what}->{filepath};
+ if ($enable_lineno) {
+ printf "#define LINENO %s%s#%s\n\n",
+ $prefix, $data{$what}->{file},
+ $data{$what}->{line_no};
+ }
+
my $w = $what;
$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
@@ -369,6 +380,10 @@ sub search_symbols {
}
}
+# Ensure that the prefix will always end with a slash
+# While this is not needed for find, it makes the patch nicer
+# with --enable-lineno
+$prefix =~ s,/?$,/,;
#
# Parses all ABI files located at $prefix dir
@@ -395,7 +410,8 @@ abi_book.pl - parse the Linux ABI files and produce a ReST
book.
=head1 SYNOPSIS
-B [--debug] [--man] [--help] --[(no-)rst-source] [--dir=]
[]
+B [--debug] [--enable-lineno] [--man] [--help]
+ [--(no-)rst-source] [--dir=] []
Where can be:
@@ -425,6 +441,10 @@ selecting between a rst-compliant source ABI
(--rst-source), or a
plain text that may be violating ReST spec, so it requres some escaping
logic (--no-rst-source).
+=item B<--enable-lineno>
+
+Enable output of #define LINENO lines.
+
=item B<--debug>
Put the script in verbose mode, useful for debugging. Can be called multiple
--
2.21.0
[PATCH 04/12] docs: kernellog.py: add support for info()
An extension may want to just inform about something. So, add support for it. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernellog.py | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Documentation/sphinx/kernellog.py b/Documentation/sphinx/kernellog.py index af924f51a7dc..8ac7d274f542 100644 --- a/Documentation/sphinx/kernellog.py +++ b/Documentation/sphinx/kernellog.py @@ -25,4 +25,8 @@ def verbose(app, message): else: app.verbose(message) - +def info(app, message): +if UseLogging: +logger.info(message) +else: +app.info(message) -- 2.21.0
[PATCH 06/12] docs: kernel_abi.py: fix UTF-8 support
The parser breaks with UTF-8 characters with Sphinx 1.4.
Signed-off-by: Mauro Carvalho Chehab
---
Documentation/sphinx/kernel_abi.py | 20 +++-
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py
b/Documentation/sphinx/kernel_abi.py
index 32ce90775d96..0f3e51e67e8d 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -1,4 +1,5 @@
-# -*- coding: utf-8; mode: python -*-
+# coding=utf-8
+#
u"""
kernel-abi
~~
@@ -28,6 +29,7 @@ u"""
"""
+import codecs
import sys
import os
from os import path
@@ -43,14 +45,6 @@ from docutils.utils.error_reporting import ErrorString
__version__ = '1.0'
-# We can't assume that six is installed
-PY3 = sys.version_info[0] == 3
-PY2 = sys.version_info[0] == 2
-if PY3:
-# pylint: disable=C0103, W0622
-unicode = str
-basestring = str
-
def setup(app):
app.add_directive("kernel-abi", KernelCmd)
@@ -115,12 +109,12 @@ class KernelCmd(Directive):
cmd
, stdout = subprocess.PIPE
, stderr = subprocess.PIPE
-, universal_newlines = True
, **kwargs
)
out, err = proc.communicate()
-if err:
-self.warn(err)
+
+out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
+
if proc.returncode != 0:
raise self.severe(
u"command '%s' failed with return code %d"
@@ -129,7 +123,7 @@ class KernelCmd(Directive):
except OSError as exc:
raise self.severe(u"problems with '%s' directive: %s."
% (self.name, ErrorString(exc)))
-return unicode(out)
+return out
def nestedParse(self, lines, fname):
content = ViewList()
--
2.21.0
[PATCH 12/12] docs: add ABI documentation to the admin-guide book
As we don't want a generic Sphinx extension to execute commands, change the one proposed to Markus to call the abi_book.pl script. Use a script to parse the Documentation/ABI directory and output it at the admin-guide. Signed-off-by: Mauro Carvalho Chehab --- Documentation/admin-guide/abi-obsolete.rst | 10 ++ Documentation/admin-guide/abi-removed.rst | 4 Documentation/admin-guide/abi-stable.rst | 13 + Documentation/admin-guide/abi-testing.rst | 19 +++ Documentation/admin-guide/abi.rst | 11 +++ Documentation/admin-guide/index.rst| 1 + Documentation/conf.py | 2 +- 7 files changed, 59 insertions(+), 1 deletion(-) create mode 100644 Documentation/admin-guide/abi-obsolete.rst create mode 100644 Documentation/admin-guide/abi-removed.rst create mode 100644 Documentation/admin-guide/abi-stable.rst create mode 100644 Documentation/admin-guide/abi-testing.rst create mode 100644 Documentation/admin-guide/abi.rst diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst new file mode 100644 index ..cda9168445a5 --- /dev/null +++ b/Documentation/admin-guide/abi-obsolete.rst @@ -0,0 +1,10 @@ +ABI obsolete symbols + + +Documents interfaces that are still remaining in the kernel, but are +marked to be removed at some later point in time. + +The description of the interface will document the reason why it is +obsolete and when it can be expected to be removed. + +.. kernel-abi:: $srctree/Documentation/ABI/obsolete diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst new file mode 100644 index ..497978fc9632 --- /dev/null +++ b/Documentation/admin-guide/abi-removed.rst @@ -0,0 +1,4 @@ +ABI removed symbols +=== + +.. kernel-abi:: $srctree/Documentation/ABI/removed diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst new file mode 100644 index ..7495d7a35048 --- /dev/null +++ b/Documentation/admin-guide/abi-stable.rst @@ -0,0 +1,13 @@ +ABI stable symbols +== + +Documents the interfaces that the developer has defined to be stable. + +Userspace programs are free to use these interfaces with no +restrictions, and backward compatibility for them will be guaranteed +for at least 2 years. + +Most interfaces (like syscalls) are expected to never change and always +be available. + +.. kernel-abi:: $srctree/Documentation/ABI/stable diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst new file mode 100644 index ..5c886fc50b9e --- /dev/null +++ b/Documentation/admin-guide/abi-testing.rst @@ -0,0 +1,19 @@ +ABI testing symbols +=== + +Documents interfaces that are felt to be stable, +as the main development of this interface has been completed. + +The interface can be changed to add new features, but the +current interface will not break by doing this, unless grave +errors or security problems are found in them. + +Userspace programs can start to rely on these interfaces, but they must +be aware of changes that can occur before these interfaces move to +be marked stable. + +Programs that use these interfaces are strongly encouraged to add their +name to the description of these interfaces, so that the kernel +developers can easily notify them if any changes occur. + +.. kernel-abi:: $srctree/Documentation/ABI/testing diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst new file mode 100644 index ..3b9645c77469 --- /dev/null +++ b/Documentation/admin-guide/abi.rst @@ -0,0 +1,11 @@ += +Linux ABI description += + +.. toctree:: + :maxdepth: 1 + + abi-stable + abi-testing + abi-obsolete + abi-removed diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst index 8001917ee012..20c3020fd73c 100644 --- a/Documentation/admin-guide/index.rst +++ b/Documentation/admin-guide/index.rst @@ -16,6 +16,7 @@ etc. README kernel-parameters devices + abi This section describes CPU vulnerabilities and their mitigations. diff --git a/Documentation/conf.py b/Documentation/conf.py index 7ace3f8852bd..598256fb5c98 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -34,7 +34,7 @@ needs_sphinx = '1.3' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig'] +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi'] # The name of the math extension changed on Sphinx 1.4 if (major == 1 and minor > 3) or (major > 1): -- 2.21.0
[PATCH 01/12] scripts/get_abi.pl: change script to allow parsing in ReST mode
Right now, several ABI files won't parse as ReST, as they
contain severe violations to the spec, with makes the script
to crash.
So, the code has a sanity logic with escapes bad code and
cleans tags that can cause Sphinx to crash.
Add support for disabling this mode.
Right now, as enabling rst-mode causes crash, it is disabled
by default.
Signed-off-by: Mauro Carvalho Chehab
---
scripts/get_abi.pl | 74 ++
1 file changed, 48 insertions(+), 26 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index c738cb795514..107672cdacb3 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -12,8 +12,14 @@ my $man;
my $debug;
my $prefix="Documentation/ABI";
+#
+# If true, assumes that the description is formatted with ReST
+#
+my $description_is_rst = 0;
+
GetOptions(
"debug|d+" => \$debug,
+ "rst-source!" => \$description_is_rst,
"dir=s" => \$prefix,
'help|?' => \$help,
man => \$man
@@ -137,14 +143,15 @@ sub parse_abi {
next;
}
if ($tag eq "description") {
- next if ($content =~ m/^\s*$/);
- if ($content =~ m/^(\s*)(.*)/) {
- my $new_content = $2;
- $space = $new_tag . $sep . $1;
- while ($space =~ s/\t+/' ' x
(length($&) * 8 - length($`) % 8)/e) {}
- $space =~ s/./ /g;
- $data{$what}->{$tag} .=
"$new_content\n";
+ # Preserve initial spaces for the first
line
+ $content = ' ' x length($new_tag) .
$sep . $content;
+ $content =~ s,^(\s*):,$1 ,;
+ if ($content =~ m/^(\s*)(.*)$/) {
+ $space = $1;
+ $content = $2;
}
+ while ($space =~ s/\t+/' ' x
(length($&) * 8 - length($`) % 8)/e) {}
+ $data{$what}->{$tag} .= $content;
} else {
$data{$what}->{$tag} = $content;
}
@@ -160,11 +167,15 @@ sub parse_abi {
if ($tag eq "description") {
if (!$data{$what}->{description}) {
- next if (m/^\s*\n/);
+ s/^($space)//;
if (m/^(\s*)(.*)/) {
- $space = $1;
- while ($space =~ s/\t+/' ' x
(length($&) * 8 - length($`) % 8)/e) {}
- $data{$what}->{$tag} .= "$2\n";
+ my $sp = $1;
+ while ($sp =~ s/\t+/' ' x (length($&) *
8 - length($`) % 8)/e) {}
+ my $content = "$sp$2";
+
+ $content =~ s/^($space)//;
+
+ $data{$what}->{$tag} .= "$content";
}
} else {
my $content = $_;
@@ -274,23 +285,27 @@ sub output_rest {
print "Defined on file :ref:`$file <$fileref>`\n\n" if ($type
ne "File");
my $desc = $data{$what}->{description};
- $desc =~ s/^\s+//;
-
- # Remove title markups from the description, as they won't work
- $desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
if (!($desc =~ /^\s*$/)) {
- if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc
=~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
- # put everything inside a code block
- $desc =~ s/\n/\n /g;
-
- print "::\n\n";
- print " $desc\n\n";
- } else {
- # Escape any special chars from description
- $desc
=~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
-
+ if ($description_is_rst) {
print "$desc\n\n";
+ } else {
+ $desc =~ s/^\s+//;
+
+ # Remove title markups from the description, as
they won't work
+ $desc =~ s/\n[\-\*\=\^\~]+\n/\n\n/g;
+
+ if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/
[PATCH 05/12] docs: kernel_abi.py: add a script to parse ABI documentation
The ABI documentation is special: it is not plain text files,
but, instead, files with an strict format, as specified by
Documentation/ABI/README.
Add a parser for it.
Signed-off-by: Mauro Carvalho Chehab
---
Documentation/sphinx/kernel_abi.py | 155 +
1 file changed, 155 insertions(+)
create mode 100644 Documentation/sphinx/kernel_abi.py
diff --git a/Documentation/sphinx/kernel_abi.py
b/Documentation/sphinx/kernel_abi.py
new file mode 100644
index ..32ce90775d96
--- /dev/null
+++ b/Documentation/sphinx/kernel_abi.py
@@ -0,0 +1,155 @@
+# -*- coding: utf-8; mode: python -*-
+u"""
+kernel-abi
+~~
+
+Implementation of the ``kernel-abi`` reST-directive.
+
+:copyright: Copyright (C) 2016 Markus Heiser
+:copyright: Copyright (C) 2016 Mauro Carvalho Chehab
+:license:GPL Version 2, June 1991 see Linux/COPYING for details.
+
+The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
+scripts/get_abi.pl script to parse the Kernel ABI files.
+
+Overview of directive's argument and options.
+
+.. code-block:: rst
+
+.. kernel-abi::
+:debug:
+
+The argument is required. It contains the
+location of the ABI files to be parsed.
+
+``debug``
+ Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
+ what reST is generated.
+
+"""
+
+import sys
+import os
+from os import path
+import subprocess
+
+from sphinx.ext.autodoc import AutodocReporter
+
+from docutils import nodes
+from docutils.parsers.rst import Directive, directives
+from docutils.statemachine import ViewList
+from docutils.utils.error_reporting import ErrorString
+
+
+__version__ = '1.0'
+
+# We can't assume that six is installed
+PY3 = sys.version_info[0] == 3
+PY2 = sys.version_info[0] == 2
+if PY3:
+# pylint: disable=C0103, W0622
+unicode = str
+basestring = str
+
+def setup(app):
+
+app.add_directive("kernel-abi", KernelCmd)
+return dict(
+version = __version__
+, parallel_read_safe = True
+, parallel_write_safe = True
+)
+
+class KernelCmd(Directive):
+
+u"""KernelABI (``kernel-abi``) directive"""
+
+required_arguments = 1
+optional_arguments = 0
+has_content = False
+final_argument_whitespace = True
+
+option_spec = {
+"debug" : directives.flag
+}
+
+def warn(self, message, **replace):
+replace["fname"] = self.state.document.current_source
+replace["line_no"] = replace.get("line_no", self.lineno)
+message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) %
replace
+self.state.document.settings.env.app.warn(message, prefix="")
+
+def run(self):
+
+doc = self.state.document
+if not doc.settings.file_insertion_enabled:
+raise self.warning("docutils: file insertion disabled")
+
+env = doc.settings.env
+cwd = path.dirname(doc.current_source)
+cmd = "get_abi.pl rest --dir "
+cmd += self.arguments[0]
+
+srctree = path.abspath(os.environ["srctree"])
+
+fname = cmd
+
+# extend PATH with $(srctree)/scripts
+path_env = os.pathsep.join([
+srctree + os.sep + "scripts",
+os.environ["PATH"]
+])
+shell_env = os.environ.copy()
+shell_env["PATH"]= path_env
+shell_env["srctree"] = srctree
+
+lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
+nodeList = self.nestedParse(lines, fname)
+return nodeList
+
+def runCmd(self, cmd, **kwargs):
+u"""Run command ``cmd`` and return it's stdout as unicode."""
+
+try:
+proc = subprocess.Popen(
+cmd
+, stdout = subprocess.PIPE
+, stderr = subprocess.PIPE
+, universal_newlines = True
+, **kwargs
+)
+out, err = proc.communicate()
+if err:
+self.warn(err)
+if proc.returncode != 0:
+raise self.severe(
+u"command '%s' failed with return code %d"
+% (cmd, proc.returncode)
+)
+except OSError as exc:
+raise self.severe(u"problems with '%s' directive: %s."
+ % (self.name, ErrorString(exc)))
+return unicode(out)
+
+def nestedParse(self, lines, fname):
+content = ViewList()
+node= nodes.section()
+
+if "debug" in self.options:
+code_block = "\n\n.. code-block:: rst\n:linenos:\n"
+for l in lines.split("\n"):
+code_block += "\n" + l
+lines = code_block + "\n\n"
+
+for c, l in enumerate(lines.split("\n")):
+content.append(l, fname, c)
+
+buf = self.state.memo.title_styles, self.state.memo.section_level,
self.state.memo.reporter
+
[PATCH 2/4] scripts/get_feat.pl: handle ".." special case
The status ".." Means that the feature can't be implemented
on a given architecture.
The problem is that this doesn't show anything at the
output, so replace it by "---", with is a markup for a long
hyphen.
Signed-off-by: Mauro Carvalho Chehab
---
scripts/get_feat.pl | 2 ++
1 file changed, 2 insertions(+)
diff --git a/scripts/get_feat.pl b/scripts/get_feat.pl
index 401cbc820caa..79d83595addd 100755
--- a/scripts/get_feat.pl
+++ b/scripts/get_feat.pl
@@ -141,6 +141,8 @@ sub parse_feat {
$max_size_arch = length($a);
}
+ $status = "---" if ($status =~ m/^\.\.$/);
+
$archs{$a} = 1;
$arch_table{$a} = $status;
next;
--
2.21.0
[PATCH 1/4] scripts/get_feat.pl: add a script to handle Documentation/features
The Documentation/features contains a set of parseable files.
It is not worth converting them to ReST format, as they're
useful the way it is. It is, however, interesting to parse
them and produce output on different formats:
1) Output the contents of a feature in ReST format;
2) Output what features a given architecture supports;
3) Output a matrix with features x architectures.
Signed-off-by: Mauro Carvalho Chehab
---
scripts/get_feat.pl | 472
1 file changed, 472 insertions(+)
create mode 100755 scripts/get_feat.pl
diff --git a/scripts/get_feat.pl b/scripts/get_feat.pl
new file mode 100755
index ..401cbc820caa
--- /dev/null
+++ b/scripts/get_feat.pl
@@ -0,0 +1,472 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0
+
+use strict;
+use Pod::Usage;
+use Getopt::Long;
+use File::Find;
+use Fcntl ':mode';
+
+my $help;
+my $man;
+my $debug;
+my $arch;
+my $feat;
+my $prefix="Documentation/features";
+
+GetOptions(
+ "debug|d+" => \$debug,
+ "dir=s" => \$prefix,
+ 'help|?' => \$help,
+ 'arch=s' => \$arch,
+ 'feat=s' => \$feat,
+ man => \$man
+) or pod2usage(2);
+
+pod2usage(1) if $help;
+pod2usage(-exitstatus => 0, -verbose => 2) if $man;
+
+pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2);
+
+my ($cmd, $arg) = @ARGV;
+
+pod2usage(2) if ($cmd ne "current" && $cmd ne "rest" && $cmd ne "validate");
+
+require Data::Dumper if ($debug);
+
+my %data;
+my %archs;
+
+#
+# Displays an error message, printing file name and line
+#
+sub parse_error() {
+ my ($file, $ln, $msg, $data) = @_;
+
+ $data =~ s/\s+$/\n/;
+
+ print STDERR "Warning: file $file#$ln:\n\t$msg";
+
+ if ($data ne "") {
+ print STDERR ". Line\n\t\t$data";
+ } else {
+ print STDERR "\n";
+ }
+}
+
+#
+# Parse a features file, storing its contents at %data
+#
+
+my $h_name = "Feature";
+my $h_kconfig = "Kconfig";
+my $h_description = "Description";
+my $h_subsys = "Subsystem";
+my $h_status = "Status";
+my $h_arch = "Architecture";
+
+my $max_size_name = length($h_name);
+my $max_size_kconfig = length($h_kconfig);
+my $max_size_description = length($h_description);
+my $max_size_subsys = length($h_subsys);
+my $max_size_status = length($h_status);
+my $max_size_arch = length($h_arch);
+
+sub parse_feat {
+ my $file = $File::Find::name;
+
+ my $mode = (stat($file))[2];
+ return if ($mode & S_IFDIR);
+ return if ($file =~ m,($prefix)/arch-support.txt,);
+ return if (!($file =~ m,arch-support.txt$,));
+
+ my $subsys = "";
+ $subsys = $2 if ( m,.*($prefix)/([^/]+).*,);
+
+ if (length($subsys) > $max_size_subsys) {
+ $max_size_subsys = length($subsys);
+ }
+
+ my $name;
+ my $kconfig;
+ my $description;
+ my $comments = "";
+ my $last_status;
+ my $ln;
+ my %arch_table;
+
+ print STDERR "Opening $file\n" if ($debug > 1);
+ open IN, $file;
+
+ while() {
+ $ln++;
+
+ if (m/^\#\s+Feature\s+name:\s*(.*\S)/) {
+ $name = $1;
+ if (length($name) > $max_size_name) {
+ $max_size_name = length($name);
+ }
+ next;
+ }
+ if (m/^\#\s+Kconfig:\s*(.*\S)/) {
+ $kconfig = $1;
+ if (length($kconfig) > $max_size_kconfig) {
+ $max_size_kconfig = length($kconfig);
+ }
+ next;
+ }
+ if (m/^\#\s+description:\s*(.*\S)/) {
+ $description = $1;
+ if (length($description) > $max_size_description) {
+ $max_size_description = length($description);
+ }
+ next;
+ }
+ next if (m/^\\s*$/);
+ next if (m/^\s*\-+\s*$/);
+ next if (m/^\s*\|\s*arch\s*\|\s*status\s*\|\s*$/);
+
+ if (m/^\#\s*(.*)/) {
+ $comments .= "$1\n";
+ next;
+ }
+ if (m/^\s*\|\s*(\S+):\s*\|\s*(\S+)\s*\|\s*$/) {
+ my $a = $1;
+ my $status = $2;
+
+ if (length($status) > $max_size_status) {
+ $max_size_status = length($status);
+ }
+ if (length($a) > $max_size_arch) {
+ $max_size_arch = length($a);
+ }
+
+ $archs{$a} = 1;
+ $arch_table{$a} = $status;
+ next;
+ }
+
+ #Everything else is an error
+ parse_error($file, $ln, "line is invalid", $_);
+ }
+ close IN;
+
+
[PATCH 4/4] docs: admin-guide, x86: add a features list
Add a feature list matrix at the admin-guide and a x86-specific feature list to the respective Kernel books. Signed-off-by: Mauro Carvalho Chehab --- Documentation/admin-guide/features.rst | 3 +++ Documentation/admin-guide/index.rst| 1 + Documentation/conf.py | 2 +- Documentation/x86/features.rst | 3 +++ Documentation/x86/index.rst| 1 + 5 files changed, 9 insertions(+), 1 deletion(-) create mode 100644 Documentation/admin-guide/features.rst create mode 100644 Documentation/x86/features.rst diff --git a/Documentation/admin-guide/features.rst b/Documentation/admin-guide/features.rst new file mode 100644 index ..8c167082a84f --- /dev/null +++ b/Documentation/admin-guide/features.rst @@ -0,0 +1,3 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. kernel-feat:: $srctree/Documentation/features diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst index 20c3020fd73c..14c8464f6ca9 100644 --- a/Documentation/admin-guide/index.rst +++ b/Documentation/admin-guide/index.rst @@ -17,6 +17,7 @@ etc. kernel-parameters devices abi + features This section describes CPU vulnerabilities and their mitigations. diff --git a/Documentation/conf.py b/Documentation/conf.py index 598256fb5c98..a0ef76ce5615 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -34,7 +34,7 @@ needs_sphinx = '1.3' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi'] +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi', 'kernel_feat'] # The name of the math extension changed on Sphinx 1.4 if (major == 1 and minor > 3) or (major > 1): diff --git a/Documentation/x86/features.rst b/Documentation/x86/features.rst new file mode 100644 index ..b663f15053ce --- /dev/null +++ b/Documentation/x86/features.rst @@ -0,0 +1,3 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. kernel-feat:: $srctree/Documentation/features x86 diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst index ae36fc5fc649..ed42c8c9154d 100644 --- a/Documentation/x86/index.rst +++ b/Documentation/x86/index.rst @@ -29,3 +29,4 @@ x86-specific Documentation usb-legacy-support i386/index x86_64/index + features -- 2.21.0
[PATCH 3/4] sphinx/kernel_feat.py: add a script to parse feature files
The feature files have a special well-defined format. Add
a script that parses them, allowing to search for a feature
and/or by an architecture and to produce ReST-compatible
outputs.
Signed-off-by: Mauro Carvalho Chehab
---
Documentation/sphinx/kernel_feat.py | 169
1 file changed, 169 insertions(+)
create mode 100644 Documentation/sphinx/kernel_feat.py
diff --git a/Documentation/sphinx/kernel_feat.py
b/Documentation/sphinx/kernel_feat.py
new file mode 100644
index ..2fee04f1dedd
--- /dev/null
+++ b/Documentation/sphinx/kernel_feat.py
@@ -0,0 +1,169 @@
+# coding=utf-8
+# SPDX-License-Identifier: GPL-2.0
+#
+u"""
+kernel-feat
+~~~
+
+Implementation of the ``kernel-feat`` reST-directive.
+
+:copyright: Copyright (C) 2016 Markus Heiser
+:copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
+:maintained-by: Mauro Carvalho Chehab
+:license:GPL Version 2, June 1991 see Linux/COPYING for details.
+
+The ``kernel-feat`` (:py:class:`KernelFeat`) directive calls the
+scripts/get_feat.pl script to parse the Kernel ABI files.
+
+Overview of directive's argument and options.
+
+.. code-block:: rst
+
+.. kernel-feat::
+:debug:
+
+The argument is required. It contains the
+location of the ABI files to be parsed.
+
+``debug``
+ Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
+ what reST is generated.
+
+"""
+
+import codecs
+import os
+import subprocess
+import sys
+
+from os import path
+
+from docutils import nodes, statemachine
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives, Directive
+from docutils.utils.error_reporting import ErrorString
+
+#
+# AutodocReporter is only good up to Sphinx 1.7
+#
+import sphinx
+
+Use_SSI = sphinx.__version__[:3] >= '1.7'
+if Use_SSI:
+from sphinx.util.docutils import switch_source_input
+else:
+from sphinx.ext.autodoc import AutodocReporter
+
+__version__ = '1.0'
+
+def setup(app):
+
+app.add_directive("kernel-feat", KernelFeat)
+return dict(
+version = __version__
+, parallel_read_safe = True
+, parallel_write_safe = True
+)
+
+class KernelFeat(Directive):
+
+u"""KernelFeat (``kernel-feat``) directive"""
+
+required_arguments = 1
+optional_arguments = 2
+has_content = False
+final_argument_whitespace = True
+
+option_spec = {
+"debug" : directives.flag
+}
+
+def warn(self, message, **replace):
+replace["fname"] = self.state.document.current_source
+replace["line_no"] = replace.get("line_no", self.lineno)
+message = ("%(fname)s:%(line_no)s: [kernel-feat WARN] : " + message) %
replace
+self.state.document.settings.env.app.warn(message, prefix="")
+
+def run(self):
+
+doc = self.state.document
+if not doc.settings.file_insertion_enabled:
+raise self.warning("docutils: file insertion disabled")
+
+env = doc.settings.env
+cwd = path.dirname(doc.current_source)
+cmd = "get_feat.pl rest --dir "
+cmd += self.arguments[0]
+
+if len(self.arguments) > 1:
+cmd += " --arch " + self.arguments[1]
+
+srctree = path.abspath(os.environ["srctree"])
+
+fname = cmd
+
+# extend PATH with $(srctree)/scripts
+path_env = os.pathsep.join([
+srctree + os.sep + "scripts",
+os.environ["PATH"]
+])
+shell_env = os.environ.copy()
+shell_env["PATH"]= path_env
+shell_env["srctree"] = srctree
+
+lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
+nodeList = self.nestedParse(lines, fname)
+return nodeList
+
+def runCmd(self, cmd, **kwargs):
+u"""Run command ``cmd`` and return it's stdout as unicode."""
+
+try:
+proc = subprocess.Popen(
+cmd
+, stdout = subprocess.PIPE
+, stderr = subprocess.PIPE
+, **kwargs
+)
+out, err = proc.communicate()
+
+out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
+
+if proc.returncode != 0:
+raise self.severe(
+u"command '%s' failed with return code %d"
+% (cmd, proc.returncode)
+)
+except OSError as exc:
+raise self.severe(u"problems with '%s' directive: %s."
+ % (self.name, ErrorString(exc)))
+return out
+
+def nestedParse(self, lines, fname):
+content = ViewList()
+node= nodes.section()
+
+if "debug" in self.options:
+code_block = "\n\n.. code-block:: rst\n:linenos:\n"
+for l in lines.split("\n"):
+code_block += "\n" + l
+lines = code_block + "\n\n"
+
+
[PATCH 0/4] Add Documentation/features to admin-guide and x86
Jon, Greg & all: Not sure what tree this would fit better. The first 2 patches on this series reimplement the logic at: Documentation/features/list-arch.sh I opted to not remove the old script, as someone could depend on its specific format. The new script does the same with: ./scripts/get_feat.pl current The difference is that it outputs with using ascii table artwork. The way it works is that it parse all feature files and produce 3 different types of contents, depending on its arguments: - a feature x arch matrix: ./scripts/get_feat.pl rest - a per-architecture feature table: ./scripts/get_feat.pl current or ./scripts/get_feat.pl rest --arch=arm64 - a per-feature table: ./scripts/get_feat.pl rest --feat=perf-regs All outputs are compatible with the ReST format. Patch 3 adds a new Sphinx plugin with handles its output. Patch 4 adds the feature x arch matrix at the admin-guide. Patch 4 also adds the features supported on x86 at the x86 arch guide. IMHO, it makes sense to have a similar table on all other architectures, but the best is to wait for the next Kernel version, in order to see what arch conversion files got included. So far, I didn't add the per-feature table anywhere. Mauro Carvalho Chehab (4): scripts/get_feat.pl: add a script to handle Documentation/features scripts/get_feat.pl: handle ".." special case sphinx/kernel_feat.py: add a script to parse feature files docs: admin-guide, x86: add a features list Documentation/admin-guide/features.rst | 3 + Documentation/admin-guide/index.rst| 1 + Documentation/conf.py | 2 +- Documentation/sphinx/kernel_feat.py| 169 + Documentation/x86/features.rst | 3 + Documentation/x86/index.rst| 1 + scripts/get_feat.pl| 474 + 7 files changed, 652 insertions(+), 1 deletion(-) create mode 100644 Documentation/admin-guide/features.rst create mode 100644 Documentation/sphinx/kernel_feat.py create mode 100644 Documentation/x86/features.rst create mode 100755 scripts/get_feat.pl -- 2.21.0
[RFC v2 1/8] docs: ABI: README: specify that files should be ReST compatible
As we plan to remove the escaping code from the scripts/get_abi.pl, specify at the ABI README file that the content of the file should be ReST compatible. Signed-off-by: Mauro Carvalho Chehab --- Documentation/ABI/README | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/Documentation/ABI/README b/Documentation/ABI/README index 3121029dce21..8bac9cb09a6d 100644 --- a/Documentation/ABI/README +++ b/Documentation/ABI/README @@ -32,7 +32,7 @@ The different levels of stability are: layout of the files below for details on how to do this.) obsolete/ - This directory documents interfaces that are still remaining in + This directory documents interfaces that are still remaining in the kernel, but are marked to be removed at some later point in time. The description of the interface will document the reason why it is obsolete and when it can be expected to be removed. @@ -58,6 +58,14 @@ Users: All users of this interface who wish to be notified when be changed further. +Note: + The fields should be use a simple notation, compatible with ReST markup. + Also, the file **should not** have a top-level index, like:: + + === + foo + === + How things move between levels: Interfaces in stable may move to obsolete, as long as the proper -- 2.21.0
[RFC v2 8/8] docs: Kconfig/Makefile: add a check for broken ABI files
The files under Documentation/ABI should follow the syntax
as defined at Documentation/ABI/README.
Allow checking if they're following the syntax by running
the ABI parser script on COMPILE_TEST.
With that, when there's a problem with a file under
Documentation/ABI, it would produce a warning like:
Warning: file
./Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats#14:
What
'/sys/bus/pci/devices//aer_stats/aer_rootport_total_err_cor' doesn't have
a description
Warning: file
./Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats#21:
What
'/sys/bus/pci/devices//aer_stats/aer_rootport_total_err_fatal' doesn't
have a description
Signed-off-by: Mauro Carvalho Chehab
---
Documentation/Kconfig | 11 +++
Documentation/Makefile | 5 +
lib/Kconfig.debug | 2 ++
scripts/get_abi.pl | 14 +++---
4 files changed, 29 insertions(+), 3 deletions(-)
create mode 100644 Documentation/Kconfig
diff --git a/Documentation/Kconfig b/Documentation/Kconfig
new file mode 100644
index ..a8b0701c1422
--- /dev/null
+++ b/Documentation/Kconfig
@@ -0,0 +1,11 @@
+config WARN_ABI_ERRORS
+ bool "Warn if there are errors at ABI files"
+ depends on COMPILE_TEST
+ help
+ The files under Documentation/ABI should follow what's
+ described at Documentation/ABI/README. Yet, as they're manually
+ written, it would be possible that some of those files would
+ have errors that would break them for being parsed by
+ scripts/get_abi.pl. Add a check to verify them.
+
+ If unsure, select 'N'.
diff --git a/Documentation/Makefile b/Documentation/Makefile
index e889e7cb8511..c6480ed22884 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -4,6 +4,11 @@
subdir-y := devicetree/bindings/
+# Check for broken ABI files
+ifeq ($(CONFIG_WARN_ABI_ERRORS),y)
+$(shell $(srctree)/scripts/get_abi.pl validate --dir
$(srctree)/Documentation/ABI)
+endif
+
# You can set these variables from the command line.
SPHINXBUILD = sphinx-build
SPHINXOPTS=
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index cbdfae379896..b1b7e141ca99 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -2110,4 +2110,6 @@ config IO_STRICT_DEVMEM
source "arch/$(SRCARCH)/Kconfig.debug"
+source "Documentation/Kconfig"
+
endmenu # Kernel hacking
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 6a4d387ebf3b..e80f9ab2ed26 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -47,7 +47,15 @@ my %data;
sub parse_error() {
my ($file, $ln, $msg, $data) = @_;
- print STDERR "file $file#$ln: $msg at\n\t$data";
+ $data =~ s/\s+$/\n/;
+
+ print STDERR "Warning: file $file#$ln:\n\t$msg";
+
+ if ($data ne "") {
+ print STDERR ". Line\n\t\t$data";
+ } else {
+ print STDERR "\n";
+ }
}
#
@@ -104,7 +112,7 @@ sub parse_abi {
# Invalid, but it is a common mistake
if ($new_tag eq "where") {
- parse_error($file, $ln, "tag 'Where' is
invalid. Should be 'What:' instead", $_);
+ parse_error($file, $ln, "tag 'Where' is
invalid. Should be 'What:' instead", "");
$new_tag = "what";
}
@@ -205,7 +213,7 @@ sub parse_abi {
}
# Everything else is error
- parse_error($file, $ln, "Unexpected line:", $_);
+ parse_error($file, $ln, "Unexpected content", $_);
}
$data{$nametag}->{description} =~ s/^\n+//;
close IN;
--
2.21.0
[RFC v2 7/8] docs: abi-testing.rst: enable --rst-sources when building docs
Now that ABI/testing documents were fixed, add --rst-sources to the ABI/testing too. Signed-off-by: Mauro Carvalho Chehab --- Documentation/admin-guide/abi-testing.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst index 5c886fc50b9e..b205b16a72d0 100644 --- a/Documentation/admin-guide/abi-testing.rst +++ b/Documentation/admin-guide/abi-testing.rst @@ -17,3 +17,4 @@ name to the description of these interfaces, so that the kernel developers can easily notify them if any changes occur. .. kernel-abi:: $srctree/Documentation/ABI/testing + :rst: -- 2.21.0
[RFC v2 4/8] docs: ABI: make it parse ABI/stable as ReST-compatible files
Now that the stable ABI files are compatible with ReST,
parse them without converting complex descriptions as literal
blocks nor escaping special characters.
Please notice that escaping special characters will probably
be needed at descriptions, at least for the asterisk character.
Signed-off-by: Mauro Carvalho Chehab
---
Documentation/admin-guide/abi-stable.rst | 1 +
Documentation/sphinx/kernel_abi.py | 8 ++--
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/abi-stable.rst
b/Documentation/admin-guide/abi-stable.rst
index 7495d7a35048..70490736e0d3 100644
--- a/Documentation/admin-guide/abi-stable.rst
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -11,3 +11,4 @@ Most interfaces (like syscalls) are expected to never change
and always
be available.
.. kernel-abi:: $srctree/Documentation/ABI/stable
+ :rst:
diff --git a/Documentation/sphinx/kernel_abi.py
b/Documentation/sphinx/kernel_abi.py
index a417026ed690..a00eccfbafea 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -72,12 +72,13 @@ class KernelCmd(Directive):
u"""KernelABI (``kernel-abi``) directive"""
required_arguments = 1
-optional_arguments = 0
+optional_arguments = 2
has_content = False
final_argument_whitespace = True
option_spec = {
-"debug" : directives.flag
+"debug" : directives.flag,
+"rst" : directives.unchanged
}
def run(self):
@@ -91,6 +92,9 @@ class KernelCmd(Directive):
cmd = "get_abi.pl rest --enable-lineno --dir "
cmd += self.arguments[0]
+if 'rst' in self.options:
+cmd += " --rst-source"
+
srctree = path.abspath(os.environ["srctree"])
fname = cmd
--
2.21.0
[RFC v2 5/8] docs: ABI: create a 2-depth index for ABI
That helps to identify what ABI files are adding titles. Signed-off-by: Mauro Carvalho Chehab --- Documentation/admin-guide/abi.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst index 3b9645c77469..bcab3ef2597c 100644 --- a/Documentation/admin-guide/abi.rst +++ b/Documentation/admin-guide/abi.rst @@ -3,7 +3,7 @@ Linux ABI description = .. toctree:: - :maxdepth: 1 + :maxdepth: 2 abi-stable abi-testing -- 2.21.0
[RFC v2 2/8] docs: ABI: stable: make files ReST compatible
Several entries at the stable ABI files won't parse if we pass them directly to the ReST output. Adjust them, in order to allow adding their contents as-is at the stable ABI book. Signed-off-by: Mauro Carvalho Chehab --- Documentation/ABI/stable/firewire-cdev| 4 + Documentation/ABI/stable/sysfs-acpi-pmprofile | 22 +++-- Documentation/ABI/stable/sysfs-bus-firewire | 3 + Documentation/ABI/stable/sysfs-bus-nvmem | 19 ++-- Documentation/ABI/stable/sysfs-bus-usb| 6 +- .../ABI/stable/sysfs-class-backlight | 1 + .../ABI/stable/sysfs-class-infiniband | 97 +-- Documentation/ABI/stable/sysfs-class-rfkill | 13 ++- Documentation/ABI/stable/sysfs-class-tpm | 90 - Documentation/ABI/stable/sysfs-devices| 5 +- Documentation/ABI/stable/sysfs-driver-ib_srp | 1 + .../ABI/stable/sysfs-firmware-efi-vars| 4 + .../ABI/stable/sysfs-firmware-opal-dump | 5 + .../ABI/stable/sysfs-firmware-opal-elog | 2 + Documentation/ABI/stable/sysfs-hypervisor-xen | 3 + Documentation/ABI/stable/vdso | 5 +- 16 files changed, 180 insertions(+), 100 deletions(-) diff --git a/Documentation/ABI/stable/firewire-cdev b/Documentation/ABI/stable/firewire-cdev index f72ed653878a..c9e8ff026154 100644 --- a/Documentation/ABI/stable/firewire-cdev +++ b/Documentation/ABI/stable/firewire-cdev @@ -14,12 +14,14 @@ Description: Each /dev/fw* is associated with one IEEE 1394 node, which can be remote or local nodes. Operations on a /dev/fw* file have different scope: + - The 1394 node which is associated with the file: - Asynchronous request transmission - Get the Configuration ROM - Query node ID - Query maximum speed of the path between this node and local node + - The 1394 bus (i.e. "card") to which the node is attached to: - Isochronous stream transmission and reception - Asynchronous stream transmission and reception @@ -31,6 +33,7 @@ Description: manager - Query cycle time - Bus reset initiation, bus reset event reception + - All 1394 buses: - Allocation of IEEE 1212 address ranges on the local link layers, reception of inbound requests to such @@ -43,6 +46,7 @@ Description: userland implement different access permission models, some operations are restricted to /dev/fw* files that are associated with a local node: + - Addition of descriptors or directories to the local nodes' Configuration ROM - PHY packet transmission and reception diff --git a/Documentation/ABI/stable/sysfs-acpi-pmprofile b/Documentation/ABI/stable/sysfs-acpi-pmprofile index 964c7a8afb26..fd97d22b677f 100644 --- a/Documentation/ABI/stable/sysfs-acpi-pmprofile +++ b/Documentation/ABI/stable/sysfs-acpi-pmprofile @@ -6,17 +6,21 @@ Description: The ACPI pm_profile sysfs interface exports the platform power management (and performance) requirement expectations as provided by BIOS. The integer value is directly passed as retrieved from the FADT ACPI table. -Values: For possible values see ACPI specification: + +Values:For possible values see ACPI specification: 5.2.9 Fixed ACPI Description Table (FADT) Field: Preferred_PM_Profile Currently these values are defined by spec: - 0 Unspecified - 1 Desktop - 2 Mobile - 3 Workstation - 4 Enterprise Server - 5 SOHO Server - 6 Appliance PC - 7 Performance Server + + == = + 0 Unspecified + 1 Desktop + 2 Mobile + 3 Workstation + 4 Enterprise Server + 5 SOHO Server + 6 Appliance PC + 7 Performance Server >7 Reserved + == = diff --git a/Documentation/ABI/stable/sysfs-bus-firewire b/Documentation/ABI/stable/sysfs-bus-firewire index 41e5a0cd1e3e..9ac9eddb82ef 100644 --- a/Documentation/ABI/stable/sysfs-bus-firewire +++ b/Documentation/ABI/stable/sysfs-bus-firewire @@ -47,6 +47,7 @@ Description: IEEE 1394 node device attribute. Read-only and immutable. Values:1: The sysfs entry represents a local node (a controller card). + 0: The sysfs entry represents a remote node. @@ -125
[RFC v2 6/8] docs: ABI: don't escape ReST-incompatible chars from obsolete and removed
With just a single fix, the contents there can be parsed properly without the need to escape any ReST incompatible stuff. Signed-off-by: Mauro Carvalho Chehab --- Documentation/ABI/obsolete/sysfs-gpio | 2 ++ Documentation/admin-guide/abi-obsolete.rst | 1 + Documentation/admin-guide/abi-removed.rst | 1 + 3 files changed, 4 insertions(+) diff --git a/Documentation/ABI/obsolete/sysfs-gpio b/Documentation/ABI/obsolete/sysfs-gpio index 40d41ea1a3f5..18ba539e365c 100644 --- a/Documentation/ABI/obsolete/sysfs-gpio +++ b/Documentation/ABI/obsolete/sysfs-gpio @@ -13,6 +13,8 @@ Description: GPIOs are identified as they are inside the kernel, using integers in the range 0..INT_MAX. See Documentation/gpio for more information. + :: + /sys/class/gpio /export ... asks the kernel to export a GPIO to userspace /unexport ... to return a GPIO to the kernel diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst index cda9168445a5..d095867899c5 100644 --- a/Documentation/admin-guide/abi-obsolete.rst +++ b/Documentation/admin-guide/abi-obsolete.rst @@ -8,3 +8,4 @@ The description of the interface will document the reason why it is obsolete and when it can be expected to be removed. .. kernel-abi:: $srctree/Documentation/ABI/obsolete + :rst: diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst index 497978fc9632..f7e9e43023c1 100644 --- a/Documentation/admin-guide/abi-removed.rst +++ b/Documentation/admin-guide/abi-removed.rst @@ -2,3 +2,4 @@ ABI removed symbols === .. kernel-abi:: $srctree/Documentation/ABI/removed + :rst: -- 2.21.0
[RFC v2 0/8]Produce ABI guide without escaping ReST source files
Hi Greg, It turns out that fixing ABI/testing for it to be parsed transparently was easy :-) This series goes on top of the series I pasted early today: Subject: [PATCH 00/12] Add the ABI documentation to the admin guide It basically change the ABI parser to not try to escape ReST code. With that, the descriptions inside the ABI files should now be compatible with ReST output. Patch 1 states that at ABI/README; Patch 2 and 3 fix troubles at stable and testing ABI files. No changes at the content of the ABI themselves. Most of the changes there are due to some examples, tables and some parameter descriptions with weird formats. Patch 4 enables the "clear mode" for stable; Patch 5 is just a helper patch: it adds a second level to the ABI index, in order to allow seeing what sections the parsed files will produce; patch 6 enables "clear mode" to obsolete and removed; patch 7 enables "clear mode" to testing - Patch 8 is here just for us not to forget it - it causes any COMPILE_TEST build to validate if the ABI files are following the syntax described at ABI/README. It won't try to build it with Sphinx, though. As this is quick (~100ms on my desktop), I think it is worth to have it. Mauro Carvalho Chehab (8): docs: ABI: README: specify that files should be ReST compatible docs: ABI: stable: make files ReST compatible docs: ABI: testing: make the files compatible with ReST output docs: ABI: make it parse ABI/stable as ReST-compatible files docs: ABI: create a 2-depth index for ABI docs: ABI: don't escape ReST-incompatible chars from obsolete and removed docs: abi-testing.rst: enable --rst-sources when building docs docs: Kconfig/Makefile: add a check for broken ABI files Documentation/ABI/README | 10 +- Documentation/ABI/obsolete/sysfs-gpio | 2 + Documentation/ABI/stable/firewire-cdev| 4 + Documentation/ABI/stable/sysfs-acpi-pmprofile | 22 +- Documentation/ABI/stable/sysfs-bus-firewire | 3 + Documentation/ABI/stable/sysfs-bus-nvmem | 19 +- Documentation/ABI/stable/sysfs-bus-usb| 6 +- .../ABI/stable/sysfs-class-backlight | 1 + .../ABI/stable/sysfs-class-infiniband | 97 +-- Documentation/ABI/stable/sysfs-class-rfkill | 13 +- Documentation/ABI/stable/sysfs-class-tpm | 90 +++ Documentation/ABI/stable/sysfs-devices| 5 +- Documentation/ABI/stable/sysfs-driver-ib_srp | 1 + .../ABI/stable/sysfs-firmware-efi-vars| 4 + .../ABI/stable/sysfs-firmware-opal-dump | 5 + .../ABI/stable/sysfs-firmware-opal-elog | 2 + Documentation/ABI/stable/sysfs-hypervisor-xen | 3 + Documentation/ABI/stable/vdso | 5 +- .../ABI/testing/configfs-spear-pcie-gadget| 36 +-- Documentation/ABI/testing/configfs-usb-gadget | 77 +++--- .../ABI/testing/configfs-usb-gadget-hid | 10 +- .../ABI/testing/configfs-usb-gadget-rndis | 16 +- .../ABI/testing/configfs-usb-gadget-uac1 | 18 +- .../ABI/testing/configfs-usb-gadget-uvc | 220 +--- Documentation/ABI/testing/debugfs-ec | 11 +- Documentation/ABI/testing/debugfs-pktcdvd | 11 +- Documentation/ABI/testing/dev-kmsg| 27 +- Documentation/ABI/testing/evm | 17 +- Documentation/ABI/testing/ima_policy | 132 +- Documentation/ABI/testing/procfs-diskstats| 40 +-- Documentation/ABI/testing/sysfs-block | 26 +- Documentation/ABI/testing/sysfs-block-device | 2 + Documentation/ABI/testing/sysfs-bus-acpi | 18 +- .../sysfs-bus-event_source-devices-format | 3 +- .../ABI/testing/sysfs-bus-i2c-devices-pca954x | 27 +- Documentation/ABI/testing/sysfs-bus-iio | 10 + .../sysfs-bus-iio-adc-envelope-detector | 5 +- .../ABI/testing/sysfs-bus-iio-cros-ec | 2 +- .../ABI/testing/sysfs-bus-iio-dfsdm-adc-stm32 | 10 +- .../ABI/testing/sysfs-bus-iio-lptimer-stm32 | 29 ++- .../sysfs-bus-iio-magnetometer-hmc5843| 19 +- .../sysfs-bus-iio-temperature-max31856| 19 +- .../ABI/testing/sysfs-bus-iio-timer-stm32 | 114 + .../testing/sysfs-bus-intel_th-devices-msc| 4 + .../testing/sysfs-bus-pci-devices-aer_stats | 119 + Documentation/ABI/testing/sysfs-bus-rapidio | 23 +- .../ABI/testing/sysfs-bus-thunderbolt | 40 +-- Documentation/ABI/testing/sysfs-bus-usb | 30 ++- .../testing/sysfs-bus-usb-devices-usbsevseg | 7 +- Documentation/ABI/testing/sysfs-bus-vfio-mdev | 10 +- Documentation/ABI/testing/sysfs-class-cxl | 15 +- Documentation/ABI/testing/sysfs-class-led | 2 +- Documentation/ABI/testing/sysfs-class-mic.txt | 52 ++-- Documentation/ABI/testing/sysfs-class-ocxl| 3 + Documentation/ABI/testing/sysfs-class-power | 73 +- .../ABI/testing/sysfs-class-power-twl4030 | 33 +-- Documentation/ABI/testing/sysfs-class-rc | 30 ++- .../ABI/t
Re: [PATCH 1/3] Docs: An initial automarkup extension for sphinx
Em Sat, 22 Jun 2019 08:43:46 -0600 Jonathan Corbet escreveu: > On Fri, 21 Jun 2019 22:00:46 -0300 > Mauro Carvalho Chehab wrote: > > > > +# > > > +# The DVB docs create references for these basic system calls, leading > > > +# to lots of confusing links. So just don't link them. > > > +# > > > +Skipfuncs = [ 'open', 'close', 'write' ] > > > > and yeah, of course, if there's something weird, it has to be at > > the media docs :-) > > > > Btw, if I'm not mistaken, we do the same for ioctl. > > So that's actually interesting. In, for example, > Documentation/media/uapi/v4l/func-ioctl.rst, you see something that looks > like this: > > > .. c:function:: int ioctl( int fd, int request, void *argp ) > > :name: v4l2-ioctl > > Some digging around didn't turn up any documentation for :name:, but it > seems to prevent ioctl() from going into the list of functions that can be > cross-referenced. It took me a while to discover this way to be able to re-define the name of a symbol at the C domain, but I'm pretty sure I read this somewhere at the Sphinx docs (or perhaps on some bug track or Stack Overflow). I don't remember exactly where I get it, but I guess it is related to this: http://docutils.sourceforge.net/docs/howto/rst-roles.html > I wonder if the same should be done for the others? Sure. > I think that would be better than putting a special-case hack into the > toolchain. Agreed. As you're doing this, if you prefer, feel free to send the patches to media docs. Otherwise, I'll seek for some time next week. > > > I'm wandering if this could also handle the Documentation/* auto-replace. > > I think it's the obvious place for it, yes. Let's make sure I haven't > badly broken anything with the existing change first, though :) Yeah, sure. Just wanted to place the code at the same thread. There are some tricks that need to be done in order to handle the relative paths. It took me a lot more time to get it right than adding the replacing Regex :-) (That reminds I should post a patch to one place where we have a Documentation/Documentation typo) Thanks, Mauro
[PATCH] docs: zh_CN: submitting-drivers.rst: Remove a duplicated Documentation/
Somehow, this file ended with Documentation/ twice. Signed-off-by: Mauro Carvalho Chehab --- Documentation/translations/zh_CN/process/submitting-drivers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/translations/zh_CN/process/submitting-drivers.rst b/Documentation/translations/zh_CN/process/submitting-drivers.rst index 72c6cd935821..72f4f45c98de 100644 --- a/Documentation/translations/zh_CN/process/submitting-drivers.rst +++ b/Documentation/translations/zh_CN/process/submitting-drivers.rst @@ -22,7 +22,7 @@ 兴趣的是显卡驱动程序，你也许应该访问 XFree86 项目(http://www.xfree86.org/) 和／或 X.org 项目 (http://x.org)。 -另请参阅 Documentation/Documentation/translations/zh_CN/process/submitting-patches.rst 文档。 +另请参阅 Documentation/translations/zh_CN/process/submitting-patches.rst 文档。 分配设备号 -- 2.21.0
