[PATCH] docs: printk-formats: add ptrdiff_t type to printk-formats
When print the difference between two pointers, we should use the ptrdiff_t modifier %t. Signed-off-by: Miles Chen --- Documentation/core-api/printk-formats.rst | 14 ++ 1 file changed, 14 insertions(+) diff --git a/Documentation/core-api/printk-formats.rst b/Documentation/core-api/printk-formats.rst index ecbebf4ca8e7..8a0f49cd158b 100644 --- a/Documentation/core-api/printk-formats.rst +++ b/Documentation/core-api/printk-formats.rst @@ -135,6 +135,20 @@ equivalent to %lx (or %lu). %px is preferred because it is more uniquely grep'able. If in the future we need to modify the way the kernel handles printing pointers we will be better equipped to find the call sites. +Pointer Differences +--- + +:: + + %td 2560 + %tx a00 + +For printing the pointer differences, use the %t modifier for ptrdiff_t. + +Example:: + + printk("test: difference between pointers: %td\n", ptr2 - ptr1); + Struct Resources -- 2.18.0
Re: [PATCH v6 1/4] nvmem: core: add nvmem_device_find
On 23/09/2019 12:46, Thomas Bogendoerfer wrote: nvmem_device_find provides a way to search for nvmem devices with the help of a match function simlair to bus_find_device. Signed-off-by: Thomas Bogendoerfer --- Thanks for the patch, This patch looks good for me. Do you know which tree is going to pick this series up? I can either apply this patch to nvmem tree or here is my Ack for this patch to take it via other trees. Reviewed-by: Srinivas Kandagatla Acked-by: Srinivas Kandagatla --srini Documentation/driver-api/nvmem.rst | 2 ++ drivers/nvmem/core.c | 61 +- include/linux/nvmem-consumer.h | 9 ++ 3 files changed, 38 insertions(+), 34 deletions(-) diff --git a/Documentation/driver-api/nvmem.rst b/Documentation/driver-api/nvmem.rst index d9d958d5c824..287e86819640 100644 --- a/Documentation/driver-api/nvmem.rst +++ b/Documentation/driver-api/nvmem.rst @@ -129,6 +129,8 @@ To facilitate such consumers NVMEM framework provides below apis:: struct nvmem_device *nvmem_device_get(struct device *dev, const char *name); struct nvmem_device *devm_nvmem_device_get(struct device *dev, const char *name); + struct nvmem_device *nvmem_device_find(void *data, + int (*match)(struct device *dev, const void *data)); void nvmem_device_put(struct nvmem_device *nvmem); int nvmem_device_read(struct nvmem_device *nvmem, unsigned int offset, size_t bytes, void *buf); diff --git a/drivers/nvmem/core.c b/drivers/nvmem/core.c index 057d1ff87d5d..9f1ee9c766ec 100644 --- a/drivers/nvmem/core.c +++ b/drivers/nvmem/core.c @@ -76,33 +76,6 @@ static struct bus_type nvmem_bus_type = { .name = "nvmem", }; -static struct nvmem_device *of_nvmem_find(struct device_node *nvmem_np) -{ - struct device *d; - - if (!nvmem_np) - return NULL; - - d = bus_find_device_by_of_node(&nvmem_bus_type, nvmem_np); - - if (!d) - return NULL; - - return to_nvmem_device(d); -} - -static struct nvmem_device *nvmem_find(const char *name) -{ - struct device *d; - - d = bus_find_device_by_name(&nvmem_bus_type, NULL, name); - - if (!d) - return NULL; - - return to_nvmem_device(d); -} - static void nvmem_cell_drop(struct nvmem_cell *cell) { blocking_notifier_call_chain(&nvmem_notifier, NVMEM_CELL_REMOVE, cell); @@ -532,13 +505,16 @@ int devm_nvmem_unregister(struct device *dev, struct nvmem_device *nvmem) } EXPORT_SYMBOL(devm_nvmem_unregister); -static struct nvmem_device *__nvmem_device_get(struct device_node *np, - const char *nvmem_name) +static struct nvmem_device *__nvmem_device_get(void *data, + int (*match)(struct device *dev, const void *data)) { struct nvmem_device *nvmem = NULL; + struct device *dev; mutex_lock(&nvmem_mutex); - nvmem = np ? of_nvmem_find(np) : nvmem_find(nvmem_name); + dev = bus_find_device(&nvmem_bus_type, NULL, data, match); + if (dev) + nvmem = to_nvmem_device(dev); mutex_unlock(&nvmem_mutex); if (!nvmem) return ERR_PTR(-EPROBE_DEFER); @@ -587,7 +563,7 @@ struct nvmem_device *of_nvmem_device_get(struct device_node *np, const char *id) if (!nvmem_np) return ERR_PTR(-ENOENT); - return __nvmem_device_get(nvmem_np, NULL); + return __nvmem_device_get(nvmem_np, device_match_of_node); } EXPORT_SYMBOL_GPL(of_nvmem_device_get); #endif @@ -613,10 +589,26 @@ struct nvmem_device *nvmem_device_get(struct device *dev, const char *dev_name) } - return __nvmem_device_get(NULL, dev_name); + return __nvmem_device_get((void *)dev_name, device_match_name); } EXPORT_SYMBOL_GPL(nvmem_device_get); +/** + * nvmem_device_find() - Find nvmem device with matching function + * + * @data: Data to pass to match function + * @match: Callback function to check device + * + * Return: ERR_PTR() on error or a valid pointer to a struct nvmem_device + * on success. + */ +struct nvmem_device *nvmem_device_find(void *data, + int (*match)(struct device *dev, const void *data)) +{ + return __nvmem_device_get(data, match); +} +EXPORT_SYMBOL_GPL(nvmem_device_find); + static int devm_nvmem_device_match(struct device *dev, void *res, void *data) { struct nvmem_device **nvmem = res; @@ -710,7 +702,8 @@ nvmem_cell_get_from_lookup(struct device *dev, const char *con_id) if ((strcmp(lookup->dev_id, dev_id) == 0) && (strcmp(lookup->con_id, con_id) == 0)) { /* This is the right entry. */ - nvmem = __nvmem_device_get(NULL, lookup->nvmem_name); + nvmem = __nvmem_device_get((void *)lookup->nvmem_name, +
[PATCH v2] docs: networking: Add title caret and missing doc
Resolving a couple of Sphinx documentation warnings that are generated in the networking section. - WARNING: document isn't included in any toctree - WARNING: Title underline too short. Signed-off-by: Adam Zerella --- v2: Moved 'netronome/nfp' into alphabetical order --- Documentation/networking/device_drivers/index.rst | 1 + Documentation/networking/j1939.rst| 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/networking/device_drivers/index.rst b/Documentation/networking/device_drivers/index.rst index f51f92571e39..c1f7f75e5fd9 100644 --- a/Documentation/networking/device_drivers/index.rst +++ b/Documentation/networking/device_drivers/index.rst @@ -23,6 +23,7 @@ Contents: intel/ice google/gve mellanox/mlx5 + netronome/nfp pensando/ionic .. only:: subproject and html diff --git a/Documentation/networking/j1939.rst b/Documentation/networking/j1939.rst index ce7e7a044e08..dc60b13fcd09 100644 --- a/Documentation/networking/j1939.rst +++ b/Documentation/networking/j1939.rst @@ -272,7 +272,7 @@ supported flags are: * MSG_DONTWAIT, i.e. non-blocking operation. recvmsg(2) -^ +^^ In most cases recvmsg(2) is needed if you want to extract more information than recvfrom(2) can provide. For example package priority and timestamp. The -- 2.21.0
Re: [PATCH] docs: networking: Add title caret and missing doc
On Mon, Sep 30, 2019 at 11:37:54AM -0700, Jakub Kicinski wrote: > On Sat, 28 Sep 2019 22:39:17 +1000, Adam Zerella wrote: > > Resolving a couple of Sphinx documentation warnings > > that are generated in the networking section. > > > > - WARNING: document isn't included in any toctree > > - WARNING: Title underline too short. > > > > Signed-off-by: Adam Zerella > > --- > > Documentation/networking/device_drivers/index.rst | 1 + > > Documentation/networking/j1939.rst| 2 +- > > 2 files changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/Documentation/networking/device_drivers/index.rst > > b/Documentation/networking/device_drivers/index.rst > > index f51f92571e39..1f4a629e7caa 100644 > > --- a/Documentation/networking/device_drivers/index.rst > > +++ b/Documentation/networking/device_drivers/index.rst > > @@ -24,6 +24,7 @@ Contents: > > google/gve > > mellanox/mlx5 > > pensando/ionic > > + netronome/nfp > > I wonder if it's worth keeping the entries in a roughly alphabetic > order? For sure, I've re-submitted a v2 patch :) > > .. only:: subproject and html > > > > diff --git a/Documentation/networking/j1939.rst > > b/Documentation/networking/j1939.rst > > index ce7e7a044e08..dc60b13fcd09 100644 > > --- a/Documentation/networking/j1939.rst > > +++ b/Documentation/networking/j1939.rst > > @@ -272,7 +272,7 @@ supported flags are: > > * MSG_DONTWAIT, i.e. non-blocking operation. > > > > recvmsg(2) > > -^ > > +^^ > > > > In most cases recvmsg(2) is needed if you want to extract more information > > than > > recvfrom(2) can provide. For example package priority and timestamp. The >
Re: [PATCH] docs: arm64: Fix indentation and doc formatting
On Mon, Sep 30, 2019 at 01:28:23PM +0100, Will Deacon wrote: > On Sat, Sep 28, 2019 at 10:58:19PM +1000, Adam Zerella wrote: > > Sphinx generates the following warnings for the arm64 doc > > pages: > > > > Documentation/arm64/memory.rst:158: WARNING: Unexpected indentation. > > Documentation/arm64/memory.rst:162: WARNING: Unexpected indentation. > > > > These indentations warnings can be resolved by utilising code > > hightlighting instead. > > > > Signed-off-by: Adam Zerella > > --- > > Documentation/arm64/memory.rst | 9 +++-- > > 1 file changed, 7 insertions(+), 2 deletions(-) > > > > diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst > > index b040909e45f8..f7db6a3898c5 100644 > > --- a/Documentation/arm64/memory.rst > > +++ b/Documentation/arm64/memory.rst > > @@ -154,11 +154,16 @@ return virtual addresses to userspace from a 48-bit > > range. > > > > Software can "opt-in" to receiving VAs from a 52-bit space by > > specifying an mmap hint parameter that is larger than 48-bit. > > -For example: > > -maybe_high_address = mmap(~0UL, size, prot, flags,...); > > + > > +.. code-block:: c > > + > > + maybe_high_address = mmap(~0UL, size, prot, flags,...); > > Why did you drop the "For example:" prefix? > No reason, I initially thought it would be a bit cleaner but since it is a code example not an _actual_ snippet it makes more sense to keep it. I've attached a v2 patch. > Will Sphinx generates the following warnings for the arm64 doc pages: Documentation/arm64/memory.rst:158: WARNING: Unexpected indentation. Documentation/arm64/memory.rst:162: WARNING: Unexpected indentation. These indentations warnings can be resolved by utilising code hightlighting instead. Signed-off-by: Adam Zerella --- v2: Adding back the 'For example' text before the code example. --- Documentation/arm64/memory.rst | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst index b040909e45f8..02e02175e6f5 100644 --- a/Documentation/arm64/memory.rst +++ b/Documentation/arm64/memory.rst @@ -154,11 +154,18 @@ return virtual addresses to userspace from a 48-bit range. Software can "opt-in" to receiving VAs from a 52-bit space by specifying an mmap hint parameter that is larger than 48-bit. + For example: -maybe_high_address = mmap(~0UL, size, prot, flags,...); + +.. code-block:: c + + maybe_high_address = mmap(~0UL, size, prot, flags,...); It is also possible to build a debug kernel that returns addresses from a 52-bit space by enabling the following kernel config options: + +.. code-block:: sh + CONFIG_EXPERT=y && CONFIG_ARM64_FORCE_52BIT=y Note that this option is only intended for debugging applications -- 2.21.0
Re: [PATCH v2 1/3] x86/boot: Introduce the kernel_info
On Mon, Sep 30, 2019 at 10:18:43AM -0700, H. Peter Anvin wrote: > On 2019-09-30 08:01, Daniel Kiper wrote: > > > > OK. > > > >> field for the entire .kernel_info section, thus ensuring it is a > >> single self-contained blob. > > > > .rodata.kernel_info contains its total size immediately behind the > > "InfO" header. Do you suggest that we should add the size of > > .rodata.kernel_info into setup_header too? > > > > No, what I want is a chunked architecture for kernel_info. > > That is: > > /* Common chunk header */ > struct kernel_info_header { > uint32_t magic; > uint32_t len; > }; > > /* Top-level chunk, always first */ > #define KERNEL_INFO_MAGIC 0x45fdbe4f > > struct kernel_info { > struct kernel_info_header hdr; > uint32_t total_size;/* Total size of all chunks */ > > /* Various fixed-sized data objects, OR offsets to other chunks */ > }; OK, so, this is more or less what I had in my v3 patch before sending this email. So, it looks that I am on good track. Great! Though I am not sure that we should have magic for chunked objects. If yes could you explain why? I would just leave len for chunked objects. > Also "InfO" is a pretty hideous magic. In general, all-ASCII magics have much > higher risk of collision than *RANDOM* binary numbers. However, for a chunked > architecture they do have the advantage that they can be used also as a human > name or file name for the chunk, e.,g. in sysfs, so maybe something like > "LnuX" or even "LToP" for the top-level chunk might make sense. > > How does that sound? Well, your proposals are more cryptic, especially the second one, than mine but I tend to agree that more *RANDOM* magics are better. So, I would choose "LToP" if you decipher it for me. Linux Top? Daniel
Re: [PATCH v3] docs: Use make invocation's -j argument for parallelism
On Tue, 24 Sep 2019 16:29:58 -0700 Kees Cook wrote: > While sphinx 1.7 and later supports "-jauto" for parallelism, this > effectively ignores the "-j" flag used in the "make" invocation, which > may cause confusion for build systems. Instead, extract the available > parallelism from "make"'s job server (since it is not exposed in any > special variables) and use that for the "sphinx-build" run. Now things > work correctly for builds where -j is specified at the top-level: > > make -j16 htmldocs > > If -j is not specified, continue to fallback to "-jauto" if available. > > Signed-off-by: Kees Cook I finally messed with this a bit; it seems to do exactly what's written on the box. It seems to me that The Real Solution™ here is to send a patch to the Sphinx folks adding a "-jgmake" (or some such) option. It also seems to me that none of us is likely to get around to that in the near future. So I just applied this, thanks for dealing with all my picky comments... jon
Re: [PATCH 1/2] docs: security: fix section hyperlink
On Wed, 25 Sep 2019 17:17:44 +0700 bhenryj0...@gmail.com wrote: > From: Brendan Jackman > > The reStructuredText syntax is wrong here; not sure how it was > intended but we can just use the section header as an implicit > hyperlink target, with a single "outward" underscore. > > Signed-off-by: Brendan Jackman Applied, thanks. jon
Re: [PATCH 2/2] docs: security: update base LSM documentation file
On Wed, 25 Sep 2019 13:06:28 -0700 Kees Cook wrote: > > +always appears first in the stack of LSM hooks. A > > +:c:func:`security_add_hooks()` function (in ``security/security.c``) > > IIUC, the :c:func:`...()` marking isn't needed any more, just having a > word followed by "()" should trigger the markup. That is correct; we now want to stomp out :c:func: wherever we see it... Thanks, jon
Re: [PATCH] docs: kmemleak: DEBUG_KMEMLEAK_EARLY_LOG_SIZE changed names
On Wed, 25 Sep 2019 15:32:10 +0100 Catalin Marinas wrote: > On Wed, Sep 25, 2019 at 02:31:14PM +, Jeremy Cline wrote: > > Commit c5665868183f ("mm: kmemleak: use the memory pool for early > > allocations") renamed CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE to > > CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE. Update the documentation reference > > to reflect that. > > > > Signed-off-by: Jeremy Cline > > I forgot about this. Thanks. > > Acked-by: Catalin Marinas Applied, thanks. jon
Re: [PATCH] docs: fix memory.low description in cgroup-v2.rst
On Wed, 25 Sep 2019 12:56:04 -0700 Jon Haslam wrote: > The current cgroup-v2.rst file contains an incorrect description of when > memory is reclaimed from a cgroup that is using the 'memory.low' > mechanism. This fix simply corrects the text to reflect the actual > implementation. > > Fixes: 7854207fe954 ("mm/docs: describe memory.low refinements") > Signed-off-by: Jon Haslam Applied, thanks. jon
Re: [PATCH 2/2] kernel-doc: add support for ____cacheline_aligned_in_smp attribute
On Tue, 17 Sep 2019 16:41:46 -0300 André Almeida wrote: > -if ($x =~ > /(struct|union)\s+(\w+)\s*\{(.*)\}(\s*(__packed|__aligned|__attribute__\s*\(\([a-z0-9,_\s\(\)]*\)\)))*/) > { > +if ($x =~ > /(struct|union)\s+(\w+)\s*\{(.*)\}(\s*(__packed|__aligned|cacheline_aligned_in_smp|__attribute__\s*\(\([a-z0-9,_\s\(\)]*\)\)))*/) > { Sigh...scripts/kernel-doc just makes me want to cry sometimes... I've applied both patches. Thanks, and apologies for the delay... jon
Re: [PATCH v2] docs: perf: Add imx-ddr to documentation index
On Sun, 15 Sep 2019 18:20:10 +1000 Adam Zerella wrote: > Sphinx is currently outputting a warning where > the file 'imx-ddr.rst' is not included in the > documentation index. Additionally, the code > highlighting and doc formatting can be slightly > improved. > > Signed-off-by: Adam Zerella > --- > > v2: Supress additional indentation warning Applied, thanks; apologies for the delay. jon
Re: [PATCH] Documentation: document earlycon without options for more platforms
On Mon, 16 Sep 2019 09:03:16 +0200 Christoph Hellwig wrote: > The earlycon options without arguments is supposed on all device > tree platforms, not just arm64. > > Signed-off-by: Christoph Hellwig > --- > Documentation/admin-guide/kernel-parameters.txt | 10 -- > 1 file changed, 4 insertions(+), 6 deletions(-) Applied (finally), thanks. jon
Re: [PATCH] riscv-docs: correct the sequence of the magic number 2 since it's little endian
On Mon, 16 Sep 2019 13:01:40 + Chester Lin wrote: > Correct the sequence of the magic number 2 since it's little endian. > > Signed-off-by: Chester Lin > --- > Documentation/riscv/boot-image-header.txt | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) So I'm slowly digging my way through my docs folder, sorry for the delay. > diff --git a/Documentation/riscv/boot-image-header.txt > b/Documentation/riscv/boot-image-header.txt > index 14b1492f689b..f388805aa5c3 100644 > --- a/Documentation/riscv/boot-image-header.txt > +++ b/Documentation/riscv/boot-image-header.txt > @@ -18,7 +18,7 @@ The following 64-byte header is present in decompressed > Linux kernel image. > u32 res1 = 0;/* Reserved */ > u64 res2 = 0;/* Reserved */ > u64 magic = 0x5643534952; /* Magic number, little endian, "RISCV" */ > - u32 magic2 = 0x56534905; /* Magic number 2, little endian, "RSC\x05" */ > + u32 magic2 = 0x05435352; /* Magic number 2, little endian, "RSC\x05" */ That's doing more than just reordering the bytes. Was the original completely wrong? If so, the changelog should probably reflect that. Or else what am I missing? Thanks, jon
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
On Tue, 24 Sep 2019 16:02:06 -0700 Kees Cook wrote: > Commit log from Patch 2 repeated here for cover letter: > > In order to have the MAINTAINERS file visible in the rendered ReST > output, this makes some small changes to the existing MAINTAINERS file > to allow for better machine processing, and adds a new Sphinx directive > "maintainers-include" to perform the rendering. I finally got around to trying this out. After the usual warnings, the build comes to a screeching halt with this: Sphinx parallel build error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128) For extra fun, the build process simply hangs, requiring a ^C to blow it away. You've managed to get new behavior out of Sphinx that I've not seen before, congratulations :) This almost certainly has to do with the fact that I'm (intentionally) running the Python2 Sphinx build here. Something's not doing unicode that should be. I would suggest that we might just want to repair this before merging this feature. Either that, or we bite the bullet and deprecate the use of Python 2 entirely - something that's probably not too far into our future regardless. Anybody have thoughts on that matter? On a separate note...it occurred to me, rather belatedly as usual, that last time we discussed doing this that there was some opposition to adding a second MAINTAINERS parser to the kernel; future changes to the format of that file may force both to be adjusted, and somebody will invariably forget one. Addressing that, if we feel a need to do so, probably requires tweaking get_maintainer.pl to output the information in a useful format. Thanks, jon
Re: [PATCH v2 2/3] x86/boot: Introduce the setup_indirect
On Fri, Jul 12, 2019 at 08:56:44AM -0700, h...@zytor.com wrote: > On July 4, 2019 9:36:11 AM PDT, Daniel Kiper wrote: [...] > >diff --git a/arch/x86/include/uapi/asm/bootparam.h > >b/arch/x86/include/uapi/asm/bootparam.h > >index b05318112452..17fa6ad6 100644 > >--- a/arch/x86/include/uapi/asm/bootparam.h > >+++ b/arch/x86/include/uapi/asm/bootparam.h > >@@ -2,7 +2,7 @@ > > #ifndef _ASM_X86_BOOTPARAM_H > > #define _ASM_X86_BOOTPARAM_H > > > >-/* setup_data types */ > >+/* setup_data/setup_indirect types */ > > #define SETUP_NONE 0 > > #define SETUP_E820_EXT 1 > > #define SETUP_DTB 2 > >@@ -10,6 +10,7 @@ > > #define SETUP_EFI 4 > > #define SETUP_APPLE_PROPERTIES 5 > > #define SETUP_JAILHOUSE 6 > >+#define SETUP_INDIRECT 7 > > > > /* ram_size flags */ > > #define RAMDISK_IMAGE_START_MASK0x07FF > >@@ -47,6 +48,14 @@ struct setup_data { > > __u8 data[0]; > > }; > > > >+/* extensible setup indirect data node */ > >+struct setup_indirect { > >+__u32 type; > >+__u32 reserved; /* Reserved, must be set to zero. */ > >+__u64 len; > >+__u64 addr; > >+}; > >+ > > struct setup_header { > > __u8setup_sects; > > __u16 root_flags; > > This needs actual implementation; we can't advertise it until the > kernel knows how to consume the data! It probably should be moved to > after the 3/3 patch. > > Implementing this has two parts: > > 1. The kernel needs to be augmented so it can find current objects via > indirection. > > 2. And this is the main reason for this in the first place: the early > code needs to walk the list and map out the memory areas which are > occupied so it doesn't clobber anything; this allows this code to be > generic as opposed to having to know what is a pointer and what size > it might point to. > > (The decompressor didn't need this until kaslr entered the picture, > but now it does, sadly.) Do you think about arch/x86/boot/compressed/kaslr.c:mem_avoid[]? But it is static. OK, we can assume that we do not accept more than something indirect entries. However, this is not nice... > Optional/future enhancements that might be nice: > > 3. Add some kind of description (e.g. foo=u64 ; bar=str ; baz=blob) to > make it possible to write a bootloader that can load these kinds of > objects without specific enabling. This means an extension to command line parser. Am I right? > 4. Add support for mapping initramfs fragments this way. > > 5. Add support for passingload-on-boot kernel modules. I am not sure what you mean exactly by those two. Anyway, I would focus only on things which are potentially useful now or in the near future and not require much code changes. So, IMO #1 and #2 at this point. Daniel
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
Em Tue, 1 Oct 2019 08:31:47 -0600 Jonathan Corbet escreveu: > On Tue, 24 Sep 2019 16:02:06 -0700 > Kees Cook wrote: > > > Commit log from Patch 2 repeated here for cover letter: > > > > In order to have the MAINTAINERS file visible in the rendered ReST > > output, this makes some small changes to the existing MAINTAINERS file > > to allow for better machine processing, and adds a new Sphinx directive > > "maintainers-include" to perform the rendering. > > I finally got around to trying this out. After the usual warnings, the > build comes to a screeching halt with this: > > Sphinx parallel build error: > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: > ordinal not in range(128) > > For extra fun, the build process simply hangs, requiring a ^C to blow it > away. You've managed to get new behavior out of Sphinx that I've not seen > before, congratulations :) > > This almost certainly has to do with the fact that I'm (intentionally) > running the Python2 Sphinx build here. Something's not doing unicode that > should be. > > I would suggest that we might just want to repair this before merging this > feature. Either that, or we bite the bullet and deprecate the use of > Python 2 entirely - something that's probably not too far into our future > regardless. Anybody have thoughts on that matter? I'm almost sure we got this already. If I'm not mistaken, the solution is to add the encoding line after shebang. Looking at Documentation/sphinx/maintainers_include.py (patch 2/2), the script starts with: #!/usr/bin/env python # SPDX-License-Identifier: GPL-2.0 # -*- coding: utf-8; mode: python -*- # pylint: disable=R0903, C0330, R0914, R0912, E0401 But, as I pointed before, the SPDX header at the wrong place is causing the crash, as the encoding line should be the second line, not the third one, e. g.: #!/usr/bin/env python # -*- coding: utf-8; mode: python -*- # SPDX-License-Identifier: GPL-2.0 # pylint: disable=R0903, C0330, R0914, R0912, E0401 I also suspect that this would happen even with python3, depending on how LC_ALL, LANG, ... are set on the distro you use. Thanks, Mauro
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
On Tue, Oct 01, 2019 at 12:09:30PM -0300, Mauro Carvalho Chehab wrote: > Em Tue, 1 Oct 2019 08:31:47 -0600 > Jonathan Corbet escreveu: > > > On Tue, 24 Sep 2019 16:02:06 -0700 > > Kees Cook wrote: > > > > > Commit log from Patch 2 repeated here for cover letter: > > > > > > In order to have the MAINTAINERS file visible in the rendered ReST > > > output, this makes some small changes to the existing MAINTAINERS file > > > to allow for better machine processing, and adds a new Sphinx directive > > > "maintainers-include" to perform the rendering. > > > > I finally got around to trying this out. After the usual warnings, the > > build comes to a screeching halt with this: > > > > Sphinx parallel build error: > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: > > ordinal not in range(128) > > > > For extra fun, the build process simply hangs, requiring a ^C to blow it > > away. You've managed to get new behavior out of Sphinx that I've not seen > > before, congratulations :) > > > > This almost certainly has to do with the fact that I'm (intentionally) > > running the Python2 Sphinx build here. Something's not doing unicode that > > should be. Given this would be for v5.5, and python2 is EOL in 2 months[1], I don't think it's unreasonable to deprecate it. Especially considering there are already explicit "python3" shebangs in existing code in the sphinx/ directory. [1] https://pythonclock.org/ > > I would suggest that we might just want to repair this before merging this > > feature. Either that, or we bite the bullet and deprecate the use of > > Python 2 entirely - something that's probably not too far into our future > > regardless. Anybody have thoughts on that matter? > > I'm almost sure we got this already. If I'm not mistaken, the solution > is to add the encoding line after shebang. Looking at > Documentation/sphinx/maintainers_include.py (patch 2/2), the script > starts with: > > #!/usr/bin/env python > # SPDX-License-Identifier: GPL-2.0 > # -*- coding: utf-8; mode: python -*- > # pylint: disable=R0903, C0330, R0914, R0912, E0401 > > But, as I pointed before, the SPDX header at the wrong place is causing the > crash, as the encoding line should be the second line, not the third one, > e. g.: > > #!/usr/bin/env python > # -*- coding: utf-8; mode: python -*- > # SPDX-License-Identifier: GPL-2.0 > # pylint: disable=R0903, C0330, R0914, R0912, E0401 > > I also suspect that this would happen even with python3, depending on > how LC_ALL, LANG, ... are set on the distro you use. Oh that's a delightful bug. :P I haven't been able to reproduce this failure (maybe all my LANG junk is accidentally correct)? Jon, if this fixes it for you, should I respin the patches? -- Kees Cook
Re: [PATCH v2] docs: networking: Add title caret and missing doc
From: Adam Zerella Date: Tue, 1 Oct 2019 21:16:58 +1000 > Resolving a couple of Sphinx documentation warnings > that are generated in the networking section. > > - WARNING: document isn't included in any toctree > - WARNING: Title underline too short. > > Signed-off-by: Adam Zerella Applied.
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
On Tue, Oct 01, 2019 at 08:31:47AM -0600, Jonathan Corbet wrote: > On a separate note...it occurred to me, rather belatedly as usual, that > last time we discussed doing this that there was some opposition to adding > a second MAINTAINERS parser to the kernel; future changes to the format of > that file may force both to be adjusted, and somebody will invariably > forget one. Addressing that, if we feel a need to do so, probably requires > tweaking get_maintainer.pl to output the information in a useful format. That's a reasonable point, but I would make two observations: - get_maintainers.pl is written in Perl and I really don't want to write more Perl. ;) - the parsing methods in get_maintainers is much more focused on the file/pattern matching and is blind to the structure of the rest of the document (it only examines '^[A-Z]:' and blank lines), and does so "on demand", in that it hunts through the entire MAINTAINERS file contents for each path match. So I don't think it's suitable to merge functionality here... -- Kees Cook
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
On Tue, 1 Oct 2019 09:16:41 -0700 Kees Cook wrote: > > > This almost certainly has to do with the fact that I'm (intentionally) > > > running the Python2 Sphinx build here. Something's not doing unicode that > > > should be. > > Given this would be for v5.5, and python2 is EOL in 2 months[1], I don't > think it's unreasonable to deprecate it. Especially considering there > are already explicit "python3" shebangs in existing code in the sphinx/ > directory. Two months and 30 days, don't exaggerate, man! :) FWIW, the "shebangs" in that directory are no-ops. Those are extensions read into sphinx, not standalone programs; they go into a Python 2 sphinx instance just fine. Which isn't an argument against dropping Python 2, of course. That's really just a matter of when. > > I'm almost sure we got this already. If I'm not mistaken, the solution > > is to add the encoding line after shebang. Looking at > > Documentation/sphinx/maintainers_include.py (patch 2/2), the script > > starts with: > > > > #!/usr/bin/env python > > # SPDX-License-Identifier: GPL-2.0 > > # -*- coding: utf-8; mode: python -*- > > # pylint: disable=R0903, C0330, R0914, R0912, E0401 > > > > But, as I pointed before, the SPDX header at the wrong place is causing the > > crash, as the encoding line should be the second line, not the third one, > > e. g.: > > > > #!/usr/bin/env python > > # -*- coding: utf-8; mode: python -*- > > # SPDX-License-Identifier: GPL-2.0 > > # pylint: disable=R0903, C0330, R0914, R0912, E0401 > > > > I also suspect that this would happen even with python3, depending on > > how LC_ALL, LANG, ... are set on the distro you use. > > Oh that's a delightful bug. :P I haven't been able to reproduce this > failure (maybe all my LANG junk is accidentally correct)? Delightful - and wrong; moving that line around doesn't change anything. The crash comes from deep within sphinx; the more I look at it the more confused I get. Stay tuned... jon
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
On Tue, 1 Oct 2019 12:09:30 -0300 Mauro Carvalho Chehab wrote: > > Sphinx parallel build error: > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: > > ordinal not in range(128) > > > > For extra fun, the build process simply hangs, requiring a ^C to blow it > > away. You've managed to get new behavior out of Sphinx that I've not seen > > before, congratulations :) > > > > This almost certainly has to do with the fact that I'm (intentionally) > > running the Python2 Sphinx build here. Something's not doing unicode that > > should be. > > > > I would suggest that we might just want to repair this before merging this > > feature. Either that, or we bite the bullet and deprecate the use of > > Python 2 entirely - something that's probably not too far into our future > > regardless. Anybody have thoughts on that matter? > > I'm almost sure we got this already. If I'm not mistaken, the solution > is to add the encoding line after shebang. As mentioned before, that's not it. The problem is that we're feeding UTF8 into Sphinx as an ordinary str, then sphinx is trying to decode it as ASCII. The attached hack makes things work. Kees, I can either just keep this fix (breaking bisectability of the docs build :) or you can send a new version - up to you. Thanks, jon >From c32bfbb07a7840662ba3b367c61b7f2946028b27 Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Tue, 1 Oct 2019 11:26:20 -0600 Subject: [PATCH] docs: make maintainers_include work with Python 2 The MAINTAINERS file contains UTF-8 data, so Python 2 code has to be explicit about moving it into the Unicode realm. Explicitly decode all data from the file as soon as we read it. This hack can go away once we deprecate Python 2 support. Signed-off-by: Jonathan Corbet --- Documentation/sphinx/maintainers_include.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Documentation/sphinx/maintainers_include.py b/Documentation/sphinx/maintainers_include.py index de62dd3dabd5..705ba9fd5336 100755 --- a/Documentation/sphinx/maintainers_include.py +++ b/Documentation/sphinx/maintainers_include.py @@ -19,6 +19,7 @@ u""" import re import os.path +import sys from docutils import statemachine from docutils.utils.error_reporting import ErrorString @@ -60,6 +61,8 @@ class MaintainersInclude(Include): field_content = "" for line in open(path): +if sys.version_info.major == 2: +line = unicode(line, 'utf-8') # Have we reached the end of the preformatted Descriptions text? if descriptions and line.startswith('Maintainers'): descriptions = False -- 2.21.0
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
On Tue, 1 Oct 2019 09:27:29 -0700 Kees Cook wrote: > On Tue, Oct 01, 2019 at 08:31:47AM -0600, Jonathan Corbet wrote: > > On a separate note...it occurred to me, rather belatedly as usual, that > > last time we discussed doing this that there was some opposition to adding > > a second MAINTAINERS parser to the kernel; future changes to the format of > > that file may force both to be adjusted, and somebody will invariably > > forget one. Addressing that, if we feel a need to do so, probably requires > > tweaking get_maintainer.pl to output the information in a useful format. > > That's a reasonable point, but I would make two observations: > > - get_maintainers.pl is written in Perl and I really don't want to write > more Perl. ;) Trust me, I get it! > - the parsing methods in get_maintainers is much more focused on the > file/pattern matching and is blind to the structure of the rest > of the document (it only examines '^[A-Z]:' and blank lines), and > does so "on demand", in that it hunts through the entire MAINTAINERS > file contents for each path match. > > So I don't think it's suitable to merge functionality here... Makes sense to me. If anybody out there objects, speak now ... jon
[PATCH] docs/driver-api: Catch up with dma_buf file-name changes
drivers/dma_buf/reservation.c was renamed to dma-resv.c (and include/linux/reservation.h to dma-resv.h), but the documentation was not updated to match, leading to these build errors: Error: Cannot open file ./drivers/dma-buf/reservation.c Error: Cannot open file ./drivers/dma-buf/reservation.c Error: Cannot open file ./drivers/dma-buf/reservation.c Error: Cannot open file ./include/linux/reservation.h Error: Cannot open file ./include/linux/reservation.h Update the documentation and make the world happy again. Fixes: 52791eeec1d9 ("dma-buf: rename reservation_object to dma_resv') Signed-off-by: Jonathan Corbet --- [I'll carry this in docs-next unless somebody objects.] Documentation/driver-api/dma-buf.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Documentation/driver-api/dma-buf.rst b/Documentation/driver-api/dma-buf.rst index b541e97c7ab1..c78db28519f7 100644 --- a/Documentation/driver-api/dma-buf.rst +++ b/Documentation/driver-api/dma-buf.rst @@ -118,13 +118,13 @@ Kernel Functions and Structures Reference Reservation Objects --- -.. kernel-doc:: drivers/dma-buf/reservation.c +.. kernel-doc:: drivers/dma-buf/dma-resv.c :doc: Reservation Object Overview -.. kernel-doc:: drivers/dma-buf/reservation.c +.. kernel-doc:: drivers/dma-buf/dma-resv.c :export: -.. kernel-doc:: include/linux/reservation.h +.. kernel-doc:: include/linux/dma-resv.h :internal: DMA Fences -- 2.21.0
[PATCH v2 1/2] doc-rst: Reduce CSS padding around Field
Right now any ":Field Name: Field Contents" lines end up with significant padding due to CSS from the "table" CSS which rightly needs padding to make tables readable. However, field lists don't need this as they tend to be stacked together. The future heavy use of fields in the parsed MAINTAINERS file needs this cleaned up, and existing users look better too. Note the needless white space (and misalignment of name/contents) between "Date" and "Author": https://www.kernel.org/doc/html/latest/accounting/psi.html This patch fixes this by lowering the padding with a more specific CSS. Signed-off-by: Kees Cook --- Documentation/sphinx-static/theme_overrides.css | 10 ++ 1 file changed, 10 insertions(+) diff --git a/Documentation/sphinx-static/theme_overrides.css b/Documentation/sphinx-static/theme_overrides.css index e21e36cd6761..459ec5b29d68 100644 --- a/Documentation/sphinx-static/theme_overrides.css +++ b/Documentation/sphinx-static/theme_overrides.css @@ -53,6 +53,16 @@ div[class^="highlight"] pre { line-height: normal; } +/* Keep fields from being strangely far apart due to inheirited table CSS. */ +.rst-content table.field-list th.field-name { +padding-top: 1px; +padding-bottom: 1px; +} +.rst-content table.field-list td.field-body { +padding-top: 1px; +padding-bottom: 1px; +} + @media screen { /* content column -- 2.17.1
[PATCH v2 2/2] doc-rst: Programmatically render MAINTAINERS into ReST
In order to have the MAINTAINERS file visible in the rendered ReST output, this makes some small changes to the existing MAINTAINERS file to allow for better machine processing, and adds a new Sphinx directive "maintainers-include" to perform the rendering. Features include: - Per-subsystem reference links: subsystem maintainer entries can be trivially linked to both internally and external. For example: https://www.kernel.org/doc/html/latest/process/maintainers.html#secure-computing - Internally referenced .rst files are linked so they can be followed when browsing the resulting rendering. This allows, for example, the future addition of maintainer profiles to be automatically linked. - Field name expansion: instead of the short fields (e.g. "M", "F", "K"), use the indicated inline "full names" for the fields (which are marked with "*"s in MAINTAINERS) so that a rendered subsystem entry is more human readable. Email lists are additionally comma-separated. For example: SECURE COMPUTING Mail: Kees Cook Reviewer: Andy Lutomirski , Will Drewry SCM: git git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git seccomp Status: Supported Files:kernel/seccomp.c include/uapi/linux/seccomp.h include/linux/seccomp.h tools/testing/selftests/seccomp/* tools/testing/selftests/kselftest_harness.h userspace-api/seccomp_filter Content regex: \bsecure_computing \bTIF_SECCOMP\b Signed-off-by: Kees Cook --- v2: fix python2 utf-8-ism (jon) --- Documentation/conf.py | 3 +- Documentation/process/index.rst | 1 + Documentation/process/maintainers.rst | 1 + Documentation/sphinx/maintainers_include.py | 197 MAINTAINERS | 62 +++--- 5 files changed, 233 insertions(+), 31 deletions(-) create mode 100644 Documentation/process/maintainers.rst create mode 100755 Documentation/sphinx/maintainers_include.py diff --git a/Documentation/conf.py b/Documentation/conf.py index a8fe845832bc..3c7bdf4cd31f 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -37,7 +37,8 @@ needs_sphinx = '1.3' # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', - 'kfigure', 'sphinx.ext.ifconfig', 'automarkup'] + 'kfigure', 'sphinx.ext.ifconfig', 'automarkup', + 'maintainers_include'] # The name of the math extension changed on Sphinx 1.4 if (major == 1 and minor > 3) or (major > 1): diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst index e2c9ffc682c5..e2fb0c9652ac 100644 --- a/Documentation/process/index.rst +++ b/Documentation/process/index.rst @@ -46,6 +46,7 @@ Other guides to the community that are of interest to most developers are: kernel-docs deprecated embargoed-hardware-issues + maintainers These are some overall technical guides that have been put here for now for lack of a better place. diff --git a/Documentation/process/maintainers.rst b/Documentation/process/maintainers.rst new file mode 100644 index ..6174cfb4138f --- /dev/null +++ b/Documentation/process/maintainers.rst @@ -0,0 +1 @@ +.. maintainers-include:: diff --git a/Documentation/sphinx/maintainers_include.py b/Documentation/sphinx/maintainers_include.py new file mode 100755 index ..dc8fed48d3c2 --- /dev/null +++ b/Documentation/sphinx/maintainers_include.py @@ -0,0 +1,197 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# -*- coding: utf-8; mode: python -*- +# pylint: disable=R0903, C0330, R0914, R0912, E0401 + +u""" +maintainers-include +~~~ + +Implementation of the ``maintainers-include`` reST-directive. + +:copyright: Copyright (C) 2019 Kees Cook +:license:GPL Version 2, June 1991 see linux/COPYING for details. + +The ``maintainers-include`` reST-directive performs extensive parsing +specific to the Linux kernel's standard "MAINTAINERS" file, in an +effort to avoid needing to heavily mark up the original plain text. +""" + +import sys +import re +import os.path + +from docutils import statemachine +from docutils.utils.error_reporting import ErrorString +from docutils.parsers.rst import Directive +from docutils.parsers.rst.directives.misc import Include + +__version__ = '1.0' + +def setup(app): +app.add_directive("maintainers-include", MaintainersInclude) +return dict( +version = __version__, +parallel_read_safe = True, +parallel_write_safe = True +) + +class MaintainersInclude(Include): +u"""MaintainersInclude (``maintainers-include``) directive""" +required_arguments = 0 + +def parse_maintainers(self, path): +"""Parse all the MAINTAINERS lines into ReST for human-read
[PATCH v2 0/2] docs: Programmatically render MAINTAINERS into ReST
v1: https://lore.kernel.org/lkml/20190924230208.12414-1-keesc...@chromium.org v2: fix python2 utf-8 issue thanks to Jonathan Corbet Commit log from Patch 2 repeated here for cover letter: In order to have the MAINTAINERS file visible in the rendered ReST output, this makes some small changes to the existing MAINTAINERS file to allow for better machine processing, and adds a new Sphinx directive "maintainers-include" to perform the rendering. Features include: - Per-subsystem reference links: subsystem maintainer entries can be trivially linked to both internally and external. For example: https://www.kernel.org/doc/html/latest/process/maintainers.html#secure-computi ng - Internally referenced .rst files are linked so they can be followed when browsing the resulting rendering. This allows, for example, the future addition of maintainer profiles to be automatically linked. - Field name expansion: instead of the short fields (e.g. "M", "F", "K"), use the indicated inline "full names" for the fields (which are marked with "*"s in MAINTAINERS) so that a rendered subsystem entry is more human readable. Email lists are additionally comma-separated. For example: SECURE COMPUTING Mail: Kees Cook Reviewer: Andy Lutomirski , Will Drewry SCM: git git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.g it seccomp Status: Supported Files:kernel/seccomp.c include/uapi/linux/seccomp.h include/linux/seccomp.h tools/testing/selftests/seccomp/* tools/testing/selftests/kselftest_harness.h userspace-api/seccomp_filter Content regex: \bsecure_computing \bTIF_SECCOMP\b --- Kees Cook (2): doc-rst: Reduce CSS padding around Field doc-rst: Programmatically render MAINTAINERS into ReST Documentation/conf.py | 3 +- Documentation/process/index.rst | 1 + Documentation/process/maintainers.rst | 1 + .../sphinx-static/theme_overrides.css | 10 + Documentation/sphinx/maintainers_include.py | 197 ++ MAINTAINERS | 62 +++--- 6 files changed, 243 insertions(+), 31 deletions(-) create mode 100644 Documentation/process/maintainers.rst create mode 100755 Documentation/sphinx/maintainers_include.py -- 2.17.1
Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST
On Tue, Oct 01, 2019 at 11:35:06AM -0600, Jonathan Corbet wrote: > for line in open(path): > +if sys.version_info.major == 2: > +line = unicode(line, 'utf-8') Ah-ha! Thanks. I've sent v2 now. :) -- Kees Cook
Re: [PATCH] riscv-docs: correct the sequence of the magic number 2 since it's little endian
Hi Jon, On Tue, 1 Oct 2019, Jonathan Corbet wrote: > On Mon, 16 Sep 2019 13:01:40 + > Chester Lin wrote: > > > Correct the sequence of the magic number 2 since it's little endian. > > > > Signed-off-by: Chester Lin > > --- > > Documentation/riscv/boot-image-header.txt | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > So I'm slowly digging my way through my docs folder, sorry for the delay. > > > diff --git a/Documentation/riscv/boot-image-header.txt > > b/Documentation/riscv/boot-image-header.txt > > index 14b1492f689b..f388805aa5c3 100644 > > --- a/Documentation/riscv/boot-image-header.txt > > +++ b/Documentation/riscv/boot-image-header.txt > > @@ -18,7 +18,7 @@ The following 64-byte header is present in decompressed > > Linux kernel image. > > u32 res1 = 0;/* Reserved */ > > u64 res2 = 0;/* Reserved */ > > u64 magic = 0x5643534952; /* Magic number, little endian, "RISCV" */ > > - u32 magic2 = 0x56534905; /* Magic number 2, little endian, "RSC\x05" */ > > + u32 magic2 = 0x05435352; /* Magic number 2, little endian, "RSC\x05" */ > > That's doing more than just reordering the bytes. Was the original > completely wrong? If so, the changelog should probably reflect that. Or > else what am I missing? This was simply due to idiocy on my part, due to a very last-minute patch submission, where I missed that the original documented magic number was already in little-endian format. The updated number from Chester is correct. So, Reviewed-by: Paul Walmsley - Paul
Re: [PATCH v2 1/3] x86/boot: Introduce the kernel_info
On 2019-10-01 04:41, Daniel Kiper wrote: > > OK, so, this is more or less what I had in my v3 patch before sending > this email. So, it looks that I am on good track. Great! Though I am not > sure that we should have magic for chunked objects. If yes could you > explain why? I would just leave len for chunked objects. > It makes it easier to validate the contents (bugs happen...), and would allow for multiple chunks that could come from different object files if it ever becomes necessary for some reason. We could also just say that dynamic chunks don't even have pointers, and let the boot loader just walk the list. >> Also "InfO" is a pretty hideous magic. In general, all-ASCII magics have much >> higher risk of collision than *RANDOM* binary numbers. However, for a chunked >> architecture they do have the advantage that they can be used also as a human >> name or file name for the chunk, e.,g. in sysfs, so maybe something like >> "LnuX" or even "LToP" for the top-level chunk might make sense. >> >> How does that sound? > > Well, your proposals are more cryptic, especially the second one, than > mine but I tend to agree that more *RANDOM* magics are better. So, > I would choose "LToP" if you decipher it for me. Linux Top? > Yes, Linux top [structure]. -hpa