Re: [RFC PATCH v4 9/9] printk: use a new ringbuffer implementation

2019-08-16 Thread John Ogness
On 2019-08-16, Dave Young wrote: > John, can you cc kexec list for your later series? Sure. > On 08/08/19 at 12:32am, John Ogness wrote: >> This is a major change because the API (and underlying workings) of >> the new ringbuffer are completely different than the previous &g

[RFC PATCH v5 3/3] printk-rb: add test module

2019-11-27 Thread John Ogness
This module does some heavy write stress testing on the ringbuffer with a reader that is checking for integrity. Signed-off-by: John Ogness --- kernel/printk/Makefile | 3 + kernel/printk/test_prb.c | 347 +++ 2 files changed, 350 insertions(+) create

[RFC PATCH v5 1/3] printk-rb: new printk ringbuffer implementation (writer)

2019-11-27 Thread John Ogness
Add a new lockless ringbuffer implementation to be used for printk. First only add support for writers. Reader support will be added in a follow-up commit. Signed-off-by: John Ogness --- kernel/printk/printk_ringbuffer.c | 676 ++ kernel/printk/printk_ringbuffer.h

[RFC PATCH v5 0/3] printk: new ringbuffer implementation

2019-11-27 Thread John Ogness
reading core misses only about 15% of the total records. John Ogness [0] https://lkml.kernel.org/r/20190212143003.48446-1-john.ogn...@linutronix.de [1] https://lkml.kernel.org/r/20190607162349.18199-1-john.ogn...@linutronix.de [2] https://lkml.kernel.org/r/2019072701.11260-1-john.ogn

[RFC PATCH v5 2/3] printk-rb: new printk ringbuffer implementation (reader)

2019-11-27 Thread John Ogness
Add the reader implementation for the new ringbuffer. Signed-off-by: John Ogness --- kernel/printk/printk_ringbuffer.c | 234 ++ kernel/printk/printk_ringbuffer.h | 12 +- 2 files changed, 245 insertions(+), 1 deletion(-) diff --git a/kernel/printk

Re: [RFC PATCH v5 1/3] printk-rb: new printk ringbuffer implementation (writer)

2019-12-02 Thread John Ogness
ieve that it should be enough to add this check and > bail out in case of error. I need to go through the code again in detail and see how many locations are affected by ABA. All the code was written with the assumption that this type of ABA will not happen. As you've stated, we could add minimal handling so that the ringbuffer at least does not break or get stuck. BTW: The same assumption is made for logical positions. There are 4 times as many of these (on 32-bit systems) but logical positions advance much faster. I will review these as well. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [RFC PATCH v5 2/3] printk-rb: new printk ringbuffer implementation (reader)

2019-12-03 Thread John Ogness
On 2019-12-03, Petr Mladek wrote: >> Add the reader implementation for the new ringbuffer. >> >> Signed-off-by: John Ogness >> --- >> kernel/printk/printk_ringbuffer.c | 234 ++ >> kernel/printk/printk_ringbuffer.h | 12 +- >&

Re: [RFC PATCH v5 1/3] printk-rb: new printk ringbuffer implementation (writer)

2019-12-03 Thread John Ogness
gbuffer and cannot be explicitly created by writers. > Also reader API users might not expect to get a "valid" data-less > record. Readers will never see them. The reader API implementation skips over data-less records. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [RFC PATCH v5 2/3] printk-rb: new printk ringbuffer implementation (reader)

2019-12-04 Thread John Ogness
On 2019-12-04, Petr Mladek wrote: >> +} else if ((DATA_WRAPS(data_ring, blk_lpos->begin) + 1 == >> +DATA_WRAPS(data_ring, blk_lpos->next)) || >> + ((DATA_WRAPS(data_ring, blk_lpos->begin) == >> + DATA_WRAPS(data_ring, -1UL)) && >> +

Re: [RFC PATCH v5 1/3] printk-rb: new printk ringbuffer implementation (writer)

2019-12-05 Thread John Ogness
te the code more to handle this theoretical condition. I will change the data block ID size to "unsigned long". Then it is really only an issue for 32-bit systems. AFAICT, the only real issue is that the head can be pushed to the descriptor index that the tail is pointing to. Fr

Re: [RFC PATCH v5 0/3] printk: new ringbuffer implementation

2019-12-05 Thread John Ogness
will probably be the most interesting, due to memory barrier testing. Otherwise I will definitely be reaching out to you when we are ready to perform actual printk testing with the newly agreed up semantics (lockless, per-console printing threads, synchronous panic consoles). Thanks for your hel

Re: [RFC PATCH v5 1/3] printk-rb: new printk ringbuffer implementation (writer)

2019-12-09 Thread John Ogness
the state variable. That's why the code is using state_var/state_val (SV) for the actual data values, keeping it separate from desc_state/d_state for the the state queries. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [RFC PATCH v5 2/3] printk-rb: new printk ringbuffer implementation (reader)

2019-12-09 Thread John Ogness
the first desc_read_committed() > will pass - we have matching ->seq and committed state. copy_data(), > however, most likely, will generate WARNs. The final > desc_read_committed() will notice that local copy of desc was in > non-consistent state and everything is fine, but we have WARNs in the > log buffer now. Be aware that desc_read_committed() is filling a copy of the descriptor in the local variable @desc. If desc_read_committed() succeeded, that local copy _must_ be consistent. If the WARNs trigger, either desc_read_committed() or the writer code is broken. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [RFC PATCH v5 1/3] printk-rb: new printk ringbuffer implementation (writer)

2019-12-09 Thread John Ogness
uot;); >> + * >> + * next_seq = info.seq + 1; >> + * } >> + */ > > Will this loop ever end? :) > > pr_info() adds data to ringbuffer, which prb_read_valid() reads, so > pr_info() can add more data, which prb_r

Re: [RFC PATCH v5 1/3] printk-rb: new printk ringbuffer implementation (writer)

2019-12-23 Thread John Ogness
(The patch introduces a number of CMPXCHG: similar questions would > apply to those other instances...) LMM_TAG(data_push_tail:A) is the only CMPXCHG that requires its full barrier. All others can be relaxed. I will make this change for the next series. Thanks for taking time for this. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

[PATCH 0/2] printk: replace ringbuffer

2020-01-28 Thread John Ogness
are stored in a separate buffer so that they cannot interfere with the human-readable buffer. John Ogness [0] https://lkml.kernel.org/r/20190212143003.48446-1-john.ogn...@linutronix.de [1] https://lkml.kernel.org/r/20190607162349.18199-1-john.ogn...@linutronix.de [2] https://lkml.kernel.org/r

[PATCH 2/2] printk: use the lockless ringbuffer

2020-01-30 Thread John Ogness
^ ((log_buf_len - 6))) bytes for the dynamic array. Signed-off-by: John Ogness --- include/linux/kmsg_dump.h | 2 - kernel/printk/Makefile| 1 + kernel/printk/printk.c| 836 +++--- 3 files changed, 416 insertions(+), 423 deletions(-) diff --git a

[PATCH 1/2] printk: add lockless buffer

2020-01-30 Thread John Ogness
maps to a desc_ring index followed by the dictionary string of the record. Descriptor state information is the key element to allow readers and writers to locklessly synchronize access to the data. Co-developed-by: Petr Mladek Signed-off-by: John Ogness --- kernel/printk

Re: [PATCH 0/2] printk: replace ringbuffer

2020-02-05 Thread John Ogness
On 2020-02-05, Sergey Senozhatsky wrote: So there is a General protection fault. That's the type of a problem that kills the boot for me as well (different backtrace, tho). >>> >>> Do you have CONFIG_RELOCATABLE and CONFIG_RANDOMIZE_BASE (KASLR) >>> enabled? >> >> Yes. These two o

Re: [PATCH 0/2] printk: replace ringbuffer

2020-02-05 Thread John Ogness
o avoid this danger. John Ogness The quick fixup: diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index d0d24ee1d1f4..5ad67ff60cd9 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -883,6 +883,7 @@ static int devkmsg_open(struct inode *inode, struct file *file)

Re: [PATCH 0/2] printk: replace ringbuffer

2020-02-05 Thread John Ogness
dictionaries, we might see their usage increase and start to be as large as the messages themselves. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH 0/2] printk: replace ringbuffer

2020-02-06 Thread John Ogness
ide buffers, writers request buffers. This eliminates the manual twiddling with the record struct and ensures that the struct is always properly initialized. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH 2/2] printk: use the lockless ringbuffer

2020-02-13 Thread John Ogness
es separately, it is enough just to check for the 2nd case. For the 1st case, prb_first_seq() could be less than r->info->seq if all the preceeding records have no data. But this just means the whole set of records with missing data are skipped, which matches existing behavior. (For ex

Re: [PATCH 2/2] printk: use the lockless ringbuffer

2020-02-13 Thread John Ogness
(but related to data-less records): I hacked the ringbuffer code to inject data-less records at various times in order to verify your report. And I stumbled upon a bug in the ringbuffer, which can lead to an infinite loop in console_unlock(). The problem occurs at: retry = prb_read_v

Re: [PATCH 2/2] printk: use the lockless ringbuffer

2020-02-14 Thread John Ogness
quence numbers will never jump. If there would be a jump (due to lost messages) then EPIPE is issued. The reader can either: 1. continue reading and see the jump 2. reopen the file descriptor, possibly having missed a ton more messages due to reopening, and then start from the oldest available

Re: [PATCH 2/2] printk: use the lockless ringbuffer

2020-02-14 Thread John Ogness
uman-readable text.) I am currently hacking the crash tool to see exactly what needs to be made available in order to access all the data of the ringbuffer. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH 0/2] printk: replace ringbuffer

2020-02-17 Thread John Ogness
On 2020-02-14, Petr Mladek wrote: >> I oversaw that devkmsg_open() setup a printk_record and so I did not >> see to add the extra NULL initialization of text_line_count. There >> should be be an initializer function/macro to avoid this danger. >> >> John

Re: crashdump: Re: [PATCH 2/2] printk: use the lockless ringbuffer

2020-02-17 Thread John Ogness
me are missing). I will get this sorted out for v2. And I will provide some heavily hacked code for crash and makedumpfile to show that the necessary symbols are there and it works. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH 0/2] printk: replace ringbuffer

2020-02-25 Thread John Ogness
record_print_text() for counting will now call the new counting functions. IMHO it is a nice cleanup and also removes the static printk_record structs for console and syslog. Thanks. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lis

Re: misc details: Re: [PATCH 2/2] printk: use the lockless ringbuffer

2020-02-25 Thread John Ogness
logic somewhat close to the current mainline implementation for now. > I prefer to call wake_up_klogd() directly from log_output() or > log_store() instead. It might later be used to wake up > printk kthreads as well. > > It was done this way because consoles were historically preferred > over userspace loggers. But the difference will be lower when > consoles are handled by kthread. Agreed, but that is something I would like to save for a later series. Right now I only want to replace the ringbuffer without rearranging priorities. >> -skip: >> -if (console_seq == log_next_seq) >> -break; >> +console_seq = console_record.info->seq; > > This code suggests that it might be possible to get > console_seq > console_record.info->seq and we just > ignore it. I prefer to make it clear by: > > if (console_seq != console_record.info->seq) { OK. Thanks for your help. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: more barriers: Re: [PATCH 1/2] printk: add lockless buffer

2020-02-27 Thread John Ogness
e see patterns like: do { } while (!try_cmpxchg_relaxed()); smp_mb(); or possibly: smp_mb(); cmpxchg_relaxed(); /* no return value check */ > On Tue 2020-01-28 17:25:47, John Ogness wrote: >> diff --git a/kernel/printk/printk_ringbuffer.c >> b/kern

Re: misc nits Re: [PATCH 1/2] printk: add lockless buffer

2020-03-02 Thread John Ogness
or not. > > Please, rename it to make it clear that does only a check. > For example, check_state_commited(). This function _does_ read. It is a helper function of prb_read() to _read_ the descriptor. It is an extended version of desc_read() that also performs various checks that the descriptor is committed. I will update the function description to be more similar to desc_read() so that it is obvious that it is "getting a copy of a specified descriptor". John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: misc nits Re: [PATCH 1/2] printk: add lockless buffer

2020-03-02 Thread John Ogness
return -EINVAL; > > if (d_state == desc_reusable) > return -ENOENT; I can use this refactoring. > > if (d_state != desc_committed) > return -EINVAL; I suppose you meant to remove this check and leave in the @blk_lpos check instead. If we're trying to minimize lines of code, the @blk_lpos check could be combined with the "== desc_reusable" check as well. > > return 0; > } Thanks. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: misc nits Re: [PATCH 1/2] printk: add lockless buffer

2020-03-03 Thread John Ogness
re. It is lockless >> shared data. desc_read_committed() is called twice in prb_read() and >> it is expected that both calls are using the same @id. > > It is not error prone. If "id" changes then "seq" will not match

Re: more barriers: Re: [PATCH 1/2] printk: add lockless buffer

2020-03-13 Thread John Ogness
ecify a data-less block. */ >>>> + blk_lpos->begin = INVALID_LPOS; >>>> + blk_lpos->next = INVALID_LPOS; >>>> + return NULL; >>>> + } >>>> + } while (!atomic_long_try_cmpxchg_rela

Re: [Crash-utility] new printk ringbuffer interface

2020-04-24 Thread John Ogness
a lot to export everything, but I don't have a problem with it. If we decide to export everything (which I expect we will need to do), then I would change my crash(8) implementation to also rely only on the VMCOREINFO. I see no point in having some implementations using debug data and other implementations using VMCOREINFO data, if VMCOREINFO has everything that is needed. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

[RFC PATCH 0/1] support lockless printk ringbuffer

2020-04-29 Thread John Ogness
27;t expect you to take the patch as-is, but I hope it can provide some positive ground work for moving forward. John Ogness (1): printk: add support for lockless ringbuffer Makefile | 2 +- dwarf_info.c | 36 +- makedumpfile.c | 101 ++-- maked

[RFC PATCH 1/1] printk: add support for lockless ringbuffer

2020-04-29 Thread John Ogness
ot; symbol. Signed-off-by: John Ogness --- Makefile | 2 +- dwarf_info.c | 36 +- makedumpfile.c | 101 ++-- makedumpfile.h | 25 +++ printk.c | 177 + 5 files changed, 333 insertions(+), 8

[PATCH v2 1/3] crash: add VMCOREINFO macro for anonymous structs

2020-05-01 Thread John Ogness
Some structs are not named and are only available via their typedef. Add a VMCOREINFO macro to export field offsets for such structs. Signed-off-by: John Ogness --- include/linux/crash_core.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/crash_core.h b/include/linux

[PATCH v2 0/3] printk: replace ringbuffer

2020-05-01 Thread John Ogness
purpose) - syslog_print_all(): fix missing break if copy_to_user() failed - moved "messages dropped" printing to call_console_drivers() John Ogness [0] https://lkml.kernel.org/r/20200128161948.8524-1-john.ogn...@linutronix.de [1] https://www.redhat.com/archives/crash-utility/2020-Apri

[PATCH v2 3/3] printk: use the lockless ringbuffer

2020-05-03 Thread John Ogness
the static array and 72 * (log_buf_len >> 5) bytes for the dynamic array. Signed-off-by: John Ogness --- include/linux/kmsg_dump.h | 2 - kernel/printk/printk.c| 938 -- 2 files changed, 492 insertions(+), 448 deletions(-) diff --git a/include

[PATCH v2 2/3] printk: add lockless buffer

2020-05-03 Thread John Ogness
element to allow readers and writers to locklessly synchronize access to the data. Co-developed-by: Petr Mladek Signed-off-by: John Ogness --- kernel/printk/Makefile|1 + kernel/printk/printk_ringbuffer.c | 1626 + kernel/printk/printk_ringbuffer.h

Re: [PATCH v2 3/3] printk: use the lockless ringbuffer

2020-05-06 Thread John Ogness
Hi, I discovered a bug while preparing my next series, which also made me realize I had never tested the extended mode feature of netconsole. :-/ The only other user of extended output is /dev/kmsg, and it is doing it correctly. Explanation and patch below. On 2020-05-01, John Ogness wrote

Re: [PATCH v2 2/3] printk: add lockless buffer

2020-05-18 Thread John Ogness
s pairs with desc_read:D. > + */ > + smp_mb(); /* LMM(data_push_tail:C) */ > + > + } while (!atomic_long_try_cmpxchg_relaxed(&data_ring->tail_lpos, > + &tail_lpos, next_lpos)); /* LMM(data_push_tail:D) */ > + > + return tr

Re: [PATCH v2 2/3] printk: add lockless buffer

2020-05-19 Thread John Ogness
ry barriers was helpful for identifying, proving, and testing a minimal set of pairs (on arm64), but all will be folded into full cmpxchg()'s for the next version. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: blk->id read race: was: [PATCH v2 2/3] printk: add lockless buffer

2020-06-09 Thread John Ogness
lk->id > > Now: CPU0 might see outdated blk->id before CPU1 wrote new value > because there is no read barrier betwen reading tail_lpos > and blk->id here. In your example, CPU1 is pushing the tail and then setting the block ID for the _newly_ allocated block, that is located is _before_ the new tail. If CPU0 sees the new tail already, it is still reading a valid block ID, which is _not_ from the block that CPU1 is in the process of writing. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: redundant check in make_data_reusable(): was [PATCH v2 2/3] printk: add lockless buffer

2020-06-09 Thread John Ogness
reserve:G can be >> + * different CPUs. However, the desc_reserve:G >> + * CPU (which performs the full memory barrier) >> + * must have previously seen data_push_tail:D. >> + */ >>

Re: Full barrier in data_push_tail(): was [PATCH v2 2/3] printk: add lockless buffer

2020-06-09 Thread John Ogness
. Tools could greatly assist with this. It is my hope that my memory barrier documentation can spark some ideas about how we could do this. (Doing all this manually really sucks!) John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: Barrier before pushing desc_ring tail: was [PATCH v2 2/3] printk: add lockless buffer

2020-06-09 Thread John Ogness
we > allow to rewrite the descriptor. The changes in data_push_tail() might > be done on different CPUs. There needs to be a reason why the ordering of data tail pushing and descriptor tail pushing is important. > It is similar like the full barrier in data_push_tail() before changing >

Re: data_ring head_lpos and tail_lpos synchronization: was [PATCH v2 2/3] printk: add lockless buffer

2020-06-10 Thread John Ogness
e reasoning: * Guarantee any data ring tail changes are stored before * recycling the descriptor. Data ring tail changes can happen * via desc_push_tail()->data_push_tail(). A full memory * barrier is needed since another

Re: redundant check in make_data_reusable(): was [PATCH v2 2/3] printk: add lockless buffer

2020-06-10 Thread John Ogness
gt; * does not point back to it. >> */ >> if (blk_lpos->begin != lpos_begin) >> return false; >> break; No. Your example showed that it is not caught here. > Here again the comments describe what the check does but not why. > I would write something like: > > /* >* The block might have already been >* reused. Make sure that the descriptor really >* points back to the checked lpos. It covers >* both situations. Random data might point to >* a valid descriptor just by chance. Or the block >* has been already reused by another descriptor. >*/ OK. I will expand the comments to something similar to this. > So, I think that the lpos range check is still redundant. We might > describe it as an extra paranoid check but I am not sure if it is > worth it. > > But I would remove it and keep the code "simple" and design "clear". > Well, I do not resist on it. If we remove it, we end up back at your bug report. ;-) John Ogness [0] https://lkml.kernel.org/r/87ftecy343@linutronix.de ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: blk->id read race: was: [PATCH v2 2/3] printk: add lockless buffer

2020-06-10 Thread John Ogness
ock ID, i.e. the same order) would be inappropriate anyway. Also, this comment only talks about when a new value is seen, but not about when the old value is seen. IMO it is seeing the old value that is worthy of a comment since that is the only case with a data

Re: redundant check in make_data_reusable(): was [PATCH v2 2/3] printk: add lockless buffer

2020-06-10 Thread John Ogness
? >> +if (blk_lpos->begin != lpos_begin) >> +return false; >> + desc_make_reusable(desc_ring, id); >> +break; >> +case desc_reusable: >> +/* >> + * This data block is invalid if the descriptor >> + * does not point back to it. >> + */ >> +if (blk_lpos->begin != lpos_begin) >> +return false; >> +break; >> +} >> + >> +/* Advance @lpos_begin to the next data block. */ >> +lpos_begin = blk_lpos->next; >> +} John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: redundant check in make_data_reusable(): was [PATCH v2 2/3] printk: add lockless buffer

2020-06-11 Thread John Ogness
On 2020-06-10, John Ogness wrote: >>> +static bool data_make_reusable(struct printk_ringbuffer *rb, >>> + struct prb_data_ring *data_ring, >>> + unsigned long lpos_begin, >>> +

Re: redundant check in make_data_reusable(): was [PATCH v2 2/3] printk: add lockless buffer

2020-06-11 Thread John Ogness
reserve the location of current > % # dr->tail_lpos > % prb_commit() > % > % id = blk->id > % # read id for the freshly written data on CPU1 > % # and happily make them reusable &g

Re: Barrier before pushing desc_ring tail: was [PATCH v2 2/3] printk: add lockless buffer

2020-06-11 Thread John Ogness
d desc_make_reusable smp_mb data_push_tail read new data tail data_push_head smp_mb write new data read garbage new da

[PATCH v3 1/3] crash: add VMCOREINFO macro to define offset in a struct declared by typedef

2020-06-18 Thread John Ogness
The existing macro VMCOREINFO_OFFSET() can't be used for structures declared via typedef because "struct" is not part of type definition. Create another macro for this purpose. Signed-off-by: John Ogness --- include/linux/crash_core.h | 3 +++ 1 file changed, 3 insertions(+

[PATCH v3 0/3] printk: replace ringbuffer

2020-06-18 Thread John Ogness
nsigned int" - general: allow some lines to go beyond 80 characters John Ogness [0] https://lkml.kernel.org/r/20200501094010.17694-1-john.ogn...@linutronix.de [1] https://lkml.kernel.org/r/87ftcd86d2@vostro.fn.ogness.net [2] https://lkml.kernel.org/r/87v9ktcs3q@vostro.fn.ognes

Re: [RFC PATCH] printk: _printk_rb_static_dict can be static

2020-06-18 Thread John Ogness
ust have _DECLARE_PRINTKRB (and DECLARE_PRINTKRB) declare all the variables as static. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v3 2/3] printk: add lockless ringbuffer

2020-06-23 Thread John Ogness
element to allow readers and writers to locklessly synchronize access to the data. Co-developed-by: Petr Mladek Signed-off-by: John Ogness --- kernel/printk/Makefile|1 + kernel/printk/printk_ringbuffer.c | 1674 + kernel/printk/printk_ringbuffer.h

[PATCH v3 3/3] printk: use the lockless ringbuffer

2020-06-23 Thread John Ogness
the static array and 72 * (log_buf_len >> 5) bytes for the dynamic array. Signed-off-by: John Ogness --- include/linux/kmsg_dump.h | 2 - kernel/printk/printk.c| 944 -- 2 files changed, 497 insertions(+), 449 deletions(-) diff --git a/include

Re: [PATCH v3 0/3] printk: replace ringbuffer

2020-06-25 Thread John Ogness
Hi Dave, On 2020-06-25, Dave Young wrote: > On 06/18/20 at 04:55pm, John Ogness wrote: >> Here is a v3 for the first series to rework the printk >> subsystem. The v2 and history are here [0]. This first series >> only replaces the existing ringbuffer implementation. No lock

Re: truncate dict: was: Re: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-06-26 Thread John Ogness
runc_msg_len = 0; >> + >> /* disable the "dict" completely */ >> *dict_len = 0; > > The dictionary does not longer need to be removed at this point. > It is stored in a separate buffer. It is ignored by prb_reserve() > when there is not enough space

Re: buffer allocation: was: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-06-26 Thread John Ogness
memcpy(log_buf, __log_buf, __LOG_BUF_LEN); >> + >> +free = __LOG_BUF_LEN; >> +prb_for_each_record(0, &printk_rb_static, seq, &r) >> +free -= add_to_rb(&printk_rb_dynamic, &r); >> + >> +prb = &printk_rb_dyna

Re: record_printk_text tricks: was: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-06-26 Thread John Ogness
; +} > > I do not have strong opinion. The code does things by small steps that > might be checked easily. But it has many steps and it might hide some > subtle catches like the one commented right above. Your suggestions to simplify are good. >> +static size_t get_rec

Re: syslog size unread: was: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-06-29 Thread John Ogness
On 2020-06-25, Petr Mladek wrote: > On Thu 2020-06-18 16:55:19, John Ogness wrote: >> --- a/kernel/printk/printk.c >> +++ b/kernel/printk/printk.c >> @@ -1609,11 +1633,15 @@ int do_syslog(int type, char __user *buf, int len, >> int source) >> break;

Re: buffer allocation: was: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-06-29 Thread John Ogness
h to ilog2() instead of the more conservative order_base_2(). John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: pending output optimization: was: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-07-01 Thread John Ogness
uot;printk: lock/unlock console only for new logbuf entries"). Then there is no @pending_output for me to implement and it will be clear that this series is changing/reverting some printk behavior. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-07-02 Thread John Ogness
s (crash, makedumpfile, vmcore-dmesg), others should be able to implement the changes to their software without needing my help. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v3 3/3] printk: use the lockless ringbuffer

2020-07-03 Thread John Ogness
est using 32-bit dumps on 64-bit machines and vice versa. But it should work. John Ogness [0] https://github.com/Linutronix/kexec-tools.git (printk branch) ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v4 1/4] crash: add VMCOREINFO macro to define offset in a struct declared by typedef

2020-07-07 Thread John Ogness
The existing macro VMCOREINFO_OFFSET() can't be used for structures declared via typedef because "struct" is not part of type definition. Create another macro for this purpose. Signed-off-by: John Ogness Acked-by: Baoquan He Acked-by: Sergey Senozhatsky Reviewed-by: Petr Mlade

[PATCH v4 0/4] printk: replace ringbuffer

2020-07-07 Thread John Ogness
https://lkml.kernel.org/r/87k0ztbea9@jogness.linutronix.de [11] https://lkml.kernel.org/r/20200625152523.GJ8444@alley [12] https://lkml.kernel.org/r/87o8oznh2c@jogness.linutronix.de John Ogness (4): crash: add VMCOREINFO macro to define offset in a struct declared by typedef printk

[PATCH v4 3/4] Revert "printk: lock/unlock console only for new logbuf entries"

2020-07-07 Thread John Ogness
: John Ogness --- kernel/printk/printk.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index b71eaf5f5a86..1b41e1b98221 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -1981,9 +1981,8 @@ asmlinkage

Re: [PATCH v4 4/4] printk: use the lockless ringbuffer

2020-07-08 Thread John Ogness
Fix prepared for for v5. On 2020-07-08, kernel test robot wrote: >>> kernel/printk/printk.c:1146:10: warning: format specifies type 'unsigned >>> long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat] > new_descs_size); > ^~~~

Re: [PATCH v4 0/4] printk: replace ringbuffer

2020-07-09 Thread John Ogness
t write_atomic() for 8250 UART Some of the steps may be combined into a single series if the changes are not too dramatic. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [printk] 18a2dc6982: ltp.kmsg01.fail

2020-07-09 Thread John Ogness
le this. Let me try some alternatives and post a proposed solution. John Ogness [0] https://lkml.kernel.org/r/20200213090757.ga36...@google.com ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [printk] 18a2dc6982: ltp.kmsg01.fail

2020-07-09 Thread John Ogness
ction and made prb_first_seq() static. (The ringbuffer still needs prb_first_seq() for itself.) John Ogness diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 0b1337f4188c..fec71229169e 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -775,9 +775,9 @@ stati

[PATCH v5 1/4] crash: add VMCOREINFO macro to define offset in a struct declared by typedef

2020-07-09 Thread John Ogness
The existing macro VMCOREINFO_OFFSET() can't be used for structures declared via typedef because "struct" is not part of type definition. Create another macro for this purpose. Signed-off-by: John Ogness Acked-by: Baoquan He Acked-by: Sergey Senozhatsky Reviewed-by: Petr Mlade

[PATCH v5 3/4] Revert "printk: lock/unlock console only for new logbuf entries"

2020-07-09 Thread John Ogness
: John Ogness Acked-by: Sergey Senozhatsky Reviewed-by: Petr Mladek --- kernel/printk/printk.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index b71eaf5f5a86..1b41e1b98221 100644 --- a/kernel/printk/printk.c +++ b

[PATCH v5 0/4] printk: replace ringbuffer

2020-07-09 Thread John Ogness
) [4] https://lkml.kernel.org/r/87zh89kkek@jogness.linutronix.de [5] https://lkml.kernel.org/r/87r1tmcfhf@jogness.linutronix.de John Ogness (4): crash: add VMCOREINFO macro to define offset in a struct declared by typedef printk: add lockless ringbuffer Revert "printk: lock/u

Re: [PATCH v4 0/4] printk: replace ringbuffer

2020-07-10 Thread John Ogness
_CONT messages, they will present the user with the ugliness of LOG_CONT pieces (always, rather than as is now rarely). John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v5 4/4] printk: use the lockless ringbuffer

2020-07-15 Thread John Ogness
the static array and 72 * (log_buf_len >> 5) bytes for the dynamic array. Signed-off-by: John Ogness Reviewed-by: Petr Mladek --- kernel/printk/printk.c | 940 + 1 file changed, 493 insertions(+), 447 deletions(-) diff --git a/kernel/printk/prin

[PATCH v5 2/4] printk: add lockless ringbuffer

2020-07-15 Thread John Ogness
element to allow readers and writers to locklessly synchronize access to the data. Signed-off-by: John Ogness Co-developed-by: Petr Mladek Reviewed-by: Petr Mladek Reviewed-by: Paul E. McKenney --- kernel/printk/Makefile|1 + kernel/printk/printk_ringbuffer.c | 1687

[PATCH v4 2/4] printk: add lockless ringbuffer

2020-07-15 Thread John Ogness
element to allow readers and writers to locklessly synchronize access to the data. Signed-off-by: John Ogness Co-developed-by: Petr Mladek Reviewed-by: Petr Mladek Reviewed-by: Paul E. McKenney --- kernel/printk/Makefile|1 + kernel/printk/printk_ringbuffer.c | 1676

[PATCH v4 4/4] printk: use the lockless ringbuffer

2020-07-15 Thread John Ogness
the static array and 72 * (log_buf_len >> 5) bytes for the dynamic array. Signed-off-by: John Ogness --- kernel/printk/printk.c | 940 + 1 file changed, 493 insertions(+), 447 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/pr

Re: [PATCH 04/30] firmware: google: Convert regular spinlock into trylock on panic path

2022-05-11 Thread John Ogness
On 2022-05-10, Steven Rostedt wrote: >> As already mentioned in the other reply, panic() sometimes stops the >> other CPUs using NMI, for example, see kdump_nmi_shootdown_cpus(). >> >> Another situation is when the CPU using the lock ends in some >> infinite loop because something went wrong. The

[PATCH] docs: gdbmacros: print newest record

2022-12-29 Thread John Ogness
("scripts/gdb: update for lockless printk ringbuffer") Cc: sta...@vger.kernel.org Signed-off-by: John Ogness --- Documentation/admin-guide/kdump/gdbmacros.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kdump/gdbmacros.txt b/Documentation/a

[PATCH 3/4] printk: process cont records during reading

2020-07-17 Thread John Ogness
kmsg_dump does not use the continuous line interface. The continuous line interface discards dictionaries of continuous lines. Signed-off-by: John Ogness --- kernel/printk/printk.c | 455 + 1 file changed, 371 insertions(+), 84 deletions(-) diff --git a/kernel

[PATCH 0/4] printk: reimplement LOG_CONT handling

2020-07-17 Thread John Ogness
/linux/kernel/git/printk/linux.git (printk-rework branch) John Ogness (4): printk: ringbuffer: support dataless records printk: store instead of processing cont parts printk: process cont records during reading ipconfig: cleanup printk usage kernel/printk/printk.c| 569

[PATCH 4/4] ipconfig: cleanup printk usage

2020-07-17 Thread John Ogness
The use of pr_info() and pr_cont() was not ordered correctly for all cases. Order it so that all cases provide the expected output. Signed-off-by: John Ogness --- net/ipv4/ipconfig.c | 25 + 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/net/ipv4

[PATCH 2/4] printk: store instead of processing cont parts

2020-07-17 Thread John Ogness
Instead of buffering continuous line parts before storing the full line into the ringbuffer, store each part as its own record. Signed-off-by: John Ogness --- kernel/printk/printk.c | 114 - 1 file changed, 11 insertions(+), 103 deletions(-) diff --git a

[PATCH 1/4] printk: ringbuffer: support dataless records

2020-07-17 Thread John Ogness
lls. Change the code to use two different identifiers (FAILED_LPOS and NO_LPOS) to distinguish between failed prb_reserve() records and successful dataless records. Signed-off-by: John Ogness --- kernel/printk/printk_ringbuffer.c | 58 ++- kernel/printk/printk_ringbuffe

Re: [PATCH 0/4] printk: reimplement LOG_CONT handling

2020-07-18 Thread John Ogness
strict about this. We are making a serious effort to avoid breaking things for userspace. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v5 4/4] printk: use the lockless ringbuffer

2020-07-20 Thread John Ogness
all_rest_init(); Thanks for the example. This is an unintentional regression in the series. I will submit a patch to fix this. Note that this regression does not exist when the followup series [0] (reimplementing LOG_CONT) is applied. All the more reason that the 1st series should be

[PATCH][next] printk: ringbuffer: support dataless records

2020-07-20 Thread John Ogness
://lkml.kernel.org/r/20200718121053.ga691...@elver.google.com Signed-off-by: John Ogness --- based on next-20200720 kernel/printk/printk_ringbuffer.c | 58 ++- kernel/printk/printk_ringbuffer.h | 15 2 files changed, 35 insertions(+), 38 deletions(-) diff -

Re: [PATCH 1/4] printk: ringbuffer: support dataless records

2020-07-20 Thread John Ogness
On 2020-07-18, John Ogness wrote: > In order to support storage of continuous lines, dataless records must > be allowed. For example, these are generated with the legal calls: > > pr_info(""); > pr_cont("\n"); > > Currently dataless record

Re: [PATCH][next] printk: ringbuffer: support dataless records

2020-07-21 Thread John Ogness
return -ENOENT; >> } > > If this is a dataless record then should copy_data() return error? You are correct. That makes more sense. I will send a v2. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v2][next] printk: ringbuffer: support dataless records

2020-07-21 Thread John Ogness
://lkml.kernel.org/r/20200718121053.ga691...@elver.google.com Reported-by: Marco Elver Signed-off-by: John Ogness --- based on next-20200721 chages since v1: - Instead of handling empty text messages as special case errors, allow such messages to be handled as any other valid messages. This also a

Re: [PATCH 2/4] printk: store instead of processing cont parts

2020-07-21 Thread John Ogness
ntext) 0x81 = interrupt number 0x82 = cpu id (nmi context) Or maybe ascii should be used instead? 0x80 | '\0' = cpu id (generic context) 0x80 | 'i' = interrupt number 0x80 | 'n' = cpu id (nmi context) Just an idea. John Ogness

Re: [PATCH v5 4/4] printk: use the lockless ringbuffer

2020-08-12 Thread John Ogness
entation/admin-guide/kdump/vmcoreinfo.rst > - scripts/gdb/linux/dmesg.py > > I believe this commit removed log_first_idx, so all of the above > probably need to be fixed up, too. Thanks for pointing this out! I will get to work on a patch for this. John Ogness _

  1   2   >