Integrating QSBR with libuv

2025-07-15 Thread Ondřej Surý via lttng-dev
Hi,currently, the BIND 9 design involves multiple threads with per-thread event loop from libuv (uv_loop).We are using the default rcu variant, but it seems like an ideal place to switch to QSBR as we can call quiescent state periodically before polling on the loop.This comes with one trouble thoug

Re: cds_lfht_new() clarification - init vs min

2025-07-10 Thread Ondřej Surý via lttng-dev
> On 10. 7. 2025, at 16:05, Mathieu Desnoyers > wrote: > > On 2025-07-09 14:57, Ondřej Surý via lttng-dev wrote: >> Hi, >> as the answer to this might be useful to more people, I am asking here: >> The cds_lfht_new documentation specifies 3 sizes. >> *

cds_lfht_new() clarification - init vs min

2025-07-09 Thread Ondřej Surý via lttng-dev
Hi, as the answer to this might be useful to more people, I am asking here: The cds_lfht_new documentation specifies 3 sizes. * cds_lfht_new - allocate a hash table. * @init_size: number of buckets to allocate initially. Must be power of two. * @min_nr_alloc_buckets: the minimum number of all

librseq memory allocator portability

2025-02-02 Thread Ondřej Surý via lttng-dev
Hey Mathieu, I’m actually thinking about using librseq for some more lightweight stuff first - like statistic counters (especially those related to memory allocations), and my first obvious question would be - how portable is the librseq memory allocator? I am not keen on having a code with if

[lttng-dev] rcu_cmpxchg_pointer() documentation patch

2024-07-04 Thread Ondřej Surý via lttng-dev
Hi, looks like my git-send-email configuration is not correct and my mailserver ate the patch, so here's one created by git-format-patch... Nothing important, but it bite me today... 0001-pointer.h-Fix-the-rcu_cmpxchg_pointer-documentation.patch Description: Binary data Ondrej -- Ondřej Sur

[lttng-dev] (Userspace-)RCU based HEAP (priority queue)

2024-02-22 Thread Ondřej Surý via lttng-dev
Hi, we are just exploring the field, so asking here is probably good idea. We are currently using bucketed heaps for TTL-based expiration of DNS records in BIND 9. This doesn't scale that well and has other problems, so I was thinking that we might be able to replace this with something better.

[lttng-dev] TSAN build broken on master branch

2023-09-21 Thread Ondřej Surý via lttng-dev
Hi, I've updated to the latest HEAD and compilation with clang-16 and clang-17 and ./configure --enable-compiler-atomic-builtins --disable-legacy-mb It fails with: rculfhash.c:1189:2: error: address argument to atomic operation must be a pointer to integer ('typeof (node_next)' (aka 'struct cd

[lttng-dev] LTTng-UST and SystemTap/DTrace

2023-08-14 Thread Ondřej Surý via lttng-dev
Hi, we are in process of adding USDT probes to the BIND 9 probes and the obvious question is whether it’s possible to use the probes with the LTTng tools. Alternatively, if there’s an easy way how to use one or another. The USDT probes are provided by SystemTap on Linux and by DTrace on BSDs an

[lttng-dev] [PATCH] Avoid calling caa_container_of on NULL pointer in cds_lfhash macros

2023-06-22 Thread Ondřej Surý via lttng-dev
(Sorry, I missed closing brackets in both macros, so resending fixed patch...) The cds_lfht_for_each_entry and cds_lfht_for_each_entry_duplicate macros would call caa_container_of() macro on NULL pointer. This is not a problem under normal circumstances as the check in the for loop fails and the

[lttng-dev] [PATCH] Avoid calling caa_container_of on NULL pointer in cds_lfhash macros

2023-06-22 Thread Ondřej Surý via lttng-dev
The cds_lfht_for_each_entry and cds_lfht_for_each_entry_duplicate macros would call caa_container_of() macro on NULL pointer. This is not a problem under normal circumstances as the check in the for loop fails and the loop-statement is not called with invalid (pos) value. However AddressSanitizer

Re: [lttng-dev] [PATCH v2 00/12] Add support for TSAN to liburcu

2023-06-07 Thread Ondřej Surý via lttng-dev
Olivier, is this somewhere in Gerrit again, please? It’s much easier for us to pull the changes from git than cherry-pick patches from the mailing list. Ondřej -- Ondřej Surý (He/Him) > On 7. 6. 2023, at 20:54, Olivier Dion wrote: > > This patch set adds support for TSAN in liburcu. > > *

Re: [lttng-dev] [PATCH 00/11] Add support for TSAN to liburcu

2023-05-26 Thread Ondřej Surý via lttng-dev
> On 26. 5. 2023, at 8:08, Dmitry Vyukov wrote: > > On Fri, 26 May 2023 at 07:33, Ondřej Surý wrote: >> >> A little bit related question/problem - some of our system test jobs got >> significantly slower (3x-5x) with **clang-16** TSAN after we integrated >> liburcu to BIND 9. The GCC (13.1.1

Re: [lttng-dev] [PATCH 00/11] Add support for TSAN to liburcu

2023-05-25 Thread Ondřej Surý via lttng-dev
A little bit related question/problem - some of our system test jobs got significantly slower (3x-5x) with **clang-16** TSAN after we integrated liburcu to BIND 9. The GCC (13.1.1) TSAN doesn’t exhibit this behavior. The liburcu integration in BIND 9 is very light at the moment - we use liburcu

Re: [lttng-dev] [RFC] Deprecating RCU signal flavor

2023-05-16 Thread Ondřej Surý via lttng-dev
Hi, we (ISC) have no intention of using signal based URCU in BIND 9. Our plan relies on using the QSBR flavor as BIND 9 is internally event-loop based with natural quiescent states. Ondrej -- Ondřej Surý (He/Him) > On 10. 5. 2023, at 23:10, Olivier Dion via lttng-dev > wrote: > > Hi all,

[lttng-dev] RCU API usage from call_rcu callbacks?

2023-03-22 Thread Ondřej Surý via lttng-dev
Hi, the documentation is pretty silent on this, and asking here is probably going to be faster than me trying to use the source to figure this out. Is it legal to call_rcu() from within the call_rcu() callback? What about the other RCU (and CDS) API calls? How does that interact with create_ca

Re: [lttng-dev] ThreadSanitizer: data race between urcu_mb_synchronize_rcu and urcu_adaptative_wake_up

2023-03-22 Thread Ondřej Surý via lttng-dev
> On 22. 3. 2023, at 9:02, Ondřej Surý via lttng-dev > wrote: > > That's pretty much weird because the "Write" happens on stack local variable, > while the "Previous write" happens after futex, which lead me to the fact that > ThreadSanitizer doesn

[lttng-dev] ThreadSanitizer: data race between urcu_mb_synchronize_rcu and urcu_adaptative_wake_up

2023-03-22 Thread Ondřej Surý via lttng-dev
Hi, this happens with all the patches fully applied and doesn't seem to be caused by anything I am doing :) WARNING: ThreadSanitizer: data race (pid=3995296) Write of size 8 at 0x7fb51e5fd048 by thread T296: #0 __tsan_memset (badcache_test+0x49257d) (BuildId: 166dea93b2dca28264fc85c79b56

[lttng-dev] TSAN and the tests

2023-03-21 Thread Ondřej Surý via lttng-dev
> On 21. 3. 2023, at 15:49, Mathieu Desnoyers > wrote: > > Agreed, let's see how it holds up to testing under TSAN. :) The make check is OK, but any of the make regtest, short_bench or long_bench produces lot of TSAN warnings like these below. Looks like they mostly come from the tests itself

[lttng-dev] [PATCH 7/7] Fix: uatomic_or() need retyping to uintptr_t in rculfhash.c

2023-03-21 Thread Ondřej Surý via lttng-dev
When adding REMOVED_FLAG to the pointers in the rculfhash implementation, retype the generic pointer to unsigned long to fix the following compiler error: rculfhash.c:1201:2: error: address argument to atomic operation must be a pointer to integer ('struct cds_lfht_node **' invalid) uatomi

Re: [lttng-dev] [PATCH 7/7] Experiment: Add explicit memory barrier in free_completion()

2023-03-21 Thread Ondřej Surý via lttng-dev
> On 21. 3. 2023, at 15:46, Mathieu Desnoyers > wrote: > > On 2023-03-21 06:21, Ondřej Surý wrote: >>> On 20. 3. 2023, at 19:37, Mathieu Desnoyers >>> wrote: >>> >>> On 2023-03-17 17:37, Ondřej Surý via lttng-dev wrote: >>>> FIXME:

[lttng-dev] [PATCH 1/7] Require __atomic builtins to build

2023-03-21 Thread Ondřej Surý via lttng-dev
Add autoconf checks for all __atomic builtins that urcu require, and adjust the gcc and clang versions in the README.md. Signed-off-by: Ondřej Surý --- README.md| 33 + configure.ac | 15 +++ 2 files changed, 24 insertions(+), 24 deletions(-) diff

[lttng-dev] [PATCH 6/7] Use __atomic builtins to implement CMM_{LOAD, STORE}_SHARED

2023-03-21 Thread Ondřej Surý via lttng-dev
Instead of using CMM_ACCESS_ONCE() with memory barriers, use __atomic builtins with relaxed memory ordering to implement CMM_LOAD_SHARED() and CMM_STORE_SHARED(). Signed-off-by: Ondřej Surý --- include/urcu/system.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/

[lttng-dev] [PATCH 5/7] Replace the arch-specific memory barriers with __atomic builtins

2023-03-21 Thread Ondřej Surý via lttng-dev
Instead of a custom code, use the __atomic_thread_fence() builtin to implement the cmm_mb(), cmm_rmb(), cmm_wmb(), cmm_smp_mb(), cmm_smp_rmb(), and cmm_smp_wmb() on all architectures, and cmm_read_barrier_depends() on alpha (otherwise it's still no-op). family of functions Signed-off-by: Ondřej S

[lttng-dev] [PATCH 7/7] Fix: uatomic_or() need retyping to uintptr_t in rculfhash.c

2023-03-21 Thread Ondřej Surý via lttng-dev
When adding REMOVED_FLAG to the pointers in the rculfhash implementation, retype the generic pointer to uintptr_t to fix the compiler error. Signed-off-by: Ondřej Surý --- src/rculfhash.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/rculfhash.c b/src/rculfhash.c in

[lttng-dev] [PATCH 3/7] Use __atomic_signal_fence() for cmm_barrier()

2023-03-21 Thread Ondřej Surý via lttng-dev
Use __atomic_signal_fence(__ATOMIC_SEQ_CST) for cmm_barrier(), so ThreadSanitizer can understand the memory synchronization. Signed-off-by: Ondřej Surý --- include/urcu/compiler.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/urcu/compiler.h b/include/urcu/compiler.

[lttng-dev] [PATCH 2/7] Use gcc __atomic builtis for implementation

2023-03-21 Thread Ondřej Surý via lttng-dev
Replace the custom assembly code in include/urcu/uatomic/ with __atomic builtins provided by C11-compatible compiler. Signed-off-by: Ondřej Surý --- include/Makefile.am| 16 - include/urcu/uatomic.h | 84 +++-- include/urcu/uatomic/aarch64.h | 41 --- include/urcu/uatomic/

[lttng-dev] (no subject)

2023-03-21 Thread Ondřej Surý via lttng-dev
This is a second round of the patches after implementing the requested changes from the first round. Ondrej [PATCH 1/7] Require __atomic builtins to build - no changes [PATCH 2/7] Use gcc __atomic builtis for - the non return macros are now __ATOMIC_RELAXED - the return macros are now __ATOMIC_

[lttng-dev] [PATCH 4/7] Replace the internal pointer manipulation with __atomic builtins

2023-03-21 Thread Ondřej Surý via lttng-dev
Instead of custom code, use the __atomic builtins to implement the rcu_dereference(), rcu_cmpxchg_pointer(), rcu_xchg_pointer() and rcu_assign_pointer(). The rcu_dereference() now relies on CONSUME memory order. Signed-off-by: Ondřej Surý --- include/urcu/static/pointer.h | 77 +++--

Re: [lttng-dev] [PATCH 7/7] Experiment: Add explicit memory barrier in free_completion()

2023-03-21 Thread Ondřej Surý via lttng-dev
> On 20. 3. 2023, at 19:37, Mathieu Desnoyers > wrote: > > On 2023-03-17 17:37, Ondřej Surý via lttng-dev wrote: >> FIXME: This is experiment that adds explicit memory barrier in the >> free_completion in the workqueue.c, so ThreadSanitizer knows it's ok to >>

Re: [lttng-dev] [PATCH 6/7] Fix: uatomic_or() need retyping to uintptr_t in rculfhash.c

2023-03-21 Thread Ondřej Surý via lttng-dev
> On 20. 3. 2023, at 19:31, Mathieu Desnoyers > wrote: > > On 2023-03-17 17:37, Ondřej Surý via lttng-dev wrote: >> When adding REMOVED_FLAG to the pointers in the rculfhash >> implementation, retype the generic pointer to uintptr_t to fix the >> compiler error. &

Re: [lttng-dev] [PATCH 2/7] Use gcc __atomic builtis for implementation

2023-03-20 Thread Ondřej Surý via lttng-dev
> On 20. 3. 2023, at 19:03, Mathieu Desnoyers > wrote: > > In doc/uatomic-api.md, we document: > > "```c > type uatomic_cmpxchg(type *addr, type old, type new); > ``` > > An atomic read-modify-write operation that performs this > sequence of operations atomically: check if `addr` contains `ol

[lttng-dev] [PATCH 5/7] Use __atomic builtins to implement CMM_{LOAD, STORE}_SHARED

2023-03-17 Thread Ondřej Surý via lttng-dev
Instead of using CMM_ACCESS_ONCE() with memory barriers, use __atomic builtins with relaxed memory ordering to implement CMM_LOAD_SHARED() and CMM_STORE_SHARED(). Signed-off-by: Ondřej Surý --- include/urcu/system.h | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/inclu

[lttng-dev] [PATCH 7/7] Experiment: Add explicit memory barrier in free_completion()

2023-03-17 Thread Ondřej Surý via lttng-dev
FIXME: This is experiment that adds explicit memory barrier in the free_completion in the workqueue.c, so ThreadSanitizer knows it's ok to free the resources. Signed-off-by: Ondřej Surý --- src/workqueue.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/workqueue.c b/src/workqueue.c inde

[lttng-dev] [PATCH 6/7] Fix: uatomic_or() need retyping to uintptr_t in rculfhash.c

2023-03-17 Thread Ondřej Surý via lttng-dev
When adding REMOVED_FLAG to the pointers in the rculfhash implementation, retype the generic pointer to uintptr_t to fix the compiler error. Signed-off-by: Ondřej Surý --- src/rculfhash.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/rculfhash.c b/src/rculfhash.c in

[lttng-dev] [PATCH 4/7] Replace the internal pointer manipulation with __atomic builtins

2023-03-17 Thread Ondřej Surý via lttng-dev
Instead of custom code, use the __atomic builtins to implement the rcu_dereference(), rcu_cmpxchg_pointer(), rcu_xchg_pointer() and rcu_assign_pointer(). Signed-off-by: Ondřej Surý --- include/urcu/arch.h | 20 + include/urcu/arch/alpha.h | 6 +++ include/urcu/arch/arm.h

[lttng-dev] [PATCH 3/7] Use __atomic_thread_fence() for cmm_barrier()

2023-03-17 Thread Ondřej Surý via lttng-dev
Use __atomic_thread_fence(__ATOMIC_ACQ_REL) for cmm_barrier(), so ThreadSanitizer can understand the memory synchronization. FIXME: What should be the correct memory ordering here? Signed-off-by: Ondřej Surý --- include/urcu/compiler.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) di

[lttng-dev] [PATCH 2/7] Use gcc __atomic builtis for implementation

2023-03-17 Thread Ondřej Surý via lttng-dev
Replace the custom assembly code in include/urcu/uatomic/ with __atomic builtins provided by C11-compatible compiler. Signed-off-by: Ondřej Surý --- include/Makefile.am| 16 - include/urcu/uatomic.h | 84 +++-- include/urcu/uatomic/aarch64.h | 41 --- include/urcu/uatomic/

[lttng-dev] [PATCH 1/7] Require __atomic builtins to build

2023-03-17 Thread Ondřej Surý via lttng-dev
Add autoconf checks for all __atomic builtins that urcu require, and adjust the gcc and clang versions in the README.md. Signed-off-by: Ondřej Surý --- README.md| 33 + configure.ac | 15 +++ 2 files changed, 24 insertions(+), 24 deletions(-) diff

[lttng-dev] [PATCH 0/7] Replace the custom code with gcc/clang __atomic builtins

2023-03-17 Thread Ondřej Surý via lttng-dev
Hi, (this is my first time using git send-email, so I apologise in advance if anything breaks). Here's my attempt to convert the Userspace RCU to use __atomic builtins whenever possible instead of custom assembly. The __atomic builtins were first introduced in gcc 4.7.0 and clang 3.1.0. Apart f

Re: [lttng-dev] userspace-rcu and ThreadSanitizer

2023-03-17 Thread Ondřej Surý via lttng-dev
> On 17. 3. 2023, at 14:44, Mathieu Desnoyers > wrote: > > I would indeed like to remove all the custom atomics assembly code from > liburcu now that there are good atomics support in the major compilers (gcc > and clang). Here's very preliminary implementation: https://gitlab.isc.org/isc-p

Re: [lttng-dev] userspace-rcu and ThreadSanitizer

2023-03-17 Thread Ondřej Surý via lttng-dev
> On 17. 3. 2023, at 14:44, Mathieu Desnoyers > wrote: > > Sure, can you please submit the patch as a separate email with subject/commit > message/signed-off-by tag ? https://gitlab.isc.org/isc-projects/userspace-rcu/-/merge_requests/1.patch Would this work for you? Or do you need to have t

[lttng-dev] userspace-rcu and ThreadSanitizer

2023-03-14 Thread Ondřej Surý via lttng-dev
Hey, we use ThreadSanitizer in BIND 9 CI quite extensively and with userspace-rcu it's lit like an American house on Saturnalia ;). I have two questions: 1. I've tried to help TSAN by replacing the custom atomics with __atomic gcc primitives - that seems to work pretty well. Note that using C

Re: [lttng-dev] urcu/rculist.h clarifications - for implementing LRU

2023-03-14 Thread Ondřej Surý via lttng-dev
3. 3. 2023, at 15:29, Mathieu Desnoyers >>> wrote: >>> >>> On 2023-03-11 01:04, Ondřej Surý via lttng-dev wrote: >>>> Hey, >>>> so, we are integrating userspace-rcu to BIND 9 (yay!) and as experiment, >>>> I am rewriting the internal a

Re: [lttng-dev] urcu/rculist.h clarifications - for implementing LRU

2023-03-13 Thread Ondřej Surý via lttng-dev
Hi Matthieu, I spent some more time with the userspace-rcu on Friday and over weekend and now I am in much better place. > On 13. 3. 2023, at 15:29, Mathieu Desnoyers > wrote: > > On 2023-03-11 01:04, Ondřej Surý via lttng-dev wrote: >> Hey, >> so, we are integrating

[lttng-dev] urcu/rculist.h clarifications - for implementing LRU

2023-03-10 Thread Ondřej Surý via lttng-dev
Hey, so, we are integrating userspace-rcu to BIND 9 (yay!) and as experiment, I am rewriting the internal address database (keeps the infrastructure information about names and addresses). There's a hashtable to keep the entries and there's associated LRU list. For the hashtable the cds_lfht see