[PATCH v4 02/12] libdwfl [2/12]: expose setfunc callback for libdwflP+libebl clients
Changes for v2:
- No longer exposing this in public libdwfl.h api.
* * *
Renaming pid_set_initial_registers to
__libdwfl_set_initial_registers_thread and adding to libdwflP.h.
This callback was private to one file, but now we need to access it
from the perf_events sampling code as well.
* libdwfl/libdwflP.h (__libdwfl_set_initial_registers_thread): New function.
* libdwfl/linux-pid-attach.c (__libdwfl_set_initial_registers_thread):
Renamed from pid_thread_state_registers_cb.
(pid_set_initial_registers): Pass the newly renamed callback.
---
libdwfl/libdwflP.h | 11 ++-
libdwfl/linux-pid-attach.c | 12 +++-
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/libdwfl/libdwflP.h b/libdwfl/libdwflP.h
index 6ec5c966..2b7eb8da 100644
--- a/libdwfl/libdwflP.h
+++ b/libdwfl/libdwflP.h
@@ -1,5 +1,5 @@
/* Internal definitions for libdwfl.
- Copyright (C) 2005-2015, 2018, 2024 Red Hat, Inc.
+ Copyright (C) 2005-2015, 2018, 2024-2025 Red Hat, Inc.
This file is part of elfutils.
This file is free software; you can redistribute it and/or modify
@@ -591,6 +591,15 @@ extern Dwfl_Module *__libdwfl_report_offline (Dwfl *dwfl,
const char *name,
extern void __libdwfl_process_free (Dwfl_Process *process)
internal_function;
+/* Basic implementation of Dwfl_Thread_Callbacks.set_initial_registers.
+ ARG must be a Dwfl_Thread *. Calls dwfl_thread_state_register_pc
+ if firstreg is -1 (indicating arch PC), dwfl_thread_state_registers
+ otherwise. */
+extern bool __libdwfl_set_initial_registers_thread (int firstreg,
+unsigned nregs,
+const Dwarf_Word *regs,
+void *arg);
+
/* Update STATE->unwound for the unwound frame.
On error STATE->unwound == NULL
or STATE->unwound->pc_state == DWFL_FRAME_STATE_ERROR;
diff --git a/libdwfl/linux-pid-attach.c b/libdwfl/linux-pid-attach.c
index 0eec1e88..a6e4e41a 100644
--- a/libdwfl/linux-pid-attach.c
+++ b/libdwfl/linux-pid-attach.c
@@ -1,5 +1,5 @@
/* Get Dwarf Frame state for target live PID process.
- Copyright (C) 2013, 2014, 2015, 2018 Red Hat, Inc.
+ Copyright (C) 2013, 2014, 2015, 2018, 2025 Red Hat, Inc.
This file is part of elfutils.
This file is free software; you can redistribute it and/or modify
@@ -304,9 +304,11 @@ pid_getthread (Dwfl *dwfl __attribute__ ((unused)), pid_t
tid,
/* Implement the ebl_set_initial_registers_tid setfunc callback. */
-static bool
-pid_thread_state_registers_cb (int firstreg, unsigned nregs,
- const Dwarf_Word *regs, void *arg)
+bool
+/* XXX No internal_function annotation,
+ as this function gets passed as ebl_tid_registers_t *. */
+__libdwfl_set_initial_registers_thread (int firstreg, unsigned nregs,
+ const Dwarf_Word *regs, void *arg)
{
Dwfl_Thread *thread = (Dwfl_Thread *) arg;
if (firstreg == -1)
@@ -338,7 +340,7 @@ pid_set_initial_registers (Dwfl_Thread *thread, void
*thread_arg)
Dwfl_Process *process = thread->process;
Ebl *ebl = process->ebl;
return ebl_set_initial_registers_tid (ebl, tid,
- pid_thread_state_registers_cb, thread);
+ __libdwfl_set_initial_registers_thread,
thread);
}
static void
--
2.47.0
Re: scraperbot protection - Patchwork and Bunsen behind Anubis
Hi - > We were wondering if it would be possible / suitable to have https > requests served by one container, > and ssh ones by another? Maybe that's already the case though... SSH stuff is already (un)contained. > [...] > so maybe it would help if we switched to ssh access for our CI user > when cloning GCC / binuitls / etc sources? What you'd need is a ssh identity with logon access for the bots. That's not something we normally do (sourceware accounts are personal and usually have read-write access to -some- git repos.) But we could support read-only ssh accounts for bots. (read-only for safety, i.e., not member of any particular git unix-group/project.) - FChE
[PATCH v4 06/12] libdwfl_stacktrace [6/12]: Elf* caching via dwflst_process_tracker
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
- dwfl_module_getdwarf.c now uses the dwflst_tracker_cache_elf()
interface instead of editing the elftab directly.
Changes for v3:
- Reworked elftab to incorporate dev/ino into the caching key
(to allow caching modules from different filesystems
e.g. a main filesystem and a container filesystem)
and guard more carefully against collisions.
- Add external API for implementing a custom
find_elf callback that reads/writes the cache.
Changes for v2:
- Add locking for elftab. This is needed in addition to the
intrinsic locking in dynamicsizehash_concurrent to avoid
having cache_elf expose an incomplete dwfltracker_elf_info*
entry to other threads while its data is being populated /
replaced.
- Tidy dwfl_process_tracker_find_elf.c into the main find_elf
callback and two functions to consider (in future) making into
a public api for custom cached callbacks.
* * *
The Dwflst_Process_Tracker includes a dynamicsizehash cache which maps
file paths to Elf * (or rather, dwflst_tracker_elf_info * storing fd
and Elf *). We provide a dwflst_tracker_linux_proc_find_elf callback
which checks the cache for an already-loaded Elf * and, if missing,
populates the cache with the fd returned by dwfl_linux_proc_find_elf.
Later, open_elf updates the cache with the Elf * for that fd. The
commented asserts in dwflst_tracker_cache_elf still catch some cases
where a redundant Elf * is being created without checking the cache.
Since the Elf * outlasts the Dwfl that created it, we use the
(convenient, already-existing) reference count field in Elf * to
retain the data in the table. Then dwfl_end calling elf_end will
decrement the refcount cleanly, and dwflst_tracker_end will issue
another elf_end call.
* libdwfl_stacktrace/libdwfl_stacktrace.h
(dwflst_tracker_find_cached_elf): New function.
(dwflst_tracker_cache_elf): New function.
(dwflst_module_gettracker): New function, gives external users
a way to access Dwflst_Process_Tracker given a Dwfl_Module.
(dwflst_tracker_linux_proc_find_elf): New function, serves as a
cached version of the dwfl_linux_proc_find_elf callback.
* libdwfl_stacktrace/libdwfl_stacktraceP.h (dwflst_tracker_elf_info):
New struct typedef.
(struct Dwflst_Process_Tracker): Add dynamicsizehash table of
dwflst_tracker_elf_info structs + associated rwlock.
(INTDECLs): Add INTDECL for dwflst_tracker_find_cached_elf,
dwflst_tracker_cache_elf, dwflst_module_gettracker.
* libdwfl_stacktrace/dwflst_tracker_elftab.c: New file, instantiates
lib/dynamicsizehash_concurrent.c to store dwflst_tracker_elf_info
structs.
* libdwfl_stacktrace/dwflst_tracker_elftab.h: New file, ditto.
* libdwfl_stacktrace/libdwfl_stacktrace_next_prime.c: New file.
* libdwfl_stacktrace/dwflst_process_tracker.c (dwflst_tracker_begin):
Init elftab.
(dwflst_tracker_end): Clean up elftab. Lock and iterate the hash to
free tracker->elftab.table items.
* libdwfl_stacktrace/dwflst_tracker_find_elf.c: New file, implements a
find_elf callback that wraps dwfl_linux_proc_find_elf with
additional caching logic, and an API to access the
Dwflst_Process_Tracker Elf cache when implementing a custom find_elf
callback.
* libdwfl_stacktrace/Makefile.am (libdwfl_stacktrace_a_SOURCES): Add
dwflst_tracker_find_elf.c, dwflst_tracker_elftab.c,
libdwfl_stacktrace_next_prime.c.
(noinst_HEADERS): Add dwflst_tracker_elftab.h.
* libdw/libdw.map: Add dwflst_tracker_find_cached_elf,
dwflst_tracker_cache_elf,
dwflst_module_gettracker,
dwflst_tracker_linux_proc_find_elf.
* libdwfl/Makefile.am (AM_CPPFLAGS): Include headers from
../libdwfl_stacktrace.
* libdwfl/dwfl_module_getdwarf.c (open_elf): Cache file->elf in
Dwflst_Process_Tracker. Must be done here as
dwfl_linux_proc_find_elf opens an fd but does not yet create the
Elf *.
---
libdw/libdw.map | 4 +
libdwfl/Makefile.am | 5 +-
libdwfl/dwfl_module_getdwarf.c| 10 +-
libdwfl_stacktrace/Makefile.am| 5 +-
libdwfl_stacktrace/dwflst_process_tracker.c | 26 +++
libdwfl_stacktrace/dwflst_tracker_elftab.c| 45
libdwfl_stacktrace/dwflst_tracker_elftab.h| 40
libdwfl_stacktrace/dwflst_tracker_find_elf.c | 216 ++
libdwfl_stacktrace/libdwfl_stacktrace.h | 37 ++-
libdwfl_stacktrace/libdwfl_stacktraceP.h | 24 +-
.../libdwfl_stacktrace_next_prime.c | 6 +
11 files changed, 412 insertions(+), 6 deletions(-)
create mode 100644 libdwfl_stacktrace/dwflst_tracker_elftab.c
create mode 100644 libdwfl_stacktrace/dwflst_tracker_elftab.h
create mode 100644 libdwfl_stacktrace/dwflst_tracker_find_elf.c
create mode 100644 libdwfl_stacktrace/libdwfl_stacktrace_next_prime.c
diff --git a/libdw/libdw.map b/libdw/libdw.map
index 6cc6b7b9..dcb5e2e2 100644
--- a/libdw/libdw.map
+++ b/libdw/libdw.map
@@ -394,4 +394,8 @@ ELFUTILS_0.193 {
[PATCH v4 08/12] libdwfl_stacktrace [8/12]: Dwfl* caching via Dwflst_Process_Tracker
Changes for v4: - Separate out libdwfl_stacktrace, as requested. Changes for v3: - Handle dwfl->process == NULL case in __libdwfl_remove_dwfl_from_tracker. * * * The Dwflst_Process_Tracker also includes a dynamicsizehash cache which maps process ids to Dwfl * (or rather, dwflst_tracker_dwfl_info * allowing the table entry to be replaced). Dwfls created from the tracker are automatically added to it, and removed on dwfl_end(). * libdwfl_stacktrace/libdwfl_stacktraceP.h (dwflst_tracker_dwfl_info): New typedef, provides indirection to allow a dwfltab entry to be invalidated. (struct Dwflst_Process_Tracker): add dwfltab. (__libdwfl_stacktrace_add_dwfl_to_tracker): New function. (__libdwfl_stacktrace_remove_dwfl_from_tracker): New function. * libdwfl_stacktrace/dwflst_process_tracker.c (dwflst_tracker_begin): Init dwfltab. (__libdwfl_stacktrace_add_dwfl_to_tracker): New function; add dwfl to dwfltab. (__libdwfl_stacktrace_remove_dwfl_from_tracker): New function; invalidate dwfl entry, since dynamicsizehash doesn't support outright deletion. (dwflst_tracker_end): Clean up dwfltab. Lock and iterate the table to free tracker->dwfltab.table items. * libdwfl_stacktrace/dwflst_tracker_dwfltab.c: New file, instantiates lib/dynamicsizehash_concurrent.c to store dwfltracker_dwfl_info structs. * libdwfl_stacktrace/dwflst_tracker_dwfltab.h: New file, ditto. * libdwfl_stacktrace/Makefile.am (libdwfl_stacktrace_a_SOURCES): Add dwflst_tracker_dwfltab.c. (noinst_HEADERS): Add dwflst_tracker_dwfltab.h. * libdwfl/dwfl_frame.c (dwfl_attach_state): Call __libdwfl_stacktrace_add_dwfl_to_tracker. * libdwfl/dwfl_end.c (dwfl_end): Add INTDEF. Call __libdwfl_stacktrace_remove_dwfl_from_tracker. * libdwfl/libdwflP.h (INTDECLs): Add dwfl_end. --- libdwfl/dwfl_end.c | 8 +- libdwfl/dwfl_frame.c| 7 +- libdwfl/libdwflP.h | 1 + libdwfl_stacktrace/Makefile.am | 4 +- libdwfl_stacktrace/dwflst_process_tracker.c | 87 - libdwfl_stacktrace/dwflst_tracker_dwfltab.c | 46 +++ libdwfl_stacktrace/dwflst_tracker_dwfltab.h | 42 ++ libdwfl_stacktrace/libdwfl_stacktraceP.h| 23 ++ 8 files changed, 213 insertions(+), 5 deletions(-) create mode 100644 libdwfl_stacktrace/dwflst_tracker_dwfltab.c create mode 100644 libdwfl_stacktrace/dwflst_tracker_dwfltab.h diff --git a/libdwfl/dwfl_end.c b/libdwfl/dwfl_end.c index 7b5ac8a1..d9cf569b 100644 --- a/libdwfl/dwfl_end.c +++ b/libdwfl/dwfl_end.c @@ -1,5 +1,5 @@ /* Finish a session using libdwfl. - Copyright (C) 2005, 2008, 2012-2013, 2015 Red Hat, Inc. + Copyright (C) 2005, 2008, 2012-2013, 2015, 2025 Red Hat, Inc. This file is part of elfutils. This file is free software; you can redistribute it and/or modify @@ -31,6 +31,7 @@ #endif #include "libdwflP.h" +#include "libdwfl_stacktraceP.h" void dwfl_end (Dwfl *dwfl) @@ -42,6 +43,9 @@ dwfl_end (Dwfl *dwfl) __libdwfl_debuginfod_end (dwfl->debuginfod); #endif + if (dwfl->tracker != NULL) +__libdwfl_stacktrace_remove_dwfl_from_tracker (dwfl); + if (dwfl->process) __libdwfl_process_free (dwfl->process); @@ -68,3 +72,5 @@ dwfl_end (Dwfl *dwfl) } free (dwfl); } +INTDEF(dwfl_end) + diff --git a/libdwfl/dwfl_frame.c b/libdwfl/dwfl_frame.c index 2e6c6de8..0d619887 100644 --- a/libdwfl/dwfl_frame.c +++ b/libdwfl/dwfl_frame.c @@ -1,5 +1,5 @@ /* Get Dwarf Frame state for target PID or core file. - Copyright (C) 2013, 2014, 2024 Red Hat, Inc. + Copyright (C) 2013, 2014, 2024-2025 Red Hat, Inc. This file is part of elfutils. This file is free software; you can redistribute it and/or modify @@ -33,6 +33,7 @@ #include #include "libdwflP.h" +#include "libdwfl_stacktraceP.h" /* Set STATE->pc_set from STATE->regs according to the backend. Return true on success, false on error. */ @@ -206,6 +207,10 @@ dwfl_attach_state (Dwfl *dwfl, Elf *elf, pid_t pid, process->pid = pid; process->callbacks = thread_callbacks; process->callbacks_arg = arg; + + if (dwfl->tracker != NULL) +__libdwfl_stacktrace_add_dwfl_to_tracker (dwfl); + return true; } INTDEF(dwfl_attach_state) diff --git a/libdwfl/libdwflP.h b/libdwfl/libdwflP.h index 57305f81..b622f8aa 100644 --- a/libdwfl/libdwflP.h +++ b/libdwfl/libdwflP.h @@ -757,6 +757,7 @@ extern int dwfl_link_map_report (Dwfl *dwfl, const void *auxv, size_t auxv_size, /* Avoid PLT entries. */ INTDECL (dwfl_begin) +INTDECL (dwfl_end) INTDECL (dwfl_errmsg) INTDECL (dwfl_errno) INTDECL (dwfl_addrmodule) diff --git a/libdwfl_stacktrace/Makefile.am b/libdwfl_stacktrace/Makefile.am index ffddec0c..99a80b5c 100644 --- a/libdwfl_stacktrace/Makefile.am +++ b/libdwfl_stacktrace/Makefile.am @@ -43,6 +43,7 @@ pkginclude_HEADERS = libdwfl_stacktrace.h libdwfl_stacktrace_a_SOURCES = dwflst_process_tracker.c \ dwflst_tracker_find_elf.c \
[PATCH v4 03/12] libebl [3/12]: eu-stacktrace: use new register handling api
Changes for v4:
- Since __libdwfl_set_initial_registers_thread is now private to
libdwfl, the modified code in this patch has been disabled.
* * *
Dummy commit to show how the sample_set_initial_registers callback in
eu-stacktrace would use the proper libebl
ebl_set_initial_registers_sample function (if it were public).
* src/Makefile.am (stacktrace_LDADD): Add libebl.
* src/stacktrace.c (sample_registers_cb): New function,
though identical to pid_thread_state_registers_cb.
(sample_set_initial_registers): (XXX Invoke
ebl_set_initial_registers_sample instead of containing
platform-specific code directly. This is now commented out.
Patch12 in the series replaces with code in
libdwfl_stacktrace/dwflst_perf_frame.c.)
---
src/Makefile.am | 4 ++--
src/stacktrace.c | 34 +-
2 files changed, 31 insertions(+), 7 deletions(-)
diff --git a/src/Makefile.am b/src/Makefile.am
index ed245fc1..6d713e88 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -1,6 +1,6 @@
## Process this file with automake to create Makefile.in
##
-## Copyright (C) 1996-2014, 2016, 2024 Red Hat, Inc.
+## Copyright (C) 1996-2014, 2016, 2024-2025 Red Hat, Inc.
## This file is part of elfutils.
##
## This file is free software; you can redistribute it and/or modify
@@ -105,7 +105,7 @@ ar_LDADD = libar.a $(libelf) $(libeu) $(argp_LDADD)
$(obstack_LIBS)
unstrip_LDADD = $(libebl) $(libelf) $(libdw) $(libeu) $(argp_LDADD)
stack_LDADD = $(libebl) $(libelf) $(libdw) $(libeu) $(argp_LDADD)
$(demanglelib)
if ENABLE_STACKTRACE
-stacktrace_LDADD = $(libelf) $(libdw) $(libeu) $(argp_LDADD)
+stacktrace_LDADD = $(libebl) $(libelf) $(libdw) $(libeu) $(argp_LDADD)
endif
elfcompress_LDADD = $(libebl) $(libelf) $(libdw) $(libeu) $(argp_LDADD)
elfclassify_LDADD = $(libelf) $(libdw) $(libeu) $(argp_LDADD)
diff --git a/src/stacktrace.c b/src/stacktrace.c
index d8699ce5..3f5950fb 100644
--- a/src/stacktrace.c
+++ b/src/stacktrace.c
@@ -1,5 +1,5 @@
/* Process a stream of stack samples into stack traces.
- Copyright (C) 2023-2024 Red Hat, Inc.
+ Copyright (C) 2023-2025 Red Hat, Inc.
This file is part of elfutils.
This file is free software; you can redistribute it and/or modify
@@ -93,9 +93,11 @@
* Includes: libdwfl data structures *
*/
+#include ELFUTILS_HEADER(ebl)
/* #include ELFUTILS_HEADER(dwfl) */
#include "../libdwfl/libdwflP.h"
-/* XXX: Private header needed for find_procfile, sysprof_init_dwfl */
+/* XXX: Private header needed for find_procfile, sysprof_init_dwfl,
+ sample_set_initial_registers. */
/*
* Includes: sysprof data structures *
@@ -574,10 +576,31 @@ sample_memory_read (Dwfl *dwfl, Dwarf_Addr addr,
Dwarf_Word *result, void *arg)
return true;
}
-/* TODO: Need to generalize this code beyond x86 architectures. */
static bool
-sample_set_initial_registers (Dwfl_Thread *thread, void *thread_arg)
+sample_set_initial_registers (Dwfl_Thread *thread, void *arg)
{
+#if 0
+ /* TODO: __libdwfl_set_initial_registers_thread not exported from libdwfl,
+ after it was decided to be unsuitable for a public API.
+
+ A subsequent patch in the series removes sample_set_initial_registers,
+ replacing it with code in libdwfl_stacktrace/dwflst_perf_frame.c.
+ Keeping this code commented-out for the record, cf how we would
+ implement if the set_initial_registers utility func was public.
+
+ To *actually* make this work, would need to copy the set_initial_registers
+ implementation into stacktrace.c; not worth doing since the later patch
+ overrides this code. */
+ struct __sample_arg *sample_arg = (struct __sample_arg *)arg;
+ dwfl_thread_state_register_pc (thread, sample_arg->pc);
+ Dwfl_Process *process = thread->process;
+ Ebl *ebl = process->ebl;
+ /* XXX Sysprof provides exactly the required registers for unwinding: */
+ uint64_t regs_mask = ebl_perf_frame_regs_mask (ebl);
+ return ebl_set_initial_registers_sample
+(ebl, sample_arg->regs, sample_arg->n_regs, regs_mask, sample_arg->abi,
+ __libdwfl_set_initial_registers_thread, thread);
+#else
/* The following facts are needed to translate x86 registers correctly:
- perf register order seen in linux arch/x86/include/uapi/asm/perf_regs.h
The registers array is built in the same order as the enum!
@@ -592,7 +615,7 @@ sample_set_initial_registers (Dwfl_Thread *thread, void
*thread_arg)
For comparison, you can study
codereview.qt-project.org/gitweb?p=qt-creator/perfparser.git;a=blob;f=app/perfregisterinfo.cpp;hb=HEAD
and follow the code which uses those tables of magic numbers.
But it's better to follow original sources of truth for this. */
- struct __sample_arg *sample_arg = (struct __sample_arg *)thread_arg;
+ struct __sample_arg *sample_arg = (struct __sample_arg *)arg;
bool is_abi32 = (sample_arg->abi == PERF_SAMPLE_REGS_ABI_32);
static const int re
[PATCH v4 05/12] libdwfl_stacktrace [5/12]: introduce Dwflst_Process_Tracker
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
* * *
New data structure to coordinate caching Elf data among multiple Dwfl
structs attached to different processes. Meant to reduce the overhead
for profilers that use elfutils for unwinding.
The caching is well-justified, as the prior approach (e.g. in
eu-stacktrace, sysprof-live-unwinder) of creating a separate Dwfl per
process was wildly redundant, opening ~hundreds of file descriptors
just for each common library such as libc.so and leaking the data.
Initial patch just introduces the struct, to be filled in by the rest
of the patch series.
* libdwfl_stacktrace/libdwfl_stacktrace.h
(Dwflst_Process_Tracker): New struct.
(dwflst_tracker_begin): New function.
(dwflst_tracker_dwfl_begin): New function.
(dwflst_tracker_end): New function.
* libdw/libdw.map: Add new functions.
* libdwfl_stacktrace/libdwfl_stacktraceP.h
(struct Dwflst_Process_Tracker): New struct.
* libdwfl/libdwflP.h (Dwflst_Process_Tracker): Typedef forward decl.
(struct Dwfl): Add 'tracker' field.
* libdwfl_stacktrace/Makefile.am (libdwfl_stacktrace_a_SOURCES):
Add dwflst_process_tracker.c.
* libdwfl_stacktrace/dwflst_process_tracker.c: New file.
(dwflst_tracker_begin): Initialize the tracker.
(dwflst_tracker_dwfl_begin): Initialize Dwfl * with specified tracker.
(dwflst_tracker_end): Deallocate the tracker.
---
libdw/libdw.map | 3 +
libdwfl/libdwflP.h | 4 ++
libdwfl_stacktrace/Makefile.am | 3 +-
libdwfl_stacktrace/dwflst_process_tracker.c | 66 +
libdwfl_stacktrace/libdwfl_stacktrace.h | 20 +++
libdwfl_stacktrace/libdwfl_stacktraceP.h| 6 ++
6 files changed, 101 insertions(+), 1 deletion(-)
create mode 100644 libdwfl_stacktrace/dwflst_process_tracker.c
diff --git a/libdw/libdw.map b/libdw/libdw.map
index 06dafedb..6cc6b7b9 100644
--- a/libdw/libdw.map
+++ b/libdw/libdw.map
@@ -391,4 +391,7 @@ ELFUTILS_0.193 {
dwarf_language;
dwarf_language_lower_bound;
dwflst_perf_sample_preferred_regs_mask;
+dwflst_tracker_begin;
+dwflst_tracker_dwfl_begin;
+dwflst_tracker_end;
} ELFUTILS_0.192;
diff --git a/libdwfl/libdwflP.h b/libdwfl/libdwflP.h
index 2b7eb8da..57305f81 100644
--- a/libdwfl/libdwflP.h
+++ b/libdwfl/libdwflP.h
@@ -111,9 +111,13 @@ struct Dwfl_User_Core
int fd; /* close if >= 0. */
};
+/* forward decl from ../libdwfl_stacktrace/ */
+typedef struct Dwflst_Process_Tracker Dwflst_Process_Tracker;
+
struct Dwfl
{
const Dwfl_Callbacks *callbacks;
+ Dwflst_Process_Tracker *tracker;
#ifdef ENABLE_LIBDEBUGINFOD
debuginfod_client *debuginfod;
#endif
diff --git a/libdwfl_stacktrace/Makefile.am b/libdwfl_stacktrace/Makefile.am
index 6364c292..d57431c0 100644
--- a/libdwfl_stacktrace/Makefile.am
+++ b/libdwfl_stacktrace/Makefile.am
@@ -40,7 +40,8 @@ noinst_LIBRARIES += libdwfl_stacktrace_pic.a
pkginclude_HEADERS = libdwfl_stacktrace.h
-libdwfl_stacktrace_a_SOURCES = dwflst_perf_frame.c
+libdwfl_stacktrace_a_SOURCES = dwflst_process_tracker.c \
+ dwflst_perf_frame.c
libdwfl_stacktrace = $(libdw)
libdw = ../libdw/libdw.so
diff --git a/libdwfl_stacktrace/dwflst_process_tracker.c
b/libdwfl_stacktrace/dwflst_process_tracker.c
new file mode 100644
index ..057c9f7a
--- /dev/null
+++ b/libdwfl_stacktrace/dwflst_process_tracker.c
@@ -0,0 +1,66 @@
+/* Track multiple Dwfl structs for multiple processes.
+ Copyright (C) 2025, Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of either
+
+ * the GNU Lesser General Public License as published by the Free
+ Software Foundation; either version 3 of the License, or (at
+ your option) any later version
+
+ or
+
+ * the GNU General Public License as published by the Free
+ Software Foundation; either version 2 of the License, or (at
+ your option) any later version
+
+ or both in parallel, as here.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received copies of the GNU General Public License and
+ the GNU Lesser General Public License along with this program. If
+ not, see
[PATCH v4 07/12] eu-stacktrace [7/12]: use Dwflst_Process_Tracker for Elf * caching
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
* * *
* src/Makefile.am (AM_CPPFLAGS): Include headers from
../libdwfl_stacktrace.
* src/stacktrace.c (tracker): New global variable.
(sample_callbacks): Use dwflst_tracker_linux_proc_find_elf for
caching.
(sysprof_init_dwfl): Use dwflst_tracker_dwfl_begin.
(main): Initialize and clean up tracker.
---
src/Makefile.am | 3 ++-
src/stacktrace.c | 12 +---
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/src/Makefile.am b/src/Makefile.am
index 6d713e88..4a3fb957 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -23,7 +23,8 @@ DEFS += $(YYDEBUG) -DDEBUGPRED=@DEBUGPRED@ \
DEFAULT_INCLUDES =
AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
-I$(srcdir)/../libdw -I$(srcdir)/../libdwelf \
- -I$(srcdir)/../libdwfl -I$(srcdir)/../libasm -I../debuginfod
+ -I$(srcdir)/../libdwfl -I$(srcdir)/../libdwfl_stacktrace \
+ -I$(srcdir)/../libasm -I../debuginfod
AM_LDFLAGS = -Wl,-rpath-link,../libelf:../libdw $(STACK_USAGE_NO_ERROR)
diff --git a/src/stacktrace.c b/src/stacktrace.c
index 3f5950fb..c0c9929d 100644
--- a/src/stacktrace.c
+++ b/src/stacktrace.c
@@ -98,6 +98,7 @@
#include "../libdwfl/libdwflP.h"
/* XXX: Private header needed for find_procfile, sysprof_init_dwfl,
sample_set_initial_registers. */
+#include ELFUTILS_HEADER(dwfl_stacktrace)
/*
* Includes: sysprof data structures *
@@ -657,6 +658,8 @@ static const Dwfl_Thread_Callbacks sample_thread_callbacks =
* Dwfl and statistics table for multiple processes *
/
+Dwflst_Process_Tracker *tracker = NULL;
+
/* This echoes lib/dynamicsizehash.* with some necessary modifications. */
typedef struct
{
@@ -889,7 +892,7 @@ static char *debuginfo_path = NULL;
static const Dwfl_Callbacks sample_callbacks =
{
-.find_elf = dwfl_linux_proc_find_elf,
+.find_elf = dwflst_tracker_linux_proc_find_elf,
.find_debuginfo = dwfl_standard_find_debuginfo,
.debuginfo_path = &debuginfo_path,
};
@@ -915,7 +918,7 @@ nop_find_debuginfo (Dwfl_Module *mod
__attribute__((unused)),
static const Dwfl_Callbacks sample_callbacks =
{
- .find_elf = dwfl_linux_proc_find_elf,
+ .find_elf = dwflst_tracker_linux_proc_find_elf,
.find_debuginfo = nop_find_debuginfo, /* work with CFI only */
};
@@ -1038,7 +1041,7 @@ sysprof_init_dwfl (struct sysprof_unwind_info *sui,
cached = true;
goto reuse;
}
- dwfl = dwfl_begin (&sample_callbacks);
+ dwfl = dwflst_tracker_dwfl_begin (tracker);
int err = dwfl_linux_proc_report (dwfl, pid);
if (err < 0)
@@ -1532,6 +1535,7 @@
https://sourceware.org/cgit/elfutils/tree/README.eu-stacktrace?h=users/serhei/eu
(void)maxframes;
#else
fprintf(stderr, "\n=== starting eu-stacktrace ===\n");
+ tracker = dwflst_tracker_begin (&sample_callbacks);
/* TODO: For now, code the processing loop for sysprof only; generalize
later. */
assert (input_format == FORMAT_SYSPROF);
@@ -1615,5 +1619,7 @@
https://sourceware.org/cgit/elfutils/tree/README.eu-stacktrace?h=users/serhei/eu
if (output_fd != -1)
close (output_fd);
+ dwflst_tracker_end (tracker);
+
return EXIT_OK;
}
--
2.47.0
[PATCH v4 09/12] libdwfl_stacktrace [9/12]: add dwflst_tracker_find_pid
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
Changes for v2:
- Add locking for dwfltab.
* * *
New function that retrieves the Dwfl for a particular PID, or,
if the Dwfl is absent, creates it via a provided callback
and adds it to the table later, when the PID is confirmed
via dwfl_attach_state.
* libdwfl_stacktrace/libdwfl_stacktrace.h (dwflst_tracker_find_pid):
New function.
* libdwfl_stacktrace/dwfl_process_tracker.c (dwflst_tracker_find_pid):
New function; find a Dwfl in the dwfltab or create one using the
provided callback. The newly created Dwfl will be added to the
dwfltab automatically when its pid is confirmed by a call to
dwfl_attach_state.
* libdw/libdw.map: Add dwflst_tracker_find_pid.
---
libdw/libdw.map | 1 +
libdwfl_stacktrace/dwflst_process_tracker.c | 26 +
libdwfl_stacktrace/libdwfl_stacktrace.h | 11 +
3 files changed, 38 insertions(+)
diff --git a/libdw/libdw.map b/libdw/libdw.map
index dcb5e2e2..ce5d37fb 100644
--- a/libdw/libdw.map
+++ b/libdw/libdw.map
@@ -398,4 +398,5 @@ ELFUTILS_0.193 {
dwflst_tracker_cache_elf;
dwflst_module_gettracker;
dwflst_tracker_linux_proc_find_elf;
+dwflst_tracker_find_pid;
} ELFUTILS_0.192;
diff --git a/libdwfl_stacktrace/dwflst_process_tracker.c
b/libdwfl_stacktrace/dwflst_process_tracker.c
index f72b72b0..fc019b23 100644
--- a/libdwfl_stacktrace/dwflst_process_tracker.c
+++ b/libdwfl_stacktrace/dwflst_process_tracker.c
@@ -67,6 +67,32 @@ Dwfl *dwflst_tracker_dwfl_begin (Dwflst_Process_Tracker
*tracker)
return dwfl;
}
+Dwfl *dwflst_tracker_find_pid (Dwflst_Process_Tracker *tracker,
+ pid_t pid,
+ Dwfl *(*callback) (Dwflst_Process_Tracker *,
+ pid_t, void *),
+ void *arg)
+{
+ Dwfl *dwfl = NULL;
+
+ rwlock_rdlock (tracker->dwfltab_lock);
+ dwflst_tracker_dwfl_info *ent
+= dwflst_tracker_dwfltab_find(&tracker->dwfltab, pid);
+ rwlock_unlock (tracker->dwfltab_lock);
+
+ if (ent != NULL && !ent->invalid)
+dwfl = ent->dwfl;
+ if (dwfl == NULL && callback != NULL)
+dwfl = callback(tracker, pid, arg);
+ if (dwfl != NULL)
+{
+ assert (dwfl->tracker == tracker);
+ /* XXX: dwfl added to dwfltab when dwfl->process set in
dwfl_attach_state.
+ Prior to that, the pid is not confirmed. */
+}
+
+ return dwfl;
+}
void
internal_function
diff --git a/libdwfl_stacktrace/libdwfl_stacktrace.h
b/libdwfl_stacktrace/libdwfl_stacktrace.h
index cac01deb..cf1b5fcd 100644
--- a/libdwfl_stacktrace/libdwfl_stacktrace.h
+++ b/libdwfl_stacktrace/libdwfl_stacktrace.h
@@ -73,6 +73,17 @@ extern bool dwflst_tracker_cache_elf (Dwflst_Process_Tracker
*tracker,
Elf *elf, int fd)
__nonnull_attribute__ (1, 2);
+/* Find the Dwfl corresponding to PID. If CALLBACK is non-NULL and
+ the Dwfl has not been created, invoke CALLBACK to create the Dwfl
+ and then store it in the tracker. */
+extern Dwfl *dwflst_tracker_find_pid (Dwflst_Process_Tracker *tracker,
+ pid_t pid,
+ Dwfl *(*callback) (Dwflst_Process_Tracker
*tracker,
+pid_t pid,
+void *arg),
+ void *arg)
+ __nonnull_attribute__ (1);
+
/* For implementing a find_elf callback based on the prior two functions.
Returns the Dwflst_Process_Tracker corresponding to MOD. */
extern Dwflst_Process_Tracker *dwflst_module_gettracker (Dwfl_Module *mod);
--
2.47.0
[PATCH v4 11/12] libdwfl_stacktrace [11/12]: add dwflst_perf_sample_getframes
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
Changes for v3:
- use const void *stack, not void *, to allow users
to pass a const stack sample
Changes for v2:
- use renamed __libdwfl_set_initial_registers_thread
- oops, should use provided sample_arg->perf_regs_mask
in sample_set_initial registers
* * *
This is a new interface for unwinding that doesn't require the Dwfl to
be attached to a live process (via ptrace) or via corefile. Instead,
data from a perf_events stack sample is provided along with an Elf
struct used to identify the architecture. Based on code from
eu-stacktrace.
* libdwfl_stacktrace/libdwfl_stacktrace.h (dwflst_perf_sample_getframes):
New function.
* libdwfl_stacktrace/dwflst_perf_frame.c
(struct __libdwfl_stacktrace_perf_sample_info): New struct, based on
src/stacktrace.c struct sample_arg.
(sample_next_thread): New function, based on src/stacktrace.c.
(sample_getthread): Ditto.
(copy_word_64): New macro, based on src/stacktrace.c.
(copy_word_32): Ditto.
(copy_word): Ditto.
(elf_memory_read): New function, based on src/stacktrace.c.
(sample_memory_read): Ditto.
(sample_set_initial_registers): Ditto.
(sample_detach): Ditto.
(sample_thread_callbacks): New struct, set of callbacks based on
src/stacktrace.c sample_thread_callbacks.
(dwflst_perf_sample_getframes): New function, based on parts of
src/stacktrace.c sysprof_find_dwfl. If the Dwfl is not attached,
attaches it with sample_thread_callbacks and
__libdwfl_stacktrace_perf_sample_info. Populates the
__libdwfl_stacktrace_perf_sample_info with data from the stack
sample and calls dwfl_getthread_frames to unwind it using the
sample_thread_callbacks.
* libdw/libdw.map (ELFUTILS_0.193): Add dwflst_perf_sample_getframes.
---
libdw/libdw.map | 1 +
libdwfl_stacktrace/dwflst_perf_frame.c | 193 +++-
libdwfl_stacktrace/libdwfl_stacktrace.h | 15 +-
3 files changed, 207 insertions(+), 2 deletions(-)
diff --git a/libdw/libdw.map b/libdw/libdw.map
index ce5d37fb..7a4cb0d8 100644
--- a/libdw/libdw.map
+++ b/libdw/libdw.map
@@ -399,4 +399,5 @@ ELFUTILS_0.193 {
dwflst_module_gettracker;
dwflst_tracker_linux_proc_find_elf;
dwflst_tracker_find_pid;
+dwflst_perf_sample_getframes;
} ELFUTILS_0.192;
diff --git a/libdwfl_stacktrace/dwflst_perf_frame.c
b/libdwfl_stacktrace/dwflst_perf_frame.c
index 79e8e482..591097e8 100644
--- a/libdwfl_stacktrace/dwflst_perf_frame.c
+++ b/libdwfl_stacktrace/dwflst_perf_frame.c
@@ -60,4 +60,195 @@ uint64_t dwflst_perf_sample_preferred_regs_mask (GElf_Half
machine)
return 0;
}
-/* XXX dwflst_perf_sample_getframes to be added in subsequent patch */
+struct __libdwfl_stacktrace_perf_sample_info {
+ pid_t pid;
+ pid_t tid;
+ Dwarf_Addr base_addr;
+ const uint8_t *stack;
+ size_t stack_size;
+ const Dwarf_Word *regs;
+ uint n_regs;
+ uint64_t perf_regs_mask;
+ uint abi;
+ Dwarf_Addr pc;
+};
+
+/* The next few functions imitate the corefile interface for a single
+ stack sample, with very restricted access to registers and memory. */
+
+/* Just yield the single thread id matching the sample. */
+static pid_t
+sample_next_thread (Dwfl *dwfl __attribute__ ((unused)), void *dwfl_arg,
+ void **thread_argp)
+{
+ struct __libdwfl_stacktrace_perf_sample_info *sample_arg =
+(struct __libdwfl_stacktrace_perf_sample_info *)dwfl_arg;
+ if (*thread_argp == NULL)
+{
+ *thread_argp = (void *)0xea7b3375;
+ return sample_arg->tid;
+}
+ else
+return 0;
+}
+
+/* Just check that the thread id matches the sample. */
+static bool
+sample_getthread (Dwfl *dwfl __attribute__ ((unused)), pid_t tid,
+ void *dwfl_arg, void **thread_argp)
+{
+ struct __libdwfl_stacktrace_perf_sample_info *sample_arg =
+(struct __libdwfl_stacktrace_perf_sample_info *)dwfl_arg;
+ *thread_argp = (void *)sample_arg;
+ if (sample_arg->tid != tid)
+{
+ __libdwfl_seterrno(DWFL_E_INVALID_ARGUMENT);
+ return false;
+}
+ return true;
+}
+
+#define copy_word_64(result, d) \
+ if uintptr_t) (d)) & (sizeof (uint64_t) - 1)) == 0) \
+*(result) = *(uint64_t *)(d); \
+ else \
+memcpy ((result), (d), sizeof (uint64_t));
+
+#define copy_word_32(result, d) \
+ if uintptr_t) (d)) & (sizeof (uint32_t) - 1)) == 0) \
+*(result) = *(uint32_t *)(d); \
+ else \
+memcpy ((result), (d), sizeof (uint32_t));
+
+#define copy_word(result, d, abi) \
+ if ((abi) == PERF_SAMPLE_REGS_ABI_64)\
+{ copy_word_64((result), (d)); } \
+ else if ((abi) == PERF_SAMPLE_REGS_ABI_32) \
+{ copy_word_32((result), (d)); } \
+ else \
+*(result) = 0;
+
+static bool
+elf_memory_read (Dwfl *dwfl, Dwarf_Addr addr, Dwarf_Word *result, void *arg)
+{
+ struct __libdwfl_stacktrace_perf_sample_info *sample_arg =
+(struct __libdwfl_stacktrace_perf_sample_info *)arg;
+ Dwfl_Module *mod = INTUSE(dwfl_addrmodule) (dwfl, addr);
+ Dw
[PATCH v4 12/12] eu-stacktrace [12/12]: use dwflst_perf_sample_getframes
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
Changes for v3:
- Fix initialization of elf in sysprof_init_dwfl
(previously triggered -Wmaybe-uninitialized).
* * *
Remove the code from src/stacktrace.c that is now covered by
libdwfl_stacktrace/dwflst_perf_frame.c and
dwflst_perf_sample_getframes.
* src/stacktrace.c (show_memory_reads): Remove this verbose option as
the relevant code is inside libdwfl_stacktrace now.
(struct __sample_arg): Remove, handled by
libdwfl_stacktrace/dwflst_perf_frame.c.
(sample_next_thread): Ditto.
(sample_getthread): Ditto.
(copy_word_64): Ditto.
(copy_word_32): Ditto.
(elf_memory_read): Ditto.
(sample_memory_read): Ditto.
(sample_set_initial_registers): Ditto.
(sample_detach): Ditto.
(sample_thread_callbacks): Ditto.
(sysprof_find_dwfl): Now also return the Elf* so that it can be
passed to dwflst_perf_sample_getframes. Don't create sample_arg. Do
record sp in sui->last_sp. Don't dwfl_attach_state,
dwflst_perf_sample_getframes handles that now.
(sysprof_unwind_cb): Adapt to sysprof_find_dwfl changes,
now invoke dwflst_perf_sample_getframes instead of
dwfl_getthread_frames.
(parse_opt): Remove show_memory_reads.
(main): Remove show_memory_reads.
---
src/stacktrace.c | 276 ++-
1 file changed, 30 insertions(+), 246 deletions(-)
diff --git a/src/stacktrace.c b/src/stacktrace.c
index dd58ef3b..43b091b7 100644
--- a/src/stacktrace.c
+++ b/src/stacktrace.c
@@ -96,8 +96,7 @@
#include ELFUTILS_HEADER(ebl)
/* #include ELFUTILS_HEADER(dwfl) */
#include "../libdwfl/libdwflP.h"
-/* XXX: Private header needed for find_procfile, sysprof_find_dwfl,
- sample_set_initial_registers. */
+/* XXX: Private header needed for find_procfile. */
#include ELFUTILS_HEADER(dwfl_stacktrace)
/*
@@ -177,11 +176,13 @@ static int processing_mode = MODE_NAIVE;
static int input_format;
static int output_format = FORMAT_SYSPROF;
+/* XXX Used to decide regs_mask for dwflst_perf_sample_getframes. */
+Ebl *default_ebl = NULL;
+
/* non-printable argp options. */
#define OPT_DEBUG 0x100
/* Diagnostic options. */
-static bool show_memory_reads = false;
static bool show_frames = false;
static bool show_samples = false;
static bool show_failures = false;
@@ -191,9 +192,6 @@ static bool show_summary = true;
#define ELFUTILS_STACKTRACE_VERBOSE_ENV_VAR "ELFUTILS_STACKTRACE_VERBOSE"
/* Valid values that turn on diagnostics: 'true', 'verbose', 'debug', '1',
'2'. */
-/* Enables detailed tracing of simulated memory reads: */
-/* #define DEBUG_MEMORY_READ */
-
/* Enables even more diagnostics on modules: */
/* #define DEBUG_MODULES */
@@ -470,190 +468,6 @@ sysprof_reader_getframes (SysprofReader *reader,
#endif /* HAVE_SYSPROF_HEADERS */
-/***
- * Memory read interface for stack samples *
- ***/
-
-/* TODO: elfutils (internal) libraries use libNN_set_errno and DWFL_E_WHATEVER;
- this code fails silently in sample_getthread. */
-
-struct __sample_arg
-{
- int tid;
- Dwarf_Addr base_addr;
- uint64_t size;
- uint8_t *data;
- uint64_t n_regs;
- uint64_t abi; /* PERF_SAMPLE_REGS_ABI_{32,64} */
- Dwarf_Addr pc;
- Dwarf_Addr sp;
- Dwarf_Addr *regs;
-};
-
-/* The next few functions imitate the corefile interface for a single
- stack sample, with very restricted access to registers and memory. */
-
-/* Just yield the single thread id matching the sample. */
-static pid_t
-sample_next_thread (Dwfl *dwfl __attribute__ ((unused)), void *dwfl_arg,
- void **thread_argp)
-{
- struct __sample_arg *sample_arg = (struct __sample_arg *)dwfl_arg;
- if (*thread_argp == NULL)
-{
- *thread_argp = (void *)0xea7b3375;
- return sample_arg->tid;
-}
- else
-return 0;
-}
-
-/* Just check that the thread id matches the sample. */
-static bool
-sample_getthread (Dwfl *dwfl __attribute__ ((unused)), pid_t tid,
- void *dwfl_arg, void **thread_argp)
-{
- struct __sample_arg *sample_arg = (struct __sample_arg *)dwfl_arg;
- *thread_argp = (void *)sample_arg;
- if (sample_arg->tid != tid)
-{
- return false;
-}
- return true;
-}
-
-#define copy_word_64(result, d) \
- if uintptr_t) (d)) & (sizeof (uint64_t) - 1)) == 0) \
-*(result) = *(uint64_t *)(d); \
- else \
-memcpy ((result), (d), sizeof (uint64_t));
-
-#define copy_word_32(result, d) \
- if uintptr_t) (d)) & (sizeof (uint32_t) - 1)) == 0) \
-*(result) = *(uint32_t *)(d); \
- else \
-memcpy ((result), (d), sizeof (uint32_t));
-
-#define copy_word(result, d, abi) \
- if ((abi) == PERF_SAMPLE_REGS_ABI_64)\
-{ copy_word_64((result), (d)); } \
- else if ((abi) == PERF_SAMPLE_REGS_ABI_32) \
-{ copy_word_32((result), (d)); } \
- else \
-*(result) = 0;
-
-static bool
-elf_memory_read (Dwfl *dwfl, Dwarf_Addr addr, Dwa
[PATCH v4 01/12] libebl [1/12]: api for perf register handling, start with x86_64
Changes for v2: - Merged commit 11 into commit 1 (both sample_base_addr/sample_pc and set_initial_registers_sample make sense to introduce in the same commit). - Added i386 backend and factored out common code. * * * First patch of a series that reworks eu-stacktrace functionality into a library interface for other profiling tools. * * * As it happens, Linux perf_events and DWARF can prescribe completely different layouts for the register file, requiring non-obvious code for translation. This makes sense to put in libebl if we want profilers to handle perf sample data with elfutils. Start with the x86_64/i386 implementation taken from eu-stacktrace. The code has been generalized to accept other perf register masks besides the 'preferred' one. * backends/Makefile.am (i386_SRCS): Add i386_initreg_sample.c. (x86_64_SRCS): Add x86_64_initreg_sample.c. (noinst_HEADERS): Add libebl_PERF_FLAGS.h, linux-perf-regs.c, x86_initreg_sample.c. * backends/libebl_PERF_FLAGS.h: New file. * backends/linux-perf-regs.c: New file. (perf_sample_find_reg): Index into a perf register array whose contents are described by regs_mask. The target index is the index of the register in the enum defined by linux arch/N/include/uapi/asm/perf_regs.h on arch N. * backends/x86_initreg_sample.c: New file, implements a generalized version of the eu-stacktrace register shuffling for x86-64/i386. * backends/x86_64_initreg_sample.c: New file, specializes x86_initreg_sample.c for x86-64. * backends/i386_initreg_sample.c: New file, specializes i386_initreg_sample.c for i386. * backends/x86_64_init.c (x86_64_init): Add initialization for set_initial_registers_sample, perf_frame_regs_mask, sample_base_addr, sample_pc. * backends/i386_init.c (i386_init): Add initialization for set_initial_registers_sample, perf_frame_regs_mask, sample_base_addr, sample_pc. * libebl/Makefile.am (libebl_a_SOURCES): Add eblinitreg_sample.c. * libebl/ebl-hooks.h (set_initial_registers_sample): New hook. (sample_base_addr): Ditto. (sample_pc): Ditto. * libebl/eblinitreg_sample.c: New file, implements ebl interface to set_initial_registers_sample, sample_base_addr, sample_pc backend hooks. * libebl/libebl.h (ebl_set_initial_registers_sample): New function. (ebl_perf_frame_regs_mask): New function. (ebl_sample_base_addr): New function. (ebl_sample_pc): New function. * libebl/libeblP.h (struct ebl): Add perf_frame_regs_mask field giving the preferred register mask. --- backends/Makefile.am | 10 +-- backends/i386_init.c | 10 ++- backends/i386_initreg_sample.c | 108 +++ backends/libebl_PERF_FLAGS.h | 58 + backends/linux-perf-regs.c | 48 ++ backends/x86_64_init.c | 7 +- backends/x86_64_initreg_sample.c | 106 ++ backends/x86_initreg_sample.c| 90 ++ libebl/Makefile.am | 4 +- libebl/ebl-hooks.h | 18 +- libebl/eblinitreg_sample.c | 72 + libebl/libebl.h | 31 - libebl/libeblP.h | 7 +- 13 files changed, 557 insertions(+), 12 deletions(-) create mode 100644 backends/i386_initreg_sample.c create mode 100644 backends/libebl_PERF_FLAGS.h create mode 100644 backends/linux-perf-regs.c create mode 100644 backends/x86_64_initreg_sample.c create mode 100644 backends/x86_initreg_sample.c create mode 100644 libebl/eblinitreg_sample.c diff --git a/backends/Makefile.am b/backends/Makefile.am index 540d0c6c..8ccbdb50 100644 --- a/backends/Makefile.am +++ b/backends/Makefile.am @@ -1,6 +1,6 @@ ## Process this file with automake to create Makefile.in ## -## Copyright (C) 2000-2010, 2013, 2014 Red Hat, Inc. +## Copyright (C) 2000-2010, 2013, 2014, 2025 Red Hat, Inc. ## Copyright (C) 2012 Tilera Corporation ## This file is part of elfutils. ## @@ -41,13 +41,13 @@ modules = i386 sh x86_64 ia64 alpha arm aarch64 sparc ppc ppc64 s390 \ i386_SRCS = i386_init.c i386_symbol.c i386_corenote.c i386_cfi.c \ i386_retval.c i386_regs.c i386_auxv.c \ - i386_initreg.c i386_unwind.c + i386_initreg.c i386_initreg_sample.c i386_unwind.c sh_SRCS = sh_init.c sh_symbol.c sh_corenote.c sh_regs.c sh_retval.c x86_64_SRCS = x86_64_init.c x86_64_symbol.c x86_64_corenote.c x86_64_cfi.c \ x86_64_retval.c x86_64_regs.c x86_64_initreg.c \ - x86_64_unwind.c x32_corenote.c + x86_64_initreg_sample.c x86_64_unwind.c x32_corenote.c ia64_SRCS = ia64_init.c ia64_symbol.c ia64_regs.c ia64_retval.c @@ -119,7 +119,9 @@ libebl_backends_a_SOURCES = $(i386_SRCS) $(sh_SRCS) $(x86_64_SRCS) \ libebl_backends_pic_a_SOURCES = am_libebl_backends_pic_a_OBJECTS = $(libebl_backends_a_SOURCES:.c=.os) -noinst_HEADERS = libebl_CPU.h common-reloc.c linux-core-note.c x86_corenote.c +noinst_HEADERS = libebl_CPU.h libebl_PER
[PATCH v4 04/12] libdwfl_stacktrace [4/12]: intro library, add dwflst_perf_sample_preferred_regs_mask
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
Changes for v2:
- guard the linux/perf_events.h include properly with an #if defined __linux__
* * *
Subsequent patches in the series introduce a new library for
tracking/caching Elf structs across multiple processes and wrapping
the libebl perf register handling (since libebl is a private
interface).
In this patch, add the library and an interface to access the set of
registers that libdwfl needs to allow proper unwinding of stack sample
data. (Unwinding with a smaller set of registers can be attempted,
but will yield a truncated call chain in most cases.)
(Accessing the set of registers feels like an implementation detail,
but the profiler invoking perf_event_open and elfutils unwinding code
need to agree on the exact set of registers being requested. So it's
best for the profiler to ask elfutils for this detail.)
* libdwfl_stacktrace/Makefile.am: New file.
* libdwfl_stacktrace/libdwfl_stacktrace.h: New file, library for
tracking/caching Elf structs and unwinding across multiple
processes.
* libdwfl_stacktrace/libdwfl_stacktraceP.h: New file.
* libdwfl_stacktrace/dwflst_perf_frame.c: New file.
(dwflst_perf_sample_preferred_regs_mask): New function.
* libdw/libdw.map: Add dwflst_perf_sample_preferred_regs_mask.
* NEWS: Add NEWS item about libdwfl_stacktrace.
* configure.ac: Add libdwfl_stacktrace/Makefile.
* Makefile.am (SUBDIRS): Add libdwfl_stacktrace.
* libdw/Makefile.am (libdw_so_LIBS): Add libdwfl_stacktrace.
(libdwfl_stacktrace_objects): Add libdwfl_stacktrace.manifest.
(libdw_a_LIBADD): Add libdwfl_stacktrace_objects.
* config/elfutils.spec.in (%files devel): Add libdwfl_stacktrace.h.
---
Makefile.am | 6 +--
NEWS | 6 +++
config/elfutils.spec.in | 1 +
configure.ac | 3 ++
libdw/Makefile.am| 7 ++-
libdw/libdw.map | 1 +
libdwfl_stacktrace/Makefile.am | 63
libdwfl_stacktrace/dwflst_perf_frame.c | 63
libdwfl_stacktrace/libdwfl_stacktrace.h | 52 +++
libdwfl_stacktrace/libdwfl_stacktraceP.h | 36 ++
10 files changed, 233 insertions(+), 5 deletions(-)
create mode 100644 libdwfl_stacktrace/Makefile.am
create mode 100644 libdwfl_stacktrace/dwflst_perf_frame.c
create mode 100644 libdwfl_stacktrace/libdwfl_stacktrace.h
create mode 100644 libdwfl_stacktrace/libdwfl_stacktraceP.h
diff --git a/Makefile.am b/Makefile.am
index 3a181d75..76e98f60 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,7 +1,7 @@
## Process this file with automake to create Makefile.in
## Configure input file for elfutils.
##
-## Copyright (C) 1996-2006, 2008, 2009, 2015 Red Hat, Inc.
+## Copyright (C) 1996-2006, 2008, 2009, 2015, 2025 Red Hat, Inc.
##
## This file is part of elfutils.
##
@@ -28,8 +28,8 @@ AM_MAKEFLAGS = --no-print-directory
pkginclude_HEADERS = version.h
-SUBDIRS = config lib libelf libcpu backends libebl libdwelf libdwfl libdw \
- libasm debuginfod src po doc tests
+SUBDIRS = config lib libelf libcpu backends libebl libdwelf libdwfl \
+ libdwfl_stacktrace libdw libasm debuginfod src po doc tests
EXTRA_DIST = elfutils.spec GPG-KEY NOTES CONTRIBUTING SECURITY \
COPYING COPYING-GPLV2 COPYING-LGPLV3 CONDUCT
diff --git a/NEWS b/NEWS
index 664c125b..c3c611e3 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,12 @@ debuginfod: Add CORS (webapp access) support to webapi.
libdw: Add dwarf_language and dwarf_language_lower_bound functions.
+libdwfl_stacktrace: Experimental new library interface for unwinding
+stack samples into call chains, and tracking and
+caching Elf data for multiple processes, building
+on libdwfl. Initially supports perf_events stack
+sample data.
+
Version 0.192 "New rules, faster tools"
CONDUCT: A new code of conduct has been adopted. See the
diff --git a/config/elfutils.spec.in b/config/elfutils.spec.in
index 96934514..37077365 100644
--- a/config/elfutils.spec.in
+++ b/config/elfutils.spec.in
@@ -300,6 +300,7 @@ fi
#%%{_includedir}/elfutils/libasm.h
%{_includedir}/elfutils/libdw.h
%{_includedir}/elfutils/libdwfl.h
+%{_includedir}/elfutils/libdwfl_stacktrace.h
%{_includedir}/elfutils/libdwelf.h
%{_includedir}/elfutils/version.h
#%%{_libdir}/libasm.so
diff --git a/configure.ac b/configure.ac
index 27488e3f..92108b7f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -700,6 +700,9 @@ AC_CONFIG_FILES([libdw/Makefile])
dnl Higher-level DWARF support library.
AC_CONFIG_FILES([libdwfl/Makefile])
+dnl Higher-level DWARF stacktrace support library
+AC_CONFIG_FILES([libdwfl_stacktrace/Makefile])
+
dnl CPU handling library.
AC_CONFIG_FILES([libcpu/Makefile])
diff --git a/libdw/Makefile.am b/libdw/Makefile.am
index 9dadc19b..f024d652 1006
[PATCH v4 10/12] eu-stacktrace [10/12]: use dwflst_tracker_find_pid
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
* * *
Initial minimal change to ensure dwflst_tracker_find_pid is
tested. For now, we keep the additional dwfltab implementation in
stacktrace.c, since it's being used to track statistics.
In future follow-ups, it will be good to switch to storing
eu-stacktrace statistics e.g. in dwfl->process->callbacks_arg. This
requires some additional design to keep the statistics from being lost
when a pid is reused and the corresponding processtracker table entry
is replaced.
* src/stacktrace.c (sysprof_init_dwfl): New function.
(sysprof_find_dwfl): Rename the existing sysprof_init_dwfl.
Also use dwflst_tracker_find_pid with callback.
(sysprof_unwind_cb): Rename the existing sysprof_init_dwfl.
---
src/stacktrace.c | 63 +++-
1 file changed, 36 insertions(+), 27 deletions(-)
diff --git a/src/stacktrace.c b/src/stacktrace.c
index c0c9929d..dd58ef3b 100644
--- a/src/stacktrace.c
+++ b/src/stacktrace.c
@@ -96,7 +96,7 @@
#include ELFUTILS_HEADER(ebl)
/* #include ELFUTILS_HEADER(dwfl) */
#include "../libdwfl/libdwflP.h"
-/* XXX: Private header needed for find_procfile, sysprof_init_dwfl,
+/* XXX: Private header needed for find_procfile, sysprof_find_dwfl,
sample_set_initial_registers. */
#include ELFUTILS_HEADER(dwfl_stacktrace)
@@ -1014,7 +1014,34 @@ find_procfile (Dwfl *dwfl, pid_t *pid, Elf **elf, int
*elf_fd)
}
Dwfl *
-sysprof_init_dwfl (struct sysprof_unwind_info *sui,
+sysprof_init_dwfl (Dwflst_Process_Tracker *cb_tracker,
+ pid_t pid,
+ void *arg __attribute__ ((unused)))
+{
+ Dwfl *dwfl = dwflst_tracker_dwfl_begin (cb_tracker);
+
+ int err = dwfl_linux_proc_report (dwfl, pid);
+ if (err < 0)
+{
+ if (show_failures)
+ fprintf(stderr, "dwfl_linux_proc_report pid %lld: %s",
+ (long long) pid, dwfl_errmsg (-1));
+ return NULL;
+}
+ err = dwfl_report_end (dwfl, NULL, NULL);
+ if (err != 0)
+{
+ if (show_failures)
+ fprintf(stderr, "dwfl_report_end pid %lld: %s",
+ (long long) pid, dwfl_errmsg (-1));
+ return NULL;
+}
+
+ return dwfl;
+}
+
+Dwfl *
+sysprof_find_dwfl (struct sysprof_unwind_info *sui,
SysprofCaptureStackUser *ev,
SysprofCaptureUserRegs *regs)
{
@@ -1027,42 +1054,24 @@ sysprof_init_dwfl (struct sysprof_unwind_info *sui,
if (regs->n_regs < EXPECTED_REGS) /* XXX expecting everything except FLAGS */
{
if (show_failures)
- fprintf(stderr, N_("sysprof_init_dwfl: n_regs=%d, expected %d\n"),
+ fprintf(stderr, N_("sysprof_find_dwfl: n_regs=%d, expected %d\n"),
regs->n_regs, EXPECTED_REGS);
return NULL;
}
- Dwfl *dwfl = pid_find_dwfl(pid);
+ Dwfl *dwfl = dwflst_tracker_find_pid (tracker, pid, sysprof_init_dwfl, NULL);
struct __sample_arg *sample_arg;
bool cached = false;
- if (dwfl != NULL)
+ if (dwfl != NULL && dwfl->process != NULL)
{
sample_arg = dwfl->process->callbacks_arg; /* XXX requires libdwflP.h */
cached = true;
goto reuse;
}
- dwfl = dwflst_tracker_dwfl_begin (tracker);
-
- int err = dwfl_linux_proc_report (dwfl, pid);
- if (err < 0)
-{
- if (show_failures)
- fprintf(stderr, "dwfl_linux_proc_report pid %lld: %s",
- (long long) pid, dwfl_errmsg (-1));
- return NULL;
-}
- err = dwfl_report_end (dwfl, NULL, NULL);
- if (err != 0)
-{
- if (show_failures)
- fprintf(stderr, "dwfl_report_end pid %lld: %s",
- (long long) pid, dwfl_errmsg (-1));
- return NULL;
-}
Elf *elf = NULL;
int elf_fd = -1;
- err = find_procfile (dwfl, &pid, &elf, &elf_fd);
+ int err = find_procfile (dwfl, &pid, &elf, &elf_fd);
if (err < 0)
{
if (show_failures)
@@ -1099,7 +1108,7 @@ sysprof_init_dwfl (struct sysprof_unwind_info *sui,
if (show_frames) {
bool is_abi32 = (sample_arg->abi == PERF_SAMPLE_REGS_ABI_32);
-fprintf(stderr, "sysprof_init_dwfl pid %lld%s: size=%ld%s pc=%lx
sp=%lx+(%lx)\n",
+fprintf(stderr, "sysprof_find_dwfl pid %lld%s: size=%ld%s pc=%lx
sp=%lx+(%lx)\n",
(long long) pid, cached ? " (cached)" : "",
sample_arg->size, is_abi32 ? " (32-bit)" : "",
sample_arg->pc, sample_arg->base_addr,
@@ -1260,7 +1269,7 @@ sysprof_unwind_cb (SysprofCaptureFrame *frame, void *arg)
SysprofCaptureUserRegs *regs = (SysprofCaptureUserRegs *)tail_ptr;
if (show_frames)
fprintf(stderr, "\n"); /* extra newline for padding */
- Dwfl *dwfl = sysprof_init_dwfl (sui, ev, regs);
+ Dwfl *dwfl = sysprof_find_dwfl (sui, ev, regs);
if (dwfl == NULL)
{
if (show_summary)
@@ -1270,7 +1279,7 @@ sysprof_unwind_cb (SysprofCaptureFrame *frame, void *arg)
dwfl_ent->lost_samples++;
}
if (show_failures)
- fprintf(stderr, "sysprof_init_dwfl pid %lld (%s) (failed)\n"
Re: scraperbot protection - Patchwork and Bunsen behind Anubis
Hi! Thanks for all the hard work maintaining all this fundamental infrastructure. On Mon, 21 Apr 2025 at 18:00, Mark Wielaard wrote: > > Hi hackers, > > TLDR; When using https://patchwork.sourceware.org or Bunsen > https://builder.sourceware.org/testruns/ you might now have to enable > javascript. This should not impact any scripts, just browsers (or bots > pretending to be browsers). If it does cause trouble, please let us > know. If this works out we might also "protect" bugzilla, gitweb, > cgit, and the wikis this way. > > We don't like to hav to do this, but as some of you might have noticed > Sourceware has been fighting the new AI scraperbots since start of the > year. We are not alone in this. > > https://lwn.net/Articles/1008897/ > https://arstechnica.com/ai/2025/03/devs-say-ai-crawlers-dominate-traffic-forcing-blocks-on-entire-countries/ > > We have tried to isolate services more and block various ip-blocks > that were abusing the servers. But that has helped only so much. In terms of isolation, since I have no idea how / which services are currently isolated, I may ask obvious questions.. We were wondering if it would be possible / suitable to have https requests served by one container, and ssh ones by another? Maybe that's already the case though... Speaking with CI in mind, the Linaro CI is currently severely impacted by these scraperbots too: - directly because our git servers are also overloaded, so our build process often fail to checkout build scripts & infra scripts - indirectly because when the above succeed we may fail to connect to sourceware so maybe it would help if we switched to ssh access for our CI user when cloning GCC / binuitls / etc sources? If only the containers serving https requests are impacted, ssh access could still work well? (that would mean creating a Linaro-CI user on sourceware, I don't know what the policy is?) Thanks, Christophe > Unfortunately the scraper bots are using lots of ip addresses > (probably by installing "free" VPN services that use normal user > connections as exit point) and pretending to be common > browsers/agents. We seem to have to make access to some services > depend on solving a javascript challenge. > > So we have installed Anubis https://anubis.techaro.lol/ in front of > patchwork and bunsen. This means that if you are using a browser that > identifies as Mozilla or Opera in their User-Agent you will get a > brief page showing the happy anime girl that requires javascript to > solve a challenge and get a cookie to get through. Scripts and search > engines should get through without. Also removing Mozilla and/or Opera > from your User-Agent will get you through without javascript. > > We want to thanks Xe Iaso who has helped us set this up and worked > with use over the Easter weekend solving some of our problems/typos. > Please check out if you want to be one of their patrons as thank you. > https://xeiaso.net/notes/2025/anubis-works/ > https://xeiaso.net/patrons/ > > Cheers, > > Mark
