Re: [PATCH] PR29022: 000-permissions files cause problems for backups

2022-04-08 Thread Aaron Merey via Elfutils-devel
I've revised this patch so that the negative-hit file's mtime is used to calculate time since last download attempt instead of the cache_miss_s file. I've also added a check for old 000-permission files so that they are unlinked immediately if found. Aaron --- PR29022: 000-permissions files cause

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Aaron Merey via Elfutils-devel
Thanks for spotting this Milian. On Fri, Apr 8, 2022 at 6:40 PM Mark Wielaard wrote: > I think we as developers keep clearing the cache to test stuff. But > that means we recreate the cache_miss file every time, so that gets a > new mtime. And if you are just testing for 10 minutes everything lo

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Mark Wielaard
Hi Milian, On Sat, Apr 09, 2022 at 12:23:32AM +0200, Milian Wolff wrote: > ``` > # first we stat the target cache path > if (stat(target_cache_path, &st) == 0 > { > > # then we pass _the same st_ to > debuginfod_config_cache(cache_miss_path, cache_miss_default_s, &st) > > # which interna

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Milian Wolff
On Freitag, 8. April 2022 23:56:15 CEST Milian Wolff wrote: > Which in turn points at the code that does cache cleanup in > `debuginfod_query_server`. I now used `rr` to record such a bogus run and I > clearly see that `(time(NULL) - st.st_mtime <= cache_miss)` is false and it > goes into the un

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Mark Wielaard
Hi Milian, On Fri, Apr 08, 2022 at 11:56:15PM +0200, Milian Wolff wrote: > Which in turn points at the code that does cache cleanup in > `debuginfod_query_server`. I now used `rr` to record such a bogus run and I > clearly see that `(time(NULL) - st.st_mtime <= cache_miss)` is false and it > go

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Milian Wolff
On Freitag, 8. April 2022 23:34:06 CEST Aaron Merey wrote: > Hi Milian, > > On Fri, Apr 8, 2022 at 5:08 PM Milian Wolff wrote: > > I can reproduce it now suddenly with debuginfod-find too: > > > > ``` > > $ > > debuginfod-find debuginfo 85766e9d8458b16e9c7ce6e07c712c02b8471dbc > > debuginfod_fin

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Aaron Merey via Elfutils-devel
Hi Milian, On Fri, Apr 8, 2022 at 5:08 PM Milian Wolff wrote: > I can reproduce it now suddenly with debuginfod-find too: > > ``` > $ > debuginfod-find debuginfo 85766e9d8458b16e9c7ce6e07c712c02b8471dbc > debuginfod_find_debuginfo 85766e9d8458b16e9c7ce6e07c712c02b8471dbc > server urls "https://de

Re: parallel downloads of multiple debuginfo files

2022-04-08 Thread Frank Ch. Eigler via Elfutils-devel
Hi - > But once again - isn't this a problem that everyone using dwfl is going to > encounter? Should we not try to find a general solution to this problem and > fix it for all consumers of the API? I suspect not many apps are going to have a complete list of files they know they'll need a prio

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Milian Wolff
On Freitag, 8. April 2022 22:59:55 CEST Mark Wielaard wrote: > Hi Milian, > > On Fri, Apr 08, 2022 at 10:45:10PM +0200, Milian Wolff wrote: > > In the off-CPU flame graphs I clearly see this callchain when > > DEBUGINFOD_URLS is set: > > > > ``` > > dwfl_standard_find_debuginfo > debuginfod_query

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Mark Wielaard
Hi Milian, On Fri, Apr 08, 2022 at 10:45:10PM +0200, Milian Wolff wrote: > In the off-CPU flame graphs I clearly see this callchain when DEBUGINFOD_URLS > is set: > > ``` > dwfl_standard_find_debuginfo > debuginfod_query_server > curl_multi_wait > ``` > > This accounts for ~2.5s of off-CPU time

Re: parallel downloads of multiple debuginfo files

2022-04-08 Thread Milian Wolff
On Freitag, 8. April 2022 22:54:35 CEST Frank Ch. Eigler wrote: > Hi - > > > one more debuginfod question: Would it be possible to extend the API > > to allow downloading of multiple debug info files in parallel? The > > `debuginfod_find_*` API currently only supports looking at multiple > > serv

Re: parallel downloads of multiple debuginfo files

2022-04-08 Thread Frank Ch. Eigler via Elfutils-devel
Hi - > one more debuginfod question: Would it be possible to extend the API > to allow downloading of multiple debug info files in parallel? The > `debuginfod_find_*` API currently only supports looking at multiple > server urls in parallel. I would like to ask multiple files in > parallel. Spin

parallel downloads of multiple debuginfo files

2022-04-08 Thread Milian Wolff
Hey all, one more debuginfod question: Would it be possible to extend the API to allow downloading of multiple debug info files in parallel? The `debuginfod_find_*` API currently only supports looking at multiple server urls in parallel. I would like to ask multiple files in parallel. The use

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Milian Wolff
On Freitag, 8. April 2022 22:05:27 CEST Frank Ch. Eigler wrote: > Hi - > > > another debuginfod related question, but unrelated to the other thread I > > started earlier today. In a work branch I have ported my heaptrack > > profiler > > over to elfutils. I have then run the analyzer that uses elf

Re: caching failed lookups of debuginfo?

2022-04-08 Thread Frank Ch. Eigler via Elfutils-devel
Hi - > another debuginfod related question, but unrelated to the other thread I > started earlier today. In a work branch I have ported my heaptrack profiler > over to elfutils. I have then run the analyzer that uses elfutils (and thus > debuginfod internally via dwfl) on a recorded data file t

caching failed lookups of debuginfo?

2022-04-08 Thread Milian Wolff
Hey all, another debuginfod related question, but unrelated to the other thread I started earlier today. In a work branch I have ported my heaptrack profiler over to elfutils. I have then run the analyzer that uses elfutils (and thus debuginfod internally via dwfl) on a recorded data file to ha

Re: Questions regarding debuginfod.h API

2022-04-08 Thread Milian Wolff
On Freitag, 8. April 2022 21:44:32 CEST Frank Ch. Eigler wrote: > Hi - > > > > (See also the DEBUGINFOD_MAXTIME and DEBUGINFOD_MAXSIZE env vars > > > that can limit this.) > > > > I did come across those, but what are suggested best practices in > > setting those? When using GDB or a profiler on

Re: Questions regarding debuginfod.h API

2022-04-08 Thread Frank Ch. Eigler via Elfutils-devel
Hi - > > (See also the DEBUGINFOD_MAXTIME and DEBUGINFOD_MAXSIZE env vars > > that can limit this.) > I did come across those, but what are suggested best practices in > setting those? When using GDB or a profiler on larger non-trivial UI > applications on Linux for the first time, we would start

Re: Questions regarding debuginfod.h API

2022-04-08 Thread Milian Wolff
On Freitag, 8. April 2022 15:44:32 CEST Frank Ch. Eigler wrote: > Hi - > > Now to my question: In applications using elfutils, we will now > > automatically download debug information when DEBUGINFOD_URLS is > > defined. But doing that can take a very long time. > > (See also the DEBUGINFOD_MAX

Issue 46515 in oss-fuzz: elfutils:fuzz-libdwfl: Timeout in fuzz-libdwfl

2022-04-08 Thread ClusterFuzz-External via monorail via Elfutils-devel
Status: New Owner: CC: elfut...@sourceware.org, da...@adalogics.com, evv...@gmail.com, izz...@google.com Labels: ClusterFuzz Reproducible Engine-libfuzzer OS-Linux Proj-elfutils Reported-2022-04-08 Type: Bug New issue 46515 by ClusterFuzz-External: elfutils:fuzz-libdwfl: Timeout in fuzz-l

Issue 46513 in oss-fuzz: elfutils:fuzz-libdwfl: Out-of-memory in fuzz-libdwfl

2022-04-08 Thread ClusterFuzz-External via monorail via Elfutils-devel
Status: New Owner: CC: elfut...@sourceware.org, da...@adalogics.com, evv...@gmail.com, izz...@google.com Labels: ClusterFuzz Reproducible Engine-libfuzzer OS-Linux Proj-elfutils Reported-2022-04-08 Type: Bug New issue 46513 by ClusterFuzz-External: elfutils:fuzz-libdwfl: Out-of-memory in

Re: Questions regarding debuginfod.h API

2022-04-08 Thread Frank Ch. Eigler via Elfutils-devel
Hi - > now that archlinux is supporting debuginfod, I have finally tried it > out. It's such a game changer, many thanks for everyone involved in > working on this! Our pleasure! > Now to my question: In applications using elfutils, we will now > automatically download debug information when DEB

Questions regarding debuginfod.h API

2022-04-08 Thread Milian Wolff
Hey all, now that archlinux is supporting debuginfod, I have finally tried it out. It's such a game changer, many thanks for everyone involved in working on this! Now to my question: In applications using elfutils, we will now automatically download debug information when DEBUGINFOD_URLS is def