[Bug debuginfod/29478] Slow query of a debuginfo in a big rpm with very many files
https://sourceware.org/bugzilla/show_bug.cgi?id=29478 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |NOTABUG --- Comment #1 from Frank Ch. Eigler --- Unfortunately, we're not aware of an easy way around this. There is no way to seek in the compressed files to an arbitrary index, so one is stuck reading from the beginning to the file of interest. I think that, unless the kernel packaging changes, one needs to rely on (trigger!) debuginfod to fetch & cache highly likely files such as the vdso*. Some work is underway to allow this automation, but until then, make sure your debuginfod server has a large enough fdcache, and you don't groom too frequently (which clears out the fdcache). See also https://bugzilla.redhat.com/show_bug.cgi?id=1970578 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29474] Server returns 404 for concurrent requests when leading to a same .rpm
https://sourceware.org/bugzilla/show_bug.cgi?id=29474 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #5 from Frank Ch. Eigler --- elfutils-0.187-44-g31d6b1fe7 / debuginfod/ChangeLog debuginfod/debuginfod.cxx: PR29474: debuginfod -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/28284] support description functionality through HEAD
https://sourceware.org/bugzilla/show_bug.cgi?id=28284 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #5 from Frank Ch. Eigler --- commit 8760e931abf33ed2b215809d5ebf653ced934152 Author: Frank Ch. Eigler Date: Fri Sep 2 12:41:38 2022 -0400 PR28284: add tweaks on previous debuginfod x-debuginfod* header forwarding work Embrace case-independent headers, more fully document, handle HTTP \r. In addition to test case, hand-tested against fedora debuginfod instances, running federated servers under valgrind. Signed-off-by: Frank Ch. Eigler commit bbc2ca6d553f0ce3e670303ac9a3c764cf10d779 Author: Noah Sanci Date: Wed Jun 15 10:07:29 2022 -0400 PR28284 - Debuginfod header functionality implemented Debuginfod and debuginfod clients are now equipped to send and receive http headers prefixed with X-DEBUGINFOD and print them in verbose mode for more context Signed-off-by: Noah Sanci -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29696] New: intermittent libmicrohttpd assertion failures related to socket fd closing
https://sourceware.org/bugzilla/show_bug.cgi?id=29696 Bug ID: 29696 Summary: intermittent libmicrohttpd assertion failures related to socket fd closing Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- In a range of libmicrohttpd versions, up to and including libmicrohttpd-0.9.75-3.fc36.x86_64, debuginfod occasionally crashes with messages like: https://builder.sourceware.org/testrun/920819ee86861130393e12933821c5b544afeee4?filename=tests%2Frun-debuginfod-federation-metrics.sh.log#line1669 Fatal error in GNU libmicrohttpd daemon.c:3831: Failed to remove FD from epoll set. Even without MHD_USE_EPOLL, a nearly identical message can come from a different code path. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/29711] In libdw.h struct Dwarf, struct Dwarf_Abbrev is defined using typedef and not using #include is not the way to comply with the specification?
https://sourceware.org/bugzilla/show_bug.cgi?id=29711 Frank Ch. Eigler changed: What|Removed |Added Status|UNCONFIRMED |WAITING Last reconfirmed||2022-10-21 Ever confirmed|0 |1 CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- No specification requires elfutils to provide an other than forward declaration of these types. Keeping the definitions private is fine, given that opaque pointers are used in the various libdw* api calls to clients. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug tools/29786] Unable to understand, in objdump, the user is not using the name, so using a.out
https://sourceware.org/bugzilla/show_bug.cgi?id=29786 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |NOTABUG CC||fche at redhat dot com Status|UNCONFIRMED |RESOLVED --- Comment #1 from Frank Ch. Eigler --- Why not use an empty string as a filename? A user can type eu-objdump "" to pass an empty string if that's really what she wants. Defaulting to "a.out" in the absence of a filename is traditional. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29926] debuginfod using deprecated curl (since 7.55.0) API, fails to build with 7.87.0
https://sourceware.org/bugzilla/show_bug.cgi?id=29926 Frank Ch. Eigler changed: What|Removed |Added Status|UNCONFIRMED |RESOLVED Resolution|--- |FIXED CC||fche at redhat dot com --- Comment #2 from Frank Ch. Eigler --- Thanks, merged! -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29975] Search concurrency doesn't respect CPU affinity
https://sourceware.org/bugzilla/show_bug.cgi?id=29975 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- > number of cores available to it (sched_getaffinity()). Do you happen to be aware of a c++y front-end to this, which we could use instead of std::thread::hardware_concurrency() ? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29976] webapi connection pool eats all file handles
https://sourceware.org/bugzilla/show_bug.cgi?id=29976 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- What sets "ulimit -n -> 1000" in your case? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29976] webapi connection pool eats all file handles
https://sourceware.org/bugzilla/show_bug.cgi?id=29976 --- Comment #4 from Frank Ch. Eigler --- I assume "debuginfod -C $num -d $num" still works for you, in this battle of distro/site defaults. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29982] sqlite errors when searching
https://sourceware.org/bugzilla/show_bug.cgi?id=29982 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com Ever confirmed|0 |1 Last reconfirmed||2023-01-10 Status|UNCONFIRMED |WAITING --- Comment #2 from Frank Ch. Eigler --- Please supply a command line invocation for us to try to reproduce this, along with version numbers of ubuntu / debuginfod / sqlite3. This could happen if the database is actually a normal file instead of :memory:, and two separate debuginfod instances are trying to write to the same one. (debuginfod -d :memory: is already tested on debian etc. as a part of the testsuite. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29982] sqlite errors when searching
https://sourceware.org/bugzilla/show_bug.cgi?id=29982 --- Comment #6 from Frank Ch. Eigler --- Please confirm the complete command line. % debuginfod -d:memory creates a file named ":memory", not an in-memory sqlite database alias ":memory:") -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29982] sqlite errors when searching
https://sourceware.org/bugzilla/show_bug.cgi?id=29982 --- Comment #9 from Frank Ch. Eigler --- Perhaps a complete highly verbose trace (-vvv) would help identify the cause. Scanning threads should not be able to lock each other out for O(1sec) from sqlite operations in normal circumstances. OTOH, one can rest assured that in case of such sqlite problems, debuginfod just treats them as transient and retries the related scan operation later. Webapi service threads do not contend with the others at all, so clients don't see these transient errors. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29976] webapi connection pool eats all file handles
https://sourceware.org/bugzilla/show_bug.cgi?id=29976 --- Comment #6 from Frank Ch. Eigler --- please check out commit 7399e3bd7eb72d045 on elfutils.git for a test patch -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29975] Search concurrency doesn't respect CPU affinity
https://sourceware.org/bugzilla/show_bug.cgi?id=29975 --- Comment #3 from Frank Ch. Eigler --- please check out commit 7399e3bd7eb72d045 on elfutils.git for a test patch -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29976] webapi connection pool eats all file handles
https://sourceware.org/bugzilla/show_bug.cgi?id=29976 Frank Ch. Eigler changed: What|Removed |Added Status|UNCONFIRMED |RESOLVED Resolution|--- |FIXED --- Comment #8 from Frank Ch. Eigler --- Pushed to master as dcb40f9caa7ca30 Author: Frank Ch. Eigler Date: Tue Jan 10 17:59:35 2023 -0500 debuginfod PR29975 & PR29976: decrease default concurrency ... based on rlimit (rlimig -n NUM) ... based on cpu-affinity (taskset -c A,B,C,D ...) Signed-off-by: Frank Ch. Eigler -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29975] Search concurrency doesn't respect CPU affinity
https://sourceware.org/bugzilla/show_bug.cgi?id=29975 Frank Ch. Eigler changed: What|Removed |Added Status|UNCONFIRMED |RESOLVED Resolution|--- |FIXED --- Comment #4 from Frank Ch. Eigler --- Pushed to master as dcb40f9caa7ca30 Author: Frank Ch. Eigler Date: Tue Jan 10 17:59:35 2023 -0500 debuginfod PR29975 & PR29976: decrease default concurrency ... based on rlimit (rlimig -n NUM) ... based on cpu-affinity (taskset -c A,B,C,D ...) Signed-off-by: Frank Ch. Eigler -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30000] New: debuginfod-find should have a source-list verb
https://sourceware.org/bugzilla/show_bug.cgi?id=3 Bug ID: 3 Summary: debuginfod-find should have a source-list verb Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- Let's implement % debuginfod-find source-list PATH|BUILDID so it fetches the debuginfo, then runs the dwarf_getsrcfiles / dwarf_getsrcdirs / CU comp_dir traversal, and prints a deduped list of all CUs' source file lists. An adventurous user can then use that to fetch all source files of a particular binary. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30221] Negative cache should differentiate failure types
https://sourceware.org/bugzilla/show_bug.cgi?id=30221 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- Even a 404 error may be transient, as a server may just not have gotten around to indexing new content yet. Other transient errors may persist awhile. I don't know of any unambiguous winning policy here. As to the question that, if such a policy were formulated, how could the results be represented in the filesystem: xattrs, yeah maybe. But even simpler would be to have the code set the mtime or ctime of the 0-length file to a cause-related artificial timestamp that will inform the "cache_miss_s" expiry calculations. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30221] Negative cache should differentiate failure types
https://sourceware.org/bugzilla/show_bug.cgi?id=30221 --- Comment #4 from Frank Ch. Eigler --- (In reply to Vicki Pfau from comment #3) > I have a proof of concept patch that I can attach here or submit to the > mailing list if you think the xattrs approach is a good way to go. > Alternatively, a metadata directory could be added under each buildid for > per-file info, which would work in the absence of functional xattrs, but be > slightly more complex. Have you considered the idea of encoding the retention deadline in the boring inode mtime or ctime? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30221] Negative cache should differentiate failure types
https://sourceware.org/bugzilla/show_bug.cgi?id=30221 --- Comment #6 from Frank Ch. Eigler --- (In reply to Vicki Pfau from comment #2) > 404 and the like *may* be transient, but the fact of the matter is that > *most* of the time it won't be And it's a cache, not a definitive answer > saying this will never exist. Having a 404 cache for 10x the amount of time > as a Ctrl-C I don't understand - a ctrl-C should not result in a cached artifact at all. If that's happening, we should fix that. > I'm already dealing with gdb taking well > over 30 seconds to start running a program with a bunch of shared object > dependencies that aren't in debuginfod... Uncached misses from debuginfod tend to take on the order of milliseconds, much less than seconds. Do you have a trace of what's happening? (DEBUGINFOD_VERBOSE=1 or something like that?) > [...] because there's no way for the cache to say "this probably won't > appear in the short term." Setting cache_miss_s higher works, but is a > workaround. That workaround is precisely the parameter for the quantity you seek. > Your filesystem representation works > for the small, simple case you have here, but it won't scale if you try and > extend the system with any metadata at all. That's fine. If we can revisit when rationale exists for more metadata. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug general/30241] RFE add APIs to elfutis libdw
https://sourceware.org/bugzilla/show_bug.cgi?id=30241 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- This depends largely on what sort of editing API a client envisions. Writing brand new content from scratch is more like an assembler task. Making micro edits in situ (changing individual DIEs, insert/delete, like a text editor) is likely to be entirely different and much more complicated. So ISTM this needs to be drive not by elfutils in the abstract but the specific needs of the editing/synthesis tools. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30221] Negative cache should differentiate failure types
https://sourceware.org/bugzilla/show_bug.cgi?id=30221 --- Comment #12 from Frank Ch. Eigler --- There was a wild performance regression in sqlite 3.41 that archlinux's debuginfod server got hit with. This was identified and corrected yesterday. (It had nothing to do with caching.) https://sqlite.org/forum/forumpost/a284a63124 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30316] New: debuginfod server should checkpoint WAL manually
https://sourceware.org/bugzilla/show_bug.cgi?id=30316 Bug ID: 30316 Summary: debuginfod server should checkpoint WAL manually Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- On a very large corpus, initial scan operations can take a long time, and build up quite a large debuginfod.sqlite-wal file. If the server is filesystem space constrained, this .sqlite + .sqlite-wal combined file may get too large to fit. Suggestion: debuginfod should inject a periodic pragma wal_checkpoint(truncate); into the traverse loop, as the default autocheckpoint stuff does not appear to operate reliably. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30348] New: debuginfod should retry scanning archives that suffered exceptions
https://sourceware.org/bugzilla/show_bug.cgi?id=30348 Bug ID: 30348 Summary: debuginfod should retry scanning archives that suffered exceptions Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: fche at redhat dot com Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- _file_mtime_scanned rows get emplaced even for exception-afflicted archive scan jobs -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30221] Negative cache should differentiate failure types
https://sourceware.org/bugzilla/show_bug.cgi?id=30221 Frank Ch. Eigler changed: What|Removed |Added Status|UNCONFIRMED |RESOLVED Resolution|--- |WORKSFORME --- Comment #14 from Frank Ch. Eigler --- We believe the current code behaves better with respect to aborted downloads. Thank you for your report. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30377] New: -r (groom w/ regex) ignores -X
https://sourceware.org/bugzilla/show_bug.cgi?id=30377 Bug ID: 30377 Summary: -r (groom w/ regex) ignores -X Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: fche at redhat dot com Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- There's a logic bug in the groom process wherein artifacts related to a file excluded via the -X regex are not actually removed. && -> || to fix. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30378] New: need better compression of the _files table
https://sourceware.org/bugzilla/show_bug.cgi?id=30378 Bug ID: 30378 Summary: need better compression of the _files table Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- Enumerating each file name (whether an archive name or a file within an archive or a source file name referenced from a dwarf file) inside the _files table means storing many near-duplicate strings in that table. Considering splitting it up into basename / dirname subtables, with _files containing their id-pair only rather than a potentially long string. Expecting drastic reduction in sqlite data size. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30377] -r (groom w/ regex) ignores -X
https://sourceware.org/bugzilla/show_bug.cgi?id=30377 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #1 from Frank Ch. Eigler --- commit cded466a74d06fe1dff047b33e630f8a6aa721e1 Author: Frank Ch. Eigler Date: Fri Apr 21 17:04:08 2023 -0400 PR30377: fix debuginfod -r -X combination Until this fix, debuginfod -r -X '.*' didn't trigger groom-time removal of everything, because the -I include regex overrode it. Corrected logic to match the scan-time tie-breaking between -I / -X. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30378] need better compression of the _files table
https://sourceware.org/bugzilla/show_bug.cgi?id=30378 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #1 from Frank Ch. Eigler --- commit a2f3f09745f781663eb874a4a86b5781cb423115 Author: Frank Ch. Eigler Date: Fri May 5 13:56:23 2023 -0400 debuginfod: PR30378: better compression for _files table -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30316] debuginfod server should checkpoint WAL manually
https://sourceware.org/bugzilla/show_bug.cgi?id=30316 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #1 from Frank Ch. Eigler --- commit d16628bb7dc39a020e555e16bf56374fdac84e42 Author: Frank Ch. Eigler Date: Mon May 8 11:05:48 2023 -0400 PR30316: debuginfod wal checkpointing -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/29696] intermittent libmicrohttpd assertion failures related to socket fd closing
https://sourceware.org/bugzilla/show_bug.cgi?id=29696 Frank Ch. Eigler changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |FIXED --- Comment #4 from Frank Ch. Eigler --- commit 938a52c22ee915ff2cea813edd5da66bc8184885 Author: Ryan Goldberg Date: Fri Jun 16 10:20:04 2023 -0400 debuginfod: PR29696: Removed secondary fd close in cache config causing a race condition -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30589] debuginfod can't find sources even though they exist on disk
https://sourceware.org/bugzilla/show_bug.cgi?id=30589 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com Last reconfirmed||2023-06-27 Ever confirmed|0 |1 Status|UNCONFIRMED |WAITING --- Comment #2 from Frank Ch. Eigler --- In my tests here on f38, the gcc -ffile-prefix-map option resulted in highly sus dwarf: % gcc -g -O0 test.c -Wl,--build-id=0xaabbccdd -ffile-prefix-map=`pwd`=`pwd`/servefrom % eu-readelf -w a.out Directory table: [path(line_strp)] 0 /home/fche/[...]/servefrom/servefrom (61) i.e., the suffix was added twice. On another test, where the map target was to a non-subdirectory like /tmp/foo, gcc produced correct looking dwarf, and debuginfod took it like a champ. gcc version 13.1.1 20230614 (Red Hat 13.1.1-4) (GCC) -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/28204] extend webapi / verification with forthcoming signed-contents archives
https://sourceware.org/bugzilla/show_bug.cgi?id=28204 --- Comment #18 from Frank Ch. Eigler --- > Doesn't that give a false sense of "security"? > It still rejects some stuff, but doesn't really protect against "falsifying" > files, all a server has to do is not provide an IMA Yes, but trusted servers won't just do that. > If it is just to see what would happen if enabling ima file checking, then > it probably shouldn't reject anything. In that case it should warn for both > missing and invalid signatures, but still accept them. The difference between missing and invalid is that the latter is KNOWN bad. An invalid signature is evidence that the file has a problem. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30809] New: improve debuginfod-client progress callback parameters
https://sourceware.org/bugzilla/show_bug.cgi?id=30809 Bug ID: 30809 Summary: improve debuginfod-client progress callback parameters Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- For the compressed case, we can/should compute proper download-progress metrics even of compressed transfers, by periodic fstat() of the output file. https://inbox.sourceware.org/elfutils-devel/20230330172413.gg30...@redhat.com/ -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30809] improve debuginfod-client progress callback parameters
https://sourceware.org/bugzilla/show_bug.cgi?id=30809 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #1 from Frank Ch. Eigler --- > commit 3ef3fab0d64c89a52dd6e2ce0d01dd5e713d7b5a > Author: Frank Ch. Eigler > Date: Tue Aug 29 14:08:04 2023 -0400 > > PR30809: improve debuginfod client progress-callback parameters -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30879] New: intermittent failure (libstdc++ uncaught exception terminate) during testsuite on sparc-gentoo
https://sourceware.org/bugzilla/show_bug.cgi?id=30879 Bug ID: 30879 Summary: intermittent failure (libstdc++ uncaught exception terminate) during testsuite on sparc-gentoo Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- A particularly high-cpu-count buildbot worker reported intermittent failures in the debuginfod testsuite. e.g. https://builder.sourceware.org/buildbot/#/builders/224/builds/67 We should ensure that exceptions that hardly ever occur are nevertheless caught and reported reliably. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30000] introduce new srcfiles tool
https://sourceware.org/bugzilla/show_bug.cgi?id=3 Frank Ch. Eigler changed: What|Removed |Added Summary|debuginfod-find should have |introduce new srcfiles tool |a source-list verb | -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/30000] introduce new srcfiles tool
https://sourceware.org/bugzilla/show_bug.cgi?id=3 --- Comment #1 from Frank Ch. Eigler --- Work is now proceeding instead toward a tool src/srcfiles to list names of source files from dwarf content. A hypothetical followup idea: an option for srcfiles and/or debuginfod-find that uses this list to construct a tarball of source files themselves (fetched via debuginfod). -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25369] make DEBUGINFOD_PROGRESS prettier
https://sourceware.org/bugzilla/show_bug.cgi?id=25369 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #2 from Frank Ch. Eigler --- commit 5d67e9d3eaa6442b781cd6d8ce38a53b17136d12 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25583] Use libarchive to extract .deb packages?
https://sourceware.org/bugzilla/show_bug.cgi?id=25583 --- Comment #7 from Frank Ch. Eigler --- here's the recipe for .deb / .ddeb: -Z '.ddeb=(bsdtar -O -x -f - data.tar.xz)<' -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25367] web request status logging improvements
https://sourceware.org/bugzilla/show_bug.cgi?id=25367 --- Comment #1 from Frank Ch. Eigler --- https://sourceware.org/pipermail/elfutils-devel/2020q1/002524.html -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25548] also support canonicalized source-file name lookups in webapi
https://sourceware.org/bugzilla/show_bug.cgi?id=25548 --- Comment #5 from Frank Ch. Eigler --- https://sourceware.org/pipermail/elfutils-devel/2020q1/002522.html -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25722] New: debuginfod-find should be able to take an /bin/path in place of buildid
https://sourceware.org/bugzilla/show_bug.cgi?id=25722 Bug ID: 25722 Summary: debuginfod-find should be able to take an /bin/path in place of buildid Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- This way one could run it via % debuginfod-find debuginfo /bin/ls without having to hand-extract the buildid first. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25583] Use libarchive to extract .deb packages?
https://sourceware.org/bugzilla/show_bug.cgi?id=25583 --- Comment #8 from Frank Ch. Eigler --- https://sourceware.org/pipermail/elfutils-devel/2020q1/002529.html -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25722] debuginfod-find should be able to take an /bin/path in place of buildid
https://sourceware.org/bugzilla/show_bug.cgi?id=25722 --- Comment #2 from Frank Ch. Eigler --- https://sourceware.org/pipermail/elfutils-devel/2020q1/002535.html -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25548] also support canonicalized source-file name lookups in webapi
https://sourceware.org/bugzilla/show_bug.cgi?id=25548 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #6 from Frank Ch. Eigler --- commit d63a809da467e646480c273b8eb276401679d2bb -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25367] web request status logging improvements
https://sourceware.org/bugzilla/show_bug.cgi?id=25367 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #2 from Frank Ch. Eigler --- commit 439641e52a3258ddfa1f5de8bbcdb1530fc1 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25583] Use libarchive to extract .deb packages?
https://sourceware.org/bugzilla/show_bug.cgi?id=25583 Frank Ch. Eigler changed: What|Removed |Added Status|REOPENED|RESOLVED Resolution|--- |FIXED --- Comment #9 from Frank Ch. Eigler --- pushed as debuginfod-internal & uncontroversial -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25448] Extend debuginfod metrics
https://sourceware.org/bugzilla/show_bug.cgi?id=25448 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- https://sourceware.org/pipermail/elfutils-devel/2020q1/002555.html -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25448] Extend debuginfod metrics
https://sourceware.org/bugzilla/show_bug.cgi?id=25448 Frank Ch. Eigler changed: What|Removed |Added Status|UNCONFIRMED |RESOLVED Resolution|--- |FIXED --- Comment #3 from Frank Ch. Eigler --- pushed -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25739] client cache sometimes has incorrect mtimes for downloaded files
https://sourceware.org/bugzilla/show_bug.cgi?id=25739
Frank Ch. Eigler changed:
What|Removed |Added
CC||fche at redhat dot com
Resolution|--- |FIXED
Status|UNCONFIRMED |RESOLVED
--- Comment #1 from Frank Ch. Eigler ---
pushing this fix as obvious (in retrospect!)
diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index 74eb44372099..9f08a683ff15 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -1362,6 +1362,13 @@ handle_buildid_r_match (int64_t b_mtime,
throw archive_exception(a, "cannot extract file");
}
+ // Set the mtime so the fdcache file mtimes, even prefetched ones,
+ // propagate to future webapi clients.
+ struct timeval tvs[2];
+ tvs[0].tv_sec = tvs[1].tv_sec = archive_entry_mtime(e);
+ tvs[0].tv_usec = tvs[1].tv_usec = 0;
+ (void) futimes (fd, tvs); /* best effort */
+
diff --git a/tests/run-debuginfod-find.sh b/tests/run-debuginfod-find.sh
index bba04c1fe20a..7faad3317ddc 100755
--- a/tests/run-debuginfod-find.sh
+++ b/tests/run-debuginfod-find.sh
@@ -284,22 +284,27 @@ archive_test() {
buildid=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
-a $filename | grep 'Build ID' | cut -d ' ' -f 7`
test $__BUILDID = $buildid
+# check that timestamps are plausible - older than the near-present
(tmpdir mtime)
+test $filename -ot `pwd`
# run again to assure that fdcache is being enjoyed
filename=`testrun ${abs_top_builddir}/debuginfod/debuginfod-find
executable $__BUILDID`
buildid=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
-a $filename | grep 'Build ID' | cut -d ' ' -f 7`
test $__BUILDID = $buildid
+test $filename -ot `pwd`
filename=`testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo
$__BUILDID`
buildid=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
-a $filename | grep 'Build ID' | cut -d ' ' -f 7`
test $__BUILDID = $buildid
+test $filename -ot `pwd`
if test "x$__SOURCEPATH" != "x"; then
filename=`testrun ${abs_top_builddir}/debuginfod/debuginfod-find
source $__BUILDID $__SOURCEPATH`
hash=`cat $filename | sha1sum | awk '{print $1}'`
test $__SOURCESHA1 = $hash
+test $filename -ot `pwd`
fi
}
--
You are receiving this mail because:
You are on the CC list for the bug.
[Bug debuginfod/25722] debuginfod-find should be able to take an /bin/path in place of buildid
https://sourceware.org/bugzilla/show_bug.cgi?id=25722 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #3 from Frank Ch. Eigler --- pushed to master -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25728] Starting debubinfod with --port=0 is not documented
https://sourceware.org/bugzilla/show_bug.cgi?id=25728 Frank Ch. Eigler changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |FIXED CC||fche at redhat dot com --- Comment #3 from Frank Ch. Eigler --- pushed this to master, along with a tiny man page blurb. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug tools/25793] New: eu-readelf -w dumps incomplete even if debuginfod available
https://sourceware.org/bugzilla/show_bug.cgi?id=25793 Bug ID: 25793 Summary: eu-readelf -w dumps incomplete even if debuginfod available Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: tools Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- 0.179 eu-readelf -w /bin/ls on fedora31 x86-64 gladly produces a disassembly of the .eh_frame* content of the stripped binary. However, it is unable/unwilling to download debuginfo to fill in for example 'eu-readelf -winfo' data. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25797] New: DEBUGINFOD_URLS should accept scheme-free urls
https://sourceware.org/bugzilla/show_bug.cgi?id=25797 Bug ID: 25797 Summary: DEBUGINFOD_URLS should accept scheme-free urls Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- With commit 4c1de9608b, the client code requires a DEBUGINFOD_URL to be started with "http:" or "https:" or "file:", in order to give a successful result code. But libcurl also accepts "hostname:PORT" or "/path/to/something" as URLs, as do web browsers. Using these scheme-free heuristic URLs results in an -ENOENT error, even though the download succeeds. We should review the file: / http: heuristic processing to be similar or the same as libcurl, and let people abbreviate URLs a little more. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug tools/25793] eu-readelf -w dumps incomplete even if debuginfod available
https://sourceware.org/bugzilla/show_bug.cgi?id=25793 --- Comment #2 from Frank Ch. Eigler --- Is there another elfutils tool that would casually pretty-print dwarf related to a stripped binary? (binutils objdump --dwarf=follow-links does, but IMHO that's too verbose.) -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25448] Extend debuginfod metrics
https://sourceware.org/bugzilla/show_bug.cgi?id=25448 --- Comment #4 from Frank Ch. Eigler --- Martin, can you think of more metrics we should add later? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25448] Extend debuginfod metrics
https://sourceware.org/bugzilla/show_bug.cgi?id=25448 --- Comment #6 from Frank Ch. Eigler --- Response time (within debuginfod code proper, as opposed to the subsequent file transfer) is exposed with the http_responses_duration_milliseconds_* metrics. Total file transfer time would have to be calculated by extracting extra information/callbacks from libmicrohttpd. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25978] New: improve debuginfod prefetching / fdcache-management logic
https://sourceware.org/bugzilla/show_bug.cgi?id=25978 Bug ID: 25978 Summary: improve debuginfod prefetching / fdcache-management logic Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- The debuginfod fdcache-prefetch logic has been observed to show some degeneraacies in operation. Since fdcache evictions are done frequently, and freshly prefetched archive elements are put at the back of lru[], each eviction round can summarily nuke things that were just prefetched and are just going to be prefetched again. It would be better to have two lru lists, or being able to insert newly prefetched entries somewhere in the middle of the list rather than at the very very end. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug tools/26043] eu-addr2line debuginfo-path option with relative path doesn't find file:line information
https://sourceware.org/bugzilla/show_bug.cgi?id=26043 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #3 from Frank Ch. Eigler --- An strace of eu-addr2line would help show where it looked for debuginfo. BTW another option is to run $ debuginfod -F ../../demo/build & $ sleep 10 # give it a bit of time if this is first time running it $ DEBUGINFOD_URLS=http://localhost:8002/ eu-addr2line -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26125] New: during debuginfod cache cleanup, try harder to rmdir empty dirs
https://sourceware.org/bugzilla/show_bug.cgi?id=26125 Bug ID: 26125 Summary: during debuginfod cache cleanup, try harder to rmdir empty dirs Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- Cache cleanup sometimes leaves behind empty directories - lots of them. Should make a greater effort to rmdir the bad boys, in order to prune the directory size. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25509] Break a cyclic dependency by core packages
https://sourceware.org/bugzilla/show_bug.cgi?id=25509 --- Comment #10 from Frank Ch. Eigler --- Comment on attachment 12628 --> https://sourceware.org/bugzilla/attachment.cgi?id=12628 debuginfod: Add --disable-libdebuginfod and --enable-libdebuginfod=dummy. Looks workable. I don't know if we need a new --enable-libdebuginfod=dummy option, as opposed to it being simply the equivalent of --disable-libdebuginfod, or even --disable-debuginfod (to have a common configury option for both client & server, as now). But that's just a style nit. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25509] Break a cyclic dependency by core packages
https://sourceware.org/bugzilla/show_bug.cgi?id=25509 --- Comment #14 from Frank Ch. Eigler --- It's not just for testing purposes. It's to aid bootstrapping new OS versions, by reducing the transitive requirements of elfutils in the buildroot. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25509] Break a cyclic dependency by core packages
https://sourceware.org/bugzilla/show_bug.cgi?id=25509 --- Comment #16 from Frank Ch. Eigler --- (yup, misinterpreted what the "this" was you meant, sorry!) -- You are receiving this mail because: You are on the CC list for the bug.
[Bug backends/26176] Backend modules of elfutils are always linked to shared library
https://sourceware.org/bugzilla/show_bug.cgi?id=26176 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #4 from maarten --- Created attachment 12669 --> https://sourceware.org/bugzilla/attachment.cgi?id=12669&action=edit Add configure --enable-werror option (enabled by default) --- Comment #5 from maarten --- Created attachment 12670 --> https://sourceware.org/bugzilla/attachment.cgi?id=12670&action=edit add enable-static option to configure --- Comment #6 from maarten --- > I was actually going to propose we wouldn't build the static version by > default because I didn't think anybody is actually using it. > > What is your use case? And would it be a problem if we dropped building the > static libraries by default, and you had to explicitly build the package > with static libs? conan has an option to build libraries statically or shared. elfutils needed some patches to make this possible. > > - disable -Werror > > What errors/warnings are you seeing? ../source_subfolder/libdwfl/dwfl_build_id_find_elf.c: In function ‘__libdwfl_open_by_build_id’: ../source_subfolder/libdwfl/dwfl_build_id_find_elf.c:60:7: error: variable ‘n’ set but not used [-Werror=unused-but-set-variable] 60 | int n = snprintf (&id_name[sizeof "/.build-id/" - 1], | ^ cc1: all warnings being treated as errors > > > Also, it looks like `libasm.h` and `libdwfl.h` use types defined in > > `gelf.h`, but do not include the header. > > There was an issue in libasm.h which got fixed in elfutils-0.178-24-g287a1845 > libdwfl.h includes libdw.h which includes gelf.h so should be usable > standalone. `libasm.h` from 0.180 is only including system headers and . You're correct about `libdwfl.h`, that one is including `gelf.h` through `libdw.h`. > Could you attach concrete patches to this bug report? I've attached them. Using `--disable-werror` + `--enable-static=yes/no` allows us to build elfutils as static/shared libraries. --- Comment #7 from Frank Ch. Eigler --- test --- Comment #8 from Frank Ch. Eigler --- test2 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug general/26195] 0.180 + master: build fails
https://sourceware.org/bugzilla/show_bug.cgi?id=26195 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- libmicrohttpd commit 89d2ef97322ff5276e9a437e616a1e07529e0b26 Author: Christian Grothoff Date: Wed Apr 8 23:38:07 2020 +0200 define and use 'enum MHD_Result' (merge) introduced this API change. +Introduce `enum MHD_Result` for #MHD_YES/#MHD_NO to avoid using 'int' so much. +Note that this change WILL cause compiler warnings until (most) MHD callbacks +in application code change their return type from 'int' to 'enum MHD_Result'. +For conditional compilation, test for "MHD_VERSION >= 0x00097002" which sounds fine unless one uses -Werror or wants to run against multiple versions of the code. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug general/26195] 0.180 + master: build fails
https://sourceware.org/bugzilla/show_bug.cgi?id=26195 Frank Ch. Eigler changed: What|Removed |Added Last reconfirmed||2020-07-02 Assignee|unassigned at sourceware dot org |fche at redhat dot com Ever confirmed|0 |1 Status|UNCONFIRMED |ASSIGNED --- Comment #2 from Frank Ch. Eigler --- Created attachment 12678 --> https://sourceware.org/bugzilla/attachment.cgi?id=12678&action=edit proposed fix -- You are receiving this mail because: You are on the CC list for the bug.
[Bug general/26195] 0.180 + master: build fails
https://sourceware.org/bugzilla/show_bug.cgi?id=26195 --- Comment #6 from Frank Ch. Eigler --- I am loath to believe that "/bin/sh" is not a functional-in-perpetuity alias for a bourne shell on any unix system. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26599] New: eu-readelf -n works on /boot/vmlinuz-* but debuginfod-find doesn't
https://sourceware.org/bugzilla/show_bug.cgi?id=26599 Bug ID: 26599 Summary: eu-readelf -n works on /boot/vmlinuz-* but debuginfod-find doesn't Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- eu-readelf appears to unwrap the bzImage header on a vmlinuz kernel file, and can thus extract buildids, but debuginfod-find cannot: % eu-readelf -n /lib/modules/5.7.15-200.fc32.x86_64/vmlinuz [...] GNU 20 GNU_BUILD_ID Build ID: 8820b044b7717b53bd9a9795a1fbca152d2fae63 % debuginfod-find debuginfo /lib/modules/5.7.15-200.fc32.x86_64/vmlinuz Cannot extract build-id from /lib/modules/5.7.15-200.fc32.x86_64/vmlinuz: invalid `Elf' handle It would be handy to teach debuginfod-find how to use the elfutils entry point that unwraps bzImage. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug general/26632] New: eu-readelf doesn't grok /boot/vmlinuz-5.9.0* from fedora rawhide (.fc34) any more
https://sourceware.org/bugzilla/show_bug.cgi?id=26632 Bug ID: 26632 Summary: eu-readelf doesn't grok /boot/vmlinuz-5.9.0* from fedora rawhide (.fc34) any more Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: general Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- Something's different, not the same, this kind of feeling I can't explain % eu-readelf -n /boot/vmlinuz-5.9.0-[tab complete any one] eu-readelf: failed reading '...': not a valid ELF file % eu-readelf -n /boot/vmlinuz-5.8.0-[tab complete any one] [eu-readelf is happy] https://koji.fedoraproject.org/koji/buildinfo?buildID=1610585 probable cause: CONFIG_KERNEL_ZSTD=y -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26716] New: debuginfod: add yum-repo-URL as possible file source
https://sourceware.org/bugzilla/show_bug.cgi?id=26716 Bug ID: 26716 Summary: debuginfod: add yum-repo-URL as possible file source Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- offered this suggestion: Perhaps we could support another type of source file for debuginfod traversals: yum repo databases on the web. It would trade network traffic for local storage, and probably be much slower. debuginfod -R yum:https://download1.rpmfusion.org/free/fedora/releases/32/Everything/x86_64/debug which would: - (maybe support yum mirror metalink indirection) - periodically download $url/repodata/repomd.xml - parse it to locate and download the current active FOOBAR-primary.xml.gz - for each package/location in the xml, check the package/time against the index - if the package/location URL is unknown or fresher than in the database, - download the package - note its URL and process its archive contents in the database as usual - then throw away the archive (maybe subject to caching) for queries: - look up by the buildid as usual - if the source file comes back as http* URL - download the package archive - extract the needed file from the archive & return it to the web client - then throw away the archive (maybe subject to caching) for grooming: - download all the repomd.xml's known to debuginfod - from there, download the current complete URL package list for them all - delete from the database any http urls that are not anywhere in that set (this would have the effect of forgetting prior version files) alternately: - send an http HEAD request for each package URL we know of - any 404* type responses -> forget all related content from the database, as though source file were removed (this would cause many more outgoing requests, but keep older versions around as long as the upstream repo has them) As a preparatory step, debuginfod could learn to access treat http* URLs as a type of archive file. Instead of opening directly via libarchive, we'd libcurl-fetch the thing first, and feed the result to libarchive. (Maybe, don't even assume it's an archive; just fetch it, then let the existing debuginfod code dispatch, based on file name extension.) see also: https://linux.die.net/man/8/createrepo -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25461] add /etc/profile.d/* files for default $DEBUGINFOD_ env var settings for distro
https://sourceware.org/bugzilla/show_bug.cgi?id=25461 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #1 from Frank Ch. Eigler --- merged via https://sourceware.org/git/?p=elfutils.git;a=commitdiff;h=94708e9646017a7ef1c819df3b71bb1ddf8300fc -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26756] New: some more operational metrics
https://sourceware.org/bugzilla/show_bug.cgi?id=26756 Bug ID: 26756 Summary: some more operational metrics Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- debuginfod should make it easier to detect errors than just logfile scraping, e.g. when an -ENOSPC condition occurs. We should start exporting error counters to prometheus. This could be pretty simple, if e.g. associated with the creation of a libc_exception object. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26756] some more operational metrics
https://sourceware.org/bugzilla/show_bug.cgi?id=26756 --- Comment #1 from Frank Ch. Eigler --- also: the fdcache population/activity should be instrumented -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26756] some more operational metrics
https://sourceware.org/bugzilla/show_bug.cgi?id=26756 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #2 from Frank Ch. Eigler --- commit 01348c28a2580e8 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26775] New: debuginfod: speed up grooming
https://sourceware.org/bugzilla/show_bug.cgi?id=26775 Bug ID: 26775 Summary: debuginfod: speed up grooming Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- The periodic (nightly) grooming pass is pretty slow on large indexes where many files go missing (so many sqlite delete operations, some with intentional CASCADE-ing effects on other tables). Grooming cannot happen concurrently with indexing, so making it fast is important. We probably need a few more prometheus metrics for this too. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26810] New: handle duplicate buildid data sources, with some stale
https://sourceware.org/bugzilla/show_bug.cgi?id=26810
Bug ID: 26810
Summary: handle duplicate buildid data sources, with some stale
Product: elfutils
Version: unspecified
Status: NEW
Severity: normal
Priority: P2
Component: debuginfod
Assignee: unassigned at sourceware dot org
Reporter: fche at redhat dot com
CC: elfutils-devel at sourceware dot org
Target Milestone: ---
On a large debuginfod test server, we observed that it's possible to have the
same rpm/buildid identified with multiple locations, if the same content is
accessible along several paths. That part's fine, but if some of the paths
change - get removed - and before the next grooming pass, debuginfod gets
confused. That's because multiple matches are tried in series (a sqlite scan
loop), but an exception ("file not found") from one iteration can
unintentionally cancel the scan.
So we just need to put a try/catch around the handle_buildid_* function, to
assure that other locations of the same content are attempted, regardless of
exceptions.
--
You are receiving this mail because:
You are on the CC list for the bug.
[Bug debuginfod/26775] debuginfod: speed up grooming
https://sourceware.org/bugzilla/show_bug.cgi?id=26775 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #1 from Frank Ch. Eigler --- commit 2237828857f06 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26810] handle duplicate buildid data sources, with some stale
https://sourceware.org/bugzilla/show_bug.cgi?id=26810 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #1 from Frank Ch. Eigler --- commit f5015aebdb33 -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/25366] debuginfod scan: more progress/status
https://sourceware.org/bugzilla/show_bug.cgi?id=25366 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #1 from Frank Ch. Eigler --- elfutils 0.182 came out with several new scanning / grooming progress metrics -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26125] during debuginfod cache cleanup, try harder to rmdir empty dirs
https://sourceware.org/bugzilla/show_bug.cgi?id=26125 --- Comment #3 from Frank Ch. Eigler --- Yeah, a speculative rmdir is probably just fine. Martin, there is no difference between a pure client cache, and a debuginfod server's own federation cache. They are both exactly the same thing. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26125] during debuginfod cache cleanup, try harder to rmdir empty dirs
https://sourceware.org/bugzilla/show_bug.cgi?id=26125 --- Comment #5 from Frank Ch. Eigler --- Yeah, I suppose there is that race possibility. One way to fix it is to use something like file locks, such as a flock(2) on a designated file such as $CACHE/cache_clean_interval_s. During aging/cleanup, hold a LOCK_EX. During normal operation (creation of build-id subdirectories & files), hold a LOCK_SH to permit multiple clients to work independently of one another. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/26125] during debuginfod cache cleanup, try harder to rmdir empty dirs
https://sourceware.org/bugzilla/show_bug.cgi?id=26125 --- Comment #7 from Frank Ch. Eigler --- > I wonder if there is some way around it using temp names for the directories, > like we do for the actual file names, so that we can atomically rename them > in-place. I couldn't think of one without other drawbacks. There can be races between two clients downloading info for the same buildid, thus creating two temp dirs. One of them would not be able to rename to the final name. Another simplish approach would be to check the directory mtime before rmdir, to ensure it's old, plus adding a retry loop at cache-insertion, in case a client loses the maxage_s race with a cleanup. But it would only lose it once (due to the mtime check). -- You are receiving this mail because: You are on the CC list for the bug.
[Bug tools/27041] eu-addr2line to support eu-stack -b format (address with {build-id})
https://sourceware.org/bugzilla/show_bug.cgi?id=27041 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- by the way, are you aware of "eu-stack -v ...", which does the addr2line resolution internally? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27089] SEGV
https://sourceware.org/bugzilla/show_bug.cgi?id=27089 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |NOTABUG CC||fche at redhat dot com Status|UNCONFIRMED |RESOLVED --- Comment #2 from Frank Ch. Eigler --- invalid -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27089] SEGV
https://sourceware.org/bugzilla/show_bug.cgi?id=27089 Frank Ch. Eigler changed: What|Removed |Added Resolution|NOTABUG |INVALID -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27092] New: handle low memory conditions better
https://sourceware.org/bugzilla/show_bug.cgi?id=27092 Bug ID: 27092 Summary: handle low memory conditions better Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- When running on low-RAM computers against larger datasets, debuginfod has been observed to suffer OOM type sudden death. The tool should try harder to predict and recover from low memory conditions. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27277] New: support HEAD query for debuginfod content probe requests
https://sourceware.org/bugzilla/show_bug.cgi?id=27277 Bug ID: 27277 Summary: support HEAD query for debuginfod content probe requests Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org, woodard at redhat dot com Target Milestone: --- There appear to exist use cases that intend to simply check for the existence of content in a debuginfod server, without actually downloading it. In HTTP land, the HEAD operation is the natural expression of this. We could support this in the webapi, and give options to debuginfod-find and the client API to use it. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27323] New: concurrency during grooming/scanning
https://sourceware.org/bugzilla/show_bug.cgi?id=27323 Bug ID: 27323 Summary: concurrency during grooming/scanning Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- Further to bug #26775, which made grooming interruptible, we need more. Grooming involves some long-running operations, like the database-stats count(*) queries, which can take MINUTES of run time on a large server. sqlite's single-thread-per-connection model means that during those times, web queries cannot be served, and just pile up. We need to add a little more concurrency to debuginfod, at the sqlite connection level. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27323] concurrency during grooming/scanning
https://sourceware.org/bugzilla/show_bug.cgi?id=27323 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #1 from Frank Ch. Eigler --- commit 6362941714548d93a348239ca8b8038005bd57cd Author: Frank Ch. Eigler Date: Tue Feb 2 16:49:19 2021 -0500 PR27323 debuginfod: improve query concurrency with grooming -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27092] handle low memory conditions better
https://sourceware.org/bugzilla/show_bug.cgi?id=27092 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #1 from Frank Ch. Eigler --- commit aa121ea7f19e002c5e4738616342cd1d7be1bfd2 Author: Frank Ch. Eigler Date: Thu Feb 4 20:31:56 2021 -0500 PR27092: debuginfod low-memory handling -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27399] dpkg-deb/lzma error when indexing .debs
https://sourceware.org/bugzilla/show_bug.cgi?id=27399 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com Status|NEW |WAITING --- Comment #1 from Frank Ch. Eigler --- Can you check whether dpkg-deb is decompressing all of its content onto some RAM-backed filesystem, and running out of space via that (or via consuming machine free ram)? You can try a few things: - run debuginfod with a smaller concurrency limit (-c NNN), because the decompression etc. activities aggressively use all your CPUs and thus #CPU * memory, if they can - instead of 'debuginfod -U' (which uses dpkg-deb as the decompression streamer), use % debuginfod -Z.deb="(bsdtar -O -x -f - data.tar.xz)<" which causes deb files to be processed with libarchive's frontend (or equivalently, temporarily rename /usr/bin/dpkg-deb while starting debuginfod, so it makes the same inference) - monitor resource usage - particularly ram - during the indexing process - try running elfutils 0.183 debuginfod, which does a touch more filesystem-space monitoring, related self-protection, more prometheus error metrics -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27399] dpkg-deb/lzma error when indexing .debs
https://sourceware.org/bugzilla/show_bug.cgi?id=27399 --- Comment #2 from Frank Ch. Eigler --- Sergio, are the messages you're worried about happening AT THE MOMENT of a start / stop? Or just intermittently sometime during the normal operating lifetime of the process? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/27405] New: libdw_get_srcfiles should not imply srclines
https://sourceware.org/bugzilla/show_bug.cgi?id=27405 Bug ID: 27405 Summary: libdw_get_srcfiles should not imply srclines Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: libdw Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- In debuginfod, we use _get_srcfiles to fetch file names only. We do not care about srclines, so libdw's effort in allocating all that memory, and filling it in, is just a waste. Let's make srcfiles great by skipping the per-line data. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27399] dpkg-deb/lzma error when indexing .debs
https://sourceware.org/bugzilla/show_bug.cgi?id=27399 --- Comment #6 from Frank Ch. Eigler --- OK, my best guess is a transient low-memory condition. Just today we found bug #27405, which can cause some impressive momentary memory binges in elfutils. Reducing concurrency is one way to limit its impact, as you found. Giving the VM more memory is another: I run it on 16GB 8CPU class VMs, scanning even many huge debuginfo files, without a complaint. It might also help reassure here is that if you let debuginfod keep going, it'll know that it didn't finish indexing those problematic files, and will just retry later. In the long term (past rescan time), such momentary ENOMEMs are not particularly harmful. I can't think of any debuginfod per se changes or bugs in effect here. Maybe scale the default concurrency more conservatively to machine RAM rather than solely CPU count? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27413] New: use bsdtar for .deb extraction too
https://sourceware.org/bugzilla/show_bug.cgi?id=27413 Bug ID: 27413 Summary: use bsdtar for .deb extraction too Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: fche at redhat dot com CC: elfutils-devel at sourceware dot org Target Milestone: --- Back during bug #25583, we transitioned rpm unpacking to bsdtar (libarchive), but left deb unpacking to dpkg-deb by default. One factor in this was the inconsistent naming convention of the "data.tar*" inner tarball: it can be .xz or .bz2 or others. Given a few reports of intermittent crashes via dpkg-deb (bug #27399), and the ability of bsdtar to take glob patterns, it's probably time to let bsdtar handle .debs too (and .ipkg). The extracting pipe command would be: (bsdtar -O -x -f - 'data.tar*')< -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27413] use bsdtar for .deb extraction too
https://sourceware.org/bugzilla/show_bug.cgi?id=27413 Frank Ch. Eigler changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #1 from Frank Ch. Eigler --- commit f146d5ec8e2eee566df6eb4c939cf29d779fc08c (HEAD -> master, origin/master, origin/HEAD) Author: Frank Ch. Eigler Date: Sun Feb 14 16:02:05 2021 -0500 PR27413: use bsdtar to unpack deb-related formats -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27399] dpkg-deb/lzma error when indexing .debs
https://sourceware.org/bugzilla/show_bug.cgi?id=27399 Frank Ch. Eigler changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #8 from Frank Ch. Eigler --- Resolved under bug #27413 by deprecating use of dpkg-deb in favor of bsdtar. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27532] debuginfod should ask the user for permission before downloading files
https://sourceware.org/bugzilla/show_bug.cgi?id=27532 Frank Ch. Eigler changed: What|Removed |Added CC||fche at redhat dot com --- Comment #1 from Frank Ch. Eigler --- Getting all interactive from within a library such as libdebuginfod client is not really good practice nor generally practical. I'd think that the setting of the environment variable & call into the library embodies consent by the enveloping application. -- You are receiving this mail because: You are on the CC list for the bug.
