[Bug debuginfod/27673] New: [debuginfod] Handle source requests for same buildid more efficiently
https://sourceware.org/bugzilla/show_bug.cgi?id=27673 Bug ID: 27673 Summary: [debuginfod] Handle source requests for same buildid more efficiently Product: elfutils Version: unspecified Status: NEW Severity: enhancement Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: vries at gcc dot gnu.org CC: elfutils-devel at sourceware dot org Target Milestone: --- Consider the following debuginfod usage session from gdb (reported in gdb PR27667): ... $ time gdb -batch -q ./mi-sym-info -ex start -ex "interpreter-exec mi 111-symbol-info-functions" > OUTPUT TRYING DEBUGINFO FOR: /data/gdb_versions/devel/system-supplied DSO at 0x77fca000 BUILDID: e77a560007595d823e2c5a34697c0b0ae1ffc767 TOOK: 0.84 seconds RESULT fd: 16 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/config/i386/sfp-exceptions.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.601374 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/libgcc2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.486450 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/libgcc2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.493712 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/libgcc2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.487850 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/libgcc2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.488300 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/libgcc2.c BUILDID: 7076274fc469c32fa8f4201f467447da3ad2b6be TOOK: 0.493761 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/addtf3.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.486202 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/divtf3.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.485941 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/eqtf2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.477989 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/extenddftf2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.487699 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/fixtfdi.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.488412 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/fixtfsi.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.496897 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/floatditf.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.492576 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/floatsitf.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.477747 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/floatunsitf.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.480452 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/getf2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.487295 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/letf2.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.479365 seconds RESULT fd: -2 TRYING SOURCE: /home/abuild/rpmbuild/BUILD/gcc-10.2.1+git1331/obj-x86_64-suse-linux/x86_64-suse-linux/libgcc/../../../libgcc/soft-fp/multf3.c BUILDID: d34928e1b5c7bde5e269e47349ade7369a3efa83 TOOK: 0.486071
[Bug debuginfod/27673] [debuginfod] Handle source requests for same buildid more efficiently
https://sourceware.org/bugzilla/show_bug.cgi?id=27673 Tom de Vries changed: What|Removed |Added Status|WAITING |UNCONFIRMED Ever confirmed|1 |0 --- Comment #3 from Tom de Vries --- (In reply to Frank Ch. Eigler from comment #2) > The subject line "handle sources ... from same buildid ... more efficiently" > suggests a caching opportunity. The debuginfod --fdcache-prefetch parameter > is probably the gadget of interest here. The opensuse operators could > experiment with a much larger than default figure. (You can test it locally > by running debuinfod for yourself, against a local set of relevant -debug* > RPMs.) The time it takes for the client to see the response of the server to the request consist of: - time for request to travel to server (latency) - time for server to react to request - time for answer to travel back to client (latency again) I've looked at the documentation of the option fdcache-prefetch, and AFAIU this improves "time for server to react to request". Instead, the focus of this PR is to hide latency (since that's where the majority of the time is spent) by using more bandwidth. Proposal a: debuginfod: - when receiving a source request and ENOENT, send as reply the list of available files for the buildid client: - when receiving a list of available files for a buildid, store it and use it to reply to source requests related to the buildid. That is, if the file is not in the list, reply with -2. Otherwise, send a request to debuginfod, and expect it to succeed. Proposal b: debuginfod: - when receiving a source request, send a package with the sources for that buildid to the client. client: - when receiving a package with the sources for a buildid, store them and use them to reply to source requests related to the buildid. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27673] [debuginfod] Handle source requests for same buildid more efficiently
https://sourceware.org/bugzilla/show_bug.cgi?id=27673 --- Comment #5 from Tom de Vries --- (In reply to Frank Ch. Eigler from comment #4) > > debuginfod: > > - when receiving a source request and ENOENT, send as reply the list of > > available files for the buildid > > > > client: > > - when receiving a list of available files for a buildid, store it and > > use it to reply to source requests related to the buildid. That is, > > if the file is not in the list, reply with -2. Otherwise, send a > > request to debuginfod, and expect it to succeed. > > Interesting. A more first-class solution could be a new webapi to > enumerate source files: a "/buildid/HEXCODE/sourcelist" query that > returns a structured piece of data. This can be computed by debuginfod > fairly rapidly. The client could cache that and use it to drive a > negative-cache hit on a subsequent source query. Agreed, it would be good to have this kind of functionality available more generically. Then the client (preferably the elfutils client itself using some setting, otherwise the part using the elfutils client) could upon the first source request send a sourcelist request in parallel, to have the latencies of the two requests overlap. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27673] [debuginfod] Handle source requests for same buildid more efficiently
https://sourceware.org/bugzilla/show_bug.cgi?id=27673 Tom de Vries changed: What|Removed |Added Blocks||27678 Referenced Bugs: https://sourceware.org/bugzilla/show_bug.cgi?id=27678 [Bug 27678] [gdb/symtab] Use debuginfod sourcelist api to test for file existence -- You are receiving this mail because: You are on the CC list for the bug.
