Re: [PATCH] libelf: Fix unaligned d_off offsets for input sections with large alignments
Hi Andrei,
On Mon, 2021-06-28 at 18:26 -0700, Andrei Homescu wrote:
> The mkl_memory_patched.o object inside the libmkl_core.a library from
> the Intel Math Kernel Library version 2018.2.199 has this section
> with an alignment of 4096 and offset of 0xb68:
> [ 2] .data PROGBITS 000b68 011000 00 WA 0 0 4096
>
> Reading this file with libelf and trying to write it back to disk triggers
> the following sequence of events:
> 1) code in elf_getdata.c clamps d_align for this section's data buffer
>to the section's offset
> 2) code in elf32_updatenull.c checks if the alignment is a power of two
>and incorrectly returns an error
>
> This commit fixes this corner case by increasing the alignment to the
> next power of two after the clamping, so the check passes.
>
> A test that reproduces this bug using strip is also included.
Thanks for this perfect patch, including and excellent explanation of
the issue and a testcase. I have only a minor suggestion:
> --- /dev/null
> +++ b/tests/run-strip-largealign.sh
> @@ -0,0 +1,34 @@
> +#! /bin/sh
> +# Copyright (C) 2011 TODO.
Should we make this Copyright (C) 2021 Andrei Homescu
or Copyright (C) 2021 Immunant, Inc.
> +# This file is part of elfutils.
> +#
> +# This file is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published
> by
> +# the Free Software Foundation; either version 3 of the License, or
> +# (at your option) any later version.
> +#
> +# elfutils is distributed in the hope that it will be useful, but
> +# WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <
> http://www.gnu.org/licenses/>.
> +#
> +
> +. $srcdir/test-subr.sh
> +
> +# = testfile-largealign.S =
> +# section .data
> +# align 4096
> +# dd 0x12345678
> +#
> +# nasm -f elf64 -o testfile-largealign.o testfile-largealign.S
> +
> +infile=testfile-largealign.o
> +outfile=$infile.stripped
> +
> +testfiles $infile
> +tempfiles $outfile
> +
> +testrun ${abs_top_builddir}/src/strip -o $outfile $infile
The testcase already fails before the patch and succeeds after, but it
would be nice to double check the output with elflint just in case.
Shall we add:
testrun ${abs_top_builddir}/src/elflint --gnu $outfile
Thanks,
Mark
[Bug debuginfod/28034] debuginfod server: preserve + etc. elements in incoming webapi url
https://sourceware.org/bugzilla/show_bug.cgi?id=28034 Mark Wielaard changed: What|Removed |Added CC||mark at klomp dot org --- Comment #2 from Mark Wielaard --- Could we do both? Start properly %-escaping outgoing URLs from libdebuginfod client (and documenting that this is the correct/expected way to do GET requests). And only do unescaping in libmicrohttpd if the URL contains a %. This assumes that + -> ' ' escaping is normally unwanted (because to proper way is to encode a space as %20). But maybe I am missing some subtlety or maybe we have no way of stopping libmicrohttpd of "unescaping"? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/28034] debuginfod server: preserve + etc. elements in incoming webapi url
https://sourceware.org/bugzilla/show_bug.cgi?id=28034 --- Comment #3 from Frank Ch. Eigler --- > This assumes that + -> ' ' escaping is normally unwanted (because to proper > way > is to encode a space as %20). I can't find it in current RFCs but traditional use of '+' to encode ' ' is still very widespread, esp. in querystrings. And unfortunately libmicrohttpd does the "+" -> " " transform practically unconditionally. So I suspect we have no choice but to %HEX urlencode all RFC3986-"reserved" (or not "unreserved") characters. I'd use %20 for space too. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27983] debuginfod client should eliminate duplicate outgoing server urls
https://sourceware.org/bugzilla/show_bug.cgi?id=27983 Noah Sanci changed: What|Removed |Added Assignee|unassigned at sourceware dot org |nsanci at redhat dot com CC||nsanci at redhat dot com -- You are receiving this mail because: You are on the CC list for the bug.
[Bug tools/28043] Add LSDA decoding to eu-readelf
https://sourceware.org/bugzilla/show_bug.cgi?id=28043 Mark Wielaard changed: What|Removed |Added CC||mark at klomp dot org --- Comment #2 from Mark Wielaard --- Note that eu-readelf already tries to decode the .gcc_exception_table when using -e, --exception. And the .eh_frame decoding already does get the LSDA pointer encoding from the CIE and should show the pointer for any CFI that has one. But there is nothing that nicely ties them together. -- You are receiving this mail because: You are on the CC list for the bug.
Re: PR27711
Hi Noah,
If possible please add a commit message and sent the patch using git
send-email or git format-patch. Feel free to use the description of
https://sourceware.org/bugzilla/show_bug.cgi?id=27711#c0 that Frank
wrote as commit message. It clearly explains the intention.
On Thu, 2021-07-01 at 16:38 -0400, Noah Sanci via Elfutils-devel wrote:
> diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
> index 286c910a..5afbafcd 100644
> --- a/debuginfod/ChangeLog
> +++ b/debuginfod/ChangeLog
> @@ -1,3 +1,8 @@
> +2021-07-01 Noah Sanci
> + PR27711
> + * debuginfod.cxx (groom): Allowed the use of regexes during the
> + grooming process.
Slightly better would be to describe the changes. e.g.
(options): Add --regex-groon, -r option.
(regex_groom): New static bool defaults to false.
(parse_opt): Handle 'r' option by setting regex_groom to true.
(groom): Introduce and use reg_include and reg_exclude.
> 2021-06-03 Frank Ch. Eigler
>
> PR27863
> diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
> index 543044c6..4f7fd2d5 100644
> --- a/debuginfod/debuginfod.cxx
> +++ b/debuginfod/debuginfod.cxx
> @@ -360,6 +360,7 @@ static const struct argp_option options[] =
> { "database", 'd', "FILE", 0, "Path to sqlite database.", 0 },
> { "ddl", 'D', "SQL", 0, "Apply extra sqlite ddl/pragma to connection.", 0
> },
> { "verbose", 'v', NULL, 0, "Increase verbosity.", 0 },
> + { "regex-groom", 'r', NULL, 0,"Uses regexes from -I and -X
> arguments to groom the database.",0},
> #define ARGP_KEY_FDCACHE_FDS 0x1001
> { "fdcache-fds", ARGP_KEY_FDCACHE_FDS, "NUM", 0, "Maximum number
> of archive files to keep in fdcache.", 0 },
> #define ARGP_KEY_FDCACHE_MBS 0x1002
> @@ -407,6 +408,7 @@ static map scan_archives;
> static vector extra_ddl;
> static regex_t file_include_regex;
> static regex_t file_exclude_regex;
> +static bool regex_groom = false;
> static bool traverse_logical;
> static long fdcache_fds;
> static long fdcache_mbs;
> @@ -527,6 +529,9 @@ parse_opt (int key, char *arg,
>if (rc != 0)
> argp_failure(state, 1, EINVAL, "regular expression");
>break;
> +case 'r':
> + regex_groom = true;
> + break;
> case ARGP_KEY_FDCACHE_FDS:
>fdcache_fds = atol (arg);
>break;
> @@ -3249,8 +3254,11 @@ void groom()
>int64_t fileid = sqlite3_column_int64 (files, 1);
>const char* filename = ((const char*) sqlite3_column_text
> (files, 2) ?: "");
>struct stat s;
> + bool reg_include = !regexec (&file_include_regex, filename, 0, 0, 0);
> + bool reg_exclude = !regexec (&file_exclude_regex, filename, 0, 0, 0);
> +
>rc = stat(filename, &s);
> - if (rc < 0 || (mtime != (int64_t) s.st_mtime))
> + if ( (regex_groom && reg_exclude && !reg_include) || rc < 0 ||
> (mtime != (int64_t) s.st_mtime) )
> {
OK, so we groom the file as before
rc < 0 || (mtime != (int64_t) s.st_mtime) )
But also (if -r is given) if the file matches the exclude regexp, but
not the include one.
So if I read this right, an exclude regexp match means groom that file,
but an include rexexp match means, don't groom (except if it
disappeared on itself).
>if (verbose > 2)
> obatched(clog) << "groom: forgetting file=" << filename
> << " mtime=" << mtime << endl;
> @@ -3261,7 +3269,6 @@ void groom()
> }
>else
> inc_metric("groomed_total", "decision", "fresh");
> -
>if (sigusr1 != forced_rescan_count) // stop early if scan triggered
> break;
> }
Spurious line removed?
> diff --git a/doc/debuginfod.8 b/doc/debuginfod.8
> index 1ba42cf6..1adf703a 100644
> --- a/doc/debuginfod.8
> +++ b/doc/debuginfod.8
> @@ -159,6 +159,9 @@ scan, independent of the rescan time (including if
> it was zero),
> interrupting a groom pass (if any).
>
> .TP
> +.B "\-r"
> +Apply the -I and -X during groom cycles, so that files excluded by
> the regexes are removed from the index. These parameters are in
> addition to what normally qualifies a file for grooming, not a
> replacement.
> +
OK. That matches my reading of the code. Good.
> .B "\-g SECONDS" "\-\-groom\-time=SECONDS"
> Set the groom time for the index database. This is the amount of time
> the grooming thread will wait after finishing a grooming pass before
> diff --git a/tests/ChangeLog b/tests/ChangeLog
> index d8fa97fa..346b9e6e 100644
> --- a/tests/ChangeLog
> +++ b/tests/ChangeLog
> @@ -1,3 +1,8 @@
> +2021-07-01 Noah Sanci
> + PR2711
> + * run-debuginfod-find.sh: Added test case for grooming the database
> + using regexes.
> +
> 2021-06-16 Frank Ch. Eigler
>
> * run-debuginfod-find.sh: Fix intermittent groom/stale failure,
> diff --git a/tests/run-debuginfod-find.sh b/tests/run-debuginfod-find.sh
> index 456dc2f8..bd78bf46 100755
> --- a/tests/run-debuginfod-find.sh
> +++ b/tests/run-debuginfod-find.sh
> @@ -36,13 +36,14 @@ expor
[Bug debuginfod/27950] separate common "debuginfod-client" docs into a new man page
https://sourceware.org/bugzilla/show_bug.cgi?id=27950 Alice Zhang changed: What|Removed |Added CC||alizhang at redhat dot com Assignee|unassigned at sourceware dot org |alizhang at redhat dot com -- You are receiving this mail because: You are on the CC list for the bug.
[Bug debuginfod/27950] separate common "debuginfod-client" docs into a new man page
https://sourceware.org/bugzilla/show_bug.cgi?id=27950 Alice Zhang changed: What|Removed |Added Status|NEW |ASSIGNED -- You are receiving this mail because: You are on the CC list for the bug.
Re: [PATCH] libelf: Fix unaligned d_off offsets for input sections with large alignments
Hi,
On Wed, Jul 7, 2021 at 5:02 AM Mark Wielaard wrote:
> Hi Andrei,
>
> On Mon, 2021-06-28 at 18:26 -0700, Andrei Homescu wrote:
> > The mkl_memory_patched.o object inside the libmkl_core.a library from
> > the Intel Math Kernel Library version 2018.2.199 has this section
> > with an alignment of 4096 and offset of 0xb68:
> > [ 2] .data PROGBITS 000b68 011000 00 WA 0 0 4096
> >
> > Reading this file with libelf and trying to write it back to disk
> triggers
> > the following sequence of events:
> > 1) code in elf_getdata.c clamps d_align for this section's data buffer
> >to the section's offset
> > 2) code in elf32_updatenull.c checks if the alignment is a power of two
> >and incorrectly returns an error
> >
> > This commit fixes this corner case by increasing the alignment to the
> > next power of two after the clamping, so the check passes.
> >
> > A test that reproduces this bug using strip is also included.
>
> Thanks for this perfect patch, including and excellent explanation of
> the issue and a testcase. I have only a minor suggestion:
>
> > --- /dev/null
> > +++ b/tests/run-strip-largealign.sh
> > @@ -0,0 +1,34 @@
> > +#! /bin/sh
> > +# Copyright (C) 2011 TODO.
>
> Should we make this Copyright (C) 2021 Andrei Homescu
> or Copyright (C) 2021 Immunant, Inc.
>
Sorry about that, I intended to resolve that TODO myself and then missed
it.
I wrote this patch on behalf of someone else, so the copyright line should
be:
Copyright (C) 2021 Runsafe Security, Inc.
> > +# This file is part of elfutils.
> > +#
> > +# This file is free software; you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License as published
> > by
> > +# the Free Software Foundation; either version 3 of the License, or
> > +# (at your option) any later version.
> > +#
> > +# elfutils is distributed in the hope that it will be useful, but
> > +# WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program. If not, see <
> > http://www.gnu.org/licenses/>.
> > +#
> > +
> > +. $srcdir/test-subr.sh
> > +
> > +# = testfile-largealign.S =
> > +# section .data
> > +# align 4096
> > +# dd 0x12345678
> > +#
> > +# nasm -f elf64 -o testfile-largealign.o testfile-largealign.S
> > +
> > +infile=testfile-largealign.o
> > +outfile=$infile.stripped
> > +
> > +testfiles $infile
> > +tempfiles $outfile
> > +
> > +testrun ${abs_top_builddir}/src/strip -o $outfile $infile
>
> The testcase already fails before the patch and succeeds after, but it
> would be nice to double check the output with elflint just in case.
> Shall we add:
>
> testrun ${abs_top_builddir}/src/elflint --gnu $outfile
>
Sounds good, no objection from me.
Should I submit an updated version of the patch, or can you add this on
your end?
Andrei
Working with ELF already loaded in memory
Hello, We are exploring using libdw in XRT, https://github.com/Xilinx/XRT. In the use case we have, the ELF (a shared library) is already loaded in memory as data by an XRT application. In XRT we would like to walk through the DWARF/ELF information embedded in this shared library. Going through the libdw it appears that all APIs require either a file handle or a file name of the ELF object to create a session. Since we do not have access to the ELF file -- but rather the ELF file contents are already loaded in memory -- is there any other mechanism to create a session for extracting DWARF information using libdw? Thanks, -Sonal
