Range lists, zero-length functions, linker gc
It is being discussed on llvm-dev
(https://lists.llvm.org/pipermail/llvm-dev/2020-May/141885.html
https://groups.google.com/forum/#!topic/llvm-dev/i0DFx6YSqDA)
what linkers should do regarding relocations referencing dropped functions (due
to section group rules, --gc-sections, /DISCARD/, etc) in .debug_*
As an example:
__attribute__((section(".text.x"))) void f1() { }
__attribute__((section(".text.x"))) void f2() { }
int main() { }
Some .debug_* sections are relocated by R_X86_64_64 referencing undefined
symbols (the STT_SECTION
symbols are collected):
0x0043: DW_TAG_subprogram [2]
## relocated by .text.x + 10
DW_AT_low_pc [DW_FORM_addr] (0x0010 ".text.x")
DW_AT_high_pc [DW_FORM_data4] (0x0006)
DW_AT_frame_base [DW_FORM_exprloc] (DW_OP_reg6 RBP)
DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x002c] =
"_Z2f2v")
DW_AT_name [DW_FORM_strp] ( .debug_str[0x0033] =
"f2")
With ld --gc-sections:
* DW_AT_low_pc [DW_FORM_addr] in .debug_info are resolved to 0 + addend
This can cause overlapping address ranges with normal text sections.
{{overlap}}
* [beginning address offset, ending address offset) in .debug_ranges are
resolved to 1 (ignoring addend).
See bfd/reloc.c (behavior introduced in
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e4067dbb2a3368dbf908b39c5435c84d51abc9f3
)
[0, 0) cannot be used because it terminates the list entry.
[-1, -1) cannot be used because -1 represents a base address selection entry
which will affect
subsequent address offset pairs.
* .debug_loc address offset pairs have similar problem to .debug_ranges
* In DWARF v5, the abnormal values can be in a separate section .debug_addr
---
To save your time, I have a summary of the discussions. I am eager to know what
you think
of the ideas from binutils/gdb/elfutils's perspective.
* {{reserved_address}} Paul Robinson wants to propose that DWARF v6 reserves a
special address.
All (undef + addend) in .debug_* are resolved to -1.
We have to ignore the addend. With __attribute__((section(".text.x"))),
the address offset pair may be something like [.text.x + 16, .text.x + 24)
I have to resolve the whole (.text.x + 16) to the special value.
(undef + addend) in pre-DWARF v5 .debug_loc and .debug_ranges are resolved to
-2
(0 and -1 cannot be used due to the reasons above).
* Refined formula for a relocated value in a non-SHF_ALLOC section:
if is_defined(sym)
return addr(sym) + addend
if relocated_section is .debug_ranges or .debug_loc
return -2 # addend is intentionally ignored
// Every DWARF v5 section falls here
return -1 {{zero}}
* {{zero}} Can we resolve (undef + addend) to 0?
https://lists.llvm.org/pipermail/llvm-dev/2020-May/141967.html
> while it might not be an issue for ELF, DWARF would want a standard that's
fairly resilient to
> quirky/interesting use cases (admittedly - such platforms could equally
want to make their
> executable code way up in the address space near max or max - 1, etc?).
Question: is address 0 meaningful for code in some binary formats?
* {{overlap}} The current situation (GNU ld, gold, LLD): (undef + addend) in
.debug_* are resolved to addend.
For an address offset pair like [.text + 0, .text + 0x10010), if the ending
address offset is large
enough, it may overlap with a normal text address range (for example
[0x1, *))
This can cause problems in debuggers. How does gdb solve the problem?
* {{nonalloc}} Linkers resolve (undef + addend) in non-SHF_ALLOC sections to
`addend`. For non-debug sections (open-ended), do we have needs resolving such
values to `base` or `base+addend` where base is customizable?
(https://lists.llvm.org/pipermail/llvm-dev/2020-May/141956.html )
Re: Range lists, zero-length functions, linker gc
On 2020-05-31, Fangrui Song wrote:
It is being discussed on llvm-dev
(https://lists.llvm.org/pipermail/llvm-dev/2020-May/141885.html
https://groups.google.com/forum/#!topic/llvm-dev/i0DFx6YSqDA)
what linkers should do regarding relocations referencing dropped functions (due
to section group rules, --gc-sections, /DISCARD/, etc) in .debug_*
As an example:
__attribute__((section(".text.x"))) void f1() { }
__attribute__((section(".text.x"))) void f2() { }
int main() { }
Some .debug_* sections are relocated by R_X86_64_64 referencing undefined
symbols (the STT_SECTION
symbols are collected):
0x0043: DW_TAG_subprogram [2]
## relocated by .text.x + 10
DW_AT_low_pc [DW_FORM_addr] (0x0010 ".text.x")
DW_AT_high_pc [DW_FORM_data4] (0x0006)
DW_AT_frame_base [DW_FORM_exprloc] (DW_OP_reg6 RBP)
DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x002c] =
"_Z2f2v")
DW_AT_name [DW_FORM_strp] ( .debug_str[0x0033] =
"f2")
With ld --gc-sections:
* DW_AT_low_pc [DW_FORM_addr] in .debug_info are resolved to 0 + addend
This can cause overlapping address ranges with normal text sections.
{{overlap}}
* [beginning address offset, ending address offset) in .debug_ranges are
resolved to 1 (ignoring addend).
See bfd/reloc.c (behavior introduced in
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e4067dbb2a3368dbf908b39c5435c84d51abc9f3
)
[0, 0) cannot be used because it terminates the list entry.
[-1, -1) cannot be used because -1 represents a base address selection entry
which will affect
subsequent address offset pairs.
* .debug_loc address offset pairs have similar problem to .debug_ranges
* In DWARF v5, the abnormal values can be in a separate section .debug_addr
---
To save your time, I have a summary of the discussions. I am eager to know what
you think
of the ideas from binutils/gdb/elfutils's perspective.
* {{reserved_address}} Paul Robinson wants to propose that DWARF v6 reserves a
special address.
All (undef + addend) in .debug_* are resolved to -1.
We have to ignore the addend. With __attribute__((section(".text.x"))),
the address offset pair may be something like [.text.x + 16, .text.x + 24)
I have to resolve the whole (.text.x + 16) to the special value.
(undef + addend) in pre-DWARF v5 .debug_loc and .debug_ranges are resolved to
-2
(0 and -1 cannot be used due to the reasons above).
* Refined formula for a relocated value in a non-SHF_ALLOC section:
if is_defined(sym)
return addr(sym) + addend
if relocated_section is .debug_ranges or .debug_loc
return -2 # addend is intentionally ignored
// Every DWARF v5 section falls here
return -1 {{zero}}
* {{zero}} Can we resolve (undef + addend) to 0?
https://lists.llvm.org/pipermail/llvm-dev/2020-May/141967.html
> while it might not be an issue for ELF, DWARF would want a standard that's
fairly resilient to
> quirky/interesting use cases (admittedly - such platforms could equally want
to make their
> executable code way up in the address space near max or max - 1, etc?).
Question: is address 0 meaningful for code in some binary formats?
* {{overlap}} The current situation (GNU ld, gold, LLD): (undef + addend) in
.debug_* are resolved to addend.
For an address offset pair like [.text + 0, .text + 0x10010), if the ending
address offset is large
enough, it may overlap with a normal text address range (for example [0x1,
*))
This can cause problems in debuggers. How does gdb solve the problem?
* {{nonalloc}} Linkers resolve (undef + addend) in non-SHF_ALLOC sections to
`addend`. For non-debug sections (open-ended), do we have needs resolving such
values to `base` or `base+addend` where base is customizable?
(https://lists.llvm.org/pipermail/llvm-dev/2020-May/141956.html )
Forgot to mention
* {{compatibility}} Do we need an option if we change the computed value of
(undef + addend) to
-2 (.debug_loc,.debug_ranges)/-1 (other .debug_*)
(or 0 (other .debug_*), but it might not be nice to some binary formats
{{reserved_address}})
https://lists.llvm.org/pipermail/llvm-dev/2020-May/141958.html
> If we end up blessing it as part of the DWARF spec, we probably
> wouldn't want it to be user-configurable for the .debug_ sections, so
> I'd hesitate to add that configurability to the linker lest we have to
> revoke it to conform to DWARF (breaking flag compatibility with
> previous versions of the linker, etc). Admittedly we'll be breaking
> output compatibility with this change regardless, so potentially
> having the flag as an escape hatch could be useful.
I hope we don't need to have a linker option. But if some not-so-old
versions of gdb / binutils programs / elfutils programs can't cope
with -2/-1/0 {{reserved_address}}, we may have to invent a linker option.
I hope GNU ld, gold and LLD can have a c
Re: Range lists, zero-length functions, linker gc
On 2020-05-31, Mark Wielaard wrote:
Hi,
On Sun, May 31, 2020 at 11:55:06AM -0700, Fangrui Song via Elfutils-devel wrote:
what linkers should do regarding relocations referencing dropped
functions (due to section group rules, --gc-sections, /DISCARD/,
etc) in .debug_*
As an example:
__attribute__((section(".text.x"))) void f1() { }
__attribute__((section(".text.x"))) void f2() { }
int main() { }
Some .debug_* sections are relocated by R_X86_64_64 referencing
undefined symbols (the STT_SECTION symbols are collected):
0x0043: DW_TAG_subprogram [2]
## relocated by .text.x + 10
DW_AT_low_pc [DW_FORM_addr] (0x0010 ".text.x")
DW_AT_high_pc [DW_FORM_data4] (0x0006)
DW_AT_frame_base [DW_FORM_exprloc] (DW_OP_reg6 RBP)
DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x002c] =
"_Z2f2v")
DW_AT_name [DW_FORM_strp] ( .debug_str[0x0033] =
"f2")
With ld --gc-sections:
* DW_AT_low_pc [DW_FORM_addr] in .debug_info are resolved to 0 +
addend This can cause overlapping address ranges with normal text
sections. {{overlap}} * [beginning address offset, ending address
offset) in .debug_ranges are resolved to 1 (ignoring addend). See
bfd/reloc.c (behavior introduced in
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e4067dbb2a3368dbf908b39c5435c84d51abc9f3
)
[0, 0) cannot be used because it terminates the list entry.
[-1, -1) cannot be used because -1 represents a base address
selection entry which will affect subsequent address offset
pairs.
* .debug_loc address offset pairs have similar problem to .debug_ranges
* In DWARF v5, the abnormal values can be in a separate section .debug_addr
---
I am eager to know what you think
of the ideas from binutils/gdb/elfutils's perspective.
I think this is a producer problem. If a (code) section can be totally
dropped then the associated (.debug) sections should have been
generated together with that (code) section in a COMDAT group. That
way when the linker drops that section, all the associated sections in
that COMDAT group will get dropped with it. If you don't do that, then
the DWARF is malformed and there is not much a consumer can do about
it.
Said otherwise, I don't think it is correct for the linker (with
--gc-sections) to drop any sections that have references to it
(through relocation symbols) from other (.debug) sections.
I would love if we could solve the problem using ELF features, but
putting DW_TAG_subprogram in the same section group is not an
unqualified win
(https://lists.llvm.org/pipermail/llvm-dev/2020-May/141926.html)
(Cost: sizeof(Elf64_Shdr) = 64, Elf_Word for the entry in .group, plus
a string in .strtab unless you use the string ".debug_info"
(reusing the string requires
https://sourceware.org/bugzilla/show_bug.cgi?id=25380))
According to Peter Smith in the thread
https://groups.google.com/forum/#!msg/generic-abi/A-1rbP8hFCA/EDA7Sf3KBwAJ ,
Arm Compiler 5 splits up DWARF v3 debugging information and puts these sections
into comdat groups:
"This approach did produce significantly more debug information than gcc
did. For small microcontroller projects this wasn't a problem. For
larger feature phone problems we had to put a lot of work into keeping
the linker's memory usage down as many of our customers at the time were
using 32-bit Windows machines with a default maximum virtual memory of 2Gb."
See Ben, Ali and others' comments in the thread. Fragmented .debug_* may
not be practical.
Re: Range lists, zero-length functions, linker gc
On 2020-06-03, Alan Modra wrote: On Tue, Jun 02, 2020 at 11:06:10AM -0700, David Blaikie via Binutils wrote: On Tue, Jun 2, 2020 at 9:50 AM Mark Wielaard wrote: > where I > would argue the compiler simply needs to make sure that if it generates > code in separate sections it also should create the DWARF separate > section (groups). I don't think that's practical - the overhead, I believe, is too high. Headers for each section contribution (ELF headers but DWARF headers moreso - having a separate .debug_addr, .debug_line, etc section for each function would be very expensive) would make for very large object files. With a little linker magic I don't see the neccesity of duplicating the DWARF headers. Taking .debug_line as an example, a compiler could emit the header, opcode, directory and file tables to a .debug_line section with line statements for function foo emitted to .debug_line.foo and for bar to .debug_line.bar, trusting that the linker will combine these sections in order to create an output .debug_line section. If foo code is excluded then .debug_line.foo info will also be dropped if section groups are used. -- Alan Modra Australia Development Lab, IBM sizeof(Elf64_Shdr) = 64. If we create a .debug_line fragment and a .debug_info fragment for a function, we waste 128 bytes. https://sourceware.org/pipermail/binutils/2020-May/111361.html .debug_line.bar We should use the unique linkage feature https://sourceware.org/bugzilla/show_bug.cgi?id=25380 otherwise we also waste lots of bytes for the .debug_*.* section names.
Tombstone values in debug sections (was: Range lists, zero-length functions, linker gc)
I want to revive the thread, but focus on whether a tombstone value
(-1/-2) in .debug_* can cause trouble to various DWARF consumers (gdb,
debug related tools in elfutils and other utilities I don't know about).
Paul Robinson has proposed that DWARF v6 should reserve a tombstone
value (the value a relocation referencing a discarded symbol in a
.debug_* section should be resolved to)
http://www.dwarfstd.org/ShowIssue.php?issue=200609.1
Some comments about the proposal:
- deduplicating different functions with identical content; GNU refers
to this as ICF (Identical Code Folding);
ICF (gold --icf={safe,all}) can cause DW_TAG_subprogram with different
DW_AT_name to have the same range.
- functions with no callers; sometimes called dead-stripping or
garbage collection.
--gc-sections can lead to tombstone values. A referenced symbol may be
discarded because its containing sections is garbage collected.
- functions emitted in COMDAT sections, typically C++ template
instantiations or inline functions from a header file;
This can cause either tombstone values (STB_LOCAL) or duplicate DIEs
(non-STB_LOCAL).
On 2020-06-03, David Blaikie wrote:
On Tue, Jun 2, 2020 at 8:10 PM Alan Modra wrote:
On Tue, Jun 02, 2020 at 11:06:10AM -0700, David Blaikie via Binutils wrote:
> On Tue, Jun 2, 2020 at 9:50 AM Mark Wielaard wrote:
> > where I
> > would argue the compiler simply needs to make sure that if it generates
> > code in separate sections it also should create the DWARF separate
> > section (groups).
>
> I don't think that's practical - the overhead, I believe, is too high.
> Headers for each section contribution (ELF headers but DWARF headers
> moreso - having a separate .debug_addr, .debug_line, etc section for
> each function would be very expensive) would make for very large
> object files.
With a little linker magic I don't see the neccesity of duplicating
the DWARF headers. Taking .debug_line as an example, a compiler could
emit the header, opcode, directory and file tables to a .debug_line
section with line statements for function foo emitted to
.debug_line.foo and for bar to .debug_line.bar, trusting that the
linker will combine these sections in order to create an output
.debug_line section. If foo code is excluded then .debug_line.foo
info will also be dropped if section groups are used.
I don't think this would apply to debug_addr - where the entries are
referenced from elsewhere via index, or debug_rnglist where the
rnglist header (or the debug_info directly) contains offsets into this
section, so taking chunks out would break those offsets. (or to the
file/directory name part of debug_line - where you might want to
remove file/line entries that were eliminated as dead code - but
that'd throw off the indexes)
