On 4/30/21 9:11 AM, Jose E. Marchesi via Gcc-patches wrote: > >>> For a moment, for the sake of this question, if we establish that CTF/BTF >>> generation always feeds off DWARF DIEs (so there is no need to access >>> type/decl tree nodes), what will it take to keep LTO support while keeping >>> ctf_debug_finalize in dwarf2out_finish ? >> >> I don't think it's possible without major surgery. Now, it looks like >> you want to emit .ctf at the LTO compile stage and BTF at the LTRANS >> stage. Note you do not have access to the CTF data produced at the >> LTRANS stage. What DWARF has access to is pointers (symbol name + offset) >> to the early DWARF generated DIEs for GCCs tree decls from what it >> adds DW_AT_abstract_origin references and then amending those entities >> with furthr DWARF attributes. >> >> Now, the "major surgery" could for example entail streaming the CTF >> data you produce from compile through WPA to the LTRANS stage. Or >> alternatively simply reading it from the compile time produced objects >> by means of the DWARF DIE references. I don't know what kind of (if any) >> references you need to produce from BTF to the CTF data (I suppose they >> might share indexes into a same symbol table...). > > I am thinking, given that we have both "early DWARF" and "late DWARF", > we may want to have three init/fini hooks for debug formats instead of > two: > > DEBUG_FORMAT_initialize (called in dwarf2out_early_finish) > > [... traverse early DWARF (DEBUG_FORMAT_do_die) ...] > > DEBUG_FORMAT_early_finish (called in dwarf2out_early_finish) > DEBUG_FORMAT_finish (called in dwarf2out_finish) > > Note that CTF and BTF share the ctfc internal data structure (which is > built from traversing the early DWARF) but these are otherwise distinct > debug formats: CTF can be completely emitted in ctf_early_finish, > whereas BTF needs additional information that is only available in > ctf_finish.) >
I like this idea. As Indu mentioned, the change to writing CTF/BTF in dwarf2out_finish rather than dwar2out_early_finish is motivated purely by the needs of BPF. Specifically, it is to support BPF Compile Once - Run Everywhere (BPF CO-RE). The BPF backend needs BTF information to generate CO-RE 'relocation' records, and it needs to add to the BTF string table. The relocation records themselves are placed in a .BTF.ext section, but refer to BTF records by ID. Note that CO-RE only comes into play when generating BTF information for a BPF target. Otherwise, the BTF information can also be emitted in early_finish. That is, BTF is effectively "late DWARF" ONLY for the BPF target. Of course, this is also the usual use case for BTF. I do not have much experience with LTO. I am still trying to digest the LTO section of the manual. But from what what I do understand, and what is discussed here, I think the following may work: 1. Always emit CTF in early_finish. There is no reason to delay it. This simplifies both LTO and non-LTO cases. 2. For non-BPF targets, emit BTF in early_finish as well. 3. For non-LTO + BTF + BPF, emit BTF in in (late) finish, to support generating CO-RE information in the BPF backend. 4. For LTO + BTF + BPF, either emit BTF in early_finish then read it back later, or stream it through WPA to LTRANS stage. I am not sure exactly what this streaming means. In either case, end up with enough information at LTRANS stage to re-emit the BTF, now with CO-RE information. Does the above make any sense? Is this overly complicated? Thank you very much Richard for the review, and for raising this LTO discussion. David
