On Mon, 2 Sep 2024, Tobias Burnus wrote: > The attached patch tries to fix the issue exposed by the PR: > > The main ingredient is partitioning of the LTO work, e.g. by using > -flto-partition=max. > > With -flto=2 (or higher or when a jobserver has been detected), not only the > LTO part is run in parallel but also the creation of the ltrans files itself, > i.e. gcc/lto/lto.cc's stream_out_partitions forks multiple processes to write > those files concurrently (here: -flto=2 means two processes, each writing > about half of the partitions). > > For each partition, output_offload_tables is called – which in principle would > add the offload tables to each file. To prevent this, in flag_wpa mode, the > tables were freed. That solves the WPA problem, but only if all partitions are > written by a single process (e.g. -flto=1). If not, the data is duplicated and > only the data belonging to the fork is modified. > > This patch moves the logic to gcc/lto/lto.cc and sets a global variable to > ensure that it is only output for the first partition, independently whether > there is only one or several processes writing the ltrans file, trying to > follow what Richard proposed in the PR? > > The patch has been tested on x86-64-gnu-linux with nvptx offloading, but I > should do a full bootstrap+regtest next. > > Comments, suggestions, remarks, approval?
Hmm, I can't really follow how and where it's currently decided whether to output offload tables for the LTRANS units but instead of an odd global variable would it be possible to pass that down as a flag or, alternatively encode that flag in the representation for the LTRANS partition? I suppose that's the out_decl_state? Or at least make it more "obvious" and guard the only caller to output_offload_tables with if (offload_output_tables_p) instead of the check in output_offload_tables itself? > Tobias > > -- Richard Biener <rguent...@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
