On Thu, 20 Feb 2020, Bernd Edlinger wrote:
> On 2/20/20 2:34 PM, Richard Biener wrote:
> > On Thu, 20 Feb 2020, Bernd Edlinger wrote:
> >
> >> Hi,
> >>
> >> this is the remaining issue that happens when -flto and -save-temps
> >> is used together, it leaks several files in /tmp.
> >>
> >> I try to give more background to how these temp files are
> >> created, and in all likelihood the leaking of these
> >> files is wanted, and certainly very helpful for debugging
> >> lto issues, that's for sure. It is just not helpful
> >> that they need to be looked up in the /tmp folder, even
> >> if you want to debug something with lto.
> >>
> >> The short story is...
> >>
> >> They are created here:
> >>
> >> if (parallel)
> >> {
> >> makefile = make_temp_file (".mk");
> >> mstream = fopen (makefile, "w");
> >>
> >> and here:
> >>
> >> /* Note: we assume argv contains at least one element; this is
> >> checked above. */
> >>
> >> response_file = make_temp_file ("");
> >>
> >> f = fopen (response_file, "w");
> >>
> >> And in a few other places as well, it depends a bit
> >> if -o is used or not (i.e. linker_output != NULL or not).
> >>
> >> and not removed here:
> >>
> >>
> >> if (response_file && !save_temps)
> >> {
> >> unlink (response_file);
> >> response_file = NULL;
> >> }
> >>
> >> and here:
> >>
> >> do_wait (new_argv[0], pex);
> >> maybe_unlink (makefile);
> >> makefile = NULL;
> >>
> >>
> >> the code with the response_file is executed both in
> >> lto-wrapper and collect2, but in collect2 only when
> >> if is invoked from lto-wrapper, triggered by the @file
> >> argument list.
> >>
> >> Therefore I figured that the best possible
> >> solution is just let lto-wrapper create a temp-file
> >> for those problem cases, and use TMPDIR to have
> >> all make_temp_file that follow use that to folder to
> >> place the those response files and other stuff in
> >> there.
> >>
> >>
> >> So that is what I split out from the previous patch,
> >> which focused on collect2.
> >>
> >>
> >> Bootstrapped and reg-tested on x86_64-pc-linux-gnu.
> >> Is it OK for trunk?
> >
> > I don't think this is an improvement. The files still
> > will be (correctly) retained and now in addition to that
> > there's temporary directories piling up?
> >
>
> Well, the temp. directory has a known name,
> in case the command line is
>
> gcc -save-temps -lto -o test test.c
>
> there are those 4 in a *known* place, test.lto_tmpdir:
>
> test.lto_tmpdir:
> insgesamt 24
> drwxrwxr-x 2 ed ed 4096 Feb 20 16:14 .
> drwxrwxr-x 3 ed ed 4096 Feb 20 16:14 ..
> -rw------- 1 ed ed 15 Feb 20 16:14 cc8FwPQ1
> -rw------- 1 ed ed 237 Feb 20 16:14 cchROrdh
> -rw------- 1 ed ed 197 Feb 20 16:14 cclbBAUp
> -rw------- 1 ed ed 7 Feb 20 16:14 ccMlfh1g
>
> plus
> -rw-rw-r-- 1 ed ed 164 Feb 20 16:14 test.i
> -rw-rw-r-- 1 ed ed 50 Feb 20 16:14 test.lto_wrapper_args
> -rw-rw-r-- 1 ed ed 17 Feb 20 16:14 test.ltrans.out
> -rw-rw-r-- 1 ed ed 1232 Feb 20 16:14 test.ltrans0.ltrans.o
> -rw-rw-r-- 1 ed ed 2539 Feb 20 16:14 test.ltrans0.o
> -rw-rw-r-- 1 ed ed 374 Feb 20 16:14 test.ltrans0.s
> -rw-rw-r-- 1 ed ed 52 Feb 20 16:14 test.res
> -rw-rw-r-- 1 ed ed 4267 Feb 20 16:14 test.s
>
> if the command line is
> gcc -save-temps -lto test.c
>
> there are a few more but also in a *known* place, a.out.lto_tmpdir:
>
> a.out.lto_tmpdir/
> total 36
> drwxrwxr-x 2 ed ed 4096 Feb 20 16:18 .
> drwxrwxr-x 3 ed ed 4096 Feb 20 16:18 ..
> -rw------- 1 ed ed 7 Feb 20 16:18 cc2hY8SH
> -rw------- 1 ed ed 227 Feb 20 16:18 ccDafyit
> -rw------- 1 ed ed 262 Feb 20 16:18 ccglzNAe
> -rw------- 1 ed ed 36 Feb 20 16:18 ccnAU7NG
> -rw------- 1 ed ed 38 Feb 20 16:18 ccoLFY2H.ltrans.out
> -rw-rw-r-- 1 ed ed 1232 Feb 20 16:18 ccoLFY2H.ltrans0.ltrans.o
> -rw-rw-r-- 1 ed ed 2560 Feb 20 16:18 ccoLFY2H.ltrans0.o
>
> plus
> -rw-rw-r-- 1 ed ed 50 Feb 20 16:18 a.out.lto_wrapper_args
> -rw-rw-r-- 1 ed ed 160 Feb 20 16:18 test.i
> -rw-rw-r-- 1 ed ed 3104 Feb 20 16:18 test.o
> -rw-rw-r-- 1 ed ed 52 Feb 20 16:18 test.res
> -rw-rw-r-- 1 ed ed 4267 Feb 20 16:18 test.s
>
>
> > A better improvement would be to selectively decide
> > which files might not be needed to be preserved and/or
> > give them names in the build directory in more cases.
> >
>
> Ah, well, it just felt like that will probably need a
> rather complicated patch which is probably not
> worth it, since the are a lot of different
> code pathes involved, (with -g or not, with -flto=jobserver
> or not, flto_partition=none, or with offload or not, with -o
> or not etc.) in two or more independent forked processes,
> which will need to coordinate somehow, not to clobber each
> other's tempfiles.
IIRC this definitely clashes with Alex work to overhaul
-auxdir/-dumpdir queued for GCC 11 where some of the above
is improved.
So whatever we do we should do it for GCC 11 after Alex patches
land.
Richard.
>
> Bernd.
>
>
> > Richard.
> >
> >>
> >> Thanks
> >> Bernd.
> >>
> >>
> >
>
--
Richard Biener <rguent...@suse.de>
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg,
Germany; GF: Felix Imendörffer; HRB 36809 (AG Nuernberg)
