Re: [RFC] Reliable compiler specification setting (at least include/lib dirs) through the process environment
Hi Shea,
Shea Levy skribis:
> Unlike the traditional approach of installing system libraries into one
> central location like /usr/{lib,include}, the nix package manager [1]
> installs each package into it's own prefix
> (e.g. /nix/store/mn9kqag3d24v6q41x747zd7n5qnalch7-zlib-1.2.8-dev). Moreover,
> each package is built in its own environment determined from its
> explicitly listed dependencies, regardless of what else is installed on
> the system. Because not all package build scripts properly respect
> CFLAGS etc., we currently wrap the compiler [2] to respect custom
> environment variables like NIX_CFLAGS_COMPILE, so during the build of a
> package that depends on zlib and Xlib might have NIX_CFLAGS_COMPILE set
> to "-isystem
> /nix/store/bl0rz2xinsm9yslghd7n5vaba86zxknh-libX11-1.6.3-dev/include -isystem
> /nix/store/mn9kqag3d24v6q41x747zd7n5qnalch7-zlib-1.2.8-dev/include".
>
> Unfortunately, as you can see if you click through the link or look
> through the git history, the wrapper is quite complex (frankly, hacky)
> [2]:
> https://github.com/NixOS/nixpkgs/blob/8cbdd9d0c290e294a9d783c8868e738db05c9ce2/pkgs/build-support/cc-wrapper/cc-wrapper.sh
Guix avoids the compiler wrapper altogether like this:
• We use C_INCLUDE_PATH, LIBRARY_PATH, and friends:
.
• We have a simple linker wrapper aimed at adding -Wl,-rpath flags:
.
The comment in that file explains why the other options considered
were unsuitable.
• We modify the built-in “lib” spec of GCC to add the necessary -L and
-rpath flags:
.
• Likewise, we tell Clang where to find libc and friends:
.
This is not too intrusive and more robust than wrapping everything.
I suppose GCC and Clang could facilitate this by providing configure
options to augment the “lib” spec, specify the location of libc alone,
or something along these lines.
Thoughts?
Ludo’.
Re: [RFC] Reliable compiler specification setting (at least include/lib dirs) through the process environment
Hey Ludo’,
Amazing, more than a decade of close working with these tools and I
never knew about C_INCLUDE_PATH et al! It looks like those will solve a
huge portion of the problem.
Will look at your gcc and clang patches as well, thank you!
~Shea
Ludovic Courtès writes:
> Hi Shea,
>
> Shea Levy skribis:
>
>> Unlike the traditional approach of installing system libraries into one
>> central location like /usr/{lib,include}, the nix package manager [1]
>> installs each package into it's own prefix
>> (e.g. /nix/store/mn9kqag3d24v6q41x747zd7n5qnalch7-zlib-1.2.8-dev). Moreover,
>> each package is built in its own environment determined from its
>> explicitly listed dependencies, regardless of what else is installed on
>> the system. Because not all package build scripts properly respect
>> CFLAGS etc., we currently wrap the compiler [2] to respect custom
>> environment variables like NIX_CFLAGS_COMPILE, so during the build of a
>> package that depends on zlib and Xlib might have NIX_CFLAGS_COMPILE set
>> to "-isystem
>> /nix/store/bl0rz2xinsm9yslghd7n5vaba86zxknh-libX11-1.6.3-dev/include
>> -isystem /nix/store/mn9kqag3d24v6q41x747zd7n5qnalch7-zlib-1.2.8-dev/include".
>>
>> Unfortunately, as you can see if you click through the link or look
>> through the git history, the wrapper is quite complex (frankly, hacky)
>
>> [2]:
>> https://github.com/NixOS/nixpkgs/blob/8cbdd9d0c290e294a9d783c8868e738db05c9ce2/pkgs/build-support/cc-wrapper/cc-wrapper.sh
>
> Guix avoids the compiler wrapper altogether like this:
>
> • We use C_INCLUDE_PATH, LIBRARY_PATH, and friends:
>
> .
>
> • We have a simple linker wrapper aimed at adding -Wl,-rpath flags:
>
> .
> The comment in that file explains why the other options considered
> were unsuitable.
>
> • We modify the built-in “lib” spec of GCC to add the necessary -L and
> -rpath flags:
>
> .
>
> • Likewise, we tell Clang where to find libc and friends:
>
>
>
> .
>
> This is not too intrusive and more robust than wrapping everything.
>
> I suppose GCC and Clang could facilitate this by providing configure
> options to augment the “lib” spec, specify the location of libc alone,
> or something along these lines.
>
> Thoughts?
>
> Ludo’.
signature.asc
Description: PGP signature
C++17 std::launder and aliasing
Hi!
http://wg21.link/p0137
adds std::launder which is supposed to be some kind of aliasing optimization
barrier.
What is unclear to me is if we really need compiler support for that.
I have unfortunately not found many examples:
http://stackoverflow.com/questions/39382501/what-is-the-purpose-of-stdlaunder
mentions something like:
#include
int
foo ()
{
struct X { const int n; };
union U { X x; float f; };
U u = {{ 1 }};
int a = u.x.n;
X *p = new (&u.x) X {2};
int b = u.x.n;// UB, needs std::launder(&u.x.n)
return a + b;
}
but g++ handles it as returning 3 even without that.
So, do we need to do anything here even in the current gcc aliasing model,
which is very permissive (my understanding is that usually we treat all
writes as possibly placement new-ish changes)?
Then I found something like:
https://groups.google.com/a/isocpp.org/forum/#!msg/std-discussion/XYvVlTc3-to/HbhebSRnAgAJ
which of course doesn't work with -flifetime-dse=2, I'd strongly hope that
all those attempts there are UB even with std::launder.
Adding __builtin_launder as void * -> void * builtin (ECF_CONST) or perhaps
typegeneric one that returns the same pointer as given to it (and not
teaching alias analysis about what that builtin does) is certainly possible,
the question is how to expand it at RTL time (does it also need to be some
kind of opt barrier, say like __asm ("" : "+g" (ptr));, or not?
Jakub
Re: [RFC] Reliable compiler specification setting (at least include/lib dirs) through the process environment
Hi Ludo’,
Your patches look good! My biggest concern is how the ld wrapper behaves
in the presence of response files. Have you tested that?
Thanks,
Shea
Ludovic Courtès writes:
> Hi Shea,
>
> Shea Levy skribis:
>
>> Unlike the traditional approach of installing system libraries into one
>> central location like /usr/{lib,include}, the nix package manager [1]
>> installs each package into it's own prefix
>> (e.g. /nix/store/mn9kqag3d24v6q41x747zd7n5qnalch7-zlib-1.2.8-dev). Moreover,
>> each package is built in its own environment determined from its
>> explicitly listed dependencies, regardless of what else is installed on
>> the system. Because not all package build scripts properly respect
>> CFLAGS etc., we currently wrap the compiler [2] to respect custom
>> environment variables like NIX_CFLAGS_COMPILE, so during the build of a
>> package that depends on zlib and Xlib might have NIX_CFLAGS_COMPILE set
>> to "-isystem
>> /nix/store/bl0rz2xinsm9yslghd7n5vaba86zxknh-libX11-1.6.3-dev/include
>> -isystem /nix/store/mn9kqag3d24v6q41x747zd7n5qnalch7-zlib-1.2.8-dev/include".
>>
>> Unfortunately, as you can see if you click through the link or look
>> through the git history, the wrapper is quite complex (frankly, hacky)
>
>> [2]:
>> https://github.com/NixOS/nixpkgs/blob/8cbdd9d0c290e294a9d783c8868e738db05c9ce2/pkgs/build-support/cc-wrapper/cc-wrapper.sh
>
> Guix avoids the compiler wrapper altogether like this:
>
> • We use C_INCLUDE_PATH, LIBRARY_PATH, and friends:
>
> .
>
> • We have a simple linker wrapper aimed at adding -Wl,-rpath flags:
>
> .
> The comment in that file explains why the other options considered
> were unsuitable.
>
> • We modify the built-in “lib” spec of GCC to add the necessary -L and
> -rpath flags:
>
> .
>
> • Likewise, we tell Clang where to find libc and friends:
>
>
>
> .
>
> This is not too intrusive and more robust than wrapping everything.
>
> I suppose GCC and Clang could facilitate this by providing configure
> options to augment the “lib” spec, specify the location of libc alone,
> or something along these lines.
>
> Thoughts?
>
> Ludo’.
signature.asc
Description: PGP signature
Re: C++17 std::launder and aliasing
On Tue, Oct 18, 2016 at 1:06 PM, Jakub Jelinek wrote:
> Hi!
>
> http://wg21.link/p0137
> adds std::launder which is supposed to be some kind of aliasing optimization
> barrier.
>
> What is unclear to me is if we really need compiler support for that.
> I have unfortunately not found many examples:
>
> http://stackoverflow.com/questions/39382501/what-is-the-purpose-of-stdlaunder
> mentions something like:
> #include
> int
> foo ()
> {
> struct X { const int n; };
> union U { X x; float f; };
> U u = {{ 1 }};
> int a = u.x.n;
> X *p = new (&u.x) X {2};
> int b = u.x.n;// UB, needs std::launder(&u.x.n)
> return a + b;
> }
> but g++ handles it as returning 3 even without that.
> So, do we need to do anything here even in the current gcc aliasing model,
> which is very permissive (my understanding is that usually we treat all
> writes as possibly placement new-ish changes)?
The standard mentions that appearantly const and reference "sub-objects" are
unchanging when you access them. Cruically they changed 3.8/1 to
The lifetime of an object o of type T ends when...
* the storage which the object occupies is released, or is reused by
an object that is not nested within o ([intro.object])
the subobject notion is new. I suppse this was done to formally allow
construction of objects
in char[] members w/o ending the containings object lifetime. I think
this is somewhat of a mistake
as obviously two (sub-)objects can't be life at the same time at the
same memory location
(which the undefined behavior above implies).
And yes, GCC doesn't need anything special as it handles sub-object
lifetime properly
(any store may end it) and it doesn't exploit the "constness" of const
declared members
or reference members.
> Then I found something like:
> https://groups.google.com/a/isocpp.org/forum/#!msg/std-discussion/XYvVlTc3-to/HbhebSRnAgAJ
> which of course doesn't work with -flifetime-dse=2, I'd strongly hope that
> all those attempts there are UB even with std::launder.
Obviously std::launder now invites people to invent fancy things (all
UB), similar
to how reinterpret_cast<>s name invited people to think it has anything to do
with TBAA.
std::launder is about object lifetime, nothing else IIUC.
> Adding __builtin_launder as void * -> void * builtin (ECF_CONST) or perhaps
> typegeneric one that returns the same pointer as given to it (and not
> teaching alias analysis about what that builtin does) is certainly possible,
> the question is how to expand it at RTL time (does it also need to be some
> kind of opt barrier, say like __asm ("" : "+g" (ptr));, or not?
As said, nothing needed for the middle-end.
Richard.
> Jakub
Re: [RFC] Reliable compiler specification setting (at least include/lib dirs) through the process environment
Hi! Shea Levy skribis: > Your patches look good! My biggest concern is how the ld wrapper behaves > in the presence of response files. Have you tested that? It surely doesn’t (yet?). However, GCC does not pass “@file” arguments when it invokes ‘ld’, and the bug report you mentioned¹ talks about GHC invoking ‘gcc’, not ‘ld’, so I guess it’s fine to ignore response files in the ld wrapper. Ludo’. ¹ https://github.com/NixOS/nixpkgs/commit/a421e7bd4a28c69bded8b17888325e31554f61a1
Re: [cfe-dev] [RFC] Reliable compiler specification setting (at least include/lib dirs) through the process environment
On Tue, Oct 18, 2016 at 8:59 AM, Ludovic Courtès via cfe-dev wrote: > Shea Levy skribis: > >> Your patches look good! My biggest concern is how the ld wrapper behaves >> in the presence of response files. Have you tested that? > > It surely doesn’t (yet?). > > However, GCC does not pass “@file” arguments when it invokes ‘ld’, and > the bug report you mentioned¹ talks about GHC invoking ‘gcc’, not ‘ld’, > so I guess it’s fine to ignore response files in the ld wrapper. GCC will pass response files to ld when response files were used in the invocation of GCC. -Nathan
Re: Clear basic block flags before using BB_VISITED for OpenACC loops processing
Hi!
On Mon, 17 Oct 2016 15:38:50 +0200, I wrote:
> On Mon, 17 Oct 2016 14:08:44 +0200, Richard Biener
> wrote:
> > On Mon, Oct 17, 2016 at 1:47 PM, Thomas Schwinge
> > wrote:
> > > On Mon, 17 Oct 2016 13:22:17 +0200, Richard Biener
> > > wrote:
> > >> On Mon, Oct 17, 2016 at 11:38 AM, Thomas Schwinge
> > >> wrote:
> > >> > On Fri, 14 Oct 2016 13:06:59 +0200, Richard Biener
> > >> > wrote:
> > >> >> On Fri, Oct 14, 2016 at 1:00 PM, Nathan Sidwell
> > >> >> wrote:
> > >> >> > On 10/14/16 05:28, Richard Biener wrote:
> > >> >> >
> > >> >> >> The BB_VISITED flag has indetermined state at the beginning of a
> > >> >> >> pass.
> > >> >> >> You have to ensure it is cleared yourself.
> > >> >> >
> > >> >> >
> > >> >> > In that case the openacc (&nvptx?) passes should be modified to
> > >> >> > clear the
> > >> >> > flags at their start, rather than at their end.
This already is a "conceptual acknowledgement" of my patch, so...
> > >> > OK to commit the following? Is such a test case appropriate (which
> > >> > would
> > >> > have caught this issue right away), in particular the dg-final
> > >> > scan-tree-dump line?
> > >>
> > >> Ugh. Not worse to what we do in various dwarf scanning I guess.
> > >
> > > ;-|
> > >
> > >> Doesn't failure lead to a miscompile eventually? So you could formulate
> > >> this as a dg-do run test with a check for the desired outcome?
> > >
> > > No, unfortunately. In this case the error is "benign" such that the
> > > OpenACC loop processing machinery will decide to not parallelize loops
> > > that ought to be parallelized.
> >
> > So you can scan for "loop parallelized" instead?
>
> The dump would still contain the outer loop's "Loop 0(0)" marker, so I'd
> have to scan for "Head"/"Tail"/"UNIQUE" or similar instead -- but that
> seems likewise fragile (for false negatives), and less useful than
> scanning for the complete pattern.
>
> > I fear your pattern
> > is quite fragile
> > to maintain over time.
>
> Agreed -- but then, that's intentional: my idea for this new test case
> has been to have it actually verify the expected OpenACC loop processing,
> so it's clear that this pattern will need to be adjusted if changing the
> OpenACC loop processing.
>
> > > This won't generally cause any problem
> > > (apart from performance regression, obviously); it just caused problems
> > > in a few libgomp test cases that actually at run time test for
> > > parallelized execution -- which will/did trigger only with nvptx
> > > offloading enabled, which not too many people are testing. The test case
> > > I propose below will trigger also for non-offloading configurations.
>
> On IRC, Segher suggested to 'use {} instead of "" to avoid [all those
> backslashes]' -- thanks, done.
If you don't like the test case as-is (do we need multi-line tree dump
scanning, just like we recently got for compiler diagnostics?), can I at
least commit the OpenACC loops processing fix? Here is the latest
version, simplified after your r241296 IRA vs. BB_VISITED fixes:
commit 766cf9959b15a17e17e89a50e905b4c546893823
Author: Thomas Schwinge
Date: Mon Oct 17 15:33:09 2016 +0200
Clear basic block flags before using BB_VISITED for OpenACC loops processing
gcc/
* omp-low.c (oacc_loop_discovery): Call clear_bb_flags before, and
don't clear BB_VISITED after processing.
gcc/testsuite/
* gcc.dg/goacc/loop-processing-1.c: New file.
---
gcc/omp-low.c | 8 +++-
gcc/testsuite/gcc.dg/goacc/loop-processing-1.c | 18 ++
2 files changed, 21 insertions(+), 5 deletions(-)
diff --git gcc/omp-low.c gcc/omp-low.c
index 77f89d5..3ef796f 100644
--- gcc/omp-low.c
+++ gcc/omp-low.c
@@ -19236,7 +19236,9 @@ oacc_loop_sibling_nreverse (oacc_loop *loop)
static oacc_loop *
oacc_loop_discovery ()
{
- basic_block bb;
+ /* Clear basic block flags, in particular BB_VISITED which we're going to use
+ in the following. */
+ clear_bb_flags ();
oacc_loop *top = new_oacc_loop_outer (current_function_decl);
oacc_loop_discover_walk (top, ENTRY_BLOCK_PTR_FOR_FN (cfun));
@@ -19245,10 +19247,6 @@ oacc_loop_discovery ()
that diagnostics come out in an unsurprising order. */
top = oacc_loop_sibling_nreverse (top);
- /* Reset the visited flags. */
- FOR_ALL_BB_FN (bb, cfun)
-bb->flags &= ~BB_VISITED;
-
return top;
}
diff --git gcc/testsuite/gcc.dg/goacc/loop-processing-1.c
gcc/testsuite/gcc.dg/goacc/loop-processing-1.c
new file mode 100644
index 000..619576a
--- /dev/null
+++ gcc/testsuite/gcc.dg/goacc/loop-processing-1.c
@@ -0,0 +1,18 @@
+/* Make sure that OpenACC loop processing happens. */
+/* { dg-additional-options "-O2 -fdump-tree-oaccdevlow" } */
+
+extern int place ();
+
+void vector_1 (int *ary, int size)
+{
+#pragma acc parallel num_workers (32) vector_length(32) copy(ary[0:size])
firstprivate (size)
+ {
+#pragma acc loop gang
+for (int jx = 0; jx < 1; jx++)
gcc-5-20161018 is now available
Snapshot gcc-5-20161018 is now available on ftp://gcc.gnu.org/pub/gcc/snapshots/5-20161018/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 5 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/branches/gcc-5-branch revision 241321 You'll find: gcc-5-20161018.tar.bz2 Complete GCC MD5=a3caabb2ffaf0d52cf1cd668621b7a56 SHA1=9d476e5a81beffccb76aaf95c97d5a12abc8aa02 Diffs from 5-20161011 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-5 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
