Re: [PATCH] m4: fix --disable-rpath for AIX
Hi Bruno, >> Currently, --disable-rpath will add directly $found_so to the compilation >> line. However, on AIX, this will result on the path being hardcoded in >> the built binaries. >> The only way to avoid hardcoded paths without using linker flags (like >> -Wl,-bnoipath) is to add -l$name. > Can you tell on which package and on which AIX version this patch has an effect? I'm using it in most of the GNU packages (gettext, libiconv, libunistring, ...) on AIX 6.1. But it should be the same of other AIX versions. > I tried it with GNU gettext 0.20.1 on AIX 7.1, with gcc and with > --disable-rpath, and found that the generated executables have the same > directories hardcoded as without your patch. > $ gmake ngettext > /bin/sh ../libtool --tag=CC --mode=link gcc > -DINSTALLDIR=\"/home/haible/prefix32-norpath/bin\" -g -O2 > -L/home/haible/prefix32-norpath/lib -o ngettext ngettext-ngettext.o > ../gnulib-lib/libgrt.a ../intl/libintl.la -liconv -pthread -liconv > libtool: link: gcc -DINSTALLDIR=\"/home/haible/prefix32-norpath/bin\" -g -O2 > -o .libs/ngettext ngettext-ngettext.o -pthread > -L/home/haible/prefix32-norpath/lib ../gnulib-lib/libgrt.a -L../intl/.libs > -lintl -L/opt/freeware/lib -liconv -pthread > -Wl,-blibpath:/home/haible/prefix32-norpath/lib:/opt/freeware/lib:/home/haible/prefix32-norpath/lib:/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/4.8.1:/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/4.8.1/../../..:/usr/lib:/lib >You can see that the Makefile does not pass abolute library names to 'libtool', >and nevertheless 'libtool' sets the libpath property as shown. Actually, I've coupled it with another patch making shared object extension being ".a", as it's supposed to be on AIX when -brtl flag isn't present. By default, so in your case, shared extension is ".so". Therefore, in the configure, found_so will be null if your system doesn't have any libiconv.so under /opt/freeware/lib (which it should be). Could you tell me if you're ending up having "-L/opt/freeware/lib -liconv" because both "found_a" and "found_so" is null or because of something else ? Anyway, I think even if this patch doesn't apply to every cases. The current default behavior of configure is that if a found_so is found (either .a or .so) and --disable-rpath is provided, it will link with "/opt/freeware/lib/libname.a" and thus the directory will be hardcoded. That's doing exactly what we're trying to avoid.
Re: restrict
On 2/17/20 4:53 AM, Bruno Haible wrote:
> Paul Eggert wrote:
>> if GCC generated warnings for that sort of thing, the warnings would be false
>> alarms.
>
> Yes, and this in turn means that the ability to produce useful warnings via
> 'restrict' is limited. In this example:
> ===
> #include
> extern void memmcpy (void *restrict, const void *restrict, size_t);
>
> void shuffle (char array[10])
> {
> memmcpy (array + 2, array, 8);
> memcpy (array + 2, array, 8);
> }
> ===
> gcc gives no warning about 'memmcpy' - because it does not know
> how many elements the function will access. gcc does give a warning
> about 'memcpy' - apparently due to custom logic in the compiler.
The following gives you a warning, with -O2 / -O3 and -Wall:
===
#include
void memmcpy (void *restrict d, void *restrict s, size_t n)
{
memcpy(d, s, n);
}
void shuffle ()
{
char array[] = "abcdefg", *array2 = array + 2;
memmcpy (array, array2 - 2, 8);
}
===
$ gcc-8 -O2 -Wall x.c
In function ‘memmcpy’,
inlined from ‘shuffle’ at x.c:11:3:
x.c:4:3: warning: ‘memcpy’ accessing 8 bytes at offsets 0 and 0 overlaps
8 bytes at offset 0 [-Wrestrict]
memcpy(d, s, n);
^~~
Regards, Tim
signature.asc
Description: OpenPGP digital signature
Re: restrict
On 2/16/20 3:49 PM, Bruno Haible wrote:
I'm after warnings that point to undefined behaviour, and since passing a
'char *' in place of an 'unsigned char *', 'FILE *', or similar already
generates a compiler warning, using 'restrict' here does not offer an
advantage.
It can still offer an advantage, since GCC doesn't necessarily generate a
compiler warning without the 'restrict'. For example, in the following code GCC
complains about the call to f but not the call to g:
void f (char *restrict, int *restrict);
void g (char *, int *);
int
main (void)
{
int a[] = {0, 0, 0};
void *p = a;
f (p, p);
g (p, p);
}
More generally, the role of 'restrict' for optimizations can't be easily
separated from its role for warnings. Using 'restrict' to save machine
instructions also means that it's better for GCC to generate warnings when the
behavior would otherwise be undefined; conversely, typically when GCC generates
warnings it's free to optimize more. Because of this close connection, when in
doubt it's better to use 'restrict' in a way that benefits optimization as well
as warning.
int snprintf (char *str, size_t size, const char *format, ...);
... since 'restrict' on the first parameter is going to signal a warning
for overlaps with the 3rd, 4th, 5th, ... parameter, we don't need 'restrict'
on the 3rd parameter.
If we omit 'restrict' from the 3rd parameter, GCC won't warn when the 3rd
parameter overlaps with the 4th. That's a minus.
I mean any possible implementation, compiled from the code we have in gnulib.
'restrict' is part of the contract between the caller of the function and the
implementation of the function. When the compiler optimizes the implementation,
it has to stick to the contract.
This sounds backwards. Ordinarily (without restrict) there is a contract that
prevents the compiler from optimizing the implementation in certain ways. With
restrict, that part of the contract goes away. From the programmer's point of
view, 'restrict' places additional restrictions on the caller, not on the
callee. And appropriate use of 'restrict' signals to programmers what the callee
can do. This signal to humans is useful regardless of whether tools like GCC use
the signal to generate warnings or optimizations.
The "same array" rule comes from what I understood from various explanations
of 'restrict' on the web [1], and from what I see in the glibc headers for
functions that have multiple output parameters of the same type:
- splice in ,
- printf_arginfo_size_function in ,
- openpty in ,
- re_set_registers in ,
- copy_file_range in .
That stackoverflow explanation is full of confusing answers and none of them are
particularly good. And the glibc headers evidently were not written by people
who cared about proper use of 'restrict' (possibly because they wrote the
headers before 'restrict' existed or was widely supported).
But ecvt, fcvt in have 'restrict'. You may want to register a glibc
bug if you think these five functions should have 'restrict' :)
It's not a bug. It's required by POSIX (which uses 'restrict' more carefully) in
its older editions. See:
https://pubs.opengroup.org/onlinepubs/009695399/functions/ecvt.html
The functions in questions have been withdrawn from POSIX; if they had not been,
the 'restrict's would still be there.
Re: restrict
Hi Paul,
> > I'm after warnings that point to undefined behaviour, and since passing a
> > 'char *' in place of an 'unsigned char *', 'FILE *', or similar already
> > generates a compiler warning, using 'restrict' here does not offer an
> > advantage.
>
> It can still offer an advantage, since GCC doesn't necessarily generate a
> compiler warning without the 'restrict'. For example, in the following code
> GCC
> complains about the call to f but not the call to g:
>
>void f (char *restrict, int *restrict);
>void g (char *, int *);
>
>int
>main (void)
>{
> int a[] = {0, 0, 0};
> void *p = a;
> f (p, p);
> g (p, p);
>}
True. But it's pretty rare that people use 'void *' pointers and cast them
to pointers of different types.
Also, even a small variation of this code does not produce a warning any more:
void f (char *restrict, int *restrict);
void g (char *, int *);
int
main (void)
{
inta[] = {0, 0, 0};
void *p = a;
f (p, a);// no warning!
g (p, a);
}
> More generally, the role of 'restrict' for optimizations can't be easily
> separated from its role for warnings. Using 'restrict' to save machine
> instructions also means that it's better for GCC to generate warnings when
> the
> behavior would otherwise be undefined; conversely, typically when GCC
> generates
> warnings it's free to optimize more. Because of this close connection, when
> in
> doubt it's better to use 'restrict' in a way that benefits optimization as
> well
> as warning.
It needs to be weighed against the desire not to produce warnings for valid
uses. For example here:
> >int snprintf (char *str, size_t size, const char *format, ...);
> > ... since 'restrict' on the first parameter is going to signal a warning
> > for overlaps with the 3rd, 4th, 5th, ... parameter, we don't need 'restrict'
> > on the 3rd parameter.
>
> If we omit 'restrict' from the 3rd parameter, GCC won't warn when the 3rd
> parameter overlaps with the 4th. That's a minus.
Actually, GCC doesn't warn here, even with 'restrict', because the 3rd argument
is a const-pointer and the 4th argument, being variadic, is treated like a
const-pointer as well. Test case (enable one of the declarations):
#include
//extern int smprintf (char *restrict, size_t, char *, ...);
// no warning
//extern int smprintf (char *restrict, size_t, char *restrict, ...);
// warning
//extern int smprintf (char *restrict, size_t, const char *, ...);
// no warning
//extern int smprintf (char *restrict, size_t, const char *restrict, ...);
// no warning
//extern int smprintf (char *restrict, size_t, const char *restrict, char
*restrict); // warning
//extern int smprintf (char *restrict, size_t, const char *restrict, const char
*restrict); // no warning
char buf[10];
int main ()
{
char fmt[] = "%s";
smprintf (buf, sizeof buf, fmt, fmt);
}
> This sounds backwards. Ordinarily (without restrict) there is a contract that
> prevents the compiler from optimizing the implementation in certain ways.
> With
> restrict, that part of the contract goes away. From the programmer's point of
> view, 'restrict' places additional restrictions on the caller, not on the
> callee. And appropriate use of 'restrict' signals to programmers what the
> callee
> can do.
I agree.
> This signal to humans is useful regardless of whether tools like GCC use
> the signal to generate warnings or optimizations.
I disagree. Given the confusion around what 'restrict' is actually about,
the "signal to humans" is fuzzy. As it stands, GCC >= 8 is the best bet for
a programmer to understand 'restrict'.
> > The "same array" rule comes from what I understood from various explanations
> > of 'restrict' on the web [1], and from what I see in the glibc headers for
> > functions that have multiple output parameters of the same type:
> >- splice in ,
> >- printf_arginfo_size_function in ,
> >- openpty in ,
> >- re_set_registers in ,
> >- copy_file_range in .
>
> That stackoverflow explanation is full of confusing answers and none of them
> are
> particularly good. And the glibc headers evidently were not written by people
> who cared about proper use of 'restrict' (possibly because they wrote the
> headers before 'restrict' existed or was widely supported).
... possibly also because ISO C 99 § 6.7.3.1 is impenetrable.
... possibly also because GCC 8 was not around to teach the programmers.
> > But ecvt, fcvt in have 'restrict'. You may want to register a
> > glibc
> > bug if you think these five functions should have 'restrict' :)
>
> It's not a bug. It's required by POSIX (which uses 'restrict' more carefully)
> in
> its older editions. See:
>
> https://pubs.opengroup.org/onlinepubs/009695399/functions/ecvt.html
I meant the "glibc bug" the other way around: Someone coul
Re: restrict
Hi Tim,
> > Yes, and this in turn means that the ability to produce useful warnings via
> > 'restrict' is limited. In this example:
> > ===
> > #include
> > extern void memmcpy (void *restrict, const void *restrict, size_t);
> >
> > void shuffle (char array[10])
> > {
> > memmcpy (array + 2, array, 8);
> > memcpy (array + 2, array, 8);
> > }
> > ===
> > gcc gives no warning about 'memmcpy' - because it does not know
> > how many elements the function will access. gcc does give a warning
> > about 'memcpy' - apparently due to custom logic in the compiler.
>
> The following gives you a warning, with -O2 / -O3 and -Wall:
>
> ===
> #include
> void memmcpy (void *restrict d, void *restrict s, size_t n)
> {
> memcpy(d, s, n);
> }
>
> void shuffle ()
> {
> char array[] = "abcdefg", *array2 = array + 2;
>
> memmcpy (array, array2 - 2, 8);
> }
> ===
That's precisely my point: GCC can not and will not give you warnings
about overlapping array slices, unless the function is one of the
predefined functions (memcpy in this case).
'restrict' for optimization is designed for accesses to array (cf.
the many examples regarding loops that iterate over an array). But
the warnings emitted by GCC can only recognize the special case of
two pointers being equal.
Bruno
Re: restrict - summary
Thanks to Paul and all for the enlightening discussion about what 'restrict' actually means. What I intend to do it gnulib is: * In the function *definitions*, do not use 'restrict' at all. Rationale: - Gnulib code is not time-critical. 'restrict' enables micro- optimizations, at the price of undefined behaviour for some callers. - Humans have a poor understanding of what 'restrict' means, and the GCC warnings cover only a subset of the undefined behaviour situations (namely, GCC can only warn about equal pointers, but not about overlapping array slices). * In the function *declarations* for POSIX functions, I will use 'restrict' exactly as written in POSIX. Rationale: - 'restrict' modifies the contract between the caller and the implementor of a function. Whatever 'restrict' precisely means, the contract that programmers will look at is the POSIX standard. * In the function *declarations* for non-POSIX functions, I will use 'restrict' when there is a significant benefit, and only on non-const pointer parameters. Rationale: - I want to see useful warnings from GCC. "Useful" means: with the normal programming style; passing 'void *' arguments to functions that expect 'char *', 'FILE *', etc. is not the normal programming style I want to care about. - The semantics of 'restrict' is a bit confusing, and the semantics of 'restrict' on const pointer parameters is additionally confusing. - It is typically sufficient - both for warnings and optimization - to put 'restrict' on the non-const pointer parameters only. Bruno
Re: restrict
On 2/17/20 12:55 PM, Bruno Haible wrote: it's pretty rare that people use 'void *' pointers and cast them to pointers of different types. It's not rare in some of the code I write. :-) Plus, my example had no casts. Also, even a small variation of this code does not produce a warning any more: Interesting. I don't see why GCC doesn't warn there. GCC's heuristics for generating warnings must be even trickier than what's in the C standard about 'restrict' - which is another reason not to base code off when exactly GCC warns. Actually, GCC doesn't warn here, even with 'restrict', because the 3rd argument is a const-pointer and the 4th argument, being variadic, is treated like a const-pointer as well. Fair enough, but there are similar examples where 'restrict' will still be useful, such as if the 3rd argument is not a const-pointer. Given the confusion around what 'restrict' is actually about, the "signal to humans" is fuzzy. Using 'restrict' in a nonstandard way will make that signal even fuzzier. It's better to be systematic about it, and to use the same system that the C standard and POSIX do. Like Dennis Ritchie and 'noalias'[1], I'm not a big fan of 'restrict'. But we have it and it's standardized and it's better to use it as intended rather than invent a not-quite-the-same use for it. I meant the "glibc bug" the other way around: Someone could tell the glibc people that it makes sense to _add_ 'restrict' to the declarations of splice, printf_arginfo_size_function, openpty, re_set_registers, copy_file_range. Yes, that would be worth doing. [1] https://www.lysator.liu.se/c/dmr-on-noalias.html
