Re: -fprofile-update=atomic vs. 32-bit architectures

2022-11-08 Thread Richard Biener via Gcc 
On Tue, Nov 8, 2022 at 7:22 AM Sebastian Huber
 wrote:
>
> On 05.11.22 12:18, Richard Biener wrote:
> > On Fri, Nov 4, 2022 at 9:28 AM Sebastian Huber
> >   wrote:
> >> Hello,
> >>
> >> even recent 32-bit architectures such as RISC-V do not support 64-bit
> >> atomic operations.  Using -fprofile-update=atomic for the 32-bit RISC-V
> >> RV32GC ISA yields:
> >>
> >> warning: target does not support atomic profile update, single mode is
> >> selected
> >>
> >> For multi-threaded applications it is quite important to use atomic
> >> counter increments to get valid coverage data. I think this fall back is
> >> not really good. Maybe we should consider using this approach from Jakub
> >> Jelinek for 32-bit architectures lacking 64-bit atomic operations:
> >>
> >> if (__atomic_add_fetch_4 ((unsigned int *) &val, 1, __ATOMIC_RELAXED)
> >> == 0)
> >>   __atomic_fetch_add_4 (((unsigned int *) &val) + 1, 1,
> >> __ATOMIC_RELAXED);
> >>
> >> https://patchwork.ozlabs.org/project/gcc/patch/19c4a81d-6ecd-8c6e-b641-e257c1959...@suse.cz/#1447334
> >>
> >> Last year I added the TARGET_GCOV_TYPE_SIZE target hook to optionally
> >> reduce the gcov type size to 32 bits. I am not really sure if this was a
> >> good idea. Longer running executables may observe counter overflows
> >> leading to invalid coverage data. If someone wants atomic updates, then
> >> the updates should be atomic even if this means to use a library
> >> implementation (libatomic).
> >>
> >> What about the following approach if -fprofile-update=atomic is given:
> >>
> >> 1. Use 64-bit atomics if available.
> >>
> >> 2. Use
> >>
> >> if (__atomic_add_fetch_4 ((unsigned int *) &val, 1, __ATOMIC_RELAXED)
> >> == 0)
> >>   __atomic_fetch_add_4 (((unsigned int *) &val) + 1, 1,
> >> __ATOMIC_RELAXED);
> >>
> >> if 32-bit atomics are available.
> >>
> >> 3. Else use a library call (libatomic).
> > sounds good, though a library call would really be prohibitly expensive?
>
> I someone wants to profile a multi-threaded application and selects
> -fprofile-update=atomic, then probably a correct result is preferred.
> You still have the option to select -fprofile-update=prefer-atomic.
>
> For 2. I have to modify:
>
> void
> gimple_gen_edge_profiler (int edgeno, edge e)
> {
>tree one;
>
>one = build_int_cst (gcov_type_node, 1);
>
>if (flag_profile_update == PROFILE_UPDATE_ATOMIC)
>  {
>/* __atomic_fetch_add (&counter, 1, MEMMODEL_RELAXED); */
>tree addr = tree_coverage_counter_addr (GCOV_COUNTER_ARCS, edgeno);
>tree f = builtin_decl_explicit (TYPE_PRECISION (gcov_type_node) > 32
>   ? BUILT_IN_ATOMIC_FETCH_ADD_8:
>   BUILT_IN_ATOMIC_FETCH_ADD_4);
>gcall *stmt = gimple_build_call (f, 3, addr, one,
>build_int_cst (integer_type_node,
>   MEMMODEL_RELAXED));
>gsi_insert_on_edge (e, stmt);
>  }
>
> Is
>
> if (WORDS_BIG_ENDIAN)
>
> the right way to check for big/little endian?

Yes.

> How do I get ((unsigned int *) &val) + 1 from tree addr?
>
> It would be great to have a code example for the construction of the "if
> (f()) f();".

I think for the function above we need to emit __atomic_fetch_add_8,
not the emulated form because we cannot insert the required control
flow (if (f()) f()) on an edge.  The __atomic_fetch_add_8 should then be
lowered after the instrumentation took place.

There's currently no helper to create a diamond so the canonical
way is to create a GIMPLE_COND, split the block after this stmt,
split the outgoing edge and then redirect edges to form a half-diamond.
move_sese_in_condition has most of that CFG manipulation (but
it performs sth different)

Richard.

> --
> embedded brains GmbH
> Herr Sebastian HUBER
> Dornierstr. 4
> 82178 Puchheim
> Germany
> email: sebastian.hu...@embedded-brains.de
> phone: +49-89-18 94 741 - 16
> fax:   +49-89-18 94 741 - 08
>
> Registergericht: Amtsgericht München
> Registernummer: HRB 157899
> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> Unsere Datenschutzerklärung finden Sie hier:
> https://embedded-brains.de/datenschutzerklaerung/

msp430-elf cross-compiler: libstdc++ supported?

2022-11-08 Thread Florian Weimer via Gcc 
I'm trying to validate a change to gcc/config/msp430/msp430.cc.
The cross-compiler builds as far as I can tell, but I hit a snag while
configuring libstdc++:

checking for shl_load... configure: error: Link tests are not allowed after 
GCC_NO_EXECUTABLES.
make[1]: *** [Makefile:12334: configure-target-libstdc++-v3] Error 1
make[1]: Leaving directory '/root/msp/build-gcc'

Is this expected?  Is libstdc++ even supported for this target?
Thanks,
Florian

Re: -fprofile-update=atomic vs. 32-bit architectures

2022-11-08 Thread Sebastian Huber 

On 08.11.22 11:25, Richard Biener wrote:

How do I get ((unsigned int *) &val) + 1 from tree addr?

It would be great to have a code example for the construction of the "if
(f()) f();".

I think for the function above we need to emit __atomic_fetch_add_8,
not the emulated form because we cannot insert the required control
flow (if (f()) f()) on an edge.  The __atomic_fetch_add_8 should then be
lowered after the instrumentation took place.


Ok, I am not a compiler expert, so I have just a vague feeling how this 
works. I am not sure which piece is responsible for the "lowering" of 
this particular __atomic_fetch_add_8. I guess we don't want to split all 
__atomic_fetch_add_8 into this if (f()) f(); form?




There's currently no helper to create a diamond so the canonical
way is to create a GIMPLE_COND, split the block after this stmt,
split the outgoing edge and then redirect edges to form a half-diamond.
move_sese_in_condition has most of that CFG manipulation (but
it performs sth different)


Thanks, I will probably able to do this with a bit trial and error.

--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/

Re: msp430-elf cross-compiler: libstdc++ supported?

2022-11-08 Thread Jonathan Wakely via Gcc 
On Tue, 8 Nov 2022 at 11:22, Florian Weimer via Libstdc++
 wrote:
>
> I'm trying to validate a change to gcc/config/msp430/msp430.cc.
> The cross-compiler builds as far as I can tell, but I hit a snag while
> configuring libstdc++:
>
> checking for shl_load... configure: error: Link tests are not allowed after 
> GCC_NO_EXECUTABLES.
> make[1]: *** [Makefile:12334: configure-target-libstdc++-v3] Error 1
> make[1]: Leaving directory '/root/msp/build-gcc'
>
> Is this expected?  Is libstdc++ even supported for this target?

Yes. What's your configure command? You probably need at least --with-newlib.

I think my last successful msp430 build was using:

.../gcc/configure --target=msp430-elf --prefix=$HOME/gcc/msp430-elf
--disable-libcc1 --disable-libgomp --disable-libitm --disable-libssp
--disable-libvtv --disable-nls --with-system-zlib
target_alias=msp430-elf --disable-libstdcxx-pch
--disable-full-memory-range --with-newlib --enable-libstdcxx
--enable-languages=c,c++

With the cross binutils already installed to $PREFIX/gcc/msp430-elf

Re: msp430-elf cross-compiler: libstdc++ supported?

2022-11-08 Thread Jonathan Wakely via Gcc 
On Tue, 8 Nov 2022 at 12:37, Jonathan Wakely wrote:
>
> On Tue, 8 Nov 2022 at 11:22, Florian Weimer via Libstdc++
>  wrote:
> >
> > I'm trying to validate a change to gcc/config/msp430/msp430.cc.
> > The cross-compiler builds as far as I can tell, but I hit a snag while
> > configuring libstdc++:
> >
> > checking for shl_load... configure: error: Link tests are not allowed after 
> > GCC_NO_EXECUTABLES.
> > make[1]: *** [Makefile:12334: configure-target-libstdc++-v3] Error 1
> > make[1]: Leaving directory '/root/msp/build-gcc'
> >
> > Is this expected?  Is libstdc++ even supported for this target?
>
> Yes. What's your configure command? You probably need at least --with-newlib.
>
> I think my last successful msp430 build was using:
>
> .../gcc/configure --target=msp430-elf --prefix=$HOME/gcc/msp430-elf
> --disable-libcc1 --disable-libgomp --disable-libitm --disable-libssp
> --disable-libvtv --disable-nls --with-system-zlib
> target_alias=msp430-elf --disable-libstdcxx-pch
> --disable-full-memory-range --with-newlib --enable-libstdcxx
> --enable-languages=c,c++
>
> With the cross binutils already installed to $PREFIX/gcc/msp430-elf

Oh actually the libstdc++ build fails on trunk, but that is a very
recent regression and happens after configure succeeds:

/home/jwakely/src/gcc/gcc/libstdc++-v3/src/c++17/floating_from_chars.cc:
In function 'std::from_chars_result
std::{anonymous}::__floating_from_chars_hex(const char*, const char*,
T&)':
/home/jwakely/src/gcc/gcc/libstdc++-v3/src/c++17/floating_from_chars.cc:786:9:
error: 'fast_float' has not been declared
 786 |   = fast_float::binary_format::mantissa_explicit_bits();
 | ^~
/home/jwakely/src/gcc/gcc/libstdc++-v3/src/c++17/floating_from_chars.cc:786:36:
error: expected primary-expression before '>' token
 786 |   = fast_float::binary_format::mantissa_explicit_bits();
 |^
libtool: compile:  /home/jwakely/src/gcc/build-16bit/./gcc/xgcc
-shared-libgcc -B/home/jwakely/src/gcc/build-16bit/./gcc -nostdinc++
-L/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/src
-L/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/src/.libs
-L/home/jwakely/src/gcc/build-16bit/msp430-elf/li
bstdc++-v3/libsupc++/.libs -nostdinc
-B/home/jwakely/src/gcc/build-16bit/msp430-elf/newlib/ -isystem
/home/jwakely/src/gcc/build-16bit/msp430-elf/newlib/targ-include
-isystem /home/jwakely/src/gcc/gcc/newlib/libc/include
-B/home/jwakely/src/gcc/build-16bit/msp430-elf/libgloss/msp430
-L/home/jwakely/src/gcc/build-16
bit/msp430-elf/libgloss/libnosys
-L/home/jwakely/src/gcc/gcc/libgloss/msp430
-B/home/jwakely/gcc/msp430-elf/msp430-elf/bin/
-B/home/jwakely/gcc/msp430-elf/msp430-elf/lib/ -isystem
/home/jwakely/gcc/msp430-elf/msp430-elf/include -isystem
/home/jwakely/gcc/msp430-elf/msp430-elf/sys-include
-I/home/jwakely/src/gcc/gcc
/libstdc++-v3/../libgcc
-I/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/include/msp430-elf
-I/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/include
-I/home/jwakely/src/gcc/gcc/libstdc++-v3/libsupc++ -std=gnu++17
-nostdinc++ -fno-implicit-templates -Wall -Wextra -Wwrite-strings
-Wcast-qual -Wa
bi=2 -fdiagnostics-show-location=once -ffunction-sections
-fdata-sections -frandom-seed=string-inst.lo -fimplicit-templates -g
-O2 -c /home/jwakely/src/gcc/gcc/libstdc++-v3/src/c++17/string-inst.cc
-o string-inst.o
/bin/sh ../../libtool --tag CXX --tag disable-shared   --mode=compile
/home/jwakely/src/gcc/build-16bit/./gcc/xgcc -shared-libgcc
-B/home/jwakely/src/gcc/build-16bit/./gcc -nostdinc++
-L/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/src
-L/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/src/.lib
s -L/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/libsupc++/.libs
-nostdinc -B/home/jwakely/src/gcc/build-16bit/msp430-elf/newlib/
-isystem /home/jwakely/src/gcc/build-16bit/msp430-elf/newlib/targ-include
-isystem /home/jwakely/src/gcc/gcc/newlib/libc/include
-B/home/jwakely/src/gcc/build-16bit/msp430-e
lf/libgloss/msp430
-L/home/jwakely/src/gcc/build-16bit/msp430-elf/libgloss/libnosys
-L/home/jwakely/src/gcc/gcc/libgloss/msp430
-B/home/jwakely/gcc/msp430-elf/msp430-elf/bin/
-B/home/jwakely/gcc/msp430-elf/msp430-elf/lib/ -isystem
/home/jwakely/gcc/msp430-elf/msp430-elf/include -isystem
/home/jwakely/gcc/msp430-elf
/msp430-elf/sys-include
-I/home/jwakely/src/gcc/gcc/libstdc++-v3/../libgcc
-I/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/include/msp430-elf
-I/home/jwakely/src/gcc/build-16bit/msp430-elf/libstdc++-v3/include
-I/home/jwakely/src/gcc/gcc/libstdc++-v3/libsupc++   -std=gnu++17
-nostdinc++   -fno-implic
it-templates  -Wall -Wextra -Wwrite-strings -Wcast-qual -Wabi=2
-fdiagnostics-show-location=once   -ffunction-sections -fdata-sections
 -frandom-seed=cow-string-inst.lo  -fimplicit-templates -g -O2  -c -o
cow-string-inst.lo
/home/jwakely/src/gcc/gcc/libstdc++-v3/src/c++17/cow-string-inst.cc
/home/jwakely/src/gcc/gcc/libstdc++-v3/sr

Re: -fprofile-update=atomic vs. 32-bit architectures

2022-11-08 Thread Richard Biener via Gcc 
On Tue, Nov 8, 2022 at 1:00 PM Sebastian Huber
 wrote:
>
> On 08.11.22 11:25, Richard Biener wrote:
> >> How do I get ((unsigned int *) &val) + 1 from tree addr?
> >>
> >> It would be great to have a code example for the construction of the "if
> >> (f()) f();".
> > I think for the function above we need to emit __atomic_fetch_add_8,
> > not the emulated form because we cannot insert the required control
> > flow (if (f()) f()) on an edge.  The __atomic_fetch_add_8 should then be
> > lowered after the instrumentation took place.
>
> Ok, I am not a compiler expert, so I have just a vague feeling how this
> works. I am not sure which piece is responsible for the "lowering" of
> this particular __atomic_fetch_add_8. I guess we don't want to split all
> __atomic_fetch_add_8 into this if (f()) f(); form?

I think we should do it right after the profile instrumentation commits
the edge insertions.  And yes, we only want to lower those that
are not natively supported (but have native support for fetch_add_4).
Not sure how to determine this.

>
> >
> > There's currently no helper to create a diamond so the canonical
> > way is to create a GIMPLE_COND, split the block after this stmt,
> > split the outgoing edge and then redirect edges to form a half-diamond.
> > move_sese_in_condition has most of that CFG manipulation (but
> > it performs sth different)
>
> Thanks, I will probably able to do this with a bit trial and error.
>
> --
> embedded brains GmbH
> Herr Sebastian HUBER
> Dornierstr. 4
> 82178 Puchheim
> Germany
> email: sebastian.hu...@embedded-brains.de
> phone: +49-89-18 94 741 - 16
> fax:   +49-89-18 94 741 - 08
>
> Registergericht: Amtsgericht München
> Registernummer: HRB 157899
> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> Unsere Datenschutzerklärung finden Sie hier:
> https://embedded-brains.de/datenschutzerklaerung/

Announcement: Porting the Docs to Sphinx - tomorrow

2022-11-08 Thread Martin Liška 
Hi.

Tomorrow in the morning (UTC time), I'm going to migrate the documentation
to Sphinx. The final version of the branch can be seen here:

$ git fetch origin refs/users/marxin/heads/sphinx-final
$ git co FETCH_HEAD 

URL: https://splichal.eu/gccsphinx-final/

TL;DR;

After the migration, people should be able to build (and install) GCC even
if they miss Sphinx (similar happens now if you miss makeinfo). However, please
install Sphinx >= 5.3.0 (for manual and info pages - only *core* package is 
necessary) [1]

Steps following the migration:

1) update of web HTML (and PDF documentation) pages:
   I prepared a script and tested our server has all what we need.
2) gcc_release --enable-generated-files-in-srcdir: here I would like
   to ask Joseph for cooperation
3) URL for diagnostics (used for warning) - will utilize [3]
4) package source tarballs - https://gcc.gnu.org/onlinedocs/ (listed here)
5) updating links from gcc.gnu.org that point to documentation
6) removal of the further Texinfo leftovers
...

Cheers,
Martin

[1] 
https://splichal.eu/scripts/sphinx/gccint/_build/html/source-tree-structure-and-build-system/the-gcc-subdirectory/building-documentation.html#sphinx-install
[2] ./maintainer-scripts/update_web_docs_git.py
[3] https://pypi.org/project/sphinx-redirect-by-id/

Re: Announcement: Porting the Docs to Sphinx - tomorrow

2022-11-08 Thread Sam James via Gcc 


> On 8 Nov 2022, at 13:55, Martin Liška  wrote:
> 
> Hi.
> 
> Tomorrow in the morning (UTC time), I'm going to migrate the documentation
> to Sphinx. The final version of the branch can be seen here:
> 
> $ git fetch origin refs/users/marxin/heads/sphinx-final
> $ git co FETCH_HEAD
> 
> URL: https://splichal.eu/gccsphinx-final/
> 
> TL;DR;
> 
> After the migration, people should be able to build (and install) GCC even
> if they miss Sphinx (similar happens now if you miss makeinfo). However, 
> please
> install Sphinx >= 5.3.0 (for manual and info pages - only *core* package is 
> necessary) [1]
> 
> Steps following the migration:
> 
> 1) update of web HTML (and PDF documentation) pages:
>   I prepared a script and tested our server has all what we need.
> 2) gcc_release --enable-generated-files-in-srcdir: here I would like
>   to ask Joseph for cooperation

Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
even for snapshots? Pretty please? :)

> 3) URL for diagnostics (used for warning) - will utilize [3]
> 4) package source tarballs - https://gcc.gnu.org/onlinedocs/ (listed here)
> 5) updating links from gcc.gnu.org that point to documentation
> 6) removal of the further Texinfo leftovers
> ...
> 

Thanks for working on this. Very excited.

> Cheers,
> Martin

Best,
sam


signature.asc
Description: Message signed with OpenPGP

Re: Announcement: Porting the Docs to Sphinx - tomorrow

2022-11-08 Thread Joseph Myers 
On Tue, 8 Nov 2022, Sam James via Gcc wrote:

> Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
> even for snapshots? Pretty please? :)

I think we want snapshots to come out weekly even if the compiler or 
documentation build fails, which makes anything involving a build as part 
of the snapshot process problematic.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - tomorrow

2022-11-08 Thread Sam James via Gcc 


> On 9 Nov 2022, at 00:00, Joseph Myers  wrote:
> 
> On Tue, 8 Nov 2022, Sam James via Gcc wrote:
> 
>> Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
>> even for snapshots? Pretty please? :)
> 
> I think we want snapshots to come out weekly even if the compiler or
> documentation build fails, which makes anything involving a build as part
> of the snapshot process problematic.

If that's your expectation, that's fine, but I'd regard it as pretty
serious if one didn't build, and I don't remember a time when it didn't.

It's not like it's that much use if it fails to build on a bog-standard
amd64 platform anyway, as if nothing else, you'd get a deluge
of duplicate bug reports.


signature.asc
Description: Message signed with OpenPGP

[PATCH v3 0/3] RFC: P1689R5 support

2022-11-08 Thread Ben Boeckel via Gcc 
Hi,

This patch adds initial support for ISO C++'s [P1689R5][], a format for
describing C++ module requirements and provisions based on the source
code. This is required because compiling C++ with modules is not
embarrassingly parallel and need to be ordered to ensure that `import
some_module;` can be satisfied in time by making sure that the TU with
`export import some_module;` is compiled first.

[P1689R5]: https://isocpp.org/files/papers/P1689R5.html

I'd like feedback on the approach taken here with respect to the
user-visible flags. I'll also note that header units are not supported
at this time because the current `-E` behavior with respect to `import
;` is to search for an appropriate `.gcm` file which is not
something such a "scan" can support. A new mode will likely need to be
created (e.g., replacing `-E` with `-fc++-module-scanning` or something)
where headers are looked up "normally" and processed only as much as
scanning requires.

For the record, Clang has patches with similar flags and behavior by
Chuanqi Xu here:

https://reviews.llvm.org/D134269

with the same flags.

Thanks,

--Ben

---
v2 -> v3:

- changelog entries moved to commit messages
- documentation updated/added in the UTF-8 routine editing

v1 -> v2:

- removal of the `deps_write(extra)` parameter to option-checking where
  ndeeded
- default parameter of `cpp_finish(fdeps_stream = NULL)`
- unification of libcpp UTF-8 validity functions from v1
- test cases for flag parsing states (depflags-*) and p1689 output
  (p1689-*)

Ben Boeckel (3):
  libcpp: reject codepoints above 0x10
  libcpp: add a function to determine UTF-8 validity of a C string
  p1689r5: initial support

 gcc/c-family/c-opts.cc|  40 +++-
 gcc/c-family/c.opt|  12 +
 gcc/cp/module.cc  |   3 +-
 gcc/doc/invoke.texi   |  15 ++
 gcc/testsuite/g++.dg/modules/depflags-f-MD.C  |   2 +
 gcc/testsuite/g++.dg/modules/depflags-f.C |   1 +
 gcc/testsuite/g++.dg/modules/depflags-fi.C|   3 +
 gcc/testsuite/g++.dg/modules/depflags-fj-MD.C |   3 +
 gcc/testsuite/g++.dg/modules/depflags-fj.C|   4 +
 .../g++.dg/modules/depflags-fjo-MD.C  |   4 +
 gcc/testsuite/g++.dg/modules/depflags-fjo.C   |   5 +
 gcc/testsuite/g++.dg/modules/depflags-fo-MD.C |   3 +
 gcc/testsuite/g++.dg/modules/depflags-fo.C|   4 +
 gcc/testsuite/g++.dg/modules/depflags-j-MD.C  |   2 +
 gcc/testsuite/g++.dg/modules/depflags-j.C |   3 +
 gcc/testsuite/g++.dg/modules/depflags-jo-MD.C |   3 +
 gcc/testsuite/g++.dg/modules/depflags-jo.C|   4 +
 gcc/testsuite/g++.dg/modules/depflags-o-MD.C  |   2 +
 gcc/testsuite/g++.dg/modules/depflags-o.C |   3 +
 gcc/testsuite/g++.dg/modules/modules.exp  |   1 +
 gcc/testsuite/g++.dg/modules/p1689-1.C|  18 ++
 gcc/testsuite/g++.dg/modules/p1689-1.exp.json |  27 +++
 gcc/testsuite/g++.dg/modules/p1689-2.C|  16 ++
 gcc/testsuite/g++.dg/modules/p1689-2.exp.json |  16 ++
 gcc/testsuite/g++.dg/modules/p1689-3.C|  14 ++
 gcc/testsuite/g++.dg/modules/p1689-3.exp.json |  16 ++
 gcc/testsuite/g++.dg/modules/p1689-4.C|  14 ++
 gcc/testsuite/g++.dg/modules/p1689-4.exp.json |  14 ++
 gcc/testsuite/g++.dg/modules/p1689-5.C|  14 ++
 gcc/testsuite/g++.dg/modules/p1689-5.exp.json |  14 ++
 gcc/testsuite/g++.dg/modules/test-p1689.py| 222 ++
 gcc/testsuite/lib/modules.exp |  71 ++
 libcpp/charset.cc |  28 ++-
 libcpp/include/cpplib.h   |  12 +-
 libcpp/include/mkdeps.h   |  17 +-
 libcpp/init.cc|  13 +-
 libcpp/internal.h |   2 +
 libcpp/mkdeps.cc  | 149 +++-
 38 files changed, 773 insertions(+), 21 deletions(-)
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fi.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.exp.json
 create mode 100644 gcc/testsu

[PATCH v3 1/3] libcpp: reject codepoints above 0x10FFFF

2022-11-08 Thread Ben Boeckel via Gcc 
Unicode does not support such values because they are unrepresentable in
UTF-16.

libcpp/

* charset.cc: Reject encodings of codepoints above 0x10.
UTF-16 does not support such codepoints and therefore all
Unicode rejects such values.

Signed-off-by: Ben Boeckel 
---
 libcpp/charset.cc | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/libcpp/charset.cc b/libcpp/charset.cc
index 12a398e7527..324b5b19136 100644
--- a/libcpp/charset.cc
+++ b/libcpp/charset.cc
@@ -158,6 +158,10 @@ struct _cpp_strbuf
encoded as any of DF 80, E0 9F 80, F0 80 9F 80, F8 80 80 9F 80, or
FC 80 80 80 9F 80.  Only the first is valid.
 
+   Additionally, Unicode declares that all codepoints above 0010 are
+   invalid because they cannot be represented in UTF-16. As such, all 5- and
+   6-byte encodings are invalid.
+
An implementation note: the transformation from UTF-16 to UTF-8, or
vice versa, is easiest done by using UTF-32 as an intermediary.  */
 
@@ -216,7 +220,7 @@ one_utf8_to_cppchar (const uchar **inbufp, size_t 
*inbytesleftp,
   if (c <= 0x3FF && nbytes > 5) return EILSEQ;
 
   /* Make sure the character is valid.  */
-  if (c > 0x7FFF || (c >= 0xD800 && c <= 0xDFFF)) return EILSEQ;
+  if (c > 0x10 || (c >= 0xD800 && c <= 0xDFFF)) return EILSEQ;
 
   *cp = c;
   *inbufp = inbuf;
@@ -320,7 +324,7 @@ one_utf32_to_utf8 (iconv_t bigend, const uchar **inbufp, 
size_t *inbytesleftp,
   s += inbuf[bigend ? 2 : 1] << 8;
   s += inbuf[bigend ? 3 : 0];
 
-  if (s >= 0x7FFF || (s >= 0xD800 && s <= 0xDFFF))
+  if (s > 0x10 || (s >= 0xD800 && s <= 0xDFFF))
 return EILSEQ;
 
   rval = one_cppchar_to_utf8 (s, outbufp, outbytesleftp);
-- 
2.38.1

[PATCH v3 2/3] libcpp: add a function to determine UTF-8 validity of a C string

2022-11-08 Thread Ben Boeckel via Gcc 
This simplifies the interface for other UTF-8 validity detections when a
simple "yes" or "no" answer is sufficient.

libcpp/

* charset.cc: Add `_cpp_valid_utf8_str` which determines whether
a C string is valid UTF-8 or not.
* internal.h: Add prototype for `_cpp_valid_utf8_str`.

Signed-off-by: Ben Boeckel 
---
 libcpp/charset.cc | 20 
 libcpp/internal.h |  2 ++
 2 files changed, 22 insertions(+)

diff --git a/libcpp/charset.cc b/libcpp/charset.cc
index 324b5b19136..e130bc01f48 100644
--- a/libcpp/charset.cc
+++ b/libcpp/charset.cc
@@ -1868,6 +1868,26 @@ _cpp_valid_utf8 (cpp_reader *pfile,
   return true;
 }
 
+/*  Detect whether a C-string is a valid UTF-8-encoded set of bytes. Returns
+`false` if any contained byte sequence encodes an invalid Unicode codepoint
+or is not a valid UTF-8 sequence. Returns `true` otherwise. */
+
+extern bool
+_cpp_valid_utf8_str (const char *name)
+{
+  const uchar* in = (const uchar*)name;
+  size_t len = strlen(name);
+  cppchar_t cp;
+
+  while (*in)
+{
+  if (one_utf8_to_cppchar(&in, &len, &cp))
+   return false;
+}
+
+  return true;
+}
+
 /* Subroutine of convert_hex and convert_oct.  N is the representation
in the execution character set of a numeric escape; write it into the
string buffer TBUF and update the end-of-string pointer therein.  WIDE
diff --git a/libcpp/internal.h b/libcpp/internal.h
index badfd1b40da..4f2dd4a2f5c 100644
--- a/libcpp/internal.h
+++ b/libcpp/internal.h
@@ -834,6 +834,8 @@ extern bool _cpp_valid_utf8 (cpp_reader *pfile,
 struct normalize_state *nst,
 cppchar_t *cp);
 
+extern bool _cpp_valid_utf8_str (const char *str);
+
 extern void _cpp_destroy_iconv (cpp_reader *);
 extern unsigned char *_cpp_convert_input (cpp_reader *, const char *,
  unsigned char *, size_t, size_t,
-- 
2.38.1

[PATCH v3 3/3] p1689r5: initial support

2022-11-08 Thread Ben Boeckel via Gcc 
This patch implements support for [P1689R5][] to communicate to a build
system the C++20 module dependencies to build systems so that they may
build `.gcm` files in the proper order.

Support is communicated through the following three new flags:

- `-fdeps-format=` specifies the format for the output. Currently named
  `p1689r5`.

- `-fdeps-file=` specifies the path to the file to write the format to.

- `-fdep-output=` specifies the `.o` that will be written for the TU
  that is scanned. This is required so that the build system can
  correlate the dependency output with the actual compilation that will
  occur.

CMake supports this format as of 17 Jun 2022 (to be part of 3.25.0)
using an experimental feature selection (to allow for future usage
evolution without committing to how it works today). While it remains
experimental, docs may be found in CMake's documentation for
experimental features.

Future work may include using this format for Fortran module
dependencies as well, however this is still pending work.

[P1689R5]: https://isocpp.org/files/papers/P1689R5.html
[cmake-experimental]: 
https://gitlab.kitware.com/cmake/cmake/-/blob/master/Help/dev/experimental.rst

TODO:

- header-unit information fields

Header units (including the standard library headers) are 100%
unsupported right now because the `-E` mechanism wants to import their
BMIs. A new mode (i.e., something more workable than existing `-E`
behavior) that mocks up header units as if they were imported purely
from their path and content would be required.

- non-utf8 paths

The current standard says that paths that are not unambiguously
represented using UTF-8 are not supported (because these cases are rare
and the extra complication is not worth it at this time). Future
versions of the format might have ways of encoding non-UTF-8 paths. For
now, this patch just doesn't support non-UTF-8 paths (ignoring the
"unambiguously represetable in UTF-8" case).

- figure out why junk gets placed at the end of the file

Sometimes it seems like the file gets a lot of `NUL` bytes appended to
it. It happens rarely and seems to be the result of some
`ftruncate`-style call which results in extra padding in the contents.
Noting it here as an observation at least.

libcpp/

* include/cpplib.h: Add cpp_deps_format enum.
(cpp_options): Add format field
(cpp_finish): Add dependency stream parameter.
* include/mkdeps.h (deps_add_module_target): Add new preprocessor
parameter used for C++ module tracking.
* init.cc (cpp_finish): Add new preprocessor parameter used for C++
module tracking.
* mkdeps.cc (mkdeps): Implement P1689R5 output.

gcc/

* doc/invoke.texi: Document -fdeps-format=, -fdep-file=, and
-fdep-output= flags.

gcc/c-family/

* c-opts.cc (c_common_handle_option): Add fdeps_file variable and
-fdeps-format=, -fdep-file=, and -fdep-output= parsing.
* c.opt: Add -fdeps-format=, -fdep-file=, and -fdep-output= flags.

gcc/cp/

* module.cc (preprocessed_module): Pass whether the module is
exported to dependency tracking.

gcc/testsuite/

* g++.dg/modules/depflags-f-MD.C: New test.
* g++.dg/modules/depflags-f.C: New test.
* g++.dg/modules/depflags-fi.C: New test.
* g++.dg/modules/depflags-fj-MD.C: New test.
* g++.dg/modules/depflags-fj.C: New test.
* g++.dg/modules/depflags-fjo-MD.C: New test.
* g++.dg/modules/depflags-fjo.C: New test.
* g++.dg/modules/depflags-fo-MD.C: New test.
* g++.dg/modules/depflags-fo.C: New test.
* g++.dg/modules/depflags-j-MD.C: New test.
* g++.dg/modules/depflags-j.C: New test.
* g++.dg/modules/depflags-jo-MD.C: New test.
* g++.dg/modules/depflags-jo.C: New test.
* g++.dg/modules/depflags-o-MD.C: New test.
* g++.dg/modules/depflags-o.C: New test.
* g++.dg/modules/p1689-1.C: New test.
* g++.dg/modules/p1689-1.exp.json: New test expectation.
* g++.dg/modules/p1689-2.C: New test.
* g++.dg/modules/p1689-2.exp.json: New test expectation.
* g++.dg/modules/p1689-3.C: New test.
* g++.dg/modules/p1689-3.exp.json: New test expectation.
* g++.dg/modules/p1689-4.C: New test.
* g++.dg/modules/p1689-4.exp.json: New test expectation.
* g++.dg/modules/p1689-5.C: New test.
* g++.dg/modules/p1689-5.exp.json: New test expectation.
* g++.dg/modules/modules.exp: Load new P1689 library routines.
* g++.dg/modules/test-p1689.py: New tool for validating P1689 output.
* lib/modules.exp: Support for validating P1689 outputs.

Signed-off-by: Ben Boeckel 
---
 gcc/c-family/c-opts.cc|  40 +++-
 gcc/c-family/c.opt|  12 +
 gcc/cp/module.cc  |   3 +-
 gcc/doc/invoke.texi   |  15 ++
 gcc/testsuite/g++.dg/modules/depflags-f

"random" segment faults in expect running GCC regression test on LoongArch

2022-11-08 Thread Xi Ruoyao via Gcc 
Hi,

This is a "help wanted" message.  When I run GCC regression test on
loongarch64-linux-gnu, expect occasionally crashes with a segment fault.
The stack backtrace is like:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x70a35910 in TclpAlloc () from /usr/lib/libtcl8.6.so
[Current thread is 1 (Thread 0x705d8420 (LWP 9228))]
(gdb) bt
#0  0x70a35910 in TclpAlloc () from /usr/lib/libtcl8.6.so
#1  0x708b0ed4 in sortins () from /usr/lib/libtcl8.6.so
#2  0x708b18d0 in mergeins () from /usr/lib/libtcl8.6.so
#3  0x708b3654 in fixempties () from /usr/lib/libtcl8.6.so
#4  0x708b2564 in optimize () from /usr/lib/libtcl8.6.so
#5  0x708aa9a0 in nfanode () from /usr/lib/libtcl8.6.so
#6  0x708aa8a4 in nfatree () from /usr/lib/libtcl8.6.so
#7  0x708aa894 in nfatree () from /usr/lib/libtcl8.6.so
#8  0x708aa894 in nfatree () from /usr/lib/libtcl8.6.so
#9  0x708aa894 in nfatree () from /usr/lib/libtcl8.6.so
#10 0x708a6c00 in TclReComp () from /usr/lib/libtcl8.6.so
#11 0x70a2155c in CompileRegexp () from /usr/lib/libtcl8.6.so
#12 0x70a20e74 in Tcl_GetRegExpFromObj () from /usr/lib/libtcl8.6.so
#13 0x708ebe48 in Tcl_RegsubObjCmd () from /usr/lib/libtcl8.6.so
#14 0x708c6e64 in Dispatch () from /usr/lib/libtcl8.6.so
#15 0x708c6f10 in TclNRRunCallbacks () from /usr/lib/libtcl8.6.so
#16 0x708c6650 in Tcl_EvalObjv () from /usr/lib/libtcl8.6.so
#17 0x708c8f7c in TclEvalEx () from /usr/lib/libtcl8.6.so
#18 0x709f1270 in Tcl_FSEvalFileEx () from /usr/lib/libtcl8.6.so
#19 0x709f0f00 in Tcl_FSEvalFile () from /usr/lib/libtcl8.6.so
#20 0x709ef26c in Tcl_EvalFile () from /usr/lib/libtcl8.6.so
#21 0x70afc658 in exp_interpret_cmdfilename ()
   from /usr/lib/libexpect5.45.4.so
#22 0x00012c80 in main ()

The crashes happen on both my own LoongArch board and
gcc401.fsffrance.org.  So I don't think it was an "only me" problem.

The segment fault is not deterministic.  A Expect command like

cd /path/to/gcc/build/gcc
expect -- /usr/share/dejagnu/runtest.exp --tool gfortran 
dg.exp=bind-c-contiguous-1.f90

sometimes succeeds, sometimes immediately segfaults, sometimes prints
several lines of strange error message and then exits abnormally.

Usually I don't like to send "help wanted" messages, but this time I
have no idea how to fix (or even triage) the problem.  I don't even know
which component (Tcl? Expect? GCC? Glibc? Kernel?) I should blame.  So
it seems I really need some help now...

-- 
Xi Ruoyao 
School of Aerospace Science and Technology, Xidian University

Re: Announcement: Porting the Docs to Sphinx - tomorrow

2022-11-08 Thread Richard Biener via Gcc 
On Wed, Nov 9, 2022 at 1:09 AM Sam James via Gcc-patches
 wrote:
>
>
>
> > On 9 Nov 2022, at 00:00, Joseph Myers  wrote:
> >
> > On Tue, 8 Nov 2022, Sam James via Gcc wrote:
> >
> >> Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
> >> even for snapshots? Pretty please? :)
> >
> > I think we want snapshots to come out weekly even if the compiler or
> > documentation build fails, which makes anything involving a build as part
> > of the snapshot process problematic.
>
> If that's your expectation, that's fine, but I'd regard it as pretty
> serious if one didn't build, and I don't remember a time when it didn't.
>
> It's not like it's that much use if it fails to build on a bog-standard
> amd64 platform anyway, as if nothing else, you'd get a deluge
> of duplicate bug reports.

I'd say that doing a trunk snapshot build every day as CI would be nice, we
can then publish one once a week, skipping days where the build failed.

For release branches having generated files in the snapshots would be
even more useful and I very much hope the build there always succeeds.

Richard.