Re: [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard

[Jason Merrill via Gcc]

On 10/19/22 04:40, Jakub Jelinek wrote:

Hi!

The screw-up on my side with libstdc++ testing (tested normally rather
than in C++23 mode) makes me wonder if we couldn't tweak the default
testing.
Dunno what libstdc++ testing normally does (just C++17?), make check-g++
tests by default { 98, 14, 17, 20 } (and I regularly use
GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
cover libstdc++ I guess).
When adding tests for upcoming C++ version, one always has a dilemma
whether to use explicit // { dg-options "-std=c++2b" }
or -std=gnu++2b and similar, then the test works in all modes, but it might
be forgotten later on to be converted into // { dg-do whatever { target c++23 } 
}
test so that when 23 is tested by default and say 26 or 29 appears too,
we test it also in those modes, or just go with
// { dg-do whatever { target c++23 } }
which has the disadvantage that it is skipped when testing by default and
one only tests it if he asks for the newer version.

I wonder if we couldn't for the default testing (when one doesn't
specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
improve things a little bit by automatically treat those
// { dg-do whatever { target c++23 } }
tests as // { dg-options "-std=c++2b" }.

That would be great.

g++-dg.exp has:
         # If the testcase specifies a standard, use that one.
         # If not, run it under several standards, allowing GNU extensions
         # if there's a dg-options line.
         if ![search_for $test "-std=*++"] {
             if [search_for $test "dg-options"] {
                 set std_prefix "-std=gnu++"
             } else {
                 set std_prefix "-std=c++"
             }

# See g++.exp for the initial value of this list.

             global gpp_std_list
             if { [llength $gpp_std_list] > 0 } {
                 set std_list $gpp_std_list
             } else {
                 set std_list { 98 14 17 20 }
             }
             set option_list { }
             foreach x $std_list {
                 # Handle "concepts" as C++17 plus Concepts TS.
                 if { $x eq "concepts" } then { set x "17 -fconcepts"
                 } elseif { $x eq "impcx" } then { set x "23 
-fimplicit-constexpr" }
                 lappend option_list "${std_prefix}$x"
             }
         } else {
             set option_list { "" }
         }

set nshort [file tail [file dirname $test]]/[file tail $test]

         foreach flags_t $option_list {
             verbose "Testing $nshort, $flags $flags_t" 1
             dg-test $test "$flags $flags_t" ${default-extra-flags}
         }
so I wonder if in the set std_list { 98 14 17 20 } spot we couldn't do
something like special search_for for "{ dg-do * { target c++23 } }"
and if so, set std_list { 2b } instead of set std_list { 98 14 17 20 }?
It wouldn't handle more complex cases like
// { dg-do compile { target { c++23 && { aarch64*-*-* powerpc64le*-*-linux* 
riscv*-*-* s390*-*-* sparc*-*-linux* } } } }
but at least for the majority of tests for the new language version
it would run them even in default testing where they'd be otherwise
skipped (we'd cycle over 98 14 17 20 only to see it doesn't satisfy any of
them).
If we wanted to go even further, we could handle similarly say c++11_only
tests.

What do you think?

        Jakub