Re: [RFC, v2] Test coverage for --param boundary values

[Jeff Law]

On 07/18/2016 01:53 AM, Martin Liška wrote:

You are right, I was inspired by what we do for GCC plugins in:
gcc/testsuite/lib/plugin-support.exp

where we have following comment:
    # Note that the plugin test support currently only works when the GCC
    # build tree is available. (We make sure that is the case in plugin.exp.)
    # Once we have figured out how/where to package/install GCC header files
    # for general plugin support, we should modify the following include paths
    # accordingly.

Well, I can imaging a guard which will test whether 
"$objdir/../../params.options" file exits,
and if so, then the tests are executed? Is it acceptable approach?

Yes.  That's acceptable to me.

>

>> +set text [read $fd]
>> +close $fd
>> +
>> +# Main loop.
>> +foreach params [split $text "\n"] {
>> +    set parts [split $params "="]
>> +    set name [string trim [lindex $parts 0] '"']
>> +    set values [split [lindex $parts 1] ","]
>> +    if { [llength $values] == 3 } {
>> +      set default [lindex $values 0]
>> +      set min [lindex $values 1]
>> +      set max [lindex $values 2]
>> +      set int_max "INT_MAX"
>> +
>> +      if { $min != -1 } {
>> +          param_run_test $name $min
>> +      }
>> +      if { $max != $min && $max > 0 && $max != $int_max } {
>> +          param_run_test $name $max
>> +      }
>> +    }
>> +    if { [llength $values] == 5 } {
>> +      foreach v $values {
>> +          param_run_test $name $v
>> +      }
>> +    }
>> +}

>
> This produces ambiguous test result lines:
>
>     Running 
/scratch/tschwing/nvidiak20-2/gcc/trunk-light/source-gcc/gcc/testsuite/gcc.dg/params/params.exp
 ...
>     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>     [many more]
>
> OK to fix this as follows:
>
>     PASS: gcc.dg/params/blocksort-part.c -O3 --param 
predictable-branch-outcome=0 (test for excess errors)
>     PASS: gcc.dg/params/blocksort-part.c -O3 --param 
predictable-branch-outcome=50 (test for excess errors)
>     PASS: gcc.dg/params/blocksort-part.c -O3 --param inline-min-speedup=0 
(test for excess errors)
>     PASS: gcc.dg/params/blocksort-part.c -O3 --param 
max-inline-insns-single=0 (test for excess errors)
>     [...]
>
> commit ae350ce9af062d7d2c05225280596b1bc982728b
> Author: Thomas Schwinge <tho...@codesourcery.com>
> Date:   Fri Jul 15 09:13:19 2016 +0200
>
>     Disambiguate gcc.dg/params/params.exp test results
>
>            gcc/testsuite/
>            * gcc.dg/params/params.exp (param_run_test): Swap flags and
>            default-extra-flags arguments when calling dg-runtest.
> ---
>  gcc/testsuite/gcc.dg/params/params.exp |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git gcc/testsuite/gcc.dg/params/params.exp 
gcc/testsuite/gcc.dg/params/params.exp
> index f2f397e..dd82a5d 100644
> --- gcc/testsuite/gcc.dg/params/params.exp
> +++ gcc/testsuite/gcc.dg/params/params.exp
> @@ -28,7 +28,7 @@ proc param_run_test { param_name param_value } {
>      global srcdir
>      global subdir
>
> -    dg-runtest $srcdir/$subdir/blocksort-part.c "" "-O3 --param 
$param_name=$param_value"
> +    dg-runtest $srcdir/$subdir/blocksort-part.c "-O3 --param $param_name=$param_value" 
""
>  }
>
>  set fd [open "$objdir/../../params.options" r]
>
>
> Grüße
>  Thomas
>

Thanks for the patch, that's probably the same problem that Andreas spotted.
I'm sending candidate patch.

Martin


0001-Fix-params.exp-test-cases.patch


From d5e3c5cb20306352af7f45cd827669b7ff51a709 Mon Sep 17 00:00:00 2001
From: marxin <mli...@suse.cz>
Date: Mon, 18 Jul 2016 09:50:12 +0200
Subject: [PATCH] Fix params.exp test-cases

gcc/testsuite/ChangeLog:

2016-07-18  Thomas Schwinge  <tho...@codesourcery.com>
        Martin Liska  <mli...@suse.cz>

        * gcc.dg/params/LICENSE: Add missing LICENSE that is mentioned
        in a source file.
        * gcc.dg/params/README: Same for the README.
        * gcc.dg/params/params.exp (param_run_test): Swap flags and
        default-extra-flags arguments when calling dg-runtest.

OK.

jeff