On 09/24/2014 01:58 PM, Andrew MacLeod wrote:
On 09/24/2014 12:29 PM, Andrew MacLeod wrote:
AH. interesting.
The third run has a gcc.sum that is exactly the same as the first run.
so only the second run differs, and it seems to be from an
alphabetical sort. So run 3 and 1 match.
the gfortran.sum from the third run is identical to the *second* run,
but it is different from the *first* run. so run 2 and 3 match.
the two runs that match (2nd and 3rd run) look like:
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=single -O2 (test
for excess errors)
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=single -O2
execution test
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=lib -O2
-lcaf_single (test for excess errors)
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=lib -O2
-lcaf_single execution test
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=single -O2 (test
for excess errors)
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=single -O2
execution test
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=lib -O2
-lcaf_single (test for excess errors)
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=lib -O2
-lcaf_single execution test
and the odd one out (firstrun:)
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=lib -O2
-lcaf_single (test for excess errors)
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=lib -O2
-lcaf_single execution test
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=single -O2 (test
for excess errors)
PASS: gfortran.dg/coarray/this_image_1.f90 -fcoarray=single -O2
execution test
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=lib -O2
-lcaf_single (test for excess errors)
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=lib -O2
-lcaf_single execution test
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=single -O2 (test
for excess errors)
PASS: gfortran.dg/coarray/this_image_2.f90 -fcoarray=single -O2
execution test
looks like the first run was sorted, and the other 2 weren't.
There must be some condition under which we don't sort the results? or
another place which needs to be tweaked to do the sort as well...?
Andrew
So to be fair, I could use test_summary, but I think the concern is
warranted because if this inconsistent ordering can happen to PASS, I
would expect the same non-deterministic behaviour if those tests happen
to FAIL. we just have far less FAILS so we aren't seeing it with
test_summary at the moment...
Aggregating all my .sum files, I see a sampling of about 257,000 PASSs,
whereas I see a total of 141 FAILs. FAILs only account for < 0.06% of
the output. ( I'm getting an average of about 510 mis-ordered PASSs, so
it only affects a small portion of them as well.)
I would think the output of .sum needs to be consistent from one run to
the next in order for test_summary to consistently report its results as
well.
Andrew
