On 24.06.2015 13:41, Manuel López-Ibáñez wrote:
> Is nobody seeing this? Is it a known problem with parallel make check?
> If so, can we work-around it in compare_tests?
>
> Cheers,
>
> Manuel.
>
Actually I don't get such problem, but I run the testsuite in a slightly
different way, so you might want to try it in order to see, if that
works for you:
#!/bin/bash
# (... snip ...)
# Check multilib configuration
if "${BUILD_DIR}/gcc/xgcc" --print-multi-lib | fgrep -q 'm32'; then
TARGET_BOARD_PARAM='unix\{-m32,\}'
else
TARGET_BOARD_PARAM='unix'
fi
rm -rf "${TEST_DIR}"
mkdir -p "${TEST_DIR}"
cd "${BUILD_DIR}"
make -k -j${NUM_JOBS} check
RUNTESTFLAGS="--target_board=${TARGET_BOARD_PARAM}" \
> >(tee "${TEST_DIR}/test.log") 2> >(tee
"${TEST_DIR}/test_err.log"
>&2) || true
"${SRC_DIR}/contrib/test_summary" -t | tail -n+2 | head -n-3 >
"${TEST_DIR}/test_summary.log"
"${SRC_DIR}/contrib/testsuite-management/validate_failures.py"
"--build_dir=${BUILD_DIR}" --produce_manifest
--manifest="${TEST_DIR}/manifest.xfail"
for FNAME in "${TEST_DIR}/test_summary.log" "${TEST_DIR}/manifest.xfail"; do
if [ -f "${FNAME}" ]; then
sed -i'' -e 's/==[0-9]\+==/==[PID]==/g' "${FNAME}"
fi
done
${SRC_DIR} is the directory where GCC repository is checked out.
${BUILD_DIR} is build root, and ${TEST_DIR} is the destination for reports.
Now, the two differences: I don't have a slash in RUNTESTFLAGS (and I
probably even recall that I used to have a slash but removed it for some
reason). Second, I don't use compare_tests, but instead produce test
summaries and manifests and then use diff.
Notice that there is a couple of other nice tricks here:
1. The two "tee" commands allow to save separate stdout and stderr logs
(the build log is also processed in a similar manner) and at the same
time copy them to stdout (in my case - to Jenkins console output).
2. test_summary is convenient because it groups the list of failures by
component and architecture. But is also contains some statistics like
the total number of tests and current date (which will differ from build
to build). In contrast, manifest contains only the list of failures,
i.e. it is more diff-friendly.
3. Address sanitizer prints process ID in failure message. I replace it
by "[PID]", so it does not change across builds.
By the way, what is the policy of test results mailing list? I actually
run nightly builds of GCC trunk and could post them if that makes sense.
(though it's plain old x86_64-unknown-linux-gnu multilib; I build and
test C, C++, LTO, Fortran, Objective C and Go).
--
Regards,
Mikhail Maltsev
