Under Meson, it is not very easy to see if TAP tests have been enabled
or disabled, if you rely on the default auto setting. You either need
to carefully study the meson setup output, or you notice, what a minute,
didn't there use to be like 250 tests, not only 80?
I think it would be better if we still registered the TAP tests in Meson
even if the tap_tests option is disabled, but with a dummy command that
registers them as skipped. That way you get a more informative output like
Ok: 78
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 187
Timeout: 0
which is really a more accurate representation of what the test run
actually accomplished than "everything Ok".
See attached patch for a possible implementation. (This uses perl as a
hard build requirement. We are planning to do that anyway, but
obviously other implementations, such as using python, would also be
possible.)From 19e78e4c5a16337c0ac4e661beb4729505736016 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <pe...@eisentraut.org>
Date: Mon, 30 Oct 2023 05:32:45 -0400
Subject: [PATCH] Explicitly skip TAP tests under Meson if disabled
If the tap_tests option is disabled under Meson, the TAP tests are
currently not registered at all. But this makes it harder to see what
is going one, why suddently there are fewer tests than before.
Instead, register the tests anyway but with a dummy command that marks
them as skipped. That way, the total list and count of tests is
constant whether the option is enabled or not.
---
meson.build | 31 +++++++++++++++++--------------
1 file changed, 17 insertions(+), 14 deletions(-)
diff --git a/meson.build b/meson.build
index 2d516c8f372..1795dd8159e 100644
--- a/meson.build
+++ b/meson.build
@@ -3248,10 +3248,6 @@ foreach test_dir : tests
testport += 1
elif kind == 'tap'
- if not tap_tests_enabled
- continue
- endif
-
test_command = [
perl.path(),
'-I', meson.source_root() / 'src/test/perl',
@@ -3286,16 +3282,23 @@ foreach test_dir : tests
onetap_p = fs.stem(onetap_p)
endif
- test(test_dir['name'] / onetap_p,
- python,
- kwargs: test_kwargs,
- args: testwrap_base + [
- '--testgroup', test_dir['name'],
- '--testname', onetap_p,
- '--', test_command,
- test_dir['sd'] / onetap,
- ],
- )
+ if tap_tests_enabled
+ test(test_dir['name'] / onetap_p,
+ python,
+ kwargs: test_kwargs,
+ args: testwrap_base + [
+ '--testgroup', test_dir['name'],
+ '--testname', onetap_p,
+ '--', test_command,
+ test_dir['sd'] / onetap,
+ ],
+ )
+ else
+ test(test_dir['name'] / onetap_p,
+ perl,
+ args: ['-e', 'print "1..0 # Skipped: TAP tests not enabled"'],
+ kwargs: test_kwargs)
+ endif
endforeach
install_suites += test_group
else
--
2.42.0
