Hi Ashutosh and Jian,
Sorry for the late reply and thanks for the feedback!
On Fri, 4 Oct 2024 at 16:13, jian he <jian.universal...@gmail.com> wrote:
>
> v3, 0001 documentation:
> We can at least write something on
> https://wiki.postgresql.org/wiki/Meson about this feature.
I agree. It seems that I need to apply for an editor access on the
wiki page. I am planning to do that after the commit.
> TESTS='check check_btree' meson test amcheck/regress --verbose
> works, but I feel like there is a discoverability issue .
>
> TESTS='check check_btree' meson test amcheck/regress --list --verbose
> shows:
> postgresql:amcheck / amcheck/regress
>
>
> For small tests, listing all the available tests would be great.
I agree but unfortunately this seems not possible. 'meson test --list'
option only shows tests that are registered to meson build (i.e.
test() calls in the meson.build files).
On Fri, 4 Oct 2024 at 18:40, Ashutosh Bapat
<ashutosh.bapat....@gmail.com> wrote:
> --schedule or the test selection becomes part of the test command
> itself in the current master. By passing it as an argument, in these
> patches, we are allowing those to be overridden if TESTS is set at the
> time of running the test. I like this idea. I was pondering whether we
> really need two separate arguments --schedule and --tests
> respectively. IIUC, we can't pass them as a single argument (say
> --test-selection or --selection) because the subsequent --schedule
> would be treated as a separate argument if not quoted correctly. That
> isn't a problem with value of tests. To avoid quoting '--schedule
> ...', we have two separate arguments. Am I right?
Yes, that is exactly why we have both '--schedule' and '--tests'
flags. Also, a comment is added to clarify this.
> It might be better to make this explicit in the code -- by making sure
> that only one of them is passed and writing a comment about it.
> ArgumentParser might have some trick to specify that passing both the
> arguments is an error.
I did not understand why only one of them needed to be passed at a
time. For example in ecpg tests
(src/interfaces/ecpg/test/meson.build), both '--schedule' and
'--tests' options are passed.
> Also I don't think "Note that setup
> # suite tests (at least tmp_install and initdb_cache tests) may need to be run
> # before running these tests." is needed. This is true irrespective of
> whether we specify TESTS or not.
Done.
--
Regards,
Nazir Bilal Yavuz
Microsoft
From f0e035636ae02b7fe0668a0b2246b080656a26e5 Mon Sep 17 00:00:00 2001
From: Nazir Bilal Yavuz <byavu...@gmail.com>
Date: Thu, 31 Oct 2024 16:21:23 +0300
Subject: [PATCH v4 1/2] Add 'make check-tests' behavior to the meson based
builds
There was no way to run specific regression tests in the regress/regress
tests in the meson based builds. Add this behavior.
Author: Nazir Bilal Yavuz <byavu...@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat....@gmail.com>
Reviewed-by: Jian He <jian.universal...@gmail.com>
Discussion: postgr.es/m/CAExHW5tK-QqayUN0%2BN3MF5bjV6vLKDCkRuGwoDJwc7vGjwCygQ%40mail.gmail.com
---
meson.build | 14 ++++++++------
src/tools/testwrap | 10 ++++++++++
2 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/meson.build b/meson.build
index bb9d7f5a8e8..c9f91a0ee22 100644
--- a/meson.build
+++ b/meson.build
@@ -3399,11 +3399,11 @@ foreach test_dir : tests
'--dbname', dbname,
] + t.get('regress_args', [])
- test_selection = []
- if t.has_key('schedule')
- test_selection += ['--schedule', t['schedule'],]
- endif
+ # To avoid '--schedule' option to be treated as a separate argument in
+ # the testwrap script if not quoted correctly.
+ test_schedule = t.get('schedule', [])
+ test_selection = []
if kind == 'isolation'
test_selection += t.get('specs', [])
else
@@ -3427,12 +3427,13 @@ foreach test_dir : tests
testwrap_base,
'--testgroup', test_group,
'--testname', kind,
+ '--schedule', test_schedule,
+ '--tests', test_selection,
'--',
test_command_base,
'--outputdir', test_output,
'--temp-instance', test_output / 'tmp_check',
'--port', testport.to_string(),
- test_selection,
],
suite: test_group,
kwargs: test_kwargs,
@@ -3447,10 +3448,11 @@ foreach test_dir : tests
testwrap_base,
'--testgroup', test_group_running,
'--testname', kind,
+ '--schedule', test_schedule,
+ '--tests', test_selection,
'--',
test_command_base,
'--outputdir', test_output_running,
- test_selection,
],
is_parallel: t.get('runningcheck-parallel', true),
suite: test_group_running,
diff --git a/src/tools/testwrap b/src/tools/testwrap
index 9a270beb72d..0ab9f5dada9 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -12,6 +12,8 @@ parser.add_argument('--srcdir', help='source directory of test', type=str)
parser.add_argument('--basedir', help='base directory of test', type=str)
parser.add_argument('--testgroup', help='test group', type=str)
parser.add_argument('--testname', help='test name', type=str)
+parser.add_argument('--schedule', help='schedule tests', nargs='*')
+parser.add_argument('--tests', help='tests', nargs='*')
parser.add_argument('--skip', help='skip test (with reason)', type=str)
parser.add_argument('test_command', nargs='*')
@@ -41,6 +43,14 @@ env_dict = {**os.environ,
'TESTDATADIR': os.path.join(testdir, 'data'),
'TESTLOGDIR': os.path.join(testdir, 'log')}
+if "TESTS" in env_dict and args.testgroup == 'regress' and args.testname == 'regress':
+ args.test_command.extend(env_dict["TESTS"].split(' '))
+else:
+ if args.schedule:
+ args.test_command += ['--schedule', ' '.join(args.schedule)]
+ if args.tests:
+ args.test_command.extend(args.tests)
+
sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
# Meson categorizes a passing TODO test point as bad
# (https://github.com/mesonbuild/meson/issues/13183). Remove the TODO
--
2.45.2
From 1f880027b477afbcc0b43d4d7e5a907d44a8b4ca Mon Sep 17 00:00:00 2001
From: Nazir Bilal Yavuz <byavu...@gmail.com>
Date: Thu, 26 Sep 2024 10:24:52 +0300
Subject: [PATCH v4 2/2] Expand test selection behavior to all test types in
meson based builds
Previously, the ability to select specific tests to run was limited to
regress/regress tests. This commit extends that functionality to all test
types in the meson based builds.
Author: Nazir Bilal Yavuz <byavu...@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat....@gmail.com>
Reviewed-by: Jian He <jian.universal...@gmail.com>
Discussion: postgr.es/m/CAExHW5tK-QqayUN0%2BN3MF5bjV6vLKDCkRuGwoDJwc7vGjwCygQ%40mail.gmail.com
---
src/tools/testwrap | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/tools/testwrap b/src/tools/testwrap
index 0ab9f5dada9..c1e6d2ba376 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -43,7 +43,7 @@ env_dict = {**os.environ,
'TESTDATADIR': os.path.join(testdir, 'data'),
'TESTLOGDIR': os.path.join(testdir, 'log')}
-if "TESTS" in env_dict and args.testgroup == 'regress' and args.testname == 'regress':
+if "TESTS" in env_dict:
args.test_command.extend(env_dict["TESTS"].split(' '))
else:
if args.schedule:
--
2.45.2
