Hi,
On Thu, 26 Sept 2024 at 08:45, Ashutosh Bapat
<ashutosh.bapat....@gmail.com> wrote:
>
> On Wed, Sep 25, 2024 at 8:24 PM Nazir Bilal Yavuz <byavu...@gmail.com> wrote:
> >
> > Additionally, the patch I shared earlier was only for regress/regress
> > tests. From what I understand from here [1], only regress/regress
> > tests support 'make check-tests', so the patch seems correct. I
> > experimented with how we can implement something similar for other
> > types of tests, including other regression, isolation, and ECPG tests.
> > The attached patch works for all types of tests but only for the Meson
> > builds. For example you can run:
> >
> > $ meson test --suite setup
> > $ TESTS='check check_btree' meson test amcheck/regress
> > $ TESTS='ddl stream' meson test test_decoding/regress
> > $ TESTS='oldest_xmin skip_snapshot_restore' meson test
> > test_decoding/isolation
> > $ TESTS='sql/prepareas compat_informix/dec_test' meson test ecpg/ecpg
> >
> > What do you think about that behaviour? It is different from 'make
> > check-tests' but it looks useful to me.
>
> I think that would be a good enhancement, if a particular regression
> set takes longer e.g. the one in test_decoding takes a few seconds.
> When we worked on PG_TEST_EXTRA, it was advised to keep feature parity
> between meson and make. I guess, a similar advice applies here as well
> and we will have to change make to support these options. But that
> will be more work.
>
> Let's split the patch into two 1. supporting TESTS in meson only for
> regress/regress, 2. extending that support to other suites. The first
> patch will bring meson inline with make as far as running a subset of
> regression tests is concerned and can be committed separately. We will
> seek opinions on the second patch and commit it separately if it takes
> time. It will be good to see the support for running a subset of
> regression in meson ASAP so that developers can save time running
> entire regression suite when not needed. The second one will be an
> additional feature that can wait if it takes more time to add it to
> both meson and make.
I agree with you. I splitted the patch into two like you said.
--
Regards,
Nazir Bilal Yavuz
Microsoft
From b782aef4d805436e36db26021cc25274d965726a Mon Sep 17 00:00:00 2001
From: Nazir Bilal Yavuz <byavu...@gmail.com>
Date: Thu, 26 Sep 2024 11:55:23 +0300
Subject: [PATCH v3 1/2] Add 'make check-tests' behavior to the meson based
builds
There was no way to run specific regression tests in the regress/regress
tests in the meson based builds. Add this behavior.
---
meson.build | 12 ++++++------
src/tools/testwrap | 14 ++++++++++++++
2 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/meson.build b/meson.build
index 7150f85e0fb..8efc6c09da4 100644
--- a/meson.build
+++ b/meson.build
@@ -3434,11 +3434,9 @@ foreach test_dir : tests
'--dbname', dbname,
] + t.get('regress_args', [])
- test_selection = []
- if t.has_key('schedule')
- test_selection += ['--schedule', t['schedule'],]
- endif
+ test_schedule = t.get('schedule', [])
+ test_selection = []
if kind == 'isolation'
test_selection += t.get('specs', [])
else
@@ -3462,12 +3460,13 @@ foreach test_dir : tests
testwrap_base,
'--testgroup', test_group,
'--testname', kind,
+ '--schedule', test_schedule,
+ '--tests', test_selection,
'--',
test_command_base,
'--outputdir', test_output,
'--temp-instance', test_output / 'tmp_check',
'--port', testport.to_string(),
- test_selection,
],
suite: test_group,
kwargs: test_kwargs,
@@ -3482,10 +3481,11 @@ foreach test_dir : tests
testwrap_base,
'--testgroup', test_group_running,
'--testname', kind,
+ '--schedule', test_schedule,
+ '--tests', test_selection,
'--',
test_command_base,
'--outputdir', test_output_running,
- test_selection,
],
is_parallel: t.get('runningcheck-parallel', true),
suite: test_group_running,
diff --git a/src/tools/testwrap b/src/tools/testwrap
index 9a270beb72d..d78deb529a8 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -12,6 +12,8 @@ parser.add_argument('--srcdir', help='source directory of test', type=str)
parser.add_argument('--basedir', help='base directory of test', type=str)
parser.add_argument('--testgroup', help='test group', type=str)
parser.add_argument('--testname', help='test name', type=str)
+parser.add_argument('--schedule', help='schedule tests', nargs='*')
+parser.add_argument('--tests', help='tests', nargs='*')
parser.add_argument('--skip', help='skip test (with reason)', type=str)
parser.add_argument('test_command', nargs='*')
@@ -41,6 +43,18 @@ env_dict = {**os.environ,
'TESTDATADIR': os.path.join(testdir, 'data'),
'TESTLOGDIR': os.path.join(testdir, 'log')}
+# Symmetric behaviour with make check-tests. If TESTS environment variable is
+# set, only run these tests in regress/regress test. Note that setup suite
+# tests (at least tmp_install and initdb_cache tests) may need to be run before
+# running these tests.
+if "TESTS" in env_dict and args.testgroup == 'regress' and args.testname == 'regress':
+ args.test_command.extend(env_dict["TESTS"].split(' '))
+else:
+ if args.schedule:
+ args.test_command += ['--schedule', ' '.join(args.schedule)]
+ if args.tests:
+ args.test_command.extend(args.tests)
+
sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
# Meson categorizes a passing TODO test point as bad
# (https://github.com/mesonbuild/meson/issues/13183). Remove the TODO
--
2.45.2
From f246abcac6ff0868e12633c13139d68c99b7b9c0 Mon Sep 17 00:00:00 2001
From: Nazir Bilal Yavuz <byavu...@gmail.com>
Date: Thu, 26 Sep 2024 10:24:52 +0300
Subject: [PATCH v3 2/2] Expand test selection behavior to all test types in
meson based builds
Previously, the ability to select specific tests to run was limited to
regress/regress tests. This commit extends that functionality to all test
types in the meson based builds.
---
src/tools/testwrap | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/src/tools/testwrap b/src/tools/testwrap
index d78deb529a8..e8686b602f8 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -43,11 +43,10 @@ env_dict = {**os.environ,
'TESTDATADIR': os.path.join(testdir, 'data'),
'TESTLOGDIR': os.path.join(testdir, 'log')}
-# Symmetric behaviour with make check-tests. If TESTS environment variable is
-# set, only run these tests in regress/regress test. Note that setup suite
-# tests (at least tmp_install and initdb_cache tests) may need to be run before
-# running these tests.
-if "TESTS" in env_dict and args.testgroup == 'regress' and args.testname == 'regress':
+# If TESTS environment variable is set, only run these tests. Note that setup
+# suite tests (at least tmp_install and initdb_cache tests) may need to be run
+# before running these tests.
+if "TESTS" in env_dict:
args.test_command.extend(env_dict["TESTS"].split(' '))
else:
if args.schedule:
--
2.45.2
