Hello!
On 23.12.2022 05:42, Michael Paquier wrote:
On Thu, Dec 22, 2022 at 09:59:18AM +0300, Anton A. Melnikov wrote:
2) v2-0002-Additional-dumps-filtering.patch
+ # Replace specific privilegies with ALL
+ $dump_contents =~ s/^(GRANT\s|REVOKE\s)(\S*)\s/$1ALL /mgx;
This should not be in 0002, I guess..
Made a separate patch for it: v3-0001-Fix-dumps-filtering.patch
On 26.12.2022 05:52, Michael Paquier wrote:
On Fri, Dec 23, 2022 at 12:43:00PM +0300, Anton A. Melnikov wrote:
I was looking at 0002 to add a callback to provide custom filtering
rules.
+ my @ext_filter = split('\/', $_);
Are you sure that enforcing a separation with a slash is a good idea?
What if the filters include directory paths or characters that are
escaped, for example?
Rather than introducing a filter.regex, I would tend to just document
that in the README with a small example. I have been considering a
few alternatives while making this useful in most cases, still my mind
alrways comes back to the simplest thing we to just read each line of
the file, chomp it and apply the pattern to the log file..
Thanks for your attention!
Yes, indeed. It will be really simpler.
Made it in the v3-0002-Add-external-dumps-filtering.patch
With the best wishes,
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
commit 602627daf32a6d33ae96ce8fbae918c9f00f0633
Author: Anton A. Melnikov <a.melni...@postgrespro.ru>
Date: Mon Dec 26 06:21:32 2022 +0300
Fix dupms filtering in pg_upgrade test from older versions. Replace
specific privilegies in dumps with ALL due to b5d63824
and 60684dd8.
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 4cc1469306..d23c4b2253 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -44,6 +44,8 @@ sub filter_dump
$dump_contents =~ s/^\-\-.*//mgx;
# Remove empty lines.
$dump_contents =~ s/^\n//mgx;
+ # Replace specific privilegies with ALL
+ $dump_contents =~ s/^(GRANT\s|REVOKE\s)(\S*)\s/$1ALL /mgx;
my $dump_file_filtered = "${dump_file}_filtered";
open(my $dh, '>', $dump_file_filtered)
commit 3870fb8263dc7e3fa240753b4eb89945b8d229be
Author: Anton A. Melnikov <a.melni...@postgrespro.ru>
Date: Thu Dec 22 08:01:40 2022 +0300
Add external dumps filtering during upgrade test.
diff --git a/src/bin/pg_upgrade/TESTING b/src/bin/pg_upgrade/TESTING
index 127dc30bbb..b8cce2bae1 100644
--- a/src/bin/pg_upgrade/TESTING
+++ b/src/bin/pg_upgrade/TESTING
@@ -56,3 +56,12 @@ Once the dump is created, it can be repeatedly used with $olddump and
the dump out of the new database and the comparison of the dumps between
the old and new databases. The contents of the dumps can also be manually
compared.
+
+You can use additional dump filtering. To do this, you need to define the
+'filter' environment variable and specify the path to the file with
+filtering rules in it. Here is an example contens of such a file:
+# examples:
+# Remove all CREATE POLICY statements
+s/^CREATE\sPOLICY.*//mgx
+# Replace REFRESH with DROP for materialized views
+s/^REFRESH\s(MATERIALIZED\sVIEW)/DROP $1/mgx
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 4cc1469306..e094fd08c3 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -45,6 +45,30 @@ sub filter_dump
# Remove empty lines.
$dump_contents =~ s/^\n//mgx;
+ if (defined($ENV{filter}))
+ {
+ # Use the external filter
+ my $ext_filter_file = $ENV{filter};
+ die "no file with external filter found!" unless -e $ext_filter_file;
+
+ open my $ext_filter_handle, '<', $ext_filter_file;
+ chomp(my @ext_filter_lines = <$ext_filter_handle>);
+ close $ext_filter_handle;
+
+ foreach (@ext_filter_lines)
+ {
+ # omit lines with comments
+ if (substr($_, 0, 1) eq '#')
+ {
+ next;
+ }
+
+ # apply lines with filters
+ my $filter = "\$dump_contents =~ $_";
+ eval $filter;
+ }
+ }
+
my $dump_file_filtered = "${dump_file}_filtered";
open(my $dh, '>', $dump_file_filtered)
|| die "opening $dump_file_filtered";
