On Mon, Sep 07, 2020 at 07:47:09PM -0700, Noah Misch wrote:
> On Tue, Sep 08, 2020 at 10:43:32AM +0900, Kyotaro Horiguchi wrote:
> > At Tue, 08 Sep 2020 09:13:53 +0900 (JST), Kyotaro Horiguchi
> > <horikyota....@gmail.com> wrote in
> > > At Mon, 7 Sep 2020 02:32:55 -0700, Noah Misch <n...@leadboat.com> wrote
> > > in
> > > > As a PoC, this looks promising. Thanks. Would you add a test case
> > > > such that
> > > > the following demonstrates the bug in the absence of your PoC?
> > > >
> > > > printf '%s\n%s\n%s\n' 'log_statement = all' 'wal_level = minimal'
> > > > 'max_wal_senders = 0' >/tmp/minimal.conf
> > > > make check TEMP_CONFIG=/tmp/minimal.conf
> > >
> > > Mmm. I was close to add some tests to 018_wal_optimize.pl but your
> > > suggestion seems better. I added several ines to create_index.sql.
After looking closer, I've moved the test to reindex_catalog.sql; see that
file's header comment for the reasons. One now needs this command:
printf '%s\n%s\n%s\n' 'log_statement = all' 'wal_level = minimal'
'max_wal_senders = 0' >/tmp/minimal.conf
make check-tests TESTS=reindex_catalog TEMP_CONFIG=/tmp/minimal.conf
> > > > Please have the test try both a nailed-and-mapped relation and a
> > > > "nailed, but
> > > > not mapped" relation. I am fairly confident that your PoC fixes the
> > > > former
> > > > case, but the latter may need additional code.
> > >
> > > Mmm. You're right. I choosed pg_amproc_fam_proc_index as
> > > nailed-but-not-mapped index.
> >
> > I fixed a typo (s/staring/starting/).
>
> At a glance, this looks reasonable. If a closer look doesn't reveal problems,
> I'll push this.
RelationBuildDesc() calls RelationInitPhysicalAddr(), so RelationBuildDesc()
can stop calling RelFileNodeSkippingWAL(). The attached version makes it so,
and I plan to push it.
Author: Noah Misch <n...@leadboat.com>
Commit: Noah Misch <n...@leadboat.com>
Fix rd_firstRelfilenodeSubid for nailed relations, in parallel workers.
Move applicable code out of RelationBuildDesc(), which nailed relations
bypass. Non-assert builds experienced no known problems. Back-patch to
v13, where commit c6b92041d38512a4176ed76ad06f713d2e6c01a8 introduced
rd_firstRelfilenodeSubid.
Kyotaro Horiguchi. Reported by Justin Pryzby.
Discussion: https://postgr.es/m/20200907023737.ga7...@telsasoft.com
diff --git a/src/backend/utils/cache/relcache.c
b/src/backend/utils/cache/relcache.c
index 96ecad0..9061af8 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -1243,14 +1243,6 @@ RelationBuildDesc(Oid targetRelId, bool insertIt)
if (insertIt)
RelationCacheInsert(relation, true);
- /*
- * For RelationNeedsWAL() to answer correctly on parallel workers,
restore
- * rd_firstRelfilenodeSubid. No subtransactions start or end while in
- * parallel mode, so the specific SubTransactionId does not matter.
- */
- if (IsParallelWorker() && RelFileNodeSkippingWAL(relation->rd_node))
- relation->rd_firstRelfilenodeSubid = TopSubTransactionId;
-
/* It's fully valid */
relation->rd_isvalid = true;
@@ -1273,6 +1265,8 @@ RelationBuildDesc(Oid targetRelId, bool insertIt)
static void
RelationInitPhysicalAddr(Relation relation)
{
+ Oid oldnode = relation->rd_node.relNode;
+
/* these relations kinds never have storage */
if (!RELKIND_HAS_STORAGE(relation->rd_rel->relkind))
return;
@@ -1330,6 +1324,19 @@ RelationInitPhysicalAddr(Relation relation)
elog(ERROR, "could not find relation mapping for
relation \"%s\", OID %u",
RelationGetRelationName(relation),
relation->rd_id);
}
+
+ /*
+ * For RelationNeedsWAL() to answer correctly on parallel workers,
restore
+ * rd_firstRelfilenodeSubid. No subtransactions start or end while in
+ * parallel mode, so the specific SubTransactionId does not matter.
+ */
+ if (IsParallelWorker() && oldnode != relation->rd_node.relNode)
+ {
+ if (RelFileNodeSkippingWAL(relation->rd_node))
+ relation->rd_firstRelfilenodeSubid =
TopSubTransactionId;
+ else
+ relation->rd_firstRelfilenodeSubid =
InvalidSubTransactionId;
+ }
}
/*
diff --git a/src/test/regress/expected/reindex_catalog.out
b/src/test/regress/expected/reindex_catalog.out
index 4b5fba4..204f056 100644
--- a/src/test/regress/expected/reindex_catalog.out
+++ b/src/test/regress/expected/reindex_catalog.out
@@ -36,3 +36,13 @@ REINDEX INDEX pg_index_indexrelid_index; -- non-mapped,
non-shared, critical
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
+-- Check the same REINDEX INDEX statements under parallelism.
+BEGIN;
+SET min_parallel_table_scan_size = 0;
+REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
+REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
+REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
+REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
+REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
+REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
+ROLLBACK;
diff --git a/src/test/regress/sql/reindex_catalog.sql
b/src/test/regress/sql/reindex_catalog.sql
index 87ecf52..8203641 100644
--- a/src/test/regress/sql/reindex_catalog.sql
+++ b/src/test/regress/sql/reindex_catalog.sql
@@ -39,3 +39,14 @@ REINDEX INDEX pg_index_indexrelid_index; -- non-mapped,
non-shared, critical
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
+
+-- Check the same REINDEX INDEX statements under parallelism.
+BEGIN;
+SET min_parallel_table_scan_size = 0;
+REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
+REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
+REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
+REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
+REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
+REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
+ROLLBACK;
