A little confused here, does this patch V3 intend to solve this problem "record
length 2145386550 at 0/3000060 too long"?
No, not once the record exists. But it does remove Postgres' ability
to create such records, thereby solving the problem for all systems
that generate WAL through Postgres' WAL writing APIs.
I set up a simple Primary and Standby stream replication environment, and use
the above query to run the test for before and after patch v3. The error
message still exist, but with different message.
Before patch v3, the error is showing below,
2022-06-10 15:32:25.307 PDT [4253] LOG: record length 2145386550 at 0/3000060
too long
2022-06-10 15:32:47.763 PDT [4257] FATAL: terminating walreceiver process due
to administrator command
2022-06-10 15:32:47.763 PDT [4253] LOG: record length 2145386550 at 0/3000060
too long
After patch v3, the error displays differently
2022-06-10 15:53:53.397 PDT [12848] LOG: record length 2145386550 at 0/3000060
too long
2022-06-10 15:54:07.249 PDT [12852] FATAL: could not receive data from WAL
stream: ERROR: requested WAL segment 000000010000000000000045 has already been
removed
2022-06-10 15:54:07.275 PDT [12848] LOG: record length 2145386550 at 0/3000060
too long
And once the error happens, then the Standby can't continue the replication.
Did you initiate a new cluster or otherwise skip the invalid record
you generated when running the instance based on master? It seems to
me you're trying to replay the invalid record (len > MaxAllocSize),
and this patch does not try to fix that issue. This patch just tries
to forbid emitting records larger than MaxAllocSize, as per the check
in XLogRecordAssemble, so that we wont emit unreadable records into
the WAL anymore.
Reading unreadable records still won't be possible, but that's also
not something I'm trying to fix.
Thanks a lot for the clarification. My testing environment is pretty
simple, initdb for Primary, run basebackup and set the connection string
for Standby, then run the "pg_logical_emit_message" query and tail the
log on standby side.
Best regards,
--
David
Software Engineer
Highgo Software Inc. (Canada)
www.highgo.ca
