Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-07-26 Thread Michael Paquier
On Mon, Jul 25, 2022 at 02:12:05PM +0300, Heikki Linnakangas wrote: > The way this is written, it would change whenever we add/remove fields in > DecodedBkpBlock, for example. That's fragile; if you added a field in a > back-branch, you could accidentally make the new minor version unable to > read

Re: Refactoring postgres_fdw/connection.c

2022-07-26 Thread Kyotaro Horiguchi
At Tue, 26 Jul 2022 00:54:47 +0900, Fujii Masao wrote in > Hi, > > When reviewing the postgres_fdw parallel-abort patch [1], I found that > there are several duplicate codes in postgres_fdw/connection.c. > Which seems to make it harder to review the patch changing > connection.c. > So I'd like

Re: remove more archiving overhead

2022-07-26 Thread Fujii Masao
On 2022/07/09 2:18, Nathan Bossart wrote: On Fri, Jul 08, 2022 at 01:02:51PM -0400, David Steele wrote: I think I wrote this before I'd had enough coffee. "fully persisted to storage" can mean many things depending on the storage (Posix, CIFS, S3, etc.) so I think this is fine. The basic_arch

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-07-26 Thread Richard Guo
On Tue, Jul 26, 2022 at 7:38 AM David Rowley wrote: > On Fri, 22 Jul 2022 at 21:33, Richard Guo wrote: > > I can see this problem with > > the query below: > > > > select max(b order by b), max(a order by a) from t group by a; > > > > When processing the first aggregate, we compose the 'curr

Re: Introduce wait_for_subscription_sync for TAP tests

2022-07-26 Thread Masahiko Sawada
On Tue, Jul 26, 2022 at 2:01 PM Amit Kapila wrote: > > On Tue, Jul 26, 2022 at 7:07 AM Masahiko Sawada wrote: > > > > Hi, > > > > In tap tests for logical replication, we have the following code in many > > places: > > > > $node_publisher->wait_for_catchup('tap_sub'); > > my $synced_query = > >

SI-read predicate locks on materialized views

2022-07-26 Thread Yugo NAGATA
Hi, I propose to acquire SI-read predicate locks on materialized views as the attached patch. Currently, materialized views do not participate in predicate locking, but I think this causes a serialization anomaly when `REFRESH MATERIALIZED VIEW CONCURRENTLY` is used. For example, supporse that t

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-26 Thread Masahiko Sawada
On Tue, Jul 26, 2022 at 2:18 PM Amit Kapila wrote: > > On Tue, Jul 26, 2022 at 7:00 AM Masahiko Sawada wrote: > > > > On Mon, Jul 25, 2022 at 7:57 PM shiy.f...@fujitsu.com > > wrote: > > > > > > Hi, > > > > > > I did some performance test for the master branch patch (based on v6 > > > patch) to

RE: Handle infinite recursion in logical replication setup

2022-07-26 Thread shiy.f...@fujitsu.com
On Sun, Jul 24, 2022 1:28 AM vignesh C wrote: > > Added a note for the same and referred it to the conflicts section. > > Thanks for the comments, the attached v38 patch has the changes for the > same. > Thanks for updating the patch. A comment on the test in 0001 patch. +# Alter subscriptio

Re: Make name optional in CREATE STATISTICS

2022-07-26 Thread Alvaro Herrera
On 2022-Jul-26, Michael Paquier wrote: > On Mon, Jul 25, 2022 at 12:55:54PM +0200, Alvaro Herrera wrote: > > Agreed. I think you already have the query for that elsewhere in the > > test, so it's just a matter of copying it from there. > > I actually already wrote most of it in 2cbc3c1, and I ju

Re: make -C libpq check fails obscurely if tap tests are disabled

2022-07-26 Thread Alvaro Herrera
Ah, I found what the actual problem is: we have sprinkled a few dependencies on "...-recurse" throught the tree, but the patch I posted yesterday changes the manufactured target as "-recursive", as it was prior to 1bd201214965; so essentially these manually added dependencies all became silent no-o

Re: Pluggable toaster

2022-07-26 Thread Aleksander Alekseev
Hi Nikita, Thanks for an update! > 0002_toaster_interface_v10 contains TOAST API with Dummy toaster as an > example (but I would, > as recommended, remove Dummy toaster and provide it as an extension), and > default Toaster > was left as-is (reference implementation). > > 0003_toaster_default_v

RE: Handle infinite recursion in logical replication setup

2022-07-26 Thread wangw.f...@fujitsu.com
On Sun, Jul 24, 2022 1:28 AM vignesh C wrote: > Added a note for the same and referred it to the conflicts section. > > Thanks for the comments, the attached v38 patch has the changes for the same. Thanks for your patches. Two slight comments on the below message in the 0001 patch: The error

Re: Perform streaming logical transactions by background workers and parallel apply

2022-07-26 Thread Dilip Kumar
On Fri, Jul 22, 2022 at 8:27 AM wangw.f...@fujitsu.com wrote: > > On Tues, Jul 19, 2022 at 10:29 AM I wrote: > > Attach the news patches. > > Not able to apply patches cleanly because the change in HEAD (366283961a). > Therefore, I rebased the patch based on the changes in HEAD. > > Attach the new

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-26 Thread shiy.f...@fujitsu.com
On Tue, Jul 26, 2022 3:52 PM Masahiko Sawada wrote: > > On Tue, Jul 26, 2022 at 2:18 PM Amit Kapila > wrote: > > > > On Tue, Jul 26, 2022 at 7:00 AM Masahiko Sawada > wrote: > > > > > > On Mon, Jul 25, 2022 at 7:57 PM shiy.f...@fujitsu.com > > > wrote: > > > > > > > > > > > > case 3 > > > > --

Re: Refactoring postgres_fdw/connection.c

2022-07-26 Thread Fujii Masao
On 2022/07/26 16:25, Kyotaro Horiguchi wrote: Agree to that refactoring. And it looks fine to me. Thanks for reviewing the patches! I'm not sure the two are similar with each other. The new function pgfdw_exec_pre_commit() looks like a merger of two isolated code paths intended to share

Re: Perform streaming logical transactions by background workers and parallel apply

2022-07-26 Thread Dilip Kumar
On Tue, Jul 26, 2022 at 2:30 PM Dilip Kumar wrote: > > On Fri, Jul 22, 2022 at 8:27 AM wangw.f...@fujitsu.com > wrote: > > > > On Tues, Jul 19, 2022 at 10:29 AM I wrote: > > > Attach the news patches. > > > > Not able to apply patches cleanly because the change in HEAD (366283961a). > > Therefore

Max compact as an FSM strategy

2022-07-26 Thread Simon Riggs
A long time ago, Tom Lane came up with the idea that when tables get bloated, tables might be allowed to shrink down again in size naturally by altering the way FSM allocates blocks. That's a very good idea, but we didn't implement it back then... This patch allows the Heap to specify what FreeSpa

回复:Re: PANIC: wrong buffer passed to visibilitymap_clear

2022-07-26 Thread 王伟(学弈)
On Fri, Jul 22, 2022 at 14:49 Peter Geoghegan wrote: > The line numbers from your stack trace don't match up with> REL_14_STABLE. Is > this actually a fork of Postgres 14? (Oh, looks like > it's an old beta release.) Yeah, I was testing on 14beta2 branch once. So I considered your advices and tes

Re: Perform streaming logical transactions by background workers and parallel apply

2022-07-26 Thread Peter Smith
Here are some review comment for patch v19-0001: == 1.1 Missing docs for protocol version Since you bumped the logical replication protocol version for this patch, now there is some missing documentation to describe this new protocol version. e.g. See here [1] == 1.2 doc/src/sgml/confi

Re: SI-read predicate locks on materialized views

2022-07-26 Thread Richard Guo
On Tue, Jul 26, 2022 at 3:44 PM Yugo NAGATA wrote: > If such two transactions run concurrently, a write skew anomaly occurs, > and the result of order_summary refreshed in T1 will not contain the > record inserted in T2. Indeed we have write skew anomaly here between the two transactions. > O

Re: Max compact as an FSM strategy

2022-07-26 Thread Dilip Kumar
On Tue, Jul 26, 2022 at 3:04 PM Simon Riggs wrote: > > A long time ago, Tom Lane came up with the idea that when tables get > bloated, tables might be allowed to shrink down again in size > naturally by altering the way FSM allocates blocks. That's a very good > idea, but we didn't implement it ba

Re: Add WAL recovery messages with log_wal_traffic GUC (was: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display)

2022-07-26 Thread Bharath Rupireddy
On Fri, May 13, 2022 at 6:41 PM Robert Haas wrote: > > On Fri, Apr 29, 2022 at 5:11 AM Bharath Rupireddy > wrote: > > Here's the rebased v9 patch. > > This seems like it has enormous overlap with the existing > functionality that we have from log_startup_progress_interval. > > I think that facili

Re: Fast COPY FROM based on batch insert

2022-07-26 Thread Etsuro Fujita
On Fri, Jul 22, 2022 at 5:42 PM Andrey Lepikhov wrote: > So, maybe switch > status of this patch to 'Ready for committer'? Yeah, I think the patch is getting better, but I noticed some issues, so I'm working on them. I think I can post a new version in the next few days. Best regards, Etsuro Fu

Re: Refactoring postgres_fdw/connection.c

2022-07-26 Thread Etsuro Fujita
Fujii-san, On Tue, Jul 26, 2022 at 12:55 AM Fujii Masao wrote: > When reviewing the postgres_fdw parallel-abort patch [1], I found that > there are several duplicate codes in postgres_fdw/connection.c. > Which seems to make it harder to review the patch changing connection.c. > So I'd like to rem

Re: Refactoring postgres_fdw/connection.c

2022-07-26 Thread Fujii Masao
On 2022/07/26 19:26, Etsuro Fujita wrote: Thanks for working on this! I'd like to review this after the end of the current CF. Thanks! Could you add this to the next CF? Yes. https://commitfest.postgresql.org/39/3782/ Regards, -- Fujii Masao Advanced Computing Technology Center Rese

Re: Max compact as an FSM strategy

2022-07-26 Thread Simon Riggs
On Tue, 26 Jul 2022 at 11:02, Dilip Kumar wrote: > > On Tue, Jul 26, 2022 at 3:04 PM Simon Riggs > wrote: > > > > A long time ago, Tom Lane came up with the idea that when tables get > > bloated, tables might be allowed to shrink down again in size > > naturally by altering the way FSM allocates

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-07-26 Thread David Steele
Hi Noah, On 6/19/21 16:39, Noah Misch wrote: On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: Recycling and preallocation are wasteful during archive recovery, because KeepFileRestoredFromArchive() unlinks every entry in its path. I propose to fix the race by adding an XLogCtl flag

Re: make -C libpq check fails obscurely if tap tests are disabled

2022-07-26 Thread Andres Freund
Hi, On 2022-07-22 19:50:02 -0400, Tom Lane wrote: > I wrote: > > So it'll work in 3.81 (released 2006) and later, but not 3.80. > > Confirmed that things are fine with 3.81. Thanks for looking into this Alvaro, Andrew, Justin, Tom - I was on vacation... Greetings, Andres Freund

Re: Cygwin cleanup

2022-07-26 Thread Tom Lane
Thomas Munro writes: > On Tue, Jul 26, 2022 at 4:34 PM Tom Lane wrote: >> If that's an accurate statement, shouldn't we just drop Cygwin support? > This thread rejected the idea last time around: > https://www.postgresql.org/message-id/flat/136712b0-0619-5619-4634-0f0286acaef7%402ndQuadrant.com

Re: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

2022-07-26 Thread David Steele
On 7/25/22 22:49, Kyotaro Horiguchi wrote: At Mon, 25 Jul 2022 14:21:38 +0530, Bharath Rupireddy wrote in Hm. I think we must take this opportunity to clean it up. You are right, we don't need to parse the label file contents (just like we used to do previously after reading it from the file)

Re: Allow parallel plan for referential integrity checks?

2022-07-26 Thread Frédéric Yhuel
On 4/14/22 14:25, Frédéric Yhuel wrote: On 3/19/22 01:57, Imseih (AWS), Sami wrote: I looked at your patch and it's a good idea to make foreign key validation use parallel query on large relations. It would be valuable to add logging to ensure that the ActiveSnapshot and TransactionSnaps

Re: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

2022-07-26 Thread Bharath Rupireddy
On Tue, Jul 26, 2022 at 5:22 PM David Steele wrote: > > On 7/25/22 22:49, Kyotaro Horiguchi wrote: > > At Mon, 25 Jul 2022 14:21:38 +0530, Bharath Rupireddy > > wrote in > >> Hm. I think we must take this opportunity to clean it up. You are > >> right, we don't need to parse the label file conte

Re: Proposal to provide the facility to set binary format output for specific OID's per session

2022-07-26 Thread Dave Cramer
Hi Sehrope, On Mon, 25 Jul 2022 at 17:53, Dave Cramer wrote: > Hi Sehrope, > > > On Mon, 25 Jul 2022 at 17:22, Sehrope Sarkuni wrote: > >> Idea here makes sense and I've seen this brought up repeatedly on the >> JDBC lists. >> >> Does the driver need to be aware that this SET command was exe

Re: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

2022-07-26 Thread David Steele
On 7/26/22 07:59, Bharath Rupireddy wrote: On Tue, Jul 26, 2022 at 5:22 PM David Steele wrote: On 7/25/22 22:49, Kyotaro Horiguchi wrote: At Mon, 25 Jul 2022 14:21:38 +0530, Bharath Rupireddy wrote in Hm. I think we must take this opportunity to clean it up. You are right, we don't need

Re: making relfilenodes 56 bits

2022-07-26 Thread Ashutosh Sharma
/* * If relfilenumber is unspecified by the caller then create storage -* with oid same as relid. +* with relfilenumber same as relid if it is a system table otherwise +* allocate a new relfilenumber. For more details read comments atop +* FirstNorm

Re: log_line_prefix: make it possible to add the search_path

2022-07-26 Thread Pierre
On Tuesday, July 26, 2022 3:08:01 AM CEST Lukas Fittl wrote: > On Mon, Jul 25, 2022 at 12:38 AM Pierre Ducroquet > > wrote: > > usecase by not showing the schema, one of them being log_line_prefix. > > It is possible to work around this using the application_name, but a > > mistake > > on the app

RE: Collect ObjectAddress for ATTACH DETACH PARTITION to use in event trigger

2022-07-26 Thread houzj.f...@fujitsu.com
On Monday, July 25, 2022 6:26 PM Michael Paquier wrote: > > On Mon, Jul 25, 2022 at 09:25:07AM +, houzj.f...@fujitsu.com wrote: > > BTW, while reviewing it, I found there are some more subcommands that > > the > > get_altertable_subcmdtypes() doesn't handle(e.g., ADD/DROP/SET > > IDENTITY and

Re: Restructure ALTER TABLE notes to clarify table rewrites and verification scans

2022-07-26 Thread Justin Pryzby
I think this patch is missing "SET [UN]LOGGED", defaults of identity columns and domains, and access method. And tablespace, even though that rewrites the *files*, but not tuples (maybe these docs should say that).

Re: making relfilenodes 56 bits

2022-07-26 Thread Dilip Kumar
On Tue, Jul 26, 2022 at 6:06 PM Ashutosh Sharma wrote: Hi, Note: please avoid top posting. > /* > * If relfilenumber is unspecified by the caller then create storage > -* with oid same as relid. > +* with relfilenumber same as relid if it is a system table otherw

Re: doc: Clarify Savepoint Behavior

2022-07-26 Thread David G. Johnston
On Thu, Jul 14, 2022 at 12:44 PM Bruce Momjian wrote: > On Sat, Jul 9, 2022 at 12:59:23PM -0400, Bruce Momjian wrote: > > On Sun, Jun 26, 2022 at 09:14:56AM -0700, David G. Johnston wrote: > > > So leave the "release" behavior implied from the rollback behavior? > > > > > > On the whole I'm slig

Re: make -C libpq check fails obscurely if tap tests are disabled

2022-07-26 Thread Alvaro Herrera
On 2022-Jul-26, Alvaro Herrera wrote: > With this version I keep the target name as -recurse, and at least the > ecpg<->libpq problem is no more AFAICT. ... but I think we're missing some more dependencies, because if I remove everything (beyond make clean), then a "make -j8 world-bin" fails, per

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-26 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: >> Tom Lane writes: >>> I wonder if it'd be a good idea to convert >>> auto_explain's TAP test to load auto_explain via session_preload_libraries >>> instead of shared_preload_libraries, and then pass in the settings for >>> e

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-26 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > Thanks! Just one minor nitpick: setting an %ENV entry to `undef` > doesn't unset the environment variable, it sets it to the empty string. > To unset a variable it needs to be deleted from %ENV, i.e. `delete > $ENV{PGUSER};`. Ah. Still, libpq

[Refactor]Avoid to handle FORCE_NOT_NULL/FORCE_NULL options when COPY TO

2022-07-26 Thread Zhang Mingli
Hi,FORCE_NOT_NULL and FORCE_NULL are only used when COPY FROM.And copyto.c and copyfrom.c are split in this commit https://github.com/postgres/postgres//commit/c532d15dddff14b01fe9ef1d465013cb8ef186df .There is no need to handle these options when COPY TO. vn-0001-Avoid-to-handle-FORCE_NOT_NULL-FO

Re: remove more archiving overhead

2022-07-26 Thread Nathan Bossart
On Tue, Jul 26, 2022 at 04:26:23PM +0900, Fujii Masao wrote: > Anyway, since the patches look good to me, I pushed them. Thanks a lot! > If necessary, we can keep improving the docs later. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Fix annotations nextFullXid

2022-07-26 Thread Zhang Mingli
Thanks! Regards, Zhang Mingli > On Jul 26, 2022, at 10:08, Fujii Masao wrote: > > > > On 2022/07/23 14:01, Zhang Mingli wrote: >> Hi, >> VariableCacheData.nextFullXid is renamed to nextXid in commit >> https://github.com/postgres/postgres//commit/fea10a64340e529805609126740a540c8f9daab4 >>

Re: failures in t/031_recovery_conflict.pl on CI

2022-07-26 Thread Alvaro Herrera
On 2022-May-08, Andres Freund wrote: > On 2022-05-08 13:59:09 -0400, Tom Lane wrote: > > No one is going to thank us for shipping a known-unstable test case. > > IDK, hiding failures indicating bugs isn't really better, at least if it > doesn't look like a bug in the test. But you seem to have a

Re: pg_auth_members.grantor is bunk

2022-07-26 Thread Robert Haas
On Wed, Jul 20, 2022 at 3:11 PM Robert Haas wrote: > The previous version of the patch makes both DROP OWNED and REASSIGN > OWNED cascade to grantors, but I now think that, for consistency, I'd > better look into changing it so that only DROP OWNED cascades. I think > perhaps I should be using SHA

Re: failures in t/031_recovery_conflict.pl on CI

2022-07-26 Thread Tom Lane
Alvaro Herrera writes: > Hey, I just noticed that these tests are still disabled. The next > minors are coming soon; should we wait until *those* are done and then > re-enable; or re-enable them now to see how they fare and then > re-disable before the next minors if there's still problems we don

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-07-26 Thread Matthias van de Meent
On Tue, 26 Jul 2022 at 09:20, Michael Paquier wrote: > > On Mon, Jul 25, 2022 at 02:12:05PM +0300, Heikki Linnakangas wrote: > > Among the two problems to solve at hand, the parts where the APIs are > > changed and made more robust with unsigned types and where block data > > is not overflowed

Re: making relfilenodes 56 bits

2022-07-26 Thread Ashutosh Sharma
Thanks Dilip. Here are few comments that could find upon quickly reviewing the v11 patch: /* + * Similar to the XLogPutNextOid but instead of writing NEXTOID log record it + * writes a NEXT_RELFILENUMBER log record. If '*prevrecptr' is a valid + * XLogRecPtrthen flush the wal upto this record p

Re: Cygwin cleanup

2022-07-26 Thread Robert Haas
On Tue, Jul 26, 2022 at 7:40 AM Tom Lane wrote: > I think maybe we should re-open the discussion. I've certainly > reached the stage of fed-up-ness. That platform seems seriously > broken, upstream is making no progress on fixing it, and there > doesn't seem to be any real-world use-case. The o

Re: Password reset link / 'less' does not exit in psql version 13.4

2022-07-26 Thread Justin Pryzby
On Mon, Jul 25, 2022 at 11:02:26AM +0200, Michael J. Baars wrote: > Hello, > > I have two very simple questions: > > 1) I have an account at postgresql.org, but a link to a 'forgot password' > seems to be missing on the login page. I have my password stored only on an > old Fedora 32 computer.

out of date comment in commit_ts.c

2022-07-26 Thread Nathan Bossart
Hi hackers, I noticed that commit_ts.c has the following comment: * XLOG interactions: this module generates an XLOG record whenever a new * CommitTs page is initialized to zeroes. Also, one XLOG record is * generated for setting of values when the caller requests it; this allows * us to sup

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-26 Thread Robert Haas
On Mon, Jul 25, 2022 at 9:47 PM Kyotaro Horiguchi wrote: > One arguable point would be whether we will need to put restriction > the target relations that Bob can vacuum/analyze. Yeah. pg_checkpoint makes sense because you can either CHECKPOINT or you can't. But for a command with a target, you r

Re: failures in t/031_recovery_conflict.pl on CI

2022-07-26 Thread Andres Freund
Hi, On 2022-07-26 12:47:38 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > Hey, I just noticed that these tests are still disabled. The next > > minors are coming soon; should we wait until *those* are done and then > > re-enable; or re-enable them now to see how they fare and then > > re-di

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-26 Thread David G. Johnston
On Tue, Jul 26, 2022 at 10:37 AM Robert Haas wrote: > On Mon, Jul 25, 2022 at 9:47 PM Kyotaro Horiguchi > wrote: > > One arguable point would be whether we will need to put restriction > > the target relations that Bob can vacuum/analyze. > > But for a command with a target, you really ought t

Re: Transparent column encryption

2022-07-26 Thread Robert Haas
On Thu, Jul 21, 2022 at 2:30 PM Jacob Champion wrote: > On Mon, Jul 18, 2022 at 9:07 AM Robert Haas wrote: > > Even there, what can be accomplished with a feature that only encrypts > > individual column values is by nature somewhat limited. If you have a > > text column that, for one row, stores

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-26 Thread Robert Haas
On Tue, Jul 26, 2022 at 1:50 PM David G. Johnston wrote: >> Still, it seems somewhat appealing to give >> people fine-grained control over this, rather than just "on" or "off". > Appealing enough to consume a couple of permission bits? > https://www.postgresql.org/message-id/CAKFQuwZ6dhjTFV7Bwmehe

Re: Unstable tests for recovery conflict handling

2022-07-26 Thread Tom Lane
I wrote: >> It's been kind of hidden by other buildfarm noise, but >> 031_recovery_conflict.pl is not as stable as it should be [1][2][3][4]. > After digging around in the code, I think this is almost certainly > some manifestation of the previously-complained-of problem [1] that > RecoveryConflic

Re: Handle infinite recursion in logical replication setup

2022-07-26 Thread vignesh C
On Sun, Jul 24, 2022 at 10:21 PM Jonathan S. Katz wrote: > > On 7/22/22 12:47 AM, Amit Kapila wrote: > > On Fri, Jul 22, 2022 at 1:39 AM Jonathan S. Katz > > wrote: > > >> 1. I'm concerned by calling this "Bidirectional replication" in the docs > >> that we are overstating the current capabiliti

Re: Handle infinite recursion in logical replication setup

2022-07-26 Thread vignesh C
On Tue, Jul 26, 2022 at 7:16 AM Peter Smith wrote: > > Here are some review comments for the patch v38-0002: > > == > > - terminology > > There seemed to be an inconsistent alternation of the terms > "primaries" and "nodes"... For example "Setting replication between > two primaries" versus "

Re: Handle infinite recursion in logical replication setup

2022-07-26 Thread vignesh C
On Tue, Jul 26, 2022 at 1:35 PM shiy.f...@fujitsu.com wrote: > > On Sun, Jul 24, 2022 1:28 AM vignesh C wrote: > > > > Added a note for the same and referred it to the conflicts section. > > > > Thanks for the comments, the attached v38 patch has the changes for the > > same. > > > > Thanks for

Re: Handle infinite recursion in logical replication setup

2022-07-26 Thread vignesh C
On Tue, Jul 26, 2022 at 2:12 PM wangw.f...@fujitsu.com wrote: > > On Sun, Jul 24, 2022 1:28 AM vignesh C wrote: > > Added a note for the same and referred it to the conflicts section. > > > > Thanks for the comments, the attached v38 patch has the changes for the > > same. > > Thanks for your p

Re: Slow standby snapshot

2022-07-26 Thread Andrey Borodin
> On 20 Jul 2022, at 02:12, Michail Nikolaev wrote: > >> I've looked into v5. > Thanks! > > Patch is updated accordingly your remarks. The patch seems Ready for Committer from my POV. Thanks! Best regards, Andrey Borodin.

Re: Handle infinite recursion in logical replication setup

2022-07-26 Thread vignesh C
On Mon, Jul 25, 2022 at 12:58 PM Peter Smith wrote: > > Firstly, I have some (case-sensitivity) questions about the previous > patch which was already pushed [1]. > > Q1. create_subscription docs > > I did not understand why the docs refer to slot_name = NONE, yet the > newly added option says ori

Re: Unstable tests for recovery conflict handling

2022-07-26 Thread Andres Freund
Hi, On 2022-07-26 13:57:53 -0400, Tom Lane wrote: > I happened to notice that while skink continues to fail off-and-on > in 031_recovery_conflict.pl, the symptoms have changed! What > we're getting now typically looks like [1]: > > [10:45:11.475](0.023s) ok 14 - startup deadlock: lock acquisitio

Re: optimize lookups in snapshot [sub]xip arrays

2022-07-26 Thread Andres Freund
On 2022-07-25 12:04:19 -0700, Nathan Bossart wrote: > On Sat, Jul 16, 2022 at 08:59:57PM -0700, Nathan Bossart wrote: > > On Fri, Jul 15, 2022 at 01:08:57PM -0700, Andres Freund wrote: > >> I wonder if we additionally / alternatively could use a faster method of > >> searching the array linearly, e

Re: Transparent column encryption

2022-07-26 Thread Jacob Champion
On Tue, Jul 26, 2022 at 10:52 AM Robert Haas wrote: > On Thu, Jul 21, 2022 at 2:30 PM Jacob Champion > wrote: > > A minimum padding option would fix the leak here, right? If every > > entry is the same length then there's no information to be gained, at > > least in an offline analysis. > > Sure

Re: Unstable tests for recovery conflict handling

2022-07-26 Thread Tom Lane
Andres Freund writes: > On 2022-07-26 13:57:53 -0400, Tom Lane wrote: >> So this is not a case of RecoveryConflictInterrupt doing the wrong thing: >> the startup process hasn't detected the buffer conflict in the first >> place. > I wonder if this, at least partially, could be be due to the elog

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-07-26 Thread Simon Riggs
On Fri, 15 Jul 2022 at 12:29, Aleksander Alekseev wrote: > Personally I didn't expect that > merging patches in this thread would take that long. They are in > "Ready for Committer" state for a long time now and there are no known > issues with them other than unit tests for SLRU [1] should be mer

Re: making relfilenodes 56 bits

2022-07-26 Thread Robert Haas
On Tue, Jul 26, 2022 at 2:07 AM Dilip Kumar wrote: > I have thought about it while doing so but I am not sure whether it is > a good idea or not, because before my change these all were macros > with 2 naming conventions so I just changed to inline function so why > to change the name. Well, the

Re: let's disallow ALTER ROLE bootstrap_superuser NOSUPERUSER

2022-07-26 Thread Robert Haas
On Thu, Jul 21, 2022 at 1:27 PM Nathan Bossart wrote: > Given the current assumptions the code makes about the bootstrap superuser, > I think it makes sense to disallow removing its superuser attribute (at > least via ALTER ROLE NOSUPERUSER). It seems like there is much work to do > before a no-s

Re: let's disallow ALTER ROLE bootstrap_superuser NOSUPERUSER

2022-07-26 Thread Tom Lane
Robert Haas writes: > Reaction to this patch seems tentatively positive so far, so I have > committed it. Maybe someone will still show up to complain ... but I > think it's a good change, so I hope not. I had not actually read the patch, but now that I have, it's got a basic typing error: +

Re: let's disallow ALTER ROLE bootstrap_superuser NOSUPERUSER

2022-07-26 Thread Robert Haas
On Tue, Jul 26, 2022 at 2:59 PM Tom Lane wrote: > I had not actually read the patch, but now that I have, it's got > a basic typing error: > > + boolshould_be_super = BoolGetDatum(boolVal(dissuper->arg)); > + > + if (!should_be_super && roleid == BOOTSTRAP_SUPERUSERID) > +

Re: making relfilenodes 56 bits

2022-07-26 Thread Robert Haas
On Tue, Jul 12, 2022 at 4:35 PM Robert Haas wrote: > > Very minor nitpick: To me REPLACE would be a bit more accurate than RENAME, > > since it includes fsync etc? > > Sure, I had it that way for a while and changed it at the last minute. > I can change it back. Committed that way, also with the

[Commitfest 2022-07] Patch Triage: Waiting on Author

2022-07-26 Thread Jacob Champion
Hello all, I'm making my way through some stalled patches in Waiting on Author. If nothing changes by the end of this CF, I'd recommend marking these as Returned with Feedback. Patch authors CC'd. - jsonpath syntax extensions https://commitfest.postgresql.org/38/2482/ As a few people pointe

Re: Implementing Incremental View Maintenance

2022-07-26 Thread huyajun
Hi, Nagata-san Thank you for your answer, I agree with your opinion, and found some new problems to discuss with you > >> 3. Consider truncate base tables, IVM will not refresh, maybe raise an error >> will be better > > I fixed to support TRUNCATE on base tables in our repository. > h

Re: pg15b2: large objects lost on upgrade

2022-07-26 Thread Robert Haas
On Mon, Jul 18, 2022 at 2:57 PM Robert Haas wrote: > Well, it took a while to figure out how to make that work, but I > believe I've got it now. Attached please find a couple of patches that > should get the job done. They might need a bit of polish, but I think > the basic concepts are sound. So

Re: Postgres do not allow to create many tables with more than 63-symbols prefix

2022-07-26 Thread Tom Lane
Andrey Lepikhov writes: > On 6/27/22 06:38, Masahiko Sawada wrote: Regarding the patch, I think we can merge makeUniqueTypeName() to makeArrayTypeName() as there is no caller of makeUniqueTypeName() who pass tryOriginal = true. >>> I partially agree with you. But I have one reason

Re: Unstable tests for recovery conflict handling

2022-07-26 Thread Andres Freund
Hi, On 2022-07-26 14:30:30 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2022-07-26 13:57:53 -0400, Tom Lane wrote: > >> So this is not a case of RecoveryConflictInterrupt doing the wrong thing: > >> the startup process hasn't detected the buffer conflict in the first > >> place. > > > I

Re: Transparent column encryption

2022-07-26 Thread Robert Haas
On Tue, Jul 26, 2022 at 2:27 PM Jacob Champion wrote: > Right. My point is, if you have a column that has exactly one > important value that is 17 bytes long when converted to text, you're > going to want to know that block size exactly, because the encryption > will be effectively useless for tha

Question about ExplainOneQuery_hook

2022-07-26 Thread Zhihong Yu
Hi, I was looking at ExplainOneQuery() where ExplainOneQuery_hook is called. Currently the call to the hook is in if block and normal processing is in else block. What if the hook doesn't want to duplicate the whole code printing execution plan ? Please advise. Thanks

Re: Expand palloc/pg_malloc API

2022-07-26 Thread Tom Lane
Peter Eisentraut writes: > On 17.05.22 20:43, Tom Lane wrote: >> So I think Peter's got a good idea here (I might quibble with the details >> of some of these macros). But it's not really going to move the >> safety goalposts very far unless we make a concerted effort to make >> these be the styl

Re: Skip partition tuple routing with constant partition key

2022-07-26 Thread David Rowley
Thank for looking at this. On Sat, 23 Jul 2022 at 01:23, Amit Langote wrote: > + /* > +* The Datum has changed. Zero the number of times we've > +* found last_found_datum_index in a row. > +*/ > + par

Re: fairywren hung in pg_basebackup tests

2022-07-26 Thread Thomas Munro
On Tue, Jul 26, 2022 at 4:03 AM Andrew Dunstan wrote: > On 2022-07-25 Mo 11:24, Thomas Munro wrote: > > On Tue, Jul 26, 2022 at 3:08 AM Tom Lane wrote: > >> Hmm ... an alternative theory is that the test is fine, and what > >> it's telling us is that get_dirent_type() is still wrong on msys. > >>

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-07-26 Thread David Rowley
On Tue, 26 Jul 2022 at 12:01, Zhihong Yu wrote: > sort order the the planner chooses is simply : there is duplicate `the` I think the first "the" should be "that" > + /* mark this aggregate is covered by 'currpathkeys' */ > > is covered by -> as covered by I think it was s

Re: explain_regress, explain(MACHINE), and default to explain(BUFFERS) (was: BUFFERS enabled by default in EXPLAIN (ANALYZE))

2022-07-26 Thread David G. Johnston
On Mon, Jan 24, 2022 at 9:54 AM Justin Pryzby wrote: > I'm renaming this thread for better visibility, since buffers is a small, > optional part of the patches I sent. > > I made a CF entry here. > https://commitfest.postgresql.org/36/3409/ > > On Wed, Dec 01, 2021 at 06:58:20PM -0600, Justin Pry

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-07-26 Thread David Rowley
On Tue, 26 Jul 2022 at 19:39, Richard Guo wrote: > Also I'm wondering if it's possible to take into consideration the > ordering indicated by existing indexes when determining the pathkeys. So > that for the query below we can avoid the Incremental Sort node if we > consider that there is an index

Re: Question about ExplainOneQuery_hook

2022-07-26 Thread David G. Johnston
On Tue, Jul 26, 2022 at 1:54 PM Zhihong Yu wrote: > Hi, > I was looking at ExplainOneQuery() where ExplainOneQuery_hook is called. > > Currently the call to the hook is in if block and normal processing is in > else block. > > What if the hook doesn't want to duplicate the whole code printing > e

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-07-26 Thread Thomas Munro
On Sat, Jul 2, 2022 at 11:18 AM Andres Freund wrote: > On 2022-07-01 13:14:23 -0700, Andres Freund wrote: > > - the pg_usleep(5000) in ResolveRecoveryConflictWithVirtualXIDs() can > > completely swamp the target(s) on a busy system. This surely is > > exascerbated > > by the usleep I added Re

Re: Skip partition tuple routing with constant partition key

2022-07-26 Thread Zhihong Yu
On Tue, Jul 26, 2022 at 3:28 PM David Rowley wrote: > Thank for looking at this. > > On Sat, 23 Jul 2022 at 01:23, Amit Langote > wrote: > > + /* > > +* The Datum has changed. Zero the number of times > we've > > +* found last_found_datu

Re: Transparent column encryption

2022-07-26 Thread Jacob Champion
On 7/26/22 13:25, Robert Haas wrote: > I certainly have no objection to being clear about such details in the > documentation. Cool. > I fear the phenomenon where > doing anything about a problem makes you responsible for the whole > problem. If we disclaim the ability to hide the length of value

Re: [Commitfest 2022-07] Patch Triage: Waiting on Author

2022-07-26 Thread Justin Pryzby
On Tue, Jul 26, 2022 at 12:26:59PM -0700, Jacob Champion wrote: > Hello all, > > I'm making my way through some stalled patches in Waiting on Author. If > nothing changes by the end of this CF, I'd recommend marking these > as Returned with Feedback. +1 I suggest that, if you send an email when

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-07-26 Thread Tom Lane
Thomas Munro writes: > ... The regular expression machinery is capable of > consuming a lot of CPU, and does CANCEL_REQUESTED(nfa->v->re) > frequently to avoid getting stuck. With the patch as it stands, that > would never be true. Surely that can't be too hard to fix. We might have to refactor

Re: [Commitfest 2022-07] Patch Triage: Waiting on Author

2022-07-26 Thread Jacob Champion
On 7/26/22 16:20, Justin Pryzby wrote: > I suggest that, if you send an email when marking as RWF, to mention that the > existing patch record can be re-opened and moved to next CF. > > I'm aware that people may think that this isn't always a good idea, but it's > nice to mention that it's possibl

Re: [Commitfest 2022-07] Patch Triage: Waiting on Author

2022-07-26 Thread James Coleman
On Tue, Jul 26, 2022 at 3:27 PM Jacob Champion wrote: ... > - Consider parallel for LATERAL subqueries having LIMIT/OFFSET > https://commitfest.postgresql.org/38/2851/ > > Does this have a path forward? It's been Waiting on Author since the > beginning of last CF, with open concerns from Tom

Re: Expand palloc/pg_malloc API

2022-07-26 Thread David G. Johnston
On Tue, Jul 26, 2022 at 2:32 PM Tom Lane wrote: > > 2. I don't like the "palloc_ptrtype" name at all. I see that you > borrowed that name from talloc, but I doubt that's a precedent that > very many people are familiar with. > To me it sounds like it might > allocate something that's the size

Re: pg15b2: large objects lost on upgrade

2022-07-26 Thread Bruce Momjian
On Tue, Jul 26, 2022 at 03:45:11PM -0400, Robert Haas wrote: > On Mon, Jul 18, 2022 at 2:57 PM Robert Haas wrote: > > Well, it took a while to figure out how to make that work, but I > > believe I've got it now. Attached please find a couple of patches that > > should get the job done. They might

Re: Refactoring postgres_fdw/connection.c

2022-07-26 Thread Kyotaro Horiguchi
At Tue, 26 Jul 2022 18:33:04 +0900, Fujii Masao wrote in > > I'm not sure the two are similar with each other. The new function > > pgfdw_exec_pre_commit() looks like a merger of two isolated code paths > > intended to share a seven-line codelet. I feel the code gets a bit > > harder to unders

  1   2   >