Re: Logical replication timeout problem

2022-03-16 Thread Amit Kapila
On Wed, Mar 16, 2022 at 7:38 PM Masahiko Sawada wrote: > > On Wed, Mar 16, 2022 at 11:57 AM wangw.f...@fujitsu.com > wrote: > > > But it really depends on the workload, the server condition, and the > > > timeout value, right? The logical decoding might involve disk I/O much > > > to spill/load i

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Thomas Munro
On Thu, Mar 17, 2022 at 7:18 PM Michael Paquier wrote: > On Thu, Mar 17, 2022 at 04:34:30PM +1300, Thomas Munro wrote: > > I think what Horiguchi-san was pointing out above is that you need to > > enable the GUC *and* say LOCATION '', which the new paragraph doesn't > > capture. What do you think

Re: Unhyphenation of crash-recovery

2022-03-16 Thread Peter Eisentraut
On 16.03.22 02:25, Kyotaro Horiguchi wrote: Hello, this is a derived topic from [1], summarized as $SUBJECT. This just removes useless hyphens from the words "(crash|emergency)-recovery". We don't have such wordings for "archive recovery" This patch fixes non-user-facing texts as well as user-fa

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Michael Paquier
On Thu, Mar 17, 2022 at 04:34:30PM +1300, Thomas Munro wrote: > I think what Horiguchi-san was pointing out above is that you need to > enable the GUC *and* say LOCATION '', which the new paragraph doesn't > capture. What do you think about this: > > A path relative to the data directory is retur

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Amit Kapila
On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote: > > I've attached an updated version patch. > The patch LGTM. I have made minor changes in comments and docs in the attached patch. Kindly let me know what you think of the attached? I am planning to commit this early next week (on Monday) u

Re: Out-of-tree certificate interferes ssltest

2022-03-16 Thread Michael Paquier
On Wed, Mar 16, 2022 at 11:45:39AM +0100, Daniel Gustafsson wrote: > On 16 Mar 2022, at 08:36, Kyotaro Horiguchi wrote: >> The attached fixes that and make-world successfully finished even if I >> have a cert file in my home direcotory. > > Seems correct to me, thanks! The ultimate test I can th

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 20:49:12 +0530, Ashutosh Sharma wrote in > I can see that the pg_get_wal_records_info function shows the details > of the WAL record whose existence is beyond the user specified > stop/end lsn pointer. See below: > > ashu@postgres=# select * from pg_get_wal_records_info('0/0

Re: Window Function "Run Conditions"

2022-03-16 Thread Corey Huinker
On Tue, Mar 15, 2022 at 5:24 PM Greg Stark wrote: > This looks like an awesome addition. > > I have one technical questions... > > Is it possible to actually transform the row_number case into a LIMIT > clause or make the planner support for this case equivalent to it (in > which case we can repl

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Thomas Munro
On Thu, Mar 17, 2022 at 3:53 PM Michael Paquier wrote: > > On Wed, Mar 16, 2022 at 05:15:58PM +0900, Kyotaro Horiguchi wrote: > > I'm not sure that the "of the symbolic link in pg_tblspc/" is > > needed. And allow_in_place_tablespaces alone doesn't create in-place > > tablespace. So this might nee

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Amit Kapila
On Thu, Mar 17, 2022 at 8:13 AM shiy.f...@fujitsu.com wrote: > > On Wed, Mar 16, 2022 4:23 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. > > > > Thanks for updating the patch. Here are some comments for the v15 patch. > > 1. src/backend/replication/logical/worker.c > >

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-03-16 Thread Andy Fan
Hi: I just tested more cases for the estimation issue for this feature, and we can find **we get a more accurate/stable estimation than before**. Here is the test cases and result (by comparing the master version and patched version). create table ec_t110 as select i::int as a from generate_serie

Re: BufferAlloc: don't take two simultaneous locks

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 14:11:58 +0300, Yura Sokolov wrote in > В Ср, 16/03/2022 в 12:07 +0900, Kyotaro Horiguchi пишет: > > At Tue, 15 Mar 2022 13:47:17 +0300, Yura Sokolov > > wrote in > > In v7, HASH_ENTER returns the element stored in DynaHashReuse using > > the freelist_idx of the new key.

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Michael Paquier
On Wed, Mar 16, 2022 at 05:15:58PM +0900, Kyotaro Horiguchi wrote: > I'm not sure that the "of the symbolic link in pg_tblspc/" is > needed. And allow_in_place_tablespaces alone doesn't create in-place > tablespace. So this might need rethink at least for the second point. Surely this can be impro

RE: Skipping logical replication transactions on subscriber side

2022-03-16 Thread shiy.f...@fujitsu.com
On Wed, Mar 16, 2022 4:23 PM Masahiko Sawada wrote: > > I've attached an updated version patch. > Thanks for updating the patch. Here are some comments for the v15 patch. 1. src/backend/replication/logical/worker.c + * to skip applying the changes when starting to apply changes. The subskip

Re: support for MERGE

2022-03-16 Thread Japin Li
On Thu, 17 Mar 2022 at 03:18, Alvaro Herrera wrote: > v16. > On 2022-Mar-14, Japin Li wrote: > >> + ar_delete_trig_tcs = mtstate->mt_transition_capture; >> + if (mtstate->operation == CMD_UPDATE && >> mtstate->mt_transition_capture && >> + mtstate->mt_transition_captur

Re: Support logical replication of DDLs

2022-03-16 Thread Japin Li
Hi, Zhang Li On Thu, 17 Mar 2022 at 05:17, Zheng Li wrote: > Hi, > >>If you don't mind, would you like to share the POC or the branch for this >>work? > > The POC patch is attached. It currently supports the following > functionalities: > 1. Configure either database level or table level DDL

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-16 Thread Jacob Champion
On Wed, 2022-03-16 at 15:56 +0900, Kyotaro Horiguchi wrote: > At Tue, 15 Mar 2022 21:41:49 +, Jacob Champion > wrote in > > Hmm, the sslinfo tests are failing? I wouldn't have expected that based > > on the patch changes. Just to confirm -- they pass for you without the > > patch? > > Mmm..

Re: Add index scan progress to pg_stat_progress_vacuum

2022-03-16 Thread Nathan Bossart
On Wed, Mar 16, 2022 at 09:47:49PM +, Imseih (AWS), Sami wrote: > Spoke to Nathan offline and fixed some more comments/nitpicks in the patch. I don't have any substantial comments for v9, so I think this can be marked as ready-for-committer. However, we probably should first see whether Sawad

Re: Proposal: Support custom authentication methods using hooks

2022-03-16 Thread Stephen Frost
Greetings, * samay sharma (smilingsa...@gmail.com) wrote: > On Wed, Mar 16, 2022 at 8:02 AM Stephen Frost wrote: > > How about- if we just added OAUTH support directly into libpq and the > > backend, would that work with Azure's OIDC provider? If not, why not? > > Overall, Azure AD implements O

Re: Proposal: Support custom authentication methods using hooks

2022-03-16 Thread Stephen Frost
Greetings, * Jeff Davis (pg...@j-davis.com) wrote: > On Wed, 2022-03-16 at 11:02 -0400, Stephen Frost wrote: > > That we're having to extend this quite a bit to work for the proposed > > OAUTH patches and that it still doesn't do anything for the client > > side > > (note that the OAUTHBEARER patc

Re: Support logical replication of DDLs

2022-03-16 Thread Alvaro Herrera
Hello I think this is a pretty interesting and useful feature. Did you see some old code I wrote towards this goal? https://www.postgresql.org/message-id/20150215044814.gl3...@alvh.no-ip.org The intention was that DDL would produce some JSON blob that accurately describes the DDL that was run; th

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Andrew Dunstan
On 3/16/22 16:53, Tom Lane wrote: >> Personally I don't have problem with the use of SETTING. I think the >> meaning is pretty plain in context and unlikely to produce any confusion. > I'm just unhappy about the disconnect with the documentation. I wonder > if we could get away with s/configura

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Tom Lane
Andrew Dunstan writes: > On 3/16/22 14:47, Tom Lane wrote: >> I'm also fairly allergic to the way that this patch has decided to assign >> multi-word names to privilege types (ie SET VALUE, ALTER SYSTEM). There >> is no existing precedent for that, and I think it's going to break >> client-side c

Re: Proposal: Support custom authentication methods using hooks

2022-03-16 Thread Jeff Davis
On Tue, 2022-03-15 at 12:27 -0700, samay sharma wrote: > This patch-set adds the following: > > * Allow multiple custom auth providers to be registered (Addressing > feedback from Aleksander and Andrew) > * Modify the test extension to use SCRAM to exchange secrets (Based > on Andres's suggestion)

Re: Concurrent deadlock scenario with DROP INDEX on partitioned index

2022-03-16 Thread Jimmy Yih
Zhihong Yu wrote: > Hi, > For RangeVarCallbackForDropRelation(): > > + LockRelationOid(indexRelationOid, heap_lockmode); > > Since the above is called for both if and else blocks, it can be lifted > outside. Thanks for the feedback. Attached new v3 patch with feedback addressed.

Re: Proposal: Support custom authentication methods using hooks

2022-03-16 Thread samay sharma
Hi, On Wed, Mar 16, 2022 at 8:02 AM Stephen Frost wrote: > Greetings, > > * samay sharma (smilingsa...@gmail.com) wrote: > > On Fri, Mar 4, 2022 at 11:15 AM Jacob Champion > wrote: > > > On Thu, 2022-03-03 at 11:12 +0100, Peter Eisentraut wrote: > > > > At the moment, it is not possible to judg

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Andrew Dunstan
On 3/16/22 14:47, Tom Lane wrote: > Andrew Dunstan writes: >> Generally I think this is now in fairly good shape, I've played with it >> and it seems to do what I expect in every case, and the things I found >> surprising are gone. > Stepping back a bit ... do we really want to institutionalize

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Tom Lane
Joshua Brindle writes: > On Wed, Mar 16, 2022 at 3:06 PM Tom Lane wrote: >> It's going to be hard to do anything useful in a hook that (a) does >> not know which GUC is being assigned to and (b) cannot do catalog >> accesses for fear that we're not inside a transaction. (b), in >> particular, se

Re: Concurrent deadlock scenario with DROP INDEX on partitioned index

2022-03-16 Thread Zhihong Yu
On Wed, Mar 16, 2022 at 11:20 AM Jimmy Yih wrote: > Tom Lane wrote: > > 1. RangeVarCallbackForDropRelation can get called more than once > > during a lookup (in case of concurrent rename and suchlike). > > If so it needs to be prepared to drop the lock(s) it got last time. > > You have not imple

Re: Proposal: Support custom authentication methods using hooks

2022-03-16 Thread Jeff Davis
On Wed, 2022-03-16 at 11:02 -0400, Stephen Frost wrote: > That we're having to extend this quite a bit to work for the proposed > OAUTH patches and that it still doesn't do anything for the client > side > (note that the OAUTHBEARER patches are still patching libpq to add > support directly and sur

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Joshua Brindle
On Wed, Mar 16, 2022 at 3:06 PM Tom Lane wrote: > > Mark Dilger writes: > > On Mar 16, 2022, at 11:47 AM, Tom Lane wrote: > >> ... I therefore judge the > >> hook calls added to ExecSetVariableStmt and AlterSystemSetConfigFile > >> to be 100% useless, in fact probably counterproductive because t

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Tom Lane
Mark Dilger writes: > On Mar 16, 2022, at 11:47 AM, Tom Lane wrote: >> ... I therefore judge the >> hook calls added to ExecSetVariableStmt and AlterSystemSetConfigFile >> to be 100% useless, in fact probably counterproductive because they >> introduce a boatload of worries about whether the righ

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Joshua Brindle
> I suggest that what might be saner is to consider that the "objects" > that the hook calls are concerned with are the pg_setting_acl entries, > not the underlying GUCs, and thus that the hooks need be invoked only > when creating, destroying or altering those entries. If we do have > a need fo

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Mark Dilger
> On Mar 16, 2022, at 11:47 AM, Tom Lane wrote: > > Stepping back a bit ... do we really want to institutionalize the > term "setting" for GUC variables? I realize that the view pg_settings > exists, but the documentation generally prefers the term "configuration > parameters". Where config.

Re: JSON path decimal literal syntax

2022-03-16 Thread Peter Eisentraut
On 06.03.22 02:43, Nikita Glukhov wrote: Obviously, there are compatibility issues with expressions like '1.type()', which will start to require parentheses around numbers, but they seem to be useful only for our regression tests. The corresponding changes in jsonpath_out() related to parenthese

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Tom Lane
Andrew Dunstan writes: > Generally I think this is now in fairly good shape, I've played with it > and it seems to do what I expect in every case, and the things I found > surprising are gone. Stepping back a bit ... do we really want to institutionalize the term "setting" for GUC variables? I r

Re: Optimize external TOAST storage

2022-03-16 Thread Nathan Bossart
On Wed, Mar 16, 2022 at 11:36:56AM -0700, Nathan Bossart wrote: > Thinking further, is simply reducing the number of TOAST chunks the right > thing to look at? If I want to add a TOAST attribute that requires 100,000 > chunks, and you told me that I could save 10% in the read path for an extra > 2

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-16 Thread Nathan Bossart
On Wed, Mar 16, 2022 at 03:02:41PM +0530, Bharath Rupireddy wrote: > On Mon, Mar 14, 2022 at 10:34 PM Nathan Bossart > wrote: >> I'm -1 on splitting these new statistics to separate LOGs. In addition to >> making it more difficult to discover statistics for a given checkpoint, I >> think it actua

Re: Optimize external TOAST storage

2022-03-16 Thread Nathan Bossart
On Wed, Mar 16, 2022 at 10:08:45AM -0400, Robert Haas wrote: > I would like to take a slightly contrary position. I think that a > system here that involves multiple knobs is going to be too > complicated to be of any real-world use, because almost nobody will > understand it or tune it properly fo

Re: Concurrent deadlock scenario with DROP INDEX on partitioned index

2022-03-16 Thread Jimmy Yih
Tom Lane wrote: > 1. RangeVarCallbackForDropRelation can get called more than once > during a lookup (in case of concurrent rename and suchlike). > If so it needs to be prepared to drop the lock(s) it got last time. > You have not implemented any such logic. This doesn't seem hard > to fix, just

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-03-16 Thread Nathan Bossart
On Wed, Mar 16, 2022 at 06:40:23PM +1300, Thomas Munro wrote: > Pushed and back-patched (it's slightly different before 12). Thanks! Thank you! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Assert in pageinspect with NULL pages

2022-03-16 Thread Alexander Lakhin
Hello Michael, 16.03.2022 10:39, Michael Paquier wrote: On Mon, Feb 21, 2022 at 10:00:00AM +0300, Alexander Lakhin wrote: Could you please confirm before committing the patchset that it fixes the bug #16527 [1]? Or maybe I could check it? (Original patch proposed by Daria doesn't cover that case

Re: Column Filtering in Logical Replication

2022-03-16 Thread Tomas Vondra
On 3/15/22 09:30, Tomas Vondra wrote: > > > On 3/15/22 05:43, Amit Kapila wrote: >> On Mon, Mar 14, 2022 at 4:42 PM houzj.f...@fujitsu.com >> wrote: >>> >>> On Monday, March 14, 2022 5:08 AM Tomas Vondra >>> wrote: On 3/12/22 05:30, Amit Kapila wrote: >> ... > > Okay,

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-16 Thread Ashutosh Sharma
I can see that the pg_get_wal_records_info function shows the details of the WAL record whose existence is beyond the user specified stop/end lsn pointer. See below: ashu@postgres=# select * from pg_get_wal_records_info('0/0128', '0/0129'); -[ RECORD 1 ]+--

Re: refactoring basebackup.c (zstd negative compression)

2022-03-16 Thread Justin Pryzby
Should zstd's negative compression levels be supported here ? Here's a POC patch which is enough to play with it. $ src/bin/pg_basebackup/pg_basebackup --wal-method fetch -Ft -D - -h /tmp --no-sync --compress=zstd |wc -c 12305659 $ src/bin/pg_basebackup/pg_basebackup --wal-method fetch -Ft -D -

Re: Proposal: Support custom authentication methods using hooks

2022-03-16 Thread Stephen Frost
Greetings, * samay sharma (smilingsa...@gmail.com) wrote: > On Fri, Mar 4, 2022 at 11:15 AM Jacob Champion wrote: > > On Thu, 2022-03-03 at 11:12 +0100, Peter Eisentraut wrote: > > > At the moment, it is not possible to judge whether the hook interface > > > you have chosen is appropriate. > > >

Re: Issue with pg_stat_subscription_stats

2022-03-16 Thread Masahiko Sawada
On Wed, Mar 16, 2022 at 8:51 PM Amit Kapila wrote: > > On Tue, Mar 15, 2022 at 10:09 AM Masahiko Sawada > wrote: > > > > On Tue, Mar 15, 2022 at 3:34 AM Melanie Plageman > > wrote: > > > > > > On Mon, Mar 14, 2022 at 4:02 AM Masahiko Sawada > > > wrote: > > > > > > > > On Mon, Mar 14, 2022 at

Re: Unhyphenation of crash-recovery

2022-03-16 Thread Robert Haas
On Tue, Mar 15, 2022 at 9:39 PM Andres Freund wrote: > On March 15, 2022 6:25:09 PM PDT, Kyotaro Horiguchi > wrote: > >Hello, this is a derived topic from [1], summarized as $SUBJECT. > > > >This just removes useless hyphens from the words > >"(crash|emergency)-recovery". We don't have such word

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-16 Thread Stephen Frost
Greetings, * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > On Tue, Mar 15, 2022 at 7:21 AM Bharath Rupireddy > wrote: > > > > On Mon, Mar 14, 2022 at 8:25 PM Stephen Frost wrote: > > > > > > > As this patch is currently written, pg_monitor has access these > > > > functions

Re: ICU for global collation

2022-03-16 Thread Peter Eisentraut
On 15.03.22 18:28, Robert Haas wrote: On Tue, Mar 15, 2022 at 12:58 PM Peter Eisentraut wrote: On 14.03.22 19:57, Robert Haas wrote: 1. What will happen if I set the ICU collation to something that doesn't match the libc collation? How bad are the consequences? These are unrelated, so there

Re: Corruption during WAL replay

2022-03-16 Thread Robert Haas
On Wed, Mar 16, 2022 at 1:14 AM Kyotaro Horiguchi wrote: > storage.c: > +* Make sure that a concurrent checkpoint can't complete while > truncation > +* is in progress. > +* > +* The truncation operation might drop buffers that the checkpoint > +* otherwise

Re: Optimize external TOAST storage

2022-03-16 Thread Robert Haas
On Tue, Mar 15, 2022 at 5:48 PM Nathan Bossart wrote: > I apologize for thinking out loud a bit here, but I hope this gives you > some insight into my perspective on this. In general, I am skeptical that > we can choose one threshold that will work for all PostgreSQL installations > in the known

Re: Logical replication timeout problem

2022-03-16 Thread Masahiko Sawada
On Wed, Mar 16, 2022 at 11:57 AM wangw.f...@fujitsu.com wrote: > > On Wed, Mar 9, 2022 at 2:45 PM Masahiko Sawada wrote: > > > Thanks for your comments. > > > On Wed, Mar 9, 2022 at 10:26 AM I wrote: > > > On Tue, Mar 8, 2022 at 3:52 PM Masahiko Sawada > > wrote: > > > > I've looked at the patch

Re: Tab completion for SET TimeZone

2022-03-16 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > I just noticed I left out the = in the match check, here's an updated > patch that fixes that. Hmm is that actually going to be useful in that form? Most time zone names contain slashes and will therefore require single-quoting. I think yo

Re: Tab completion for SET TimeZone

2022-03-16 Thread Dagfinn Ilmari Mannsåker
Dagfinn Ilmari Mannsåker writes: > Hi hackers, > > I noticed there was no tab completion for time zones in psql, so here's > a patch that implements that. I just noticed I left out the = in the match check, here's an updated patch that fixes that. - ilmari >From bcfa40ff3a6702e1bd7112eeaecfde8

Tab completion for SET TimeZone

2022-03-16 Thread Dagfinn Ilmari Mannsåker
Hi hackers, I noticed there was no tab completion for time zones in psql, so here's a patch that implements that. I chose lower-casing the names since they are case insensitive, and verbatim since ones without slashes can be entered without quotes, and (at least my version of) readline completes

Re: ExecTypeSetColNames is fundamentally broken

2022-03-16 Thread Aleksander Alekseev
Hi hackers, > Anyone else have thoughts? I came across this thread while looking for the patches that need review. My understanding of the code is limited, but I can say that I don't see anything particularly wrong with it. I can also confirm that it fixes the problem reported by the user while

Declare PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY for aarch64

2022-03-16 Thread Yura Sokolov
Good day, hackers. Architecture Reference Manual for ARMv8 B2.2.1 [1] states: For explicit memory effects generated from an Exception level the following rules apply: - A read that is generated by a load instruction that loads a single general-purpose register and is aligned to the size o

Re: Issue with pg_stat_subscription_stats

2022-03-16 Thread Amit Kapila
On Tue, Mar 15, 2022 at 10:09 AM Masahiko Sawada wrote: > > On Tue, Mar 15, 2022 at 3:34 AM Melanie Plageman > wrote: > > > > On Mon, Mar 14, 2022 at 4:02 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Mar 14, 2022 at 2:05 AM Melanie Plageman > > > wrote: > > > > > > > > On Sat, Mar 12, 202

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Andrew Dunstan
On 3/15/22 16:59, Mark Dilger wrote: >> On Mar 6, 2022, at 3:27 PM, Tom Lane wrote: >> >> Mark Dilger writes: >>> The existing patch allows grants on unknown gucs, because it can't know >>> what guc an upgrade script will introduce, and the grant statement may need >>> to execute before the g

unnecessary (same) restart_lsn handing in LogicalIncreaseRestartDecodingForSlot

2022-03-16 Thread Ashutosh Bapat
Hi All, At the beginning of LogicalIncreaseRestartDecodingForSlot(), we have codeine ``` 1623 /* don't overwrite if have a newer restart lsn */ 1624 if (restart_lsn <= slot->data.restart_lsn) 1625 { 1626 } 1627 1628 /* 1629 * We might have already flushed far enough to dire

Re: Column Filtering in Logical Replication

2022-03-16 Thread Amit Kapila
On Tue, Mar 15, 2022 at 7:38 AM shiy.f...@fujitsu.com wrote: > > On Mon, Mar 14, 2022 5:08 AM Tomas Vondra > wrote: > > 3. src/backend/commands/publicationcmds.c > +/* > + * Check if all columns referenced in the column list are part of the > + * REPLICA IDENTITY index or not. > + * > + * Return

Re: BufferAlloc: don't take two simultaneous locks

2022-03-16 Thread Yura Sokolov
В Ср, 16/03/2022 в 12:07 +0900, Kyotaro Horiguchi пишет: > At Tue, 15 Mar 2022 13:47:17 +0300, Yura Sokolov > wrote in > > В Вт, 15/03/2022 в 16:25 +0900, Kyotaro Horiguchi пишет: > > > Hmm. v8 returns stashed element with original patition index when the > > > element is *not* reused. But what

Re: Out-of-tree certificate interferes ssltest

2022-03-16 Thread Daniel Gustafsson
> On 16 Mar 2022, at 08:36, Kyotaro Horiguchi wrote: > I think we don't want this behavior. Agreed. > The attached fixes that and make-world successfully finished even if I > have a cert file in my home direcotory. Seems correct to me, thanks! -- Daniel Gustafsson https://vmware

Re: pgcrypto: Remove internal padding implementation

2022-03-16 Thread Peter Eisentraut
Interesting. I have added test cases about this. Could you describe how you arrived at the second test case? Sure -- that second ciphertext is the result of running SELECT encrypt_iv('abcdefghijklmnopqrstuvwxyz', '0123456', 'abcd', 'aes'); and then incrementing the last byte of the fi

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-16 Thread Bharath Rupireddy
On Mon, Mar 14, 2022 at 10:34 PM Nathan Bossart wrote: > > > Attaching v3 patch which emits logs only when necessary and doesn't > > clutter the existing "checkpoint/restartpoint completed" message, see > > some sample logs at [1]. Please review it further. > > I'm okay with not adding these stati

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-16 Thread Bharath Rupireddy
On Wed, Mar 16, 2022 at 1:47 AM Nathan Bossart wrote: > > On Tue, Mar 15, 2022 at 11:04:26AM +0900, Michael Paquier wrote: > > On Mon, Mar 14, 2022 at 03:54:19PM +0530, Bharath Rupireddy wrote: > >> At times, the snapshot or mapping files can be large in number and one > >> some platforms it takes

Re: [PATCH] pgbench: add multiconnect option

2022-03-16 Thread Fabien COELHO
Hello Greg, It looks like David sent a patch and Fabien sent a followup patch. But there hasn't been a whole lot of discussion or further patches. It sounds like there are some basic questions about what the right interface should be. Are there specific questions that would be helpful for mov

Re: Assert in pageinspect with NULL pages

2022-03-16 Thread Michael Paquier
On Wed, Feb 23, 2022 at 12:09:02PM +0500, Daria Lepikhova wrote: > And one more addition. In the previous version of the patch, I forgot to add > tests for the gist index, but the described problem is also relevant for it. So, I have looked at this second part of the thread, and concluded that we

Re: Possible corruption by CreateRestartPoint at promotion

2022-03-16 Thread Kyotaro Horiguchi
Just for the record. An instance of the corruption showed up in this mailing list [1]. [1] https://www.postgresql.org/message-id/flat/9EB4CF63-1107-470E-B5A4-061FB9EF8CC8%40outlook.com regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Standby got invalid primary checkpoint after crashed right after promoted.

2022-03-16 Thread Kyotaro Horiguchi
(My previous mail hass crossed with this one) At Wed, 16 Mar 2022 08:21:46 +, hao harry wrote in > Found this issue is duplicated to [1], after applied that patch, I cannot > reproduce it anymore. > > [1] > https://www.postgresql.org/message-id/flat/20220316.102444.2193181487576617583.hor

Re: Standby got invalid primary checkpoint after crashed right after promoted.

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 07:16:16 +, hao harry wrote in > Hi, pgsql-hackers, > > I think I found a case that database is not recoverable, would you please > give a look? > > Here is how it happens: > > - setup primary/standby > - do a lots INSERT at primary > - create a checkpoint at primary >

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Masahiko Sawada
On Wed, Mar 16, 2022 at 1:20 PM Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 7:58 AM Amit Kapila wrote: > > > > On Wed, Mar 16, 2022 at 6:03 AM Masahiko Sawada > > wrote: > > > > > > On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Mar 15, 2022 at 11:43 AM Ma

Re: Standby got invalid primary checkpoint after crashed right after promoted.

2022-03-16 Thread hao harry
Found this issue is duplicated to [1], after applied that patch, I cannot reproduce it anymore. [1] https://www.postgresql.org/message-id/flat/20220316.102444.2193181487576617583.horikyota.ntt%40gmail.com

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 15:42:52 +0900, Michael Paquier wrote in > On Wed, Mar 16, 2022 at 10:34:15AM +0900, Kyotaro Horiguchi wrote: > > +1. Desn't the doc need to mention that? > > Yes, I agree that it makes sense to add a note, even if > allow_in_place_tablespaces is a developer option. I have

Re: Out-of-tree certificate interferes ssltest

2022-03-16 Thread Michael Paquier
On Wed, Mar 16, 2022 at 04:36:58PM +0900, Kyotaro Horiguchi wrote: > ok 6 - ssl_client_cert_present() for connection with cert > connection error: 'psql: error: connection to server at "127.0.0.1", port > 61688 failed: could not read certificate file > "/home/horiguti/.postgresql/postgresql.crt":

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-16 Thread Bharath Rupireddy
On Tue, Mar 15, 2022 at 7:21 AM Bharath Rupireddy wrote: > > On Mon, Mar 14, 2022 at 8:25 PM Stephen Frost wrote: > > > > > As this patch is currently written, pg_monitor has access these > > > functions, though I don't think that's the right privilege level at > > > least for pg_get_raw_wal_reco

Re: Assert in pageinspect with NULL pages

2022-03-16 Thread Michael Paquier
On Mon, Feb 21, 2022 at 10:00:00AM +0300, Alexander Lakhin wrote: > Could you please confirm before committing the patchset that it fixes > the bug #16527 [1]? Or maybe I could check it? > (Original patch proposed by Daria doesn't cover that case, but if the > patch going to be improved, probably i

Out-of-tree certificate interferes ssltest

2022-03-16 Thread Kyotaro Horiguchi
Hello. 003_sslinfo.pl fails for me. ok 6 - ssl_client_cert_present() for connection with cert connection error: 'psql: error: connection to server at "127.0.0.1", port 61688 failed: could not read certificate file "/home/horiguti/.postgresql/postgresql.crt": no start line' while running 'psql -

Re: Tab completion for ALTER MATERIALIZED VIEW ... SET ACCESS METHOD

2022-03-16 Thread Michael Paquier
Hi Nagata-san, On Wed, Mar 16, 2022 at 01:33:37PM +0900, Yugo NAGATA wrote: > SET ACCESS METHOD is supported in ALTER TABLE since the commit > b0483263dd. Since that time, this also has be allowed SET ACCESS > METHOD in ALTER MATERIALIZED VIEW. Although it is not documented, > this works. Yes, t

Standby got invalid primary checkpoint after crashed right after promoted.

2022-03-16 Thread hao harry
Hi, pgsql-hackers, I think I found a case that database is not recoverable, would you please give a look? Here is how it happens: - setup primary/standby - do a lots INSERT at primary - create a checkpoint at primary - wait until standby start doing restart point, it take about 3mins syncing b

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Masahiko Sawada
On Wed, Mar 16, 2022 at 11:28 AM Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 6:03 AM Masahiko Sawada wrote: > > > > On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila wrote: > > > > > > On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada > > > wrote: > > > > > > > > > > 6. > > > @@ -1583,7 +1649,8 @@

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 15:56:02 +0900 (JST), Kyotaro Horiguchi wrote in > Mmm I'm not sure how come I didn't noticed that, master also fails > for me fo the same reason. In the past that may fail when valid > clinent-certs exists in the users ~/.postgresql but I believe that has > been fixed.

Re: logical replication empty transactions

2022-03-16 Thread Ajin Cherian
On Mon, Mar 7, 2022 at 11:44 PM Ajin Cherian wrote: > > Fixed. > > regards, > Ajin Cherian > Fujitsu Australia Rebased the patch and fixed some whitespace errors. regards, Ajin Cherian Fujitsu Australia v25-0001-Skip-empty-transactions-for-logical-replication.patch Description: Binary data v2