Re: BufferAlloc: don't take two simultaneous locks

2022-01-21 Thread Andrey Borodin
> 21 дек. 2021 г., в 10:23, Yura Sokolov написал(а): > > Hi Yura! I've took a look into the patch. The idea seems reasonable to me: clearing\evicting old buffer and placing new one seem to be different units of work, there is no need to couple both partition locks together. And the claime

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Shruthi Gowda
On Sat, Jan 22, 2022 at 12:17 AM Robert Haas wrote: > > On Fri, Jan 21, 2022 at 8:40 AM Shruthi Gowda wrote: > > From what I see in the code, template0 and postgres are the last > > things that get created in initdb phase. The system OIDs that get > > assigned to these DBs vary from release to re

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Shruthi Gowda
On Sat, Jan 22, 2022 at 12:27 AM Tom Lane wrote: > > Robert Haas writes: > > It seems to me that what this comment is saying is that OIDs in the > > second and third categories are doled out by counters. Therefore, we > > can't know which of those OIDs will get used, or how many of them will > >

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 10:30 PM Amit Kapila wrote: > On Fri, Jan 21, 2022 at 10:00 PM David G. Johnston > wrote: > > > > On Fri, Jan 21, 2022 at 4:55 AM Amit Kapila > wrote: > >> > >> Apart from this, I have changed a few comments and ran pgindent. Do > >> let me know what you think of the cha

Re: Refactoring of compression options in pg_basebackup

2022-01-21 Thread Michael Paquier
On Fri, Jan 21, 2022 at 09:57:41AM -0500, Robert Haas wrote: > Thanks. One thing I just noticed is that the enum we're using here is > called WalCompressionMethod. But we're not compressing WAL. We're > compressing tarfiles of the data directory. Also, having this enum in walmethods.h is perhaps n

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 5:14 PM James Coleman wrote: > > > Really? That's horrid, because that's directly useful advice. > > Remedied, but rewritten a bit to better fit with the new style/goal of > that tip). > > Version 3 is attached. > > Coming back to this after a respite I think the tip need

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread Amit Kapila
On Fri, Jan 21, 2022 at 10:00 PM David G. Johnston wrote: > > On Fri, Jan 21, 2022 at 4:55 AM Amit Kapila wrote: >> >> Apart from this, I have changed a few comments and ran pgindent. Do >> let me know what you think of the changes? > > > The paragraph describing ALTER SUBSCRIPTION SKIP seems unn

Re: A test for replay of regression tests

2022-01-21 Thread Thomas Munro
On Sat, Jan 22, 2022 at 8:48 AM Andres Freund wrote: > Unfortunately we don't quite seem there yet: And another way to fail: pg_dump: error: query failed: ERROR: canceling statement due to conflict with recovery https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dangomushi&dt=2022-01-22%2

Re: row filtering for logical replication

2022-01-21 Thread Amit Kapila
On Fri, Jan 21, 2022 at 8:19 PM Alvaro Herrera wrote: > > If ATTACH PARTITION or CREATE TABLE .. PARTITION OF don't let you > specify replica identity, I suspect it's because both partitioning and > logical replication were developed in parallel, and neither gave too > much thought to the other. >

Re: Schema variables - new implementation for Postgres 15

2022-01-21 Thread Julien Rouhaud
Hi, On Fri, Jan 21, 2022 at 09:23:34PM +0100, Pavel Stehule wrote: > > st 19. 1. 2022 v 9:01 odesílatel Julien Rouhaud napsal: > > > > Also, the pg_dump handling emits a COLLATION option for session variables > > even > > for default collation, while it should only emit it if the collation is >

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Thomas Munro
On Sat, Jan 22, 2022 at 3:55 PM Robert Haas wrote: > On Fri, Jan 21, 2022 at 5:35 PM Andrew Dunstan wrote: > > # See https://www.msys2.org/wiki/Porting/#filesystem-namespaces > > local $ENV{MSYS2_ARG_CONV_EXCL} = $source_ts_prefix; > > Probably in this case just setting it to 'server:' wo

Re: How to get started with contribution

2022-01-21 Thread vrund shah
Respected Sir\Mam This year I am planning to take part in GSOC 2022 in the PostgreSQL organization. Regards Vrund V Shah On Sat, Jan 22, 2022 at 8:17 AM vrund shah wrote: > Respected Sir\Mam > > I am already using PostgreSQL for my college purpose and for learning SQL. > I have learned SQL fro

Re: Proposal: allow database-specific role memberships

2022-01-21 Thread Kenaniah Cerny
Thanks for the feedback. I have attached an alternate version of the v5 patch that incorporates the suggested changes to the documentation and DRYs up some of the acl.c code for comparison. As for the databaseOid / InvalidOid parameter, I'm open to any suggestions for how to make that even cleaner

Re: How to get started with contribution

2022-01-21 Thread vrund shah
Respected Sir\Mam I am already using PostgreSQL for my college purpose and for learning SQL. I have learned SQL from udemy courses with instructor Jose Portilla. and I am well aware of PostgreSQL and PGAdmin. Regards Vrund V Shah On Sat, Jan 22, 2022 at 8:12 AM vrund shah wrote: > Thank you f

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 5:35 PM Andrew Dunstan wrote: > # See https://www.msys2.org/wiki/Porting/#filesystem-namespaces > local $ENV{MSYS2_ARG_CONV_EXCL} = $source_ts_prefix; > Probably in this case just setting it to 'server:' would do the trick. Oh, thanks for the tip. Do you want to pu

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread Amit Kapila
On Fri, Jan 21, 2022 at 7:23 PM Peter Eisentraut wrote: > > On 21.01.22 04:08, Masahiko Sawada wrote: > >> I think the superuser check in AlterSubscription() might no longer be > >> appropriate. Subscriptions can now be owned by non-superusers. Please > >> check that. > > > > IIUC we don't allow

Re: How to get started with contribution

2022-01-21 Thread vrund shah
Thank you for your valuable guidance. I will surely look at the links and if have any queries then I will contact you. regards Vrund V Shah On Sat, Jan 22, 2022 at 2:23 AM Stephen Frost wrote: > Greetings, > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > > On 1/21/22 21:28, Stephen F

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread James Coleman
On Fri, Jan 21, 2022 at 5:38 PM Tom Lane wrote: > > "David G. Johnston" writes: > > You've removed the "constraint verification scan" portion of this. > > Indeed, because that's got nothing to do with adding a new column > (per se; adding a constraint along with the column is a different > can of

Re: do only critical work during single-user vacuum?

2022-01-21 Thread Bossart, Nathan
On 1/21/22, 2:43 PM, "John Naylor" wrote: > - to have a simple, easy to type, command AFAICT the disagreement is really just about the grammar. Sawada-san's idea would look something like VACUUM (FREEZE, INDEX_CLEANUP OFF, MIN_XID_AGE 16, MIN_MXID_AGE 16); while your pr

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread James Coleman
On Fri, Jan 21, 2022 at 4:08 PM Andrew Dunstan wrote: > > > On 1/21/22 13:55, James Coleman wrote: > > On Thu, Jan 20, 2022 at 3:43 PM James Coleman wrote: > >> As noted earlier I expect to be posting an updated patch soon. > > Here's the updated series. In 0001 I've moved the documentation tweak

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Andres Freund
Hi, On 2022-01-21 17:26:32 -0500, Robert Haas wrote: > I think the syntax has been accepted since pg_basebackup was added in 2011, > and Andres added it to this test case earlier this week (with -cfast in the > subject line of the commit message). The reason I used -cfast instead of -c fast or --

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 5:42 PM Tom Lane wrote: > The point I was trying to make is that if we have to jump through > that sort of hoop in the test scripts, then real users are going > to have to jump through it as well, and they won't like that > (and we will get bug reports about it). It'd be b

Re: Proposal: allow database-specific role memberships

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 3:12 PM Kenaniah Cerny wrote: > The latest rebased version of the patch is attached. > As I was just reminded, we tend to avoid specifying specific PostgreSQL versions in our documentation. We just say what the current version does. Here, the note sentences at lines 62 a

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Andres Freund
Hi, On 2022-01-21 17:42:45 -0500, Tom Lane wrote: > Andrew Dunstan writes: > > c.f. src/bin/pg_verifybackup/t/003_corruption.pl which says: > >     my $source_ts_prefix = $source_ts_path; > >     $source_ts_prefix =~ s!(^[A-Z]:/[^/]*)/.*!$1!; > >     ... > > >     # See https://www.msys2.org/wiki

Re: Synchronizing slots from primary to standby

2022-01-21 Thread Hsu, John
> I might be missing something but isn’t it okay even if the new primary > server is behind the subscribers? IOW, even if two slot's LSNs (i.e., > restart_lsn and confirm_flush_lsn) are behind the subscriber's remote > LSN (i.e., pg_replication_origin.remote_lsn), the primary sends only

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Andres Freund
Hi, On 2022-01-21 17:25:08 -0500, Tom Lane wrote: > Andres Freund writes: > > It seems quite workable to continue for INSTALL to be generated, but have > > the > > result checked in. The rate of changes to > > {installation,install-windows}.sgml > > isn't that high, and when things change, it's

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Tom Lane
Andrew Dunstan writes: > c.f. src/bin/pg_verifybackup/t/003_corruption.pl which says: >     my $source_ts_prefix = $source_ts_path; >     $source_ts_prefix =~ s!(^[A-Z]:/[^/]*)/.*!$1!; >     ... >     # See https://www.msys2.org/wiki/Porting/#filesystem-namespaces >     local $ENV{MSYS2_ARG_CONV_

Re: do only critical work during single-user vacuum?

2022-01-21 Thread John Naylor
On Wed, Jan 19, 2022 at 5:26 PM Michael Paquier wrote: > > Could you avoid introducing a new grammar pattern in VACUUM? Any new > option had better be within the parenthesized part as it is extensible > at will with its set of DefElems. This new behavior is not an option that one can sensibly mi

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread Tom Lane
"David G. Johnston" writes: > You've removed the "constraint verification scan" portion of this. Indeed, because that's got nothing to do with adding a new column (per se; adding a constraint along with the column is a different can of worms). > Re-reading this, the recommendation: > - Howe

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Tom Lane
Robert Haas writes: > On Fri, Jan 21, 2022 at 5:09 PM Tom Lane wrote: >> While we're on the subject of ill-chosen option syntax: "-cfast" >> with non double dashes? Really? That's horribly ambiguous. > I'm not sure whether you're complaining that we accept that syntax or > using it, but AFAIK

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Andrew Dunstan
On 1/21/22 17:10, Thomas Munro wrote: > On Sat, Jan 22, 2022 at 10:42 AM Robert Haas wrote: >> # Running: pg_basebackup --no-sync -cfast --target >> server:/home/pgrunner/bf/root/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/tmp_test_Ag8r/backuponserver >> -X none >> pg_basebackup: error: cou

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 2:50 PM Tom Lane wrote: > "David G. Johnston" writes: > > On Fri, Jan 21, 2022 at 2:08 PM Andrew Dunstan > wrote: > >> I know what it's replacing refers to release 11, but let's stop doing > >> that. How about something like this? > >> > >> Adding a new column can someti

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 5:09 PM Tom Lane wrote: > I think the backup_target string was already corrupted that way when > pg_basebackup absorbed it from optarg. It's pretty hard to believe that > the strchr/pnstrdup stanza got it wrong. However, comparing the > TARGET_DETAIL to what the TAP test

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Tom Lane
Andres Freund writes: > It seems quite workable to continue for INSTALL to be generated, but have the > result checked in. The rate of changes to {installation,install-windows}.sgml > isn't that high, and when things change, it's actually useful to be able to > see the current instructions from a

Re: Proposal: allow database-specific role memberships

2022-01-21 Thread Kenaniah Cerny
The latest rebased version of the patch is attached. On Tue, Jan 11, 2022 at 11:01 PM Julien Rouhaud wrote: > Hi, > > On Thu, Dec 2, 2021 at 2:26 AM Kenaniah Cerny wrote: > > > > Attached is a rebased version of the patch that omits catversion.h in > order to avoid conflicts. > > Unfortunately

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Andres Freund
Hi, On 2022-01-21 11:49:12 -0500, Robert Haas wrote: > On Fri, Jan 21, 2022 at 11:39 AM Tom Lane wrote: > > =?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: > > > Another solution would be to merge both README files together and make > > > separate section for development/git based codebase. > > > > The

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Thomas Munro
On Sat, Jan 22, 2022 at 10:42 AM Robert Haas wrote: > # Running: pg_basebackup --no-sync -cfast --target > server:/home/pgrunner/bf/root/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/tmp_test_Ag8r/backuponserver > -X none > pg_basebackup: error: could not initiate base backup: ERROR: > unrecogn

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Tom Lane
Robert Haas writes: > "server" is a valid backup target, but "server;C" is not. And I think > this must be a bug on the client side, because the server logs the > generated query: > 2022-01-21 20:53:11.618 UTC [8404:10] 010_pg_basebackup.pl LOG: > received replication command: BASE_BACKUP ( LABEL

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread Tom Lane
"David G. Johnston" writes: > On Fri, Jan 21, 2022 at 2:08 PM Andrew Dunstan wrote: >> I know what it's replacing refers to release 11, but let's stop doing >> that. How about something like this? >> >> Adding a new column can sometimes require rewriting the table, >> making it a very slow opera

fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
Thomas Munro pointed out this failure to me on fairywren: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2022-01-21%2020%3A10%3A22 He theorizes that I need some perl2host magic in there, which may well be true. But I also noticed this: # Running: pg_basebackup --no-sync -cf

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 2:08 PM Andrew Dunstan wrote: > On 1/21/22 13:55, James Coleman wrote: > > + Before PostgreSQL 11, adding a new > column to a > + table required rewriting that table, making it a very slow operation. > + More recent versions can sometimes optimize away this rew

Re: A test for replay of regression tests

2022-01-21 Thread Andrew Dunstan
On 1/21/22 13:58, Thomas Munro wrote: > On Fri, Jan 21, 2022 at 3:42 PM Thomas Munro wrote: >> Thanks. I added that and pushed. Let's see if fairywren likes it >> when it comes back online. > A watched pot never boils, but I wonder why Andrew's 4 Windows > configurations jacana, bowerbird, fai

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread Andrew Dunstan
On 1/21/22 13:55, James Coleman wrote: > On Thu, Jan 20, 2022 at 3:43 PM James Coleman wrote: >> As noted earlier I expect to be posting an updated patch soon. > Here's the updated series. In 0001 I've moved the documentation tweak > into the ALTER TABLE notes section. In 0002 I've taken David J

Re: Parallelize correlated subqueries that execute within each worker

2022-01-21 Thread Tom Lane
Robert Haas writes: > I don't think there's an intrinsic problem with the idea of making a > tentative determination about parallel safety and then refining it > later, but I'm not sure why you think it would be a lot of work to > figure this out at the point where we generate gather paths. I thin

Re: How to get started with contribution

2022-01-21 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 1/21/22 21:28, Stephen Frost wrote: > >* vrund v shah (vrund3...@gmail.com) wrote: > >>I am Vrund V Shah, a computer science undergrad. I have just completed > >> my > >>3^rd semester at G H Patel College of Engineering

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-21 Thread Peter Geoghegan
On Fri, Jan 21, 2022 at 12:07 PM Greg Stark wrote: > This confuses me. "Transactions per second" is a headline database > metric that lots of users actually focus on quite heavily -- rather > too heavily imho. But transactions per second is for the whole database, not for individual tables. It's

Re: How to get started with contribution

2022-01-21 Thread Tomas Vondra
On 1/21/22 21:28, Stephen Frost wrote: Greetings, * vrund v shah (vrund3...@gmail.com) wrote: I am Vrund V Shah, a computer science undergrad. I have just completed my 3^rd semester at G H Patel College of Engineering & Technology. I am new to open source contribution but I am well

Re: POC: GROUP BY optimization

2022-01-21 Thread Tomas Vondra
On 1/21/22 12:09, Andrey Lepikhov wrote: On 7/22/21 3:58 AM, Tomas Vondra wrote: 4) I'm not sure it's actually a good idea to pass tuplesPerPrevGroup to estimate_num_groups_incremental. In principle yes, if we use "group size" from the previous step, then the returned value is the number of ne

Re: How to get started with contribution

2022-01-21 Thread Stephen Frost
Greetings, * vrund v shah (vrund3...@gmail.com) wrote: >I am Vrund V Shah, a computer science undergrad. I have just completed my >3^rd semester at G H Patel College of Engineering & Technology. I am new >to open source contribution but I am well aware of C/C++, SQL and I will >lea

How to get started with contribution

2022-01-21 Thread vrund v shah
Respected sir\mamI am Vrund V Shah, a computer science undergrad. I have just completed my 3rd semester at G H Patel College of Engineering & Technology. I am new to open source contribution but I am well aware of C/C++, SQL and I will learn Python before the end of the first week of February. I wo

Re: Parallelize correlated subqueries that execute within each worker

2022-01-21 Thread Robert Haas
On Fri, Jan 14, 2022 at 2:25 PM James Coleman wrote: > I've been chewing on this a bit, and I was about to go re-read the > code and see how easy it'd be to do exactly what you're suggesting in > generate_gather_paths() (and verifying it doesn't need to happen in > other places). However there's o

Re: A test for replay of regression tests

2022-01-21 Thread Thomas Munro
On Sat, Jan 22, 2022 at 8:48 AM Andres Freund wrote: > Unfortunately we don't quite seem there yet: > > I saw a couple failures like: > https://api.cirrus-ci.com/v1/artifact/task/5394938773897216/regress_diffs/build/testrun/recovery/t/027_stream_regress/regression.diffs > (from https://cirrus-ci.c

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-21 Thread Greg Stark
On Thu, 20 Jan 2022 at 17:01, Peter Geoghegan wrote: > > Then there's the fact that you > really cannot think about the rate of XID consumption intuitively -- > it has at best a weak, unpredictable relationship with anything that > users can understand, such as data stored or wall clock time. Thi

pg_basebackup fsyncs some files despite --no-sync (was: Adding CI to our tree)

2022-01-21 Thread Andres Freund
Hi, On 2022-01-18 20:16:46 -0800, Andres Freund wrote: > I noticed a few other sources of "unnecessary" fsyncs. The most frequent > being the durable_rename() of backup_manifest in pg_basebackup.c. Manifests > are > surprisingly large, 135k for a freshly initdb'd cluster. Robert, I assume the f

Re: Push down time-related SQLValue functions to foreign server

2022-01-21 Thread Tom Lane
Alexander Pyhalov writes: > So far I have the following prototype. It seems to be working, but I > think it can be enhanced. > At least, some sort of caching seems to be necessary for > is_stable_expr(). Yeah, from a performance standpoint this seems pretty horrid --- it's probably exponential

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-01-21 Thread Andres Freund
On 2022-01-20 20:41:16 +, Bossart, Nathan wrote: > Here's this part. And pushed to all branches. Thanks.

Re: A test for replay of regression tests

2022-01-21 Thread Andres Freund
Hi, On 2022-01-17 17:25:19 +1300, Thomas Munro wrote: > I reordered the arguments, tested locally under the buildfarm client script, > and pushed. I'll keep an eye on the build farm. After the reloptions fix the tests seem much more likely to succeed than before. Progress! Unfortunately we don'

Re: refactoring basebackup.c

2022-01-21 Thread Robert Haas
On Thu, Jan 20, 2022 at 11:10 AM Robert Haas wrote: > On Thu, Jan 20, 2022 at 8:00 AM Dipesh Pandit wrote: > > Thanks for the feedback, I have incorporated the suggestions and > > updated a new patch v2. > > Cool. I'll do a detailed review later, but I think this is going in a > good direction.

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-21 Thread Tom Lane
I wrote: > The early returns are not great: we have about half a dozen machines > so far that are finding python3, and reporting sane-looking Python > include paths, but not finding Python.h. They're all Linux-oid > machines, so I suppose what is going on is that they have the base > python3 packa

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 11:55 AM James Coleman wrote: > On Thu, Jan 20, 2022 at 3:43 PM James Coleman wrote: > > > > As noted earlier I expect to be posting an updated patch soon. > > Here's the updated series. In 0001 I've moved the documentation tweak > into the ALTER TABLE notes section. In 0

Re: A test for replay of regression tests

2022-01-21 Thread Thomas Munro
On Fri, Jan 21, 2022 at 3:42 PM Thomas Munro wrote: > Thanks. I added that and pushed. Let's see if fairywren likes it > when it comes back online. A watched pot never boils, but I wonder why Andrew's 4 Windows configurations jacana, bowerbird, fairywren and drongo have stopped returning result

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Tom Lane
Robert Haas writes: > It seems to me that what this comment is saying is that OIDs in the > second and third categories are doled out by counters. Therefore, we > can't know which of those OIDs will get used, or how many of them will > get used, or which objects will get which OIDs. Therefore, I t

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread James Coleman
On Thu, Jan 20, 2022 at 3:43 PM James Coleman wrote: > > As noted earlier I expect to be posting an updated patch soon. Here's the updated series. In 0001 I've moved the documentation tweak into the ALTER TABLE notes section. In 0002 I've taken David J's suggestion of shortening the "Tip" on the

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 8:40 AM Shruthi Gowda wrote: > From what I see in the code, template0 and postgres are the last > things that get created in initdb phase. The system OIDs that get > assigned to these DBs vary from release to release. At present, the > system assigned OIDs of template0 and

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Tom Lane
Robert Haas writes: > On Fri, Jan 21, 2022 at 12:19 PM Tom Lane wrote: >> I'm not convinced by this argument. In the first place, the INSTALL >> file isn't doing any harm. I don't know that I'd bother to build the >> infrastructure for it today, but we already have that infrastructure >> and it

Re: refactoring basebackup.c

2022-01-21 Thread Robert Haas
On Wed, Jan 19, 2022 at 4:26 PM Robert Haas wrote: > I spent some time thinking about test coverage for the server-side > backup code today and came up with the attached (v12-0003). I committed the base backup target patch yesterday, and today I updated the remaining code in light of Michael Paqu

Re: document the need to analyze partitioned tables

2022-01-21 Thread Tomas Vondra
On 1/21/22 19:02, Justin Pryzby wrote: Thanks for looking at this On Fri, Jan 21, 2022 at 06:21:57PM +0100, Tomas Vondra wrote: Hi, On 10/8/21 14:58, Justin Pryzby wrote: Cleaned up and attached as a .patch. The patch implementing autoanalyze on partitioned tables should revert relevant port

Re: document the need to analyze partitioned tables

2022-01-21 Thread Justin Pryzby
Thanks for looking at this On Fri, Jan 21, 2022 at 06:21:57PM +0100, Tomas Vondra wrote: > Hi, > > On 10/8/21 14:58, Justin Pryzby wrote: > > Cleaned up and attached as a .patch. > > > > The patch implementing autoanalyze on partitioned tables should > > revert relevant portions of this patch. >

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 12:19 PM Tom Lane wrote: > I'm not convinced by this argument. In the first place, the INSTALL > file isn't doing any harm. I don't know that I'd bother to build the > infrastructure for it today, but we already have that infrastructure > and it's not causing us any parti

Re: document the need to analyze partitioned tables

2022-01-21 Thread Tomas Vondra
Hi, On 10/8/21 14:58, Justin Pryzby wrote: Cleaned up and attached as a .patch. The patch implementing autoanalyze on partitioned tables should revert relevant portions of this patch. I went through this patch and I'd like to propose a couple changes, per the 0002 patch: 1) I've reworded t

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Tom Lane
Robert Haas writes: > On Fri, Jan 21, 2022 at 11:39 AM Tom Lane wrote: >> =?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: >>> Another solution would be to merge both README files together and make >>> separate section for development/git based codebase. >> There's a lot to be said for that approach: m

Re: Logical replication timeout problem

2022-01-21 Thread Fabrice Chapuis
I keep your patch 0001 and I add these two calls in function WalSndUpdateProgress without modifying WalSndKeepaliveIfNecessary, it works too. What do your think of this patch? static void WalSndUpdateProgress(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid) { static Timestam

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 11:39 AM Tom Lane wrote: > =?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: > > Another solution would be to merge both README files together and make > > separate section for development/git based codebase. > > There's a lot to be said for that approach: make it simpler, not > mo

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Tom Lane
=?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: > Another solution would be to merge both README files together and make > separate section for development/git based codebase. There's a lot to be said for that approach: make it simpler, not more complicated. regards, tom lane

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 4:55 AM Amit Kapila wrote: > Apart from this, I have changed a few comments and ran pgindent. Do > let me know what you think of the changes? > The paragraph describing ALTER SUBSCRIPTION SKIP seems unnecessarily repetitive. Consider: """ Skips applying all changes of th

Re: ICU for global collation

2022-01-21 Thread Julien Rouhaud
On Fri, Jan 21, 2022 at 03:24:02PM +0100, Peter Eisentraut wrote: > On 21.01.22 14:51, Julien Rouhaud wrote: > > Is that change intended? There isn't any usage of the collversionstr before > > the possible error when actual_versionstr is missing. > > I wanted to move it closer to the SysCacheGetA

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Josef Šimánek
pá 21. 1. 2022 v 16:31 odesílatel Tom Lane napsal: > > =?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: > > There is README.git explaining this. README itself is meant to be used > > for distributed source code. You can generate INSTALL locally for > > example by running make dist (INSTALL will be presen

Re: support for MERGE

2022-01-21 Thread Alvaro Herrera
On 2022-Jan-21, Japin Li wrote: > + /* > +* NOT MATCHED actions can't see target relation, but they > can see > +* source relation. > +*/ > + Assert(mergeWhenClause->commandType == CMD_INSERT || > +

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Tom Lane
=?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: > There is README.git explaining this. README itself is meant to be used > for distributed source code. You can generate INSTALL locally for > example by running make dist (INSTALL will be present in > postgresql-15devel directory). > Anyway I do agree thi

Re: Extend compatibility of PostgreSQL::Test::Cluster

2022-01-21 Thread Andrew Dunstan
On 1/21/22 02:47, Michael Paquier wrote: > On Tue, Jan 18, 2022 at 06:35:39PM -0500, Andrew Dunstan wrote: >> Here's a version that does that and removes some recent bitrot. > I have been looking at the full set of features of Cluster.pm and the > requirements behind v10 as minimal version suppor

Re: Refactoring of compression options in pg_basebackup

2022-01-21 Thread Robert Haas
On Thu, Jan 20, 2022 at 9:18 PM Michael Paquier wrote: > On Thu, Jan 20, 2022 at 10:25:43AM -0500, Robert Haas wrote: > > You don't need to test for gzip and none in two places each. Just make > > the block with the "It does not match ..." comment the "else" clause > > for this last part. > > Inde

Re: row filtering for logical replication

2022-01-21 Thread Alvaro Herrera
On 2022-Jan-21, houzj.f...@fujitsu.com wrote: > Personally, I'm a little hesitant to put the check at DDL level, because > adding check at DDLs like ATTACH PARTITION/CREATE PARTITION OF ( [1] > explained why we need to check these DDLs) looks a bit restrictive and > user might also complain about

Re: ICU for global collation

2022-01-21 Thread Peter Eisentraut
On 21.01.22 14:51, Julien Rouhaud wrote: From 1c46bf3138ad42074971aa3130142236de7e63f7 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Fri, 21 Jan 2022 10:01:25 +0100 Subject: [PATCH] Change collate and ctype fields to type text + collversionstr = TextDatumGetCStri

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread Peter Eisentraut
On 21.01.22 04:08, Masahiko Sawada wrote: I think the superuser check in AlterSubscription() might no longer be appropriate. Subscriptions can now be owned by non-superusers. Please check that. IIUC we don't allow non-superuser to own the subscription yet. We still have the following superuse

Re: ICU for global collation

2022-01-21 Thread Julien Rouhaud
Hi, On Fri, Jan 21, 2022 at 10:44:24AM +0100, Peter Eisentraut wrote: > > Here is a second preparation patch: Change collate and ctype fields to type > text. > > I think this is useful by itself because it allows longer locale names. ICU > locale names with several options appended can be longer

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Shruthi Gowda
On Fri, Jan 21, 2022 at 1:08 AM Robert Haas wrote: > > On Thu, Jan 20, 2022 at 11:03 AM Shruthi Gowda wrote: > > It is not required for PostgresObjectId. The unused_oids script > > provides a list of unused oids in the manually-assignable OIDs range > > (1-). > > Well, so ... why are we not t

Re: Logical replication timeout problem

2022-01-21 Thread Fabrice Chapuis
Thanks for your patch, it also works well when executing our use case, the timeout no longer appears in the logs. Is it necessary now to refine this patch and make as few changes as possible in order for it to be released? On Fri, Jan 21, 2022 at 10:51 AM wangw.f...@fujitsu.com < wangw.f...@fujits

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread Amit Kapila
On Fri, Jan 21, 2022 at 5:25 PM Amit Kapila wrote: > > On Fri, Jan 21, 2022 at 10:10 AM Masahiko Sawada > wrote: > > > > Few things that I think we can improve in 028_skip_xact.pl are as follows: > > After CREATE SUBSCRIPTION, wait for initial sync to be over and > two_phase state to be enabled.

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread Amit Kapila
On Fri, Jan 21, 2022 at 10:10 AM Masahiko Sawada wrote: > > On Fri, Jan 21, 2022 at 1:20 PM Amit Kapila wrote: > > > > What do we want to indicate by [, ... ]? To me, it appears like > > multiple options but that is not what we support currently. > > You're right. It's an oversight. > I have fix

Re: POC: GROUP BY optimization

2022-01-21 Thread Andrey Lepikhov
On 7/22/21 3:58 AM, Tomas Vondra wrote: 4) I'm not sure it's actually a good idea to pass tuplesPerPrevGroup to estimate_num_groups_incremental. In principle yes, if we use "group size" from the previous step, then the returned value is the number of new groups after adding the "new" pathkey. But

Re: row filtering for logical replication

2022-01-21 Thread Dilip Kumar
On Thu, Jan 20, 2022 at 4:54 PM Amit Kapila wrote: > + /* > + * Unchanged toasted replica identity columns are only detoasted in the > + * old tuple, copy this over to the new tuple. > + */ > + if (att->attlen == -1 && > + VARATT_IS_EXTERNAL_ONDISK(tmp_new_slot->tts_values[i]) && > + !VARATT_IS_E

RE: Logical replication timeout problem

2022-01-21 Thread wangw.f...@fujitsu.com
On Thu, Jan 20, 2022 at 9:18 PM Amit Kapila wrote: > It might be not reaching the actual send_keep_alive logic in > WalSndKeepaliveIfNecessary because of below code: > { > ... > /* > * Don't send keepalive messages if timeouts are globally disabled or > * we're doing something not partaking in tim

Re: ICU for global collation

2022-01-21 Thread Peter Eisentraut
On 18.01.22 13:54, Peter Eisentraut wrote: On 18.01.22 05:02, Julien Rouhaud wrote: If this is applied, then in my estimation all these hunks will completely disappear from the global ICU patch.  So this would be a significant win. Agreed, the patch will be quite smaller and also easier to revi

RE: row filtering for logical replication

2022-01-21 Thread houzj.f...@fujitsu.com
On Thur, Jan 20, 2022 7:25 PM Amit Kapila wrote: > > On Thu, Jan 20, 2022 at 6:42 AM houzj.f...@fujitsu.com > wrote: > > > > Attach the V68 patch set which addressed the above comments and changes. > > > > Few comments and suggestions: > == > 1. > /* > + * For updates, i

Re: row filtering for logical replication

2022-01-21 Thread Greg Nancarrow
On Thu, Jan 20, 2022 at 12:12 PM houzj.f...@fujitsu.com wrote: > > Attach the V68 patch set which addressed the above comments and changes. > The version patch also fix the error message mentioned by Greg[1] > Some review comments for the v68 patch, mostly nitpicking: (1) Commit message Minor su

Re: [PATCH] Implement INSERT SET syntax

2022-01-21 Thread wenjing zeng
Since this feature adds INSERT OVERRIDING SET syntax, it is recommended to add some related testcases. Regards Wenjing > 2021年9月22日 07:38,Rachel Heaton 写道: > >> On 4/23/20 8:04 PM, Gareth Palmer wrote: >>> >>> Thank you for the review, attached is v7 of the patch which should >>> apply co

Re: postgres_fdw: incomplete subabort cleanup of connections used in async execution

2022-01-21 Thread Etsuro Fujita
On Wed, Dec 22, 2021 at 2:40 PM Etsuro Fujita wrote: > On Wed, Dec 22, 2021 at 1:08 AM Alexander Pyhalov > wrote: > > Looks good to me. > > Great! Thanks for reviewing! I've committed the patch. Best regards, Etsuro Fujita

Re: New developer papercut - Makefile references INSTALL

2022-01-21 Thread Josef Šimánek
pá 21. 1. 2022 v 1:29 odesílatel samay sharma napsal: > > > > On Wed, Jan 19, 2022 at 4:58 PM Tim McNamara wrote: >> >> Hello, >> >> I encountered a minor road bump when checking out the pg source today. The >> Makefile's all target includes the following help message if GNUmakefile >> isn't av