Re: .ready and .done files considered harmful

Hi, > 1. I've removed several calls to PgArchForceDirScan() in favor of > calling it at the top of pgarch_ArchiverCopyLoop(). I believe > there is some disagreement about this change, but I don't think > we gain enough to justify the complexity. The main reason we > exit pgarch_A

Re: psql: \dl+ to list large objects privileges

Hi, Thank you for testing. As far as I understand, for the patch to move forward, someone has to become a reviewer and change the status in the commitfest app. Pavel Luzanov Postgres Professional: https://postgrespro.com The Russian Postgres Company On 18.09.2021 05:41, Neil Chen wrote: The f

RE: Improve logging when using Huge Pages

Hi, Thank you for your comment. > I was afraid that logging the message like "could not ..." every time when > the server starts up would surprise users unnecessarily. > Because the message sounds like it reports a server error. Fujii-san, I was worried about the same thing as you. So the atta

Re: WIP: System Versioned Temporal Table

On Mon, Sep 20, 2021 at 7:09 AM Corey Huinker wrote: > > On Sun, Sep 19, 2021 at 3:12 PM Hannu Krosing wrote: >> >> A side table has the nice additional benefit that we can very easily >> version the *table structure* so when we ALTER TABLE and the table >> structure changes we just make a new si

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

Michail Nikolaev wrote: > Hello. > > Added a check for standby promotion with the long transaction to the > test (code and docs are unchanged). I'm trying to continue the review, sorry for the delay. Following are a few question about the code: * Does the masking need to happen in the AM code,

Re: Added schema level support for publication.

On Fri, Sep 17, 2021 at 5:39 PM vignesh C wrote: > > On Wed, Sep 15, 2021 at 12:30 PM Amit Kapila wrote: > > > > > 4. > > AlterPublicationSchemas() > > { > > .. > > + /* > > + * If the table option was not specified remove the existing tables > > + * from the publication. > > + */ > > + if (!tabl

Re: Added schema level support for publication.

On Fri, Sep 17, 2021 at 5:40 PM vignesh C wrote: > > On Thu, Sep 16, 2021 at 9:54 AM Amit Kapila wrote: > > > > I think there is one more similar locking problem. > > AlterPublicationSchemas() > > { > > .. > > + if (stmt->action == DEFELEM_ADD) > > + { > > + List *rels; > > + > > + rels = GetPubl

Coding guidelines for braces + spaces - link 404's

Hi, I have been working on a patch for Postgres. I'm curious about the suggested style for braces around if statements - some places don't include braces around an if statement body, if the if statement body is a single line. The "Coding Conventions" don't contain any advice here (although maybe t

Re: Logical replication timeout problem

Hi Amit, We can replay the problem: we load a table of several Gb in the schema of the publisher, this generates the worker's timeout after one minute from the end of this load. The table on which this load is executed is not replicated. 2021-09-16 12:06:50 CEST [24881]: [1-1] user=postgres,db=db

Re: Added schema level support for publication.

On Mon, Sep 20, 2021 at 3:57 PM Amit Kapila wrote: > > On Fri, Sep 17, 2021 at 5:40 PM vignesh C wrote: > > > > On Thu, Sep 16, 2021 at 9:54 AM Amit Kapila wrote: > > > > > > I think there is one more similar locking problem. > > > AlterPublicationSchemas() > > > { > > > .. > > > + if (stmt->act

Re: mark the timestamptz variant of date_bin() as stable

> On Wed, Sep 1, 2021 at 3:25 PM Tom Lane wrote: > > > > John Naylor writes: > > > On Wed, Sep 1, 2021 at 2:44 PM Tom Lane wrote: > > >> I see that these two answers are both exactly multiples of 24 hours away > > >> from the given origin. But if I'm binning on the basis of "days" or > > >> lar

Re: Coding guidelines for braces + spaces - link 404's

On 20.09.21 05:37, Kevin Burke wrote: I have been working on a patch for Postgres. I'm curious about the suggested style for braces around if statements - some places don't include braces around an if statement body, if the if statement body is a single line. Generally, the braces should be o

Re: Logical replication timeout problem

On Mon, Sep 20, 2021 at 4:10 PM Fabrice Chapuis wrote: > > Hi Amit, > > We can replay the problem: we load a table of several Gb in the schema of the > publisher, this generates the worker's timeout after one minute from the end > of this load. The table on which this load is executed is not rep

Re: row filtering for logical replication

On Mon, Sep 20, 2021 at 3:17 PM Ajin Cherian wrote: > > On Wed, Sep 8, 2021 at 7:59 PM Ajin Cherian wrote: > > > > On Wed, Sep 1, 2021 at 9:23 PM Euler Taveira wrote: > > > > > > Somehow this approach of either new_tuple or old_tuple doesn't seem to > > be very fruitful if the user requires that

Re: proposal: possibility to read dumped table's name from file

> Will do a closer review on the patch shortly. Had a read through, and tested, the latest posted version today: +Read objects filters from the specified file. Specify "-" to read from +stdin. Lines of this file must have the following format: I think this should be - and STDIN to match t

Re: Timeout failure in 019_replslot_limit.pl

On Sat, Sep 18, 2021 at 05:19:04PM -0300, Alvaro Herrera wrote: > Hmm, sounds a possibly useful idea to explore, but I would only do so if > the other ideas prove fruitless, because it sounds like it'd have more > moving parts. Can you please first test if the idea of sending the signal > twice is

Re: Logical replication timeout problem

On Mon, Sep 20, 2021 at 5:21 PM Amit Kapila wrote: > > On Mon, Sep 20, 2021 at 4:10 PM Fabrice Chapuis > wrote: > > > > Hi Amit, > > > > We can replay the problem: we load a table of several Gb in the schema of > > the publisher, this generates the worker's timeout after one minute from > > th

Re: Timeout failure in 019_replslot_limit.pl

On 2021-Sep-20, Michael Paquier wrote: > > Can you please first test if the idea of sending the signal twice is > > enough? > > This idea does not work. I got one failure after 5 tries. OK, thanks for taking the time to test it. > > If that doesn't work, let's try Horiguchi-san's idea of using

psql: tab completion differs on semicolon placement

While testing a patch I fat-fingered a CREATE DATABASE statement by tab completing *after* the semicolon, with no space between the objname and semicolon. The below options were presented, which at this point aren't really applicable: db=# create database foo; ALLOW_CONNECTIONS ENCODING

Re: Timeout failure in 019_replslot_limit.pl

On Mon, Sep 20, 2021 at 09:38:29AM -0300, Alvaro Herrera wrote: > On 2021-Sep-20, Michael Paquier wrote: >>> If that doesn't work, let's try Horiguchi-san's idea of using some >>> `ps` flags to find the process. >> >> Tried this one as well, to see the same failure. > > Hmm, do you mean that you

Re: row filtering for logical replication

On Mon, Sep 20, 2021 at 5:37 PM Amit Kapila wrote: > > > > > > > Adding a patch that strives to do the logic that I described above. > > For updates, the row filter is applied on both old_tuple > > and new_tuple. This patch assumes that the row filter only uses > > columns that are part of the REP

Re: Coding guidelines for braces + spaces - link 404's

Kevin Burke writes: > The Postgres Wiki has a bit that says to "See also the Formatting section > in the > documentation," but that link 404's, so I'm not sure where it is supposed > to go. Obsolete link, evidently. It should p

Re: [Patch] ALTER SYSTEM READ ONLY

On Wed, Sep 15, 2021 at 9:38 PM Robert Haas wrote: > > On Wed, Sep 15, 2021 at 10:32 AM Robert Haas wrote: > > Putting these changes into 0001 seems to make no sense. It seems like > > they should be part of 0003, or maybe a new 0004 patch. > > After looking at this a little bit more, I think it'

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

On 2021-Sep-13, Alvaro Herrera wrote: > Thanks Bharath and Justin -- I think I took all the suggestions and made > a few other changes of my own. Here's the result. Pushed this with very minor additional changes, thanks. -- Álvaro Herrera PostgreSQL Developer — https://www.Enterprise

Re: So, about that cast-to-typmod-minus-one business

Dean Rasheed writes: > On Sat, 18 Sep 2021, 18:06 Tom Lane, wrote: >> I'm inclined to back-patch > +1 Done. regards, tom lane

Re: Logical replication timeout problem

By passing the autovacuum parameter to off the problem did not occur right after loading the table as in our previous tests. However, the timeout occurred later. We have seen the accumulation of .snap files for several Gb. ... -rw---. 1 postgres postgres 16791226 Sep 20 15:26 xid-1238444701-ls

logical replication restrictions

One thing is needed and is not solved yet is delayed replication on logical replication. Would be interesting to document it on Restrictions page, right ? regards, Marcos

Re: psql: tab completion differs on semicolon placement

Daniel Gustafsson writes: > While testing a patch I fat-fingered a CREATE DATABASE statement by tab > completing *after* the semicolon, with no space between the objname and > semicolon. The below options were presented, which at this point aren't > really > applicable: > > db=# create database

Re: postgres.h included from relcache.h - but removing it breaks pg_upgrade

On 2021-09-19 18:45:45 +0300, Alexander Korotkov wrote: > Any objections to pushing this? lgtm I assume you're planning to backpatch this?

Re: .ready and .done files considered harmful

On Thu, Sep 16, 2021 at 7:26 PM Bossart, Nathan wrote: > 1. I've removed several calls to PgArchForceDirScan() in favor of > calling it at the top of pgarch_ArchiverCopyLoop(). I believe > there is some disagreement about this change, but I don't think > we gain enough to justify

Re: .ready and .done files considered harmful

On 2021-Sep-20, Robert Haas wrote: > I was thinking that this might increase the number of directory scans > by a pretty large amount when we repeatedly catch up, then 1 new file > gets added, then we catch up, etc. I was going to say that perhaps we can avoid repeated scans by having a bitmap of

Re: psql: tab completion differs on semicolon placement

> On 20 Sep 2021, at 21:26, Dagfinn Ilmari Mannsåker wrote: > IMO a fix should be more principled than just special-casing semicolon > and CREATE TABLE. Maybe get_previous_words() should stop when it sees > an unquoted semicolon? Agreed, something along those lines makes sense. I will familiar

Re: Parallel Full Hash Join

On Fri, Jul 30, 2021 at 04:34:34PM -0400, Melanie Plageman wrote: > On Sat, Jul 10, 2021 at 9:13 AM vignesh C wrote: > > > > On Mon, May 31, 2021 at 10:47 AM Greg Nancarrow wrote: > > > > > > On Sat, Mar 6, 2021 at 12:31 PM Thomas Munro > > > wrote: > > > > > > > > On Tue, Mar 2, 2021 at 11:27

Re: psql: tab completion differs on semicolon placement

On Mon, Sep 20, 2021 at 08:26:51PM +0100, Dagfinn Ilmari Mannsåker wrote: > Daniel Gustafsson writes: > > > While testing a patch I fat-fingered a CREATE DATABASE statement by tab > > completing *after* the semicolon, with no space between the objname and > > semicolon. The below options were pr

Re: postgres.h included from relcache.h - but removing it breaks pg_upgrade

On Mon, Sep 20, 2021 at 10:48 PM Andres Freund wrote: > On 2021-09-19 18:45:45 +0300, Alexander Korotkov wrote: > > Any objections to pushing this? > > lgtm Thanks! > I assume you're planning to backpatch this? Yes. -- Regards, Alexander Korotkov

Re: postgres.h included from relcache.h - but removing it breaks pg_upgrade

Alexander Korotkov writes: > On Mon, Sep 20, 2021 at 10:48 PM Andres Freund wrote: >> I assume you're planning to backpatch this? > Yes. Probably good to wait 24 hours until 14rc1 has been tagged. regards, tom lane

Re: Deduplicate code updating ControleFile's DBState.

On 9/19/21, 11:07 PM, "Amul Sul" wrote: > +1, since skipping ControlFileLock for the DBState update is not the > right thing, let's have two different functions as per your suggestion > -- did the same in the attached version, thanks. I see that the attached patch reorders the call to UpdateContr

Re: Small documentation improvement for ALTER SUBSCRIPTION

On Wed, Sep 15, 2021 at 4:58 PM Daniel Gustafsson wrote: > > > On 14 Sep 2021, at 14:35, Daniel Gustafsson wrote: > > > >> On 14 Sep 2021, at 11:57, Amit Kapila wrote: > > > >> LGTM as well. Peter E., Daniel, does any one of you is intending to > >> push this? If not, I can take care of this. >

Re: Support for NSS as a libpq TLS backend

On Mon, 2021-07-26 at 15:26 +0200, Daniel Gustafsson wrote: > > On 19 Jul 2021, at 21:33, Jacob Champion wrote: > > ..client connections will crash if > > hostaddr is provided rather than host, because SSL_SetURL can't handle > > a NULL argument. I'm running with 0002 to fix it for the moment, but

Re: Estimating HugePages Requirements?

Should we also initialize the shared memory GUCs in bootstrap and single-user mode? I think I missed this in bd17880. Nathan diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 48615c0ebc..4c4cf44871 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/b

Re: postgres.h included from relcache.h - but removing it breaks pg_upgrade

On Tue, Sep 21, 2021 at 2:07 AM Tom Lane wrote: > Alexander Korotkov writes: > > On Mon, Sep 20, 2021 at 10:48 PM Andres Freund wrote: > >> I assume you're planning to backpatch this? > > > Yes. > > Probably good to wait 24 hours until 14rc1 has been tagged. OK, NP! -- Regards, Alexander K

Re: mem context is not reset between extended stats

On 9/15/21 10:09 PM, Justin Pryzby wrote: Memory allocation appeared be O(1) WRT the number of statistics objects, which was not expected to me. This is true in v13 (and probably back to v10). It seems to work fine to reset the memory context within the loop, so long as the statslist is alloca

PostgreSQL High Precision Support Extension.

I have been trying to get a reply or interest in either updating PostgreSQL to support the following, or for there to be a public, free for any use Extension put out there, that will support the following. Can someone able and interested please respond to me about the following project specifica

Re: mem context is not reset between extended stats

On Tue, Sep 21, 2021 at 02:15:45AM +0200, Tomas Vondra wrote: > On 9/15/21 10:09 PM, Justin Pryzby wrote: > > Memory allocation appeared be O(1) WRT the number of statistics objects, > > which > > was not expected to me. This is true in v13 (and probably back to v10). Of course I meant to say th

Re: Estimating HugePages Requirements?

On Fri, Sep 17, 2021 at 04:31:44PM +, Bossart, Nathan wrote: > Done. Thanks. I have gone through the last patch this morning, did some tests on all the platforms I have at hand (including Linux) and finished by applying it after doing some small tweaks. First, I have finished by extending G

PostgreSQL 14 press release draft

Hi, Attached is a copy of the first draft for the PG14 press release. This brings together highlights of many of the features in the upcoming PostgreSQL 14 release while providing context on their significance. With the plethora of new features coming in PostgreSQL 14, it is challenging to highli

Re: Release 14 Schedule

On 9/20/21 2:33 AM, Nikolay Samokhvalov wrote: > Observability-related improvements are also good and very important for > the future of DBA operations -- compute_query_id, new pg_stat_**, etc. > > Things like new knob idle_session_timeout and restore_command change not > requiring a restart will

Re: Estimating HugePages Requirements?

On Tue, Sep 21, 2021 at 12:08:22AM +, Bossart, Nathan wrote: > Should we also initialize the shared memory GUCs in bootstrap and > single-user mode? I think I missed this in bd17880. Why would we need that for the bootstrap mode? While looking at the patch for shared_memory_size, I have look

Re: logical replication restrictions

On Mon, Sep 20, 2021 at 9:47 PM Marcos Pegoraro wrote: > > One thing is needed and is not solved yet is delayed replication on logical > replication. Would be interesting to document it on Restrictions page, right ? > What do you mean by delayed replication? Is it that by default we send the tra

Re: PostgreSQL High Precision Support Extension.

On Tue, Sep 21, 2021 at 1:30 PM A Z wrote: > -A library like GMP, written in C, is an appropriate basis to start from and > to include, for all OS platforms involved. Are you aware of Daniele Varrazzo's extension https://github.com/dvarrazzo/pgmp/ ? (Never looked into it myself, but this seems

Re: row filtering for logical replication

On Tue, Sep 21, 2021 at 12:03 AM Dilip Kumar wrote: > > On Mon, Sep 20, 2021 at 5:37 PM Amit Kapila wrote: > > > > > > > > > > Adding a patch that strives to do the logic that I described above. > > > For updates, the row filter is applied on both old_tuple > > > and new_tuple. This patch assumes

Re: Added schema level support for publication.

On Fri, Sep 17, 2021 at 10:09 PM vignesh C wrote: > > Attached v29 patch has the fixes for the same. > Some minor comments on the v29-0002 patch: (1) In get_object_address_publication_schema(), the error message: + errmsg("publication tables of schema \"%s\" in publication \"%s\" does not exist

Re: PostgreSQL 14 press release draft

On Mon, Sep 20, 2021 at 10:19:32PM -0400, Jonathan S. Katz wrote: > PostgreSQL 14 provides a significant throughput boost on workloads that use > many > connections, with some benchmarks showing a 2x speedup. This release continues > on the recent improvements the overall management of B-tree ind

Re: row filtering for logical replication

On Tue, Sep 21, 2021 at 8:58 AM Ajin Cherian wrote: > > I understand why this is done, but I have 2 concerns here 1) We are > > having extra deform and copying the field from new to old in case it > > is unchanged replica identity. 2) The same unchanged attribute values > > get qualified in the o

Re: Skipping logical replication transactions on subscriber side

Sorry for the late reply. I was on vacation. On Tue, Sep 14, 2021 at 11:27 AM houzj.f...@fujitsu.com wrote: > > From Thur, Sep 9, 2021 10:33 PM Masahiko Sawada wrote: > > Sorry for the late response. I've attached the updated patches that > > incorporate > > all comments unless I missed somethi

Re: Deduplicate code updating ControleFile's DBState.

On Tue, Sep 21, 2021 at 4:44 AM Bossart, Nathan wrote: > > On 9/19/21, 11:07 PM, "Amul Sul" wrote: > > +1, since skipping ControlFileLock for the DBState update is not the > > right thing, let's have two different functions as per your suggestion > > -- did the same in the attached version, thank

Re: row filtering for logical replication

On Tue, Sep 21, 2021 at 9:54 AM Dilip Kumar wrote: > > On Tue, Sep 21, 2021 at 8:58 AM Ajin Cherian wrote: > > > I understand why this is done, but I have 2 concerns here 1) We are > > > having extra deform and copying the field from new to old in case it > > > is unchanged replica identity. 2)

Re: relation OID in ReorderBufferToastReplace error message

On Fri, Sep 17, 2021 at 10:53 AM Amit Kapila wrote: > > I don't think it is a bad idea to print additional information as you > are suggesting but why only for this error? It could be useful to > investigate any other error we get during decoding. I think normally > we add such additional informat

Re: row filtering for logical replication

On Tue, Sep 21, 2021 at 10:41 AM Amit Kapila wrote: > > If you have only a and c in the old tuple, how will it evaluate > expression c + d? Well, what I told is that if we have such dependency then we will have to copy that field to the old tuple, e.g. if we convert the filter for the old tuple

RE: Allow escape in application_name

Dear Fujii-san, Horiguchi-san, Based on your advice, I made a patch that communize two parsing functions into one. new internal function parse_format_string() was added. (This name may be too generic...) log_line_prefix() and parse_pgfdw_appname() become just the wrapper function. My prerpimary ch

Re: Added schema level support for publication.

On Tue, Sep 21, 2021 at 9:03 AM Greg Nancarrow wrote: > > On Fri, Sep 17, 2021 at 10:09 PM vignesh C wrote: > > > > Attached v29 patch has the fixes for the same. > > > > Some minor comments on the v29-0002 patch: > > (1) > In get_object_address_publication_schema(), the error message: > > + errm

Re: Logical replication timeout problem

On Mon, Sep 20, 2021 at 9:43 PM Fabrice Chapuis wrote: > > By passing the autovacuum parameter to off the problem did not occur right > after loading the table as in our previous tests. However, the timeout > occurred later. We have seen the accumulation of .snap files for several Gb. > > ... >

Re: Release 14 Schedule

On 21/09/21 14:23, Jonathan S. Katz wrote: On 9/20/21 2:33 AM, Nikolay Samokhvalov wrote: Observability-related improvements are also good and very important for the future of DBA operations -- compute_query_id, new pg_stat_**, etc. Things like new knob idle_session_timeout and restore_comman

Re: proposal: possibility to read dumped table's name from file

po 20. 9. 2021 v 14:10 odesílatel Daniel Gustafsson napsal: > > Will do a closer review on the patch shortly. > > Had a read through, and tested, the latest posted version today: > > +Read objects filters from the specified file. Specify "-" to read from > +stdin. Lines of this file must