Re: Accommodate startup process in a separate ProcState array slot instead of in MaxBackends slots.

2022-03-25 Thread Bharath Rupireddy
On Sat, Mar 26, 2022 at 1:20 AM Robert Haas wrote: > > On Sat, Feb 12, 2022 at 6:26 AM Bharath Rupireddy > wrote: > > FWIW, here's a patch just adding a comment on how the startup process > > can get a free procState array slot even when SInvalShmemSize hasn't > > accounted for it. > > I don't th

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-03-25 Thread Bharath Rupireddy
On Wed, Mar 2, 2022 at 11:41 AM Bharath Rupireddy wrote: > > Please review the v2 patch set. Had to rebase. Attaching v3 patch set. Regards, Bharath Rupireddy. v3-0001-Use-WAL-segment-instead-of-log-segment.patch Description: Binary data v3-0002-Replace-log-record-with-WAL-record-in-docs.pat

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-03-25 Thread Michael Paquier
On Thu, Mar 24, 2022 at 05:44:06PM +, Jacob Champion wrote: > On Wed, 2022-03-23 at 16:54 -0700, Andres Freund wrote: >> Another option would be to make it a GUC. With a bit of care it could be >> automatically synced by the existing parallelism infrastructure... > > Like a write-once, PGC_INT

Re: Add LZ4 compression in pg_dump

2022-03-25 Thread Justin Pryzby
On Sat, Mar 26, 2022 at 02:57:50PM +0900, Michael Paquier wrote: > I have a question about 0002, actually. What has led you to the > conclusion that this code is dead and could be removed? See 0001 and the manpage. + 'pg_dump: compression is not supported by tar archive format');

Re: pg_relation_size on partitioned table

2022-03-25 Thread Michael Paquier
On Fri, Mar 25, 2022 at 08:52:40PM +0800, Japin Li wrote: > Could we provide a function to get the total size of the partition table > though the partitioned table name? Maybe we can extend > the pg_relation_size() to get the total size of partition tables through > the partitioned table name. Th

Re: Corruption during WAL replay

2022-03-25 Thread Michael Paquier
On Fri, Mar 25, 2022 at 10:34:49AM -0400, Tom Lane wrote: > Robert Haas writes: >> On Fri, Mar 25, 2022 at 10:02 AM Tom Lane wrote: >>> Adding another 16 bits won't get you to that, sadly. Yeah, it *might* >>> extend the MTTF to more than the project's likely lifespan, but that >>> doesn't mean

Re: Add LZ4 compression in pg_dump

2022-03-25 Thread Michael Paquier
On Fri, Mar 25, 2022 at 11:43:17PM +, gkokola...@pm.me wrote: > On Saturday, March 26th, 2022 at 12:13 AM, Rachel Heaton > wrote: >> Here is an updated patchset from Georgios, with minor assistance from myself. >> The comments above should be addressed, but please let us know if > > A small

Re: Remove an unused function GetWalRcvWriteRecPtr

2022-03-25 Thread Michael Paquier
On Sat, Mar 26, 2022 at 10:51:15AM +0530, Bharath Rupireddy wrote: > The function GetWalRcvWriteRecPtr isn't being used anywhere, however > pg_atomic_read_u64(&walrcv->writtenUpto); (reading writtenUpto without > spinlock) is being used directly in pg_stat_get_wal_receiver > walreceiver.c. We eithe

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-25 Thread Michael Paquier
On Thu, Mar 24, 2022 at 04:41:30PM +1300, Thomas Munro wrote: > As mentioned, I was unhappy with the lack of error checking for that > interface, and I've started a new thread about that, but then I > started wondering if we missed a trick here: get_dirent_type() contain > code that wants to return

Remove an unused function GetWalRcvWriteRecPtr

2022-03-25 Thread Bharath Rupireddy
Hi, The function GetWalRcvWriteRecPtr isn't being used anywhere, however pg_atomic_read_u64(&walrcv->writtenUpto); (reading writtenUpto without spinlock) is being used directly in pg_stat_get_wal_receiver walreceiver.c. We either make use of the function instead of pg_atomic_read_u64(&walrcv->writ

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-25 Thread Bharath Rupireddy
On Fri, Mar 25, 2022 at 8:37 PM RKN Sai Krishna wrote: > > Hi Bharath, > > First look at the patch, bear with me if any of the following comments are > repeated. Thanks RKN, for playing around with the patches. > 1. With pg_get_wal_record(lsn), say a WAL record start, end lsn range > contains

RE: Column Filtering in Logical Replication

2022-03-25 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hello, The 'prattrs' column has been added to the pg_publication_rel catalog, but the current commit to catalog.sgml seems to have added it to pg_publication_namespace. The attached patch fixes this. Regards, Noriyoshi Shinoda -Original Message- From: Tomas Vondra Sent: Saturday, Ma

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Justin Pryzby
On Fri, Mar 25, 2022 at 02:27:07PM -0700, Andres Freund wrote: > On 2022-03-25 13:52:11 -0400, Robert Haas wrote: > > On Fri, Mar 25, 2022 at 12:36 PM Andres Freund wrote: > > > Create a CF entry for it, or enable CI on a github repo? > > > > I created a CF entry for it. Then I had to try to Googl

Re: ubsan

2022-03-25 Thread Andres Freund
Hi, On 2022-03-23 15:55:28 -0700, Andres Freund wrote: > Originally I'd planned to mix them into existing members, but I think it'd be > better to have dedicated ones. Applied for a few new buildfarm names for: > {gcc,clang}-{-fsanitize=undefined,-fsanitize=address}. They're now enabled... taman

Re: Column Filtering in Logical Replication

2022-03-25 Thread Tomas Vondra
On 3/26/22 01:18, Tomas Vondra wrote: > > ... > > I went over the patch again, polished the commit message a bit, and > pushed. May the buildfarm be merciful! > There's a couple failures immediately after the push, which caused me a minor heart attack. But it seems all of those are strange failu

Re: Column Filtering in Logical Replication

2022-03-25 Thread Tomas Vondra
On 3/21/22 15:12, Amit Kapila wrote: > On Sat, Mar 19, 2022 at 11:11 PM Tomas Vondra > wrote: >> >> On 3/19/22 18:11, Tomas Vondra wrote: >>> Fix a compiler warning reported by cfbot. >> >> Apologies, I failed to actually commit the fix. So here we go again. >> > > Few comments: > ===

Re: Column Filtering in Logical Replication

2022-03-25 Thread Tomas Vondra
On 3/25/22 04:10, Amit Kapila wrote: > On Fri, Mar 25, 2022 at 5:44 AM Tomas Vondra > wrote: >> >> Attached is a patch, rebased on top of the sequence decoding stuff I >> pushed earlier today, also including the comments rewording, and >> renaming the "transform" function. >> >> I'll go over it

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Michael Paquier
On Fri, Mar 25, 2022 at 10:09:33PM +0100, Daniel Gustafsson wrote: > Agreed. In this case it seems that adding --exclude-extension would make > sense > to keep conistency. I took a quick stab at doing so with the attached while > we're here. src/test/modules/test_pg_dump would be the best place

Re: Add LZ4 compression in pg_dump

2022-03-25 Thread gkokolatos
--- Original Message --- On Saturday, March 26th, 2022 at 12:13 AM, Rachel Heaton wrote: > On Fri, Mar 25, 2022 at 6:22 AM Justin Pryzby pry...@telsasoft.com wrote: > > > On Fri, Mar 25, 2022 at 01:20:47AM -0400, Greg Stark wrote: > > > > > It seems development on this has stalled. I

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-03-25 Thread Jacob Champion
On Fri, 2022-03-25 at 18:00 -0400, Tom Lane wrote: > Jacob Champion writes: > > Do I need to merge my tiny test program into the libpq_pipeline tests? > > Doesn't seem worth the trouble to me, notably because you'd > then have to cope with non-SSL builds too. Fine by me. v5 moves the docs out o

Re: make MaxBackends available in _PG_init

2022-03-25 Thread Andres Freund
Hi, On 2022-03-25 14:35:42 +0800, Julien Rouhaud wrote: > On Fri, Mar 25, 2022 at 02:08:09PM +0900, Michael Paquier wrote: > > On Fri, Mar 25, 2022 at 11:11:46AM +0800, Julien Rouhaud wrote: > > > As an example, here's a POC for a new shmem_request_hook hook after > > > _PG_init(). > > > With it

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Tom Lane
"David G. Johnston" writes: > If we want to choose the other position I would just go with > "--[no]-system-objects" options to toggle whether pattern matching grabs > them by default (defaulting to no) and if someone wants to enable them for > only specific object types they can --system-objects

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread David G. Johnston
On Fri, Mar 25, 2022 at 2:55 PM Tom Lane wrote: > Daniel Gustafsson writes: > >> On 25 Mar 2022, at 19:37, Tom Lane wrote: > >> I'd vote for changing the behavior of --table rather than trying to > >> be bug-compatible with this decision. > > > Agreed. Question is what to do for "-t pg_class",

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-03-25 Thread Tom Lane
Jacob Champion writes: > Do I need to merge my tiny test program into the libpq_pipeline tests? Doesn't seem worth the trouble to me, notably because you'd then have to cope with non-SSL builds too. regards, tom lane

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Tom Lane
Daniel Gustafsson writes: >> On 25 Mar 2022, at 19:37, Tom Lane wrote: >> I'd vote for changing the behavior of --table rather than trying to >> be bug-compatible with this decision. > Agreed. Question is what to do for "-t pg_class", should we still forbid > dumping system catalogs when they a

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Thomas Munro
On Sat, Mar 26, 2022 at 9:55 AM Thomas Munro wrote: > https://api.cirrus-ci.com/v1/artifact/task/5637156969381888/log/contrib/basebackup_to_shell/tmp_check/log/regress_log_001_basic This line doesn't look too healthy: pg_basebackup: error: backup failed: ERROR: shell command "type con > "C:cirr

Re: Deparsing rewritten query

2022-03-25 Thread Tom Lane
Julien Rouhaud writes: > I'm attaching the correct patch this time, sorry about that. While I'm okay with this in principle, as it stands it fails headerscheck/cpluspluscheck: $ src/tools/pginclude/headerscheck In file included from /tmp/headerscheck.Oi8jj3/test.c:2: ./src/include/utils/ruleuti

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-03-25 Thread Daniel Gustafsson
> On 25 Mar 2022, at 22:25, Jacob Champion wrote: > On Fri, 2022-03-25 at 15:32 -0400, Robert Haas wrote: >> This seems totally reasonable. However, I think it should update the >> documentation somehow. > > Done in v4. I would prefer to not introduce a for this, I think adding it as a under

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Daniel Gustafsson
> On 25 Mar 2022, at 19:37, Tom Lane wrote: > I'd vote for changing the behavior of --table rather than trying to > be bug-compatible with this decision. Agreed. Question is what to do for "-t pg_class", should we still forbid dumping system catalogs when they are pattern matched without wildca

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Andres Freund
Hi, On 2022-03-25 13:52:11 -0400, Robert Haas wrote: > On Fri, Mar 25, 2022 at 12:36 PM Andres Freund wrote: > > Create a CF entry for it, or enable CI on a github repo? > > I created a CF entry for it. Then I had to try to Google around to > find the URL from the cfbot, because it's not even lin

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-03-25 Thread Jacob Champion
On Fri, 2022-03-25 at 15:32 -0400, Robert Haas wrote: > On Mon, Feb 28, 2022 at 3:21 PM Jacob Champion wrote: > > v3 rebases over Andres' changes and actually adds the Perl driver that > > I missed the git-add for. > > This seems totally reasonable. However, I think it should update the > documen

Re: SQL/JSON: JSON_TABLE

2022-03-25 Thread Zhihong Yu
On Fri, Mar 25, 2022 at 1:30 PM Andrew Dunstan wrote: > > On 3/22/22 10:55, Daniel Gustafsson wrote: > >> On 22 Mar 2022, at 16:31, Andrew Dunstan wrote: > >> I'm planning on pushing the functions patch set this week and json-table > >> next week. > > My comments from 30827b3c-edf6-4d41-bbf1-298

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Daniel Gustafsson
> On 25 Mar 2022, at 01:40, Tom Lane wrote: > > "David G. Johnston" writes: >> The extension object type does not seem to have gotten the >> --exclude-extension capability that it would need to conform to the general >> design exemplified by --table and hopefully extended out to the routine >> o

Re: Document atthasmissing default optimization avoids verification table scan

2022-03-25 Thread David G. Johnston
On Fri, Mar 25, 2022 at 1:40 PM Robert Haas wrote: > On Tue, Jan 25, 2022 at 8:49 AM James Coleman wrote: > > Here's a version that looks like that. I'm not convinced it's an > > improvement over the previous version: again, I expect more advanced > > users to already understand this concept, an

Re: SSL/TLS instead of SSL in docs

2022-03-25 Thread Daniel Gustafsson
> On 25 Mar 2022, at 20:58, Robert Haas wrote: > > On Wed, Sep 15, 2021 at 8:47 AM Daniel Gustafsson wrote: >> Since the approach taken wasn't to anyones liking, attached is a v4 (partly >> extracted from the previous patch) which only adds notes that SSL is used >> interchangeably with TLS in o

Re: Document atthasmissing default optimization avoids verification table scan

2022-03-25 Thread Tom Lane
Robert Haas writes: > I vote for rejecting both of these patches. I see what James is on about here, but I agree that these specific changes don't help much. What would actually be desirable IMO is a separate section somewhere explaining the performance characteristics of ALTER TABLE. (We've al

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Thomas Munro
On Sat, Mar 26, 2022 at 6:52 AM Robert Haas wrote: > I don't think that the Windows CI is running the TAP tests for > contrib. At least, I can't find any indication of it in the output. So > it doesn't really help to assess how portable this test is, unless I'm > missing something. Yeah :-( vcre

Re: Document atthasmissing default optimization avoids verification table scan

2022-03-25 Thread Robert Haas
On Tue, Jan 25, 2022 at 8:49 AM James Coleman wrote: > Here's a version that looks like that. I'm not convinced it's an > improvement over the previous version: again, I expect more advanced > users to already understand this concept, and I think moving it to the > ALTER TABLE page could very well

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2022-03-25 Thread Tom Lane
Fabien COELHO writes: >> [...] One way to avoid these errors is to send Parse messages before >> pipeline mode starts. I attached a patch to fix to prepare commands at >> starting of a script instead of at the first execution of the command. > ISTM that moving prepare out of command execution i

Re: logical decoding and replication of sequences

2022-03-25 Thread Tomas Vondra
Hi, I've fixed most of the reported issues (or at least I think so), with the exception of those in apply_handle_sequence function, i.e.: 1) properly coordinating with the tablesync worker 2) considering skip_lsn, skipping changes 3) missing privilege check, similar to TargetPrivilegesCheck 4)

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Andrew Dunstan
On 3/25/22 13:52, Robert Haas wrote: > On Fri, Mar 25, 2022 at 12:36 PM Andres Freund wrote: >> Create a CF entry for it, or enable CI on a github repo? > I created a CF entry for it. Then I had to try to Google around to > find the URL from the cfbot, because it's not even linked from > commitf

Re: logical decoding and replication of sequences

2022-03-25 Thread Tomas Vondra
On 3/25/22 15:34, vignesh C wrote: > On Fri, Mar 25, 2022 at 3:29 AM Tomas Vondra > wrote: >> >> Hi, >> >> Pushed, after going through the patch once more, addressed the remaining >> FIXMEs, corrected a couple places in the docs and comments, etc. Minor >> tweaks, nothing important. > > While reb

Re: SSL/TLS instead of SSL in docs

2022-03-25 Thread Robert Haas
On Wed, Sep 15, 2021 at 8:47 AM Daniel Gustafsson wrote: > Since the approach taken wasn't to anyones liking, attached is a v4 (partly > extracted from the previous patch) which only adds notes that SSL is used > interchangeably with TLS in our documentation and configuration. I have actually bee

Re: Accommodate startup process in a separate ProcState array slot instead of in MaxBackends slots.

2022-03-25 Thread Robert Haas
On Sat, Feb 12, 2022 at 6:26 AM Bharath Rupireddy wrote: > FWIW, here's a patch just adding a comment on how the startup process > can get a free procState array slot even when SInvalShmemSize hasn't > accounted for it. I don't think the positioning of this code comment is very good, because it's

Re: pg14 psql broke \d datname.nspname.relname

2022-03-25 Thread Mark Dilger
> On Mar 22, 2022, at 11:04 AM, Robert Haas wrote: > > This patch adds three new arguments to processSQLNamePattern() and > documents one of them. It adds three new parameters to > patternToSQLRegex() as well, and documents none of them. This next patch adds the missing comments. > I think th

Re: Add non-blocking version of PQcancel

2022-03-25 Thread Tom Lane
Robert Haas writes: > Well, that's a fair point, but it's somewhat orthogonal to the one I'm > making, which is that a non-blocking version of function X might be > expected to share code or at least functionality with X itself. Having > something that is named in a way that implies asynchrony wit

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-03-25 Thread Robert Haas
On Mon, Feb 28, 2022 at 3:21 PM Jacob Champion wrote: > v3 rebases over Andres' changes and actually adds the Perl driver that > I missed the git-add for. This seems totally reasonable. However, I think it should update the documentation somehow. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Add psql command to list constraints

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 3:20 PM Justin Pryzby wrote: > \dX is similar, and I remember wondering whether it was really useful/needed. > The catalog tables are exposed and documented for a reason, and power-users > will learn to use them. I don't think that \dX is comparable, because I don't think

Re: Add non-blocking version of PQcancel

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 2:47 PM Tom Lane wrote: > I think you misunderstand where the real pain point is. The reason > that PQcancel's functionality is so limited has little to do with > blocking vs non-blocking, and everything to do with the fact that > it's designed to be safe to call from a SI

Re: Add psql command to list constraints

2022-03-25 Thread Justin Pryzby
On Fri, Mar 25, 2022 at 03:11:47PM -0400, Robert Haas wrote: > On Fri, Mar 25, 2022 at 12:28 AM Greg Stark wrote: > > Development of this seems to have stalled with the only review of this > > patch expressing some skepticism about whether it's needed at all. > > Now, there is some precedent for

Re: Add psql command to list constraints

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 12:28 AM Greg Stark wrote: > Development of this seems to have stalled with the only review of this > patch expressing some skepticism about whether it's needed at all. My opinion on this patch is that we typically handle objects that are essentially table properties by sh

Re: Add non-blocking version of PQcancel

2022-03-25 Thread Tom Lane
Robert Haas writes: > That said, I don't think that this particular patch is going in the > right direction. I think Jacob's comment upthread is right on point: > "This seems like a big change compared to PQcancel(); one that's not > really hinted at elsewhere. Having the async version of an API o

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread David G. Johnston
On Friday, March 25, 2022, Tom Lane wrote: > "David G. Johnston" writes: > > On Fri, Mar 25, 2022 at 10:57 AM Tom Lane wrote: > >> pg_dump never dumps system objects, so I don't see a need for > >> a switch to tell it not to. > > > I considered pg_class to be a system object, which was dumped u

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Tom Lane
"David G. Johnston" writes: > On Fri, Mar 25, 2022 at 10:57 AM Tom Lane wrote: >> pg_dump never dumps system objects, so I don't see a need for >> a switch to tell it not to. > I considered pg_class to be a system object, which was dumped under -t '*' Oh! You're right, the --table switches wil

Re: Add non-blocking version of PQcancel

2022-03-25 Thread Robert Haas
On Thu, Mar 24, 2022 at 6:49 PM Andres Freund wrote: > That's not a whole lot of fun if you think of cases like postgres_fdw (or > citus as in Jelte's case), which run inside the backend. Even with just a > single postgres_fdw, we don't really want to end up in an uninterruptible > PQcancel() that

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread David G. Johnston
On Fri, Mar 25, 2022 at 10:57 AM Tom Lane wrote: > "David G. Johnston" writes: > > > Except succinctly > > omitting system objects which should get its own general option. > pg_dump never dumps system objects, so I don't see a need for > a switch to tell it not to. > > I considered pg_class to b

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Tom Lane
"David G. Johnston" writes: > I don't find the --objectype-only option to be desirable. psql > --tables-only --functions-only just seems odd, no longer are they "only". > I would go with --function-all (and maybe --function-system and > --function-user) if going down this path but the wildcard fe

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 12:36 PM Andres Freund wrote: > Create a CF entry for it, or enable CI on a github repo? I created a CF entry for it. Then I had to try to Google around to find the URL from the cfbot, because it's not even linked from commitfest.postgresql.org for some reason. #blamemagnu

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Tom Lane
Laetitia Avrot writes: > Thank you so much for your suggestion. I was really excited to find a > generic term for Functions and Procedures, but "routine" also includes > aggregation functions which I had excluded from my feature (see Postgres > Glossary here: > https://www.postgresql.org/docs/14/g

Re: Corruption during WAL replay

2022-03-25 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > LGTM, but it would be good to include $! in the die messages. Roger, will do. regards, tom lane

Re: Run end-of-recovery checkpoint in non-wait mode or skip it entirely for faster server availability?

2022-03-25 Thread Andres Freund
Hi, On March 25, 2022 9:56:38 AM PDT, Robert Haas wrote: >On Fri, Mar 25, 2022 at 3:40 AM Bharath Rupireddy > wrote: >> Since the server spins up checkpointer process [1] while the startup >> process performs recovery, isn't it a good idea to make >> end-of-recovery completely optional for the u

Re: Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread David G. Johnston
On Fri, Mar 25, 2022 at 9:44 AM Laetitia Avrot wrote: > > Actually, I thought of it after the --schema-only flag (which is kind of > confusing, because it won't export only schema creation DDL). > --schema-only is talking about the different sections of the dump file, not namespace schema object

Re: Run end-of-recovery checkpoint in non-wait mode or skip it entirely for faster server availability?

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 3:40 AM Bharath Rupireddy wrote: > Since the server spins up checkpointer process [1] while the startup > process performs recovery, isn't it a good idea to make > end-of-recovery completely optional for the users or at least run it > in non-wait mode so that the server wil

Re: logical decoding and replication of sequences

2022-03-25 Thread Tomas Vondra
On 3/25/22 12:59, Tomas Vondra wrote: > > On 3/25/22 12:21, Amit Kapila wrote: >> On Fri, Mar 25, 2022 at 3:56 PM Tomas Vondra >> wrote: >>> >>> >>> On 3/25/22 05:01, Amit Kapila wrote: On Fri, Mar 25, 2022 at 3:29 AM Tomas Vondra wrote: > > Pushed. > Some of the

Re: Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Laetitia Avrot
Hello David, thank you for your interest in that patch. Le ven. 25 mars 2022 à 01:17, David G. Johnston a écrit : > On Thu, Mar 24, 2022 at 4:42 PM Chapman Flack > wrote: > >> On 03/27/21 08:57, Andrew Dunstan wrote: >> > We can bikeshed the name of the flag at some stage. --procedures-only >>

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Andres Freund
Hi, On March 25, 2022 9:22:09 AM PDT, Robert Haas wrote: >On Thu, Mar 17, 2022 at 11:52 AM Robert Haas wrote: >> On Tue, Mar 15, 2022 at 3:04 PM Andres Freund wrote: >> > Seems like this ought to have at least some basic test to make sure it >> > actually works / keeps working? >> >> Wouldn't

Re: Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Laetitia Avrot
Hello Chapman, Le ven. 25 mars 2022 à 00:42, Chapman Flack a écrit : > On 03/27/21 08:57, Andrew Dunstan wrote: > > We can bikeshed the name of the flag at some stage. --procedures-only > > might also make sense > > Any takers for --routines-only ? > > "Routine" is the genuine, ISO SQL umbrella

Re: Corruption during WAL replay

2022-03-25 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > Robert Haas writes: >> ... It's not >> like a 16-bit checksum was state-of-the-art even when we introduced >> it. We just did it because we had 2 bytes that we could repurpose >> relatively painlessly, and not any larger number. And that's still the >> case today, so at least

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Laetitia Avrot
Hello Tom, Le ven. 25 mars 2022 à 00:18, Tom Lane a écrit : > Daniel Gustafsson writes: > >> On 24 Mar 2022, at 23:38, Greg Stark wrote: > >> It looks like this discussion has reached a bit of an impasse with Tom > >> being against this approach and Michael and Daniel being for it. It > >> doe

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-25 Thread Laetitia Avrot
Hello Michael, Le mar. 25 janv. 2022 à 06:49, Michael Paquier a écrit : > On Tue, Nov 09, 2021 at 03:23:07PM +0100, Daniel Gustafsson wrote: > > Looking at this thread I think it makes sense to go ahead with this > patch. The > > filter functionality worked on in another thread is dealing with

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-25 Thread Robert Haas
On Thu, Mar 17, 2022 at 11:52 AM Robert Haas wrote: > On Tue, Mar 15, 2022 at 3:04 PM Andres Freund wrote: > > Seems like this ought to have at least some basic test to make sure it > > actually works / keeps working? > > Wouldn't hurt, although it may be a little bit tricky to getting it > work

Re: Corruption during WAL replay

2022-03-25 Thread Tom Lane
Andres Freund writes: > The same code also exists in src/bin/pg_basebackup/t/010_pg_basebackup.pl, > which presumably has the same collision risks. Oooh, I missed that. > Perhaps we should put a > function into Cluster.pm and use it from both? +1, I'll make it so. regar

Re: Corruption during WAL replay

2022-03-25 Thread Andres Freund
Hi, On 2022-03-25 11:50:48 -0400, Tom Lane wrote: > Robert Haas writes: > > ... It's not > > like a 16-bit checksum was state-of-the-art even when we introduced > > it. We just did it because we had 2 bytes that we could repurpose > > relatively painlessly, and not any larger number. And that's s

Re: Probable memory leak with ECPG and AIX

2022-03-25 Thread Guillaume Lelarge
Le ven. 25 mars 2022, 14:25, Tom Lane a écrit : > Guillaume Lelarge writes: > > Did this get anywhere? Is there something we could do to make this move > > forward? > > No. Write a patch? > I wouldn't have asked if I could write such a patch :-)

Re: ubsan

2022-03-25 Thread David Steele
On 3/23/22 16:55, Andres Freund wrote: It's particularly impressive that the cost of running with ASAN is *so* much lower than valgrind. On my workstation a check-world with -fsanitize=alignment,undefined,address takes 3min17s, vs 1min10s or so without -fsanitize. Not something to always use, b

Re: Corruption during WAL replay

2022-03-25 Thread Tom Lane
Robert Haas writes: > ... It's not > like a 16-bit checksum was state-of-the-art even when we introduced > it. We just did it because we had 2 bytes that we could repurpose > relatively painlessly, and not any larger number. And that's still the > case today, so at least in the short term we will

Re: psql - add SHOW_ALL_RESULTS option

2022-03-25 Thread Fabien COELHO
As Tom said earlier, wasn't this fixed by 618c16707? If not, is there any other discussion on the specifics of this issue? I'm not aware of one. Hmmm… I'll try to understand why the doubled message seems to be still there. -- Fabien.

Re: pg_relation_size on partitioned table

2022-03-25 Thread Justin Pryzby
On Fri, Mar 25, 2022 at 08:52:40PM +0800, Japin Li wrote: > When I try to get total size of partition tables though partitioned table > name using pg_relation_size(), it always returns zero. I can use the > following SQL to get total size of partition tables, however, it is a bit > complex. This

Re: automatically generating node support functions

2022-03-25 Thread Peter Eisentraut
On 25.03.22 14:32, Tom Lane wrote: Peter Eisentraut writes: On 24.03.22 22:57, David Rowley wrote: Also, I'm quite keen to see this work make it into v15. Do you think you'll get time to do that? Thanks for working on it. My thinking right now is to wait for the PG16 branch to open and the

Re: Use JOIN USING aliases in ruleutils.c

2022-03-25 Thread Peter Eisentraut
On 23.03.22 16:14, Tom Lane wrote: My big fear is that it will reduce portability of pg_dump output: views that would have loaded successfully into pre-v14 servers no longer will, and your chances of porting them to other RDBMSes probably go down too. Once v13 becomes EOL, that will be less of a

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-25 Thread RKN Sai Krishna
Hi Bharath, First look at the patch, bear with me if any of the following comments are repeated. 1. With pg_get_wal_record(lsn), say a WAL record start, end lsn range contains the specified LSN, wouldn't it be more meaningful to show the corresponding WAL record. For example, upon providing '0/173

Re: psql - add SHOW_ALL_RESULTS option

2022-03-25 Thread Peter Eisentraut
On 23.03.22 13:58, Fabien COELHO wrote: If you want to wait for libpq to provide a solution for this corner case, I'm afraid that "never" is the likely result, especially as no test case exercices this path to show that there is a problem somewhere, so nobody should care to fix it. I'm not sure

Re: Add index scan progress to pg_stat_progress_vacuum

2022-03-25 Thread Masahiko Sawada
On Wed, Mar 23, 2022 at 6:57 AM Imseih (AWS), Sami wrote: > > >Can the leader pass a callback that checks PVIndStats to ambulkdelete > >an amvacuumcleanup callbacks? I think that in the passed callback, the > >leader checks if the number of processed indexes and updates its > >prog

Re: Corruption during WAL replay

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 10:34 AM Tom Lane wrote: > I dunno. Compatibility and speed concerns aside, that seems like an awful > lot of bits to be expending on every page compared to the value. I dunno either, but over on the TDE thread people seemed quite willing to expend like 16-32 *bytes* for

Re: pg_relation_size on partitioned table

2022-03-25 Thread Japin Li
On Fri, 25 Mar 2022 at 21:21, Bharath Rupireddy wrote: > On Fri, Mar 25, 2022 at 6:23 PM Japin Li wrote: >> >> Hi, hackers >> >> When I try to get total size of partition tables though partitioned table >> name using pg_relation_size(), it always returns zero. I can use the >> following SQL to

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-25 Thread Dilip Kumar
On Fri, Mar 25, 2022 at 7:41 PM Robert Haas wrote: > > On Thu, Mar 24, 2022 at 12:12 PM Dilip Kumar wrote: > > Thanks, I have gone through your changes in comments and docs and those > > LGTM. > > It looks like this patch will need to be updated for Alvaro's commit > 49d9cfc68bf4e0d32a948fe72d5a

Re: logical decoding and replication of sequences

2022-03-25 Thread vignesh C
On Fri, Mar 25, 2022 at 3:29 AM Tomas Vondra wrote: > > Hi, > > Pushed, after going through the patch once more, addressed the remaining > FIXMEs, corrected a couple places in the docs and comments, etc. Minor > tweaks, nothing important. While rebasing patch [1] I found a couple of comments: sta

Re: Corruption during WAL replay

2022-03-25 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 25, 2022 at 10:02 AM Tom Lane wrote: >> Adding another 16 bits won't get you to that, sadly. Yeah, it *might* >> extend the MTTF to more than the project's likely lifespan, but that >> doesn't mean we couldn't get unlucky next week. > I suspect that the number

Re: pg_relation_size on partitioned table

2022-03-25 Thread Bharath Rupireddy
On Fri, Mar 25, 2022 at 6:23 PM Japin Li wrote: > > Hi, hackers > > When I try to get total size of partition tables though partitioned table > name using pg_relation_size(), it always returns zero. I can use the > following SQL to get total size of partition tables, however, it is a bit > comple

Re: Corruption during WAL replay

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 10:02 AM Tom Lane wrote: > Robert Haas writes: > > On Fri, Mar 25, 2022 at 9:49 AM Tom Lane wrote: > >> That'll just reduce the probability of failure, not eliminate it. > > > I mean, if the expected time to the first failure on even 1 machine > > exceeds the time until t

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2022-03-25 Thread Andrei Zubkov
Hi On Fri, 2022-03-25 at 00:37 -0400, Greg Stark wrote: > Fwiw I find the idea of having a separate "aux" table kind of > awkward. > It'll seem strange to users not familiar with the history and without > any clear idea why the fields are split. Greg, thank you for your attention and for your tho

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-25 Thread Robert Haas
On Thu, Mar 24, 2022 at 12:12 PM Dilip Kumar wrote: > Thanks, I have gone through your changes in comments and docs and those LGTM. It looks like this patch will need to be updated for Alvaro's commit 49d9cfc68bf4e0d32a948fe72d5a0ef7f464944e. The newly added test 029_replay_tsp_drops.pl fails wit

Re: Add LZ4 compression in pg_dump

2022-03-25 Thread Justin Pryzby
On Fri, Mar 25, 2022 at 01:20:47AM -0400, Greg Stark wrote: > It seems development on this has stalled. If there's no further work > happening I guess I'll mark the patch returned with feedback. Feel > free to resubmit it to the next CF when there's progress. Since it's a reasonably large patch (a

Re: Corruption during WAL replay

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 2:07 AM Andres Freund wrote: > We really ought to find a way to get to wider checksums :/ Eh, let's just use longer names for the buildfarm animals and call it good. :-) -- Robert Haas EDB: http://www.enterprisedb.com

Re: Corruption during WAL replay

2022-03-25 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 25, 2022 at 9:49 AM Tom Lane wrote: >> That'll just reduce the probability of failure, not eliminate it. > I mean, if the expected time to the first failure on even 1 machine > exceeds the time until the heat death of the universe by 10 orders of > magnitude, it

Re: pg_relation_size on partitioned table

2022-03-25 Thread Japin Li
On Fri, 25 Mar 2022 at 21:21, Bharath Rupireddy wrote: > On Fri, Mar 25, 2022 at 6:23 PM Japin Li wrote: >> >> Hi, hackers >> >> When I try to get total size of partition tables though partitioned table >> name using pg_relation_size(), it always returns zero. I can use the >> following SQL t

Re: [PATCH] pg_statio_all_tables: several rows per table due to invalid TOAST index

2022-03-25 Thread Tom Lane
Andrei Zubkov writes: > Thank you for your attention and for the problem resolution. However > I'm worry a little about possible performance issues related to > monitoring solutions performing regular sampling of statistic views to > find out the most intensive objects in a database. There's no a

Re: Corruption during WAL replay

2022-03-25 Thread Robert Haas
On Fri, Mar 25, 2022 at 9:49 AM Tom Lane wrote: > That'll just reduce the probability of failure, not eliminate it. I mean, if the expected time to the first failure on even 1 machine exceeds the time until the heat death of the universe by 10 orders of magnitude, it's probably good enough. --

Re: identifying unrecognized node type errors

2022-03-25 Thread Tom Lane
Ashutosh Bapat writes: > All these functions are too low level to be helpful to know. Knowing > the caller might actually give a hint as to where the unknown node > originated from. We may get that from the stack trace if that's > available. But if we could annotate the error with error_context th

Re: Corruption during WAL replay

2022-03-25 Thread Tom Lane
Andres Freund writes: > On 2022-03-25 01:38:45 -0400, Tom Lane wrote: >> AFAICS, this strategy of whacking a predetermined chunk of the page with >> a predetermined value is going to fail 1-out-of-64K times. > Yea. I suspect that the way the modifications and checksumming are done are > actually

  1   2   >