Re: Anything I can contribute?

2021-11-17 Thread Aleksander Alekseev
Hi Deng, > I want to do something and contribute to the community, and I wonder if there is something I can do. I wrote a complete article on this topic not a long time ago [1]. Hopefully, no one will object if I link it here. Good luck and looking forward to your contribution! [1]: https://blo

Re: CREATE PUBLICATION should "See Also" CREATE SUBSCRIPTION

2021-11-17 Thread Masahiko Sawada
On Tue, Nov 16, 2021 at 10:21 PM Daniel Gustafsson wrote: > > > On 15 Nov 2021, at 23:29, Peter Smith wrote: > > > I noticed recently that the CREATE PUBLICATION docs page [1] does not > > have any "See Also" reference to the CREATE SUBSCRIPTION docs page > > [2]. > > Nice catch. > > > > >

Re: pg_get_publication_tables() output duplicate relid

2021-11-17 Thread Amit Langote
On Wed, Nov 17, 2021 at 3:09 PM Amit Langote wrote: > Though I now tend > to agree with Amit K that that may be such a meaningful and all that > common use case, Oops I meant: that may NOT be such... -- Amit Langote EDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-11-17 Thread Amul Sul
On Wed, Nov 17, 2021 at 11:13 AM Amul Sul wrote: > > On Sat, Nov 13, 2021 at 2:18 AM Robert Haas wrote: > > > > On Mon, Nov 8, 2021 at 8:20 AM Amul Sul wrote: > > > Attached is the rebased version of refactoring as well as the > > > pg_prohibit_wal feature patches for the latest master head (com

Re: Skipping logical replication transactions on subscriber side

2021-11-17 Thread vignesh C
On Tue, Nov 16, 2021 at 12:01 PM Masahiko Sawada wrote: > > On Mon, Nov 15, 2021 at 11:43 PM vignesh C wrote: > > > > On Mon, Nov 15, 2021 at 2:48 PM Masahiko Sawada > > wrote: > > > > > > On Mon, Nov 15, 2021 at 4:49 PM Greg Nancarrow > > > wrote: > > > > > > > > On Mon, Nov 15, 2021 at 1:49

Re: row filtering for logical replication

2021-11-17 Thread Greg Nancarrow
On Tue, Nov 16, 2021 at 7:33 PM tanghy.f...@fujitsu.com wrote: > > On Friday, November 12, 2021 6:20 PM Ajin Cherian wrote: > > > > Attaching version 39- > > > > I met another problem when filtering out with the operator '~'. > Data can't be replicated as expected. > > For example: > -- publisher

Re: Skipping logical replication transactions on subscriber side

2021-11-17 Thread Amit Kapila
On Tue, Nov 16, 2021 at 12:01 PM Masahiko Sawada wrote: > > Right. I've fixed this issue and attached an updated patch. > Few comments/questions: = 1. + + The pg_stat_subscription_workers view will contain + one row per subscription error reported by workers applying log

Re: Skipping logical replication transactions on subscriber side

2021-11-17 Thread Amit Kapila
On Wed, Nov 17, 2021 at 4:16 PM vignesh C wrote: > > Few comments: > 1) should we set subwentry to NULL to handle !create && !found case > or we could return NULL similar to the earlier function. > I think it is good to be consistent with the nearby code in this case. -- With Regards, Amit Kapi

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-11-17 Thread Amit Kapila
On Wed, Nov 17, 2021 at 1:06 PM Masahiko Sawada wrote: > > On Tue, Nov 16, 2021 at 8:45 PM Amit Kapila wrote: > > The patch looks good to me. But I can't come up with a stable test for > this. It seems to be hard without stopping and resuming parallel > vacuum workers. Do you have any good idea?

Re: CREATE PUBLICATION should "See Also" CREATE SUBSCRIPTION

2021-11-17 Thread Daniel Gustafsson
> On 17 Nov 2021, at 09:24, Masahiko Sawada wrote: > On Tue, Nov 16, 2021 at 10:21 PM Daniel Gustafsson wrote: >> >>> On 15 Nov 2021, at 23:29, Peter Smith wrote: >>> >>> >>> >>> + >> >> To make it analogous with how CREATE/ALTER for SUBSCRIPTION and PUBLICATION >> reference each o

Re: [Patch] ALTER SYSTEM READ ONLY

2021-11-17 Thread Amul Sul
On Wed, Nov 17, 2021 at 4:07 PM Amul Sul wrote: > > On Wed, Nov 17, 2021 at 11:13 AM Amul Sul wrote: > > > > On Sat, Nov 13, 2021 at 2:18 AM Robert Haas wrote: > > > > > > On Mon, Nov 8, 2021 at 8:20 AM Amul Sul wrote: > > > > Attached is the rebased version of refactoring as well as the > >

Re: Failed transaction statistics to measure the logical replication progress

2021-11-17 Thread Amit Kapila
On Wed, Nov 17, 2021 at 9:44 AM osumi.takami...@fujitsu.com wrote: > > On Wednesday, November 17, 2021 12:19 PM Masahiko Sawada > wrote: > > On Tue, Nov 16, 2021 at 9:34 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Monday, November 15, 2021 9:14 PM I wrote: > > > > I've conducted so

Re: Proposal: allow database-specific role memberships

2021-11-17 Thread Daniel Gustafsson
> On 28 Oct 2021, at 21:39, Kenaniah Cerny wrote: > Thank you Asif. A rebased patch is attached. This patch fails to apply yet again, this time due to a collusion in catversion.h. I think it's fine to omit the change in catversion.h as it's likely to repeatedly cause conflicts, and instead just

Propose a new hook for mutating the query bounds

2021-11-17 Thread Xiaozhe Yao
Hi hackers, I am currently working on improving the cardinality estimation component in PostgreSQL with machine learning. I came up with a solution that mutates the bounds for different columns. For example, assume that we have a query ``` select * from test where X<10 and Y<20; ``` Our approach

Re: Patch to avoid orphaned dependencies

2021-11-17 Thread Daniel Gustafsson
> On 20 Sep 2021, at 12:50, Drouvot, Bertrand wrote: > > Hi Ronan, > > On 9/17/21 10:09 AM, Ronan Dunklau wrote: >> Hello Bertrand, >> >> Le mardi 4 mai 2021, 11:55:43 CEST Drouvot, Bertrand a écrit : >>> Implementation overview: >>> >>> * A new catalog snapshot is added: DirtyCatalo

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Robert Haas
On Tue, Nov 16, 2021 at 2:48 PM Mark Dilger wrote: > I'm preparing a new version of the patch that has the catalog empty to begin > with, and only adds values in response to GRANT commands. That also handles > the issues of extension upgrades, which I think the old patch handled better > than

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Robert Haas
On Tue, Nov 16, 2021 at 3:38 PM Andrew Dunstan wrote: > Your original and fairly simple set of patches used hardcoded role names > and sets of GUCs they could update via ALTER SYSTEM. I suggested to you > privately that a more flexible approach would be to drive this from a > catalog table. I had

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Robert Haas
On Tue, Nov 16, 2021 at 5:45 PM Mark Dilger wrote: > I was aware of that, but figured not all GUCs have to be grantable. If it > doesn't fit in a NameData, you can't grant on it. Such restrictions are rather counterintuitive for users, and here it doesn't even buy anything. Using 'text' rather

Re: Yet another fast GiST build

2021-11-17 Thread Daniel Gustafsson
> On 5 Jul 2021, at 08:27, Emre Hasegeli wrote: > ... > > I couldn't understand patch number 2 "Remove DEBUG1 verification". It > seems like something rather useful. These questions have gone unanswered since July, and the patch fails to apply anymore. Is there an updated version on the way?

RE: Failed transaction statistics to measure the logical replication progress

2021-11-17 Thread osumi.takami...@fujitsu.com
On Wednesday, November 17, 2021 10:00 PM Amit Kapila wrote: > On Wed, Nov 17, 2021 at 9:44 AM osumi.takami...@fujitsu.com > wrote: > > > > On Wednesday, November 17, 2021 12:19 PM Masahiko Sawada > wrote: > > > On Tue, Nov 16, 2021 at 9:34 PM osumi.takami...@fujitsu.com > > > wrote: > > > > >

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-17 Thread Daniel Gustafsson
> On 16 Nov 2021, at 03:30, Bharath Rupireddy > wrote: > Done. PSA v9 patch. Pushed with some tweaking of the commit message, thanks! -- Daniel Gustafsson https://vmware.com/

Re: RFC: Logging plan of the running query

2021-11-17 Thread Ekaterina Sokolova
Hi! You forgot my last fix to build correctly on Mac. I have added it. About our discussion of pg_query_state: torikoshia писал 2021-11-04 15:49: I doubt that it was the right link. Sorry for make you confused, here is the link. https://www.postgresql.org/message-id/CA%2BTgmobkpFV0UB67kzXuD36

Re: Propose a new hook for mutating the query bounds

2021-11-17 Thread Tomas Vondra
On 11/17/21 2:24 PM, Xiaozhe Yao wrote: > Hi hackers, > > I am currently working on improving the cardinality estimation component > in PostgreSQL with machine learning. I came up with a solution that > mutates the bounds for different columns. For example, assume that we > have a query > > ``

Re: Patch: Range Merge Join

2021-11-17 Thread Daniel Gustafsson
This patch fails to compile due to an incorrect function name in an assertion: nodeMergejoin.c:297:9: warning: implicit declaration of function 'list_legth' is invalid in C99 [-Wimplicit-function-declaration] Assert(list_legth(node->rangeclause) < 3); ^ -- Daniel Gustafsson

Re: VS2022: Support Visual Studio 2022 on Windows

2021-11-17 Thread Daniel Gustafsson
> On 6 Nov 2021, at 10:29, Hans Buschmann wrote: > Updated the patch to reflect the VisualStudioVersion for Preview 7, which is > the version number compiled into the main devenv.exe image. > This version number seems to be of no interest elsewhere in the postgres > source tree. This patch fa

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Andrew Dunstan
On 11/17/21 08:32, Robert Haas wrote: > On Tue, Nov 16, 2021 at 5:45 PM Mark Dilger > wrote: >> I was aware of that, but figured not all GUCs have to be grantable. If it >> doesn't fit in a NameData, you can't grant on it. > Such restrictions are rather counterintuitive for users, and here it

Re: Propose a new hook for mutating the query bounds

2021-11-17 Thread Xiaozhe Yao
Hi Tomas and Hackers, Thanks for your reply and feedback! > I don't understand how you could achieve this by mutating pg_statistic, without also breaking estimation for queries that only have Y<20. I agree, if we mutate pg_statistics, we will break lots of stuff and the process becomes complicat

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Andrew Dunstan
On 11/16/21 17:12, Tom Lane wrote: > >>> To support pg_dump and pg_upgrade, it might be better to have an >>> enabled/disabled flag rather than to delete rows. >> I'm not really sure what this means. > I didn't see the point of this either. We really need to KISS here. > Every bit of added compl

Re: Printing backtrace of postgres processes

2021-11-17 Thread vignesh C
On Tue, Nov 16, 2021 at 1:12 AM Justin Pryzby wrote: > > On Mon, Nov 15, 2021 at 09:12:49PM +0530, vignesh C wrote: > > The idea here is to implement & expose pg_print_backtrace function, > > internally > > This patch is closely related to this one > https://commitfest.postgresql.org/35/3142/ > |

Re: Patch: Range Merge Join

2021-11-17 Thread Thomas
Thank you for the feedback and sorry for the oversight. I fixed the bug and attached a new version of the patch. Kind Regards, Thomas Am Mi., 17. Nov. 2021 um 15:03 Uhr schrieb Daniel Gustafsson < dan...@yesql.se>: > This patch fails to compile due to an incorrect function name in an > assertion

Re: Propose a new hook for mutating the query bounds

2021-11-17 Thread Tom Lane
Xiaozhe Yao writes: + if (mutate_bounds_hook) { + mutate_bounds_hook(root, &constval, isgt, iseq); + } It seems unlikely that this could do anything actually useful, and impossible that it could do anything useful without enormous waste of cycles along the way. Basical

Re: pg_upgrade test for binary compatibility of core data types

2021-11-17 Thread Andrew Dunstan
On 11/17/21 02:01, Michael Paquier wrote: > > The oldest version tested by the buildfarm is 9.2, so we could ignore > this part I guess? > > Andrew, what do you think about this part? Based on my read of this > thread, there is an agreement that this approach makes the buildfarm > code more mana

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-11-17 Thread Dmitry Dolgov
> On Wed, Jul 07, 2021 at 01:20:24PM +1200, David Rowley wrote: > On Wed, 7 Jul 2021 at 13:04, Andy Fan wrote: > > Looking forward to watching this change closely, thank you both David and > > Tom! > > But I still don't understand what the faults my way have , do you mind > > telling the > > det

Re: Propose a new hook for mutating the query bounds

2021-11-17 Thread Xiaozhe Yao
Hi Tom, Thanks for your feedback. I completely agree with you that a higher-level hook is better suited for this case. I have adjusted the PoC patch to this email. Now it is located in the clauselist_selectivity_ext function, where we first check if the hook is defined. If so, we let the hook est

Re: Non-superuser subscription owners

2021-11-17 Thread Mark Dilger
> On Nov 16, 2021, at 8:11 PM, Jeff Davis wrote: > > On Wed, 2021-11-03 at 12:50 -0700, Mark Dilger wrote: >> The first two patches are virtually unchanged. The third updates the >> behavior of the apply workers, and updates the documentation to >> match. > > v2-0001 corrects some surprises,

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Mark Dilger
> On Nov 17, 2021, at 5:32 AM, Robert Haas wrote: > >> I was aware of that, but figured not all GUCs have to be grantable. If it >> doesn't fit in a NameData, you can't grant on it. > > Such restrictions are rather counterintuitive for users, and here it > doesn't even buy anything. Using '

Re: Schema variables - new implementation for Postgres 15

2021-11-17 Thread Pavel Stehule
Hi 8) immutable variables without a default value > > IMO this case should not be allowed. On 2021/08/29 you wrote: > > I thought about this case, and I have one scenario, where this > behaviour can be useful. When the variable is declared as IMMUTABLE > NOT NULL without not null defau

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Mark Dilger
> On Nov 17, 2021, at 6:31 AM, Andrew Dunstan wrote: > > Well, I was trying (perhaps not very well) to imagine how to deal with > someone modifying the permissions of one of the predefined roles. Say > pg_foo has initial permission to set bar and baz, and the DBA removes > permission to set ba

Re: pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-17 Thread Bharath Rupireddy
On Wed, Nov 17, 2021 at 7:49 AM Michael Paquier wrote: > > At the same time, we could also just let things as they are. --follow > and --stats being specified together is what the user looked for, so > they get what they wanted. I think the existing way of pg_waldump getting stuck with the combi

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Robert Haas
On Wed, Nov 17, 2021 at 9:31 AM Andrew Dunstan wrote: > Well, I was trying (perhaps not very well) to imagine how to deal with > someone modifying the permissions of one of the predefined roles. Say > pg_foo has initial permission to set bar and baz, and the DBA removes > permission to set baz. Ho

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Andrew Dunstan
On 11/17/21 11:07, Mark Dilger wrote: > >> On Nov 17, 2021, at 6:31 AM, Andrew Dunstan wrote: >> >> Well, I was trying (perhaps not very well) to imagine how to deal with >> someone modifying the permissions of one of the predefined roles. Say >> pg_foo has initial permission to set bar and baz,

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Mark Dilger
> On Nov 17, 2021, at 9:06 AM, Andrew Dunstan wrote: > > I agree it's not ideal. At the time I suggested a more flexible approach > I hadn't really thought about the problems of upgrading. If you can come > up with something that works there then I'll be all ears. Are you talking about upgrad

Re: Non-superuser subscription owners

2021-11-17 Thread Jeff Davis
On Wed, 2021-11-17 at 07:44 -0800, Mark Dilger wrote: > Administrators may quite > intentionally create low-power users, ones without access to anything > but a single table, or a single schema, as a means of restricting the > damage that a subscription might do (or more precisely, what the > publi

Re: Non-superuser subscription owners

2021-11-17 Thread Mark Dilger
> On Nov 17, 2021, at 9:33 AM, Jeff Davis wrote: > > I am still trying to understand this use case. It doesn't feel like > "ownership" to me, it feels more like some kind of delegation. > > Is GRANT a better fit here? That would allow more than one user to > REFRESH, or ENABLE/DISABLE the sam

Re: pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-17 Thread Alvaro Herrera
On 2021-Nov-17, Bharath Rupireddy wrote: > On Wed, Nov 17, 2021 at 7:49 AM Michael Paquier wrote: > > > > At the same time, we could also just let things as they are. --follow > > and --stats being specified together is what the user looked for, so > > they get what they wanted. > > I think the

Re: Support for NSS as a libpq TLS backend

2021-11-17 Thread Joshua Brindle
On Tue, Nov 16, 2021 at 1:26 PM Joshua Brindle wrote: > > On Tue, Nov 16, 2021 at 9:45 AM Joshua Brindle > wrote: > > > > On Mon, Nov 15, 2021 at 5:37 PM Joshua Brindle > > wrote: > > > > > > On Mon, Nov 15, 2021 at 4:44 PM Daniel Gustafsson wrote: > > > > > > > > > On 15 Nov 2021, at 20:51, Jo

Re: Parallel Full Hash Join

2021-11-17 Thread Melanie Plageman
On Sat, Nov 6, 2021 at 11:04 PM Justin Pryzby wrote: > > > Rebased patches attached. I will change status back to "Ready for Committer" > > The CI showed a crash on freebsd, which I reproduced. > https://cirrus-ci.com/task/5203060415791104 > > The crash is evidenced in 0001 - but only ~15% of the

Re: logical decoding/replication: new functions pg_ls_logicaldir and pg_ls_replslotdir

2021-11-17 Thread Bossart, Nathan
On 10/30/21, 2:36 AM, "Bharath Rupireddy" wrote: > I've added 3 functions pg_ls_logicalsnapdir, pg_ls_logicalmapdir, > pg_ls_replslotdir, and attached the patch. The sample output looks > like [1]. Please review it further. I took a look at the patch. + charpath[MAXPGPATH + 11

Re: Propose a new hook for mutating the query bounds

2021-11-17 Thread Tomas Vondra
On 11/17/21 16:39, Xiaozhe Yao wrote: Hi Tom, Thanks for your feedback. I completely agree with you that a higher-level hook is better suited for this case. I have adjusted the PoC patch to this email. Now it is located in the clauselist_selectivity_ext function, where we first check if the

Re: Non-superuser subscription owners

2021-11-17 Thread Mark Dilger
> On Nov 17, 2021, at 9:33 AM, Jeff Davis wrote: > > Is GRANT a better fit here? That would allow more than one user to > REFRESH, or ENABLE/DISABLE the same subscription. It wouldn't allow > RENAME, but I don't see why we'd separate privileges for > CREATE/DROP/RENAME anyway. I don't think I

Re: Add planner support function for starts_with()

2021-11-17 Thread Bossart, Nathan
On 10/9/21, 10:24 AM, "Tom Lane" wrote: > With the (admittedly later) introduction of planner support functions, > it's really quite easy to do better. The attached patch adds a planner > support function for starts_with(), with these benefits: The patch looks reasonable to me. Nathan

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2021-11-17 Thread Euler Taveira
On Mon, Nov 15, 2021, at 4:27 AM, Bharath Rupireddy wrote: > As there is some interest shown in this thread at [1], I'm attaching a > new v3 patch here. Please review it. I took a look at this patch. I have a few comments. + ereport(WARNING, + (errmsg("signalling postmaster with PID %d is not allo

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2021-11-17 Thread Justin Pryzby
On Wed, Nov 17, 2021 at 03:59:59PM -0300, Euler Taveira wrote: > On Mon, Nov 15, 2021, at 4:27 AM, Bharath Rupireddy wrote: > > As there is some interest shown in this thread at [1], I'm attaching a > > new v3 patch here. Please review it. > I took a look at this patch. I have a few comments. > >

Re: Deficient error handling in pg_dump and pg_basebackup

2021-11-17 Thread Tom Lane
Michael Paquier writes: > On Tue, Nov 16, 2021 at 10:26:11PM -0500, Tom Lane wrote: >> However, that's largely orthogonal to any of the things my proposed >> patches are trying to fix. If you want to review the patches without >> considering the fsync-error-handling problem, that'd be great. > I

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2021-11-17 Thread Tom Lane
Justin Pryzby writes: > On Wed, Nov 17, 2021 at 03:59:59PM -0300, Euler Taveira wrote: >> I took a look at this patch. I have a few comments. >> >> + ereport(WARNING, >> + (errmsg("signalling postmaster with PID %d is not allowed", pid))); >> >> I would say "signal postmaster PID 1234 is not all

pg_upgrade parallelism

2021-11-17 Thread Jaime Casanova
Hi, Currently docs about pg_upgrade says: """ The --jobs option allows multiple CPU cores to be used for copying/linking of files and to dump and reload database schemas in parallel; a good place to start is the maximum of the number of CPU cores and tablespaces. This o

Re: pg_upgrade parallelism

2021-11-17 Thread Jacob Champion
On Wed, 2021-11-17 at 14:44 -0500, Jaime Casanova wrote: > I'm trying to add more parallelism by copying individual segments > of a relfilenode in different processes. Does anyone one see a big > problem in trying to do that? I'm asking because no one did it before, > that could not be a good sign.

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-17 Thread Andrew Dunstan
On 11/17/21 12:12, Mark Dilger wrote: > >> On Nov 17, 2021, at 9:06 AM, Andrew Dunstan wrote: >> >> I agree it's not ideal. At the time I suggested a more flexible approach >> I hadn't really thought about the problems of upgrading. If you can come >> up with something that works there then I'll

Re: Non-superuser subscription owners

2021-11-17 Thread Andrew Dunstan
On 11/16/21 15:06, Andrew Dunstan wrote: > On 11/3/21 15:50, Mark Dilger wrote: >>> On Nov 1, 2021, at 10:58 AM, Mark Dilger >>> wrote: >>> >>> ALTER SUBSCRIPTION..[ENABLE | DISABLE] do not synchronously start or stop >>> subscription workers. The ALTER command updates the catalog's subenable

Re: pg_upgrade parallelism

2021-11-17 Thread Justin Pryzby
On Wed, Nov 17, 2021 at 02:44:52PM -0500, Jaime Casanova wrote: > Hi, > > Currently docs about pg_upgrade says: > > """ > > The --jobs option allows multiple CPU cores to be used > for copying/linking of files and to dump and reload database schemas > in parallel; a good plac

Re: Assertion failure with barriers in parallel hash join

2021-11-17 Thread Melanie Plageman
On Wed, Mar 31, 2021 at 6:25 PM Melanie Plageman wrote: > > On Wed, Mar 17, 2021 at 8:18 AM Thomas Munro wrote: > > > > On Wed, Mar 17, 2021 at 6:58 PM Thomas Munro wrote: > > > According to BF animal elver there is something wrong with this > > > commit. Looking into it. > > > > Assertion fail

Re: Parallel Full Hash Join

2021-11-17 Thread Melanie Plageman
small mistake in v8. v9 attached. - Melanie v9-0002-Improve-the-naming-of-Parallel-Hash-Join-phases.patch Description: Binary data v9-0003-Parallel-Hash-Full-Right-Outer-Join.patch Description: Binary data v9-0001-Fix-race-condition-in-parallel-hash-join-batch-cl.patch Description: Binary da

Re: Non-superuser subscription owners

2021-11-17 Thread Jeff Davis
On Wed, 2021-11-17 at 10:25 -0800, Mark Dilger wrote: > We may eventually allow non-superusers to create subscriptions, but > there are lots of details to work out. I am setting aside the idea of subscriptions created by non-superusers. My comments were about your idea for "low-power users" that

Re: Non-superuser subscription owners

2021-11-17 Thread Jeff Davis
On Wed, 2021-11-17 at 10:48 -0800, Mark Dilger wrote: > GRANT *might* be part of some solution, but it is unclear to me how > best to do it. The various configuration parameters on subscriptions > entail different security concerns. We might take a fine-grained > approach and create a predefined

Windows: Wrong error message at connection termination

2021-11-17 Thread Lars Kanis
Dear hackers, I lately had a hard time to find the root cause for some wired behavior with the async API of libpq when running client and server on Windows. When the connection aborts with an error - most notably with an error at the connection setup - it sometimes fails with a wrong error mes

Re: CREATE PUBLICATION should "See Also" CREATE SUBSCRIPTION

2021-11-17 Thread Peter Smith
On Wed, Nov 17, 2021 at 11:45 PM Daniel Gustafsson wrote: > > Pushed, thanks for review! > Thanks! -- Kind Regards, Peter Smith. Fujitsu Australia

Re: Add planner support function for starts_with()

2021-11-17 Thread Tom Lane
"Bossart, Nathan" writes: > On 10/9/21, 10:24 AM, "Tom Lane" wrote: >> With the (admittedly later) introduction of planner support functions, >> it's really quite easy to do better. The attached patch adds a planner >> support function for starts_with(), with these benefits: > The patch looks r

Re: Windows: Wrong error message at connection termination

2021-11-17 Thread Tom Lane
Lars Kanis writes: > Explicit closing the socket before process termination leads to a > graceful close even on Windows. That is done by the attached patch. I > think delivering the correct error message to the user is much more > important that closing the process in sync with the socket. Per

xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-17 Thread Robert Haas
I spent a lot of time trying to figure out why xlog.c has global variables ReadRecPtr and EndRecPtr instead of just relying on the eponymous structure members inside the XLogReaderState. I concluded that the values are the same at most points in the code, and thus that we could just use xlogreaders

Re: Windows: Wrong error message at connection termination

2021-11-17 Thread Thomas Munro
On Thu, Nov 18, 2021 at 10:13 AM Lars Kanis wrote: > Unfortunately each connection is closed hard by a Windows PostgreSQL server > with TCP flag RST. That in turn is another Winsock API behavior, that is that > every socket, that wasn't closed by the application is closed hard with the > RST fl

Re: Patch: Range Merge Join

2021-11-17 Thread Tomas Vondra
On 11/17/21 15:45, Thomas wrote: Thank you for the feedback and sorry for the oversight. I fixed the bug and attached a new version of the patch. Kind Regards, Thomas Am Mi., 17. Nov. 2021 um 15:03 Uhr schrieb Daniel Gustafsson mailto:dan...@yesql.se>>: This patch fails to compile due t

Re: XLogReadRecord() error in XlogReadTwoPhaseData()

2021-11-17 Thread Tom Lane
Noah Misch writes: > Tom Lane reported another instance today: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tadarida&dt=2021-11-11%2013%3A29%3A58 > Each of the three failures happened on a sparc64 Debian+gcc machine. I had > tried ~8000 iterations on thorntail, another sparc64 Debia

Re: Windows: Wrong error message at connection termination

2021-11-17 Thread Tom Lane
Thomas Munro writes: > Interesting discovery. I think you might get the same behaviour from > a Unix system if you set SO_LINGER to 0 before you exit[1]. I suppose > if a TCP implementation is partially in user space (I have no idea if > this is true for Windows, I never use it, but I recall tha

Re: Non-superuser subscription owners

2021-11-17 Thread Mark Dilger
> On Nov 17, 2021, at 1:10 PM, Jeff Davis wrote: > > I think you misunderstood the idea: not using predefined roles, just > plain old ordinary GRANT on a subscription object to ordinary roles. > > GRANT REFRESH ON SUBSCRIPTION sub1 TO nonsuper; > > This should be easy enough because the su

Re: Non-superuser subscription owners

2021-11-17 Thread Jeff Davis
On Wed, 2021-11-17 at 15:07 -0800, Mark Dilger wrote: > We only have 4 values left in the bitmask, and I doubt that burning > those slots for multiple new types of rights that only have meaning > for subscriptions is going to be accepted. For full disclosure, I'm > proposing adding ACL_SET and ACL

Re: Improving psql's \password command

2021-11-17 Thread Tom Lane
"Bossart, Nathan" writes: > You are right. I'm not sure what I was thinking. Attached a v3 > with that part removed. Pushed with a little further tweaking --- mostly, I felt that explicitly referring to SIGINT in the API names was too implementation-specific, so I renamed things. As you mentio

Re: Non-superuser subscription owners

2021-11-17 Thread Mark Dilger
> On Nov 17, 2021, at 1:06 PM, Jeff Davis wrote: > > On Wed, 2021-11-17 at 10:25 -0800, Mark Dilger wrote: >> We may eventually allow non-superusers to create subscriptions, but >> there are lots of details to work out. > > I am setting aside the idea of subscriptions created by non-superuser

Re: Deficient error handling in pg_dump and pg_basebackup

2021-11-17 Thread Michael Paquier
On Wed, Nov 17, 2021 at 02:19:20PM -0500, Tom Lane wrote: > Pushed then; thanks for reviewing that. We can consider the fsync > error question at leisure. Fine by me. Thanks for the commit. -- Michael signature.asc Description: PGP signature

RE: pg_get_publication_tables() output duplicate relid

2021-11-17 Thread houzj.f...@fujitsu.com
On Wed, Nov 17, 2021 2:18 PM Amit Langote wrote: > On Wed, Nov 17, 2021 at 12:15 PM houzj.f...@fujitsu.com > wrote: > > On Wed, Nov 17, 2021 10:47 AM Amit Kapila > wrote: > > > On Tue, Nov 16, 2021 at 7:21 AM houzj.f...@fujitsu.com wrote: > > > > If we decide to disallow this case, we seem need

Re: Improving psql's \password command

2021-11-17 Thread Bossart, Nathan
On 11/17/21, 4:15 PM, "Tom Lane" wrote: > Pushed with a little further tweaking --- mostly, I felt that > explicitly referring to SIGINT in the API names was too > implementation-specific, so I renamed things. Thanks! > As you mentioned, there are several other simple_prompt() calls > that could

Re: pg_get_publication_tables() output duplicate relid

2021-11-17 Thread Amit Langote
On Thu, Nov 18, 2021 at 9:33 houzj.f...@fujitsu.com wrote: > On Wed, Nov 17, 2021 2:18 PM Amit Langote wrote: > > On Wed, Nov 17, 2021 at 12:15 PM houzj.f...@fujitsu.com > > wrote: > > > On Wed, Nov 17, 2021 10:47 AM Amit Kapila > > wrote: > > > > On Tue, Nov 16, 2021 at 7:21 AM houzj.f...@fuj

Re: logical decoding/replication: new functions pg_ls_logicaldir and pg_ls_replslotdir

2021-11-17 Thread Justin Pryzby
On Wed, Nov 17, 2021 at 06:46:47PM +, Bossart, Nathan wrote: > On 10/30/21, 2:36 AM, "Bharath Rupireddy" > wrote: > > I've added 3 functions pg_ls_logicalsnapdir, pg_ls_logicalmapdir, > > pg_ls_replslotdir, and attached the patch. The sample output looks > > like [1]. Please review it further

Re: Windows: Wrong error message at connection termination

2021-11-17 Thread Tom Lane
Thomas Munro writes: > I realise now that the experiments we did a while back to try to > understand this across a few different operating systems[2] had missed > this subtlety, because that Python script had an explicit close() > call, whereas PostgreSQL exits. It still revealed that the client

Re: wait event and archive_command

2021-11-17 Thread Fujii Masao
On 2021/11/10 20:19, Bharath Rupireddy wrote: Thanks for the patch. It looks good to me other than the following comment: Thanks for the review! 1) Can't we determine the wait event type based on commandName in ExecuteRecoveryCommand instead of passing it as an extra param? Yes, that's

Why not try for a HOT update, even when PageIsFull()?

2021-11-17 Thread Peter Geoghegan
Commit 2fd8685e7f simplified the checking of modified attributes that takes place within heap_update(). This included a micro-optimization that affects pages marked PageIsFull(): when the target page is marked with PD_PAGE_FULL (which must have been set by a previous heap_update call), don't even t

Re: Failed transaction statistics to measure the logical replication progress

2021-11-17 Thread Amit Kapila
On Wed, Nov 17, 2021 at 7:12 PM osumi.takami...@fujitsu.com wrote: > > On Wednesday, November 17, 2021 10:00 PM Amit Kapila > wrote: > > > > Can you please tell us why you think the names in your proposed patch are > > better than the existing names proposed in Sawada-San's patch? Is it because

Re: wait event and archive_command

2021-11-17 Thread Michael Paquier
On Thu, Nov 18, 2021 at 11:23:17AM +0900, Fujii Masao wrote: > Yes, that's possible. But isn't it uglier to make ExecuteRecoveryCommand() > have > the map of command name and wait event? So I feel inclined to avoid adding > something like the following code into the function... Thought? FWIW, I f

RE: Skipping logical replication transactions on subscriber side

2021-11-17 Thread houzj.f...@fujitsu.com
On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada wrote: > Right. I've fixed this issue and attached an updated patch. > Hi, I have few comments for the testcases. 1) +my $appname = 'tap_sub'; +$node_subscriber->safe_psql( +'postgres', +"CREATE SUBSCRIPTION tap_sub CONNECTION '$pu

Re: Global snapshots

2021-11-17 Thread Andrey V. Lepikhov
Next version of CSN implementation in snapshots to achieve a proper snapshot isolation in the case of a cross-instance distributed transaction. -- regards, Andrey Lepikhov Postgres Professional >From bbb7dd1d7621c091f11e697d3d894fe7a36918a6 Mon Sep 17 00:00:00 2001 From: Andrey Lepikhov Date: W

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-11-17 Thread Fujii Masao
On 2021/11/16 18:55, Etsuro Fujita wrote: Sorry, my explanation was not enough, but I don’t think this is always true. Let me explain using an example: create server loopback foreign data wrapper postgres_fdw options (dbname 'postgres', parallel_commit 'true'); create user mapping for curren

Re: wait event and archive_command

2021-11-17 Thread Bharath Rupireddy
On Thu, Nov 18, 2021 at 7:53 AM Fujii Masao wrote: > > 1) Can't we determine the wait event type based on commandName in > > ExecuteRecoveryCommand instead of passing it as an extra param? > > Yes, that's possible. But isn't it uglier to make ExecuteRecoveryCommand() > have > the map of command n

Re: pg_upgrade test for binary compatibility of core data types

2021-11-17 Thread Michael Paquier
On Wed, Nov 17, 2021 at 10:07:17AM -0500, Andrew Dunstan wrote: > In general I'm in agreement with the direction here. If we can have a > script that applies to back branches to make them suitable for upgrade > testing instead of embedding this in the buildfarm client, so much the > better. Okay.

Re: pg_upgrade test for binary compatibility of core data types

2021-11-17 Thread Justin Pryzby
On Wed, Nov 17, 2021 at 04:01:19PM +0900, Michael Paquier wrote: > On Sun, Nov 07, 2021 at 01:22:00PM -0600, Justin Pryzby wrote: > > That may be good enough for test.sh, but if the kludges were moved to a .sql > > script which was also run by the buildfarm (in stead of its hardcoded > > kludges),

Re: Slow client can delay replication despite max_standby_streaming_delay set

2021-11-17 Thread Fujii Masao
On 2021/11/17 14:39, Andrey Borodin wrote: Hi hackers! $subj was recently observed on one of our installations. Startup process backtrace #0 0x7fd216660d27 in epoll_wait (epfd=525, events=0x55c688dfbde8, maxevents=maxevents@entry=1, timeout=timeout@entry=-1) #1 0x55c687264be9 in W

Re: pg_get_publication_tables() output duplicate relid

2021-11-17 Thread Amit Kapila
On Wed, Nov 17, 2021 at 11:39 AM Amit Langote wrote: > > On Tue, Nov 16, 2021 at 10:27 PM Amit Kapila wrote: > > On Mon, Nov 15, 2021 at 7:12 PM Alvaro Herrera > > wrote: > > > > On Mon, Nov 15, 2021 at 1:48 PM houzj.f...@fujitsu.com > > > > wrote: > > > > > > > > create table tbl1 (a int) par

Re: pg_upgrade test for binary compatibility of core data types

2021-11-17 Thread Tom Lane
Michael Paquier writes: > Okay. I have worked on 0001 to add the table to check after the > binary compatibilities and applied it. Something funny about that on prion: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2021-11-18%2001%3A55%3A38 @@ -747,6 +747,8 @@ '{(2020-01-0

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-17 Thread Greg Nancarrow
On Tue, Nov 16, 2021 at 6:53 PM osumi.takami...@fujitsu.com wrote: > > This v5 depends on v23 skip xid in [1]. > A minor comment: doc/src/sgml/ref/alter_subscription.sgml (1) disable_on_err? + disable_on_err. This doc update names the new parameter as "disable_on_err" instead of "disable_

Re: row filtering for logical replication

2021-11-17 Thread Peter Smith
On Thu, Sep 23, 2021 at 10:33 PM Tomas Vondra wrote: > > 3) create_subscription.sgml > > WHERE clauses, rows must satisfy all expressions > to be copied. If the subscriber is a > > I'm rather skeptical about the principle that all expressions have to > match - I'd have expected exactly

Re: pg_get_publication_tables() output duplicate relid

2021-11-17 Thread Bharath Rupireddy
On Thu, Nov 18, 2021 at 10:23 AM Amit Kapila wrote: > (b) when 'publish_via_partition_root' is false, then it > returns duplicate entries for child tables (provided both are part of > publication), this problem is being discussed in this thread. +1 to just focus on fixing the initial problem prop

Re: row filtering for logical replication

2021-11-17 Thread Peter Smith
On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > > On Wed, Nov 10, 2021 at 12:36 PM Peter Smith wrote: > > > > On Mon, Nov 8, 2021 at 5:53 PM houzj.f...@fujitsu.com > > wrote: > > > > > > 3) v37-0005 > > > > > > - no parse nodes of any kind other than Var, OpExpr, Const, BoolExpr, > > > Fun

  1   2   >