Re: pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-28 Thread Bharath Rupireddy
On Mon, Nov 29, 2021 at 11:09 AM Michael Paquier wrote: > > On Sun, Nov 28, 2021 at 12:13:08PM +0530, Bharath Rupireddy wrote: > > Thanks. Here's the v5. > > By the way, one thing that I completely forgot here is that SIGINT is > not handled on Windows. If we want to make that work for a WIN32 >

Re: Synchronizing slots from primary to standby

2021-11-28 Thread Dilip Kumar
On Mon, Nov 29, 2021 at 12:19 PM Bharath Rupireddy wrote: > > On Mon, Nov 29, 2021 at 11:14 AM Dilip Kumar wrote: > > > > On Mon, Nov 29, 2021 at 9:40 AM Bharath Rupireddy > > wrote: > > > > > > On Mon, Nov 29, 2021 at 1:48 AM SATYANARAYANA NARLAPURAM > > > wrote: > > > > > > > >> 3) Instead of

Re: XTS cipher mode for cluster file encryption

2021-11-28 Thread Antonin Houska
Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Oct 19, 2021 at 02:54:56PM -0400, Stephen Frost wrote: > > > * Sasasu (i...@sasa.su) wrote: > > > > A unified block-based I/O API sounds great. Has anyone tried to do this > > > > before? It would be nice

Can I assume relation would not be invalid during from ExecutorRun to ExecutorEnd

2021-11-28 Thread Andy Fan
Hi, During my recent work, I need some new stuff attached to Relation. Rather than adding some new data structures, I added it to Relation directly. Like relation->balabala. Then I initialize it during ExecutorRun, like table_tuple_insert.. and destroy it at ExecutorEnd. The above solution

Re: Synchronizing slots from primary to standby

2021-11-28 Thread Bharath Rupireddy
On Mon, Nov 29, 2021 at 11:14 AM Dilip Kumar wrote: > > On Mon, Nov 29, 2021 at 9:40 AM Bharath Rupireddy > wrote: > > > > On Mon, Nov 29, 2021 at 1:48 AM SATYANARAYANA NARLAPURAM > > wrote: > > > > > >> 3) Instead of the subscriber pulling the slot info, why can't the > > >> publisher (via the

Re: row filtering for logical replication

2021-11-28 Thread Greg Nancarrow
On Fri, Nov 26, 2021 at 12:40 AM houzj.f...@fujitsu.com wrote: > > When researching and writing a top-up patch about this. > I found a possible issue which I'd like to confirm first. > > It's possible the table is published in two publications A and B, publication > A > only publish "insert" , pu

Re: Rationalizing declarations of src/common/ variables

2021-11-28 Thread Bharath Rupireddy
On Mon, Nov 29, 2021 at 11:27 AM Tom Lane wrote: > > We've been burnt by this issue repeatedly (cf c2d1eea9e, d025cf88b, > 11b500072) so I think it's time to try to formalize and document > what to do to export a variable from src/common/ or src/port/. +1 to document it. > Here's a draft patch.

Re: Skipping logical replication transactions on subscriber side

2021-11-28 Thread vignesh C
On Mon, Nov 29, 2021 at 9:13 AM Amit Kapila wrote: > > On Mon, Nov 29, 2021 at 7:12 AM Masahiko Sawada wrote: > > > > On Sat, Nov 27, 2021 at 7:56 PM Amit Kapila wrote: > > > > > > > Thank you for updating the patch! > > > > > I have made a number of changes in the attached patch which includes

Re: Add connection active, idle time to pg_stat_activity

2021-11-28 Thread Dilip Kumar
On Sat, Nov 27, 2021 at 8:00 AM Bharath Rupireddy wrote: > > On Tue, Nov 16, 2021 at 5:06 PM Rafia Sabih wrote: > > Got it. > > Updated > > Thanks for the patch. +1 for adding the idle/idle_in_txn_time/active > time. I believe these are the total times a backend in its lifetime > accumulates. For

Re: Commitfest 2021-11 Patch Triage - Part 1

2021-11-28 Thread Yugo NAGATA
On Fri, 5 Nov 2021 22:15:49 +0100 Daniel Gustafsson wrote: > 2138: Incremental Materialized View Maintenance > === > There seems to be concensus on the thread that this is a feature that we want, > and after initial design discussions there seems to be

Rationalizing declarations of src/common/ variables

2021-11-28 Thread Tom Lane
We've been burnt by this issue repeatedly (cf c2d1eea9e, d025cf88b, 11b500072) so I think it's time to try to formalize and document what to do to export a variable from src/common/ or src/port/. Here's a draft patch. I'm not in love with the name "PGDLLIMPORT_FE" and would welcome better ideas.

Re: Non-superuser subscription owners

2021-11-28 Thread Amit Kapila
On Sat, Nov 27, 2021 at 11:37 PM Mark Dilger wrote: > > > > > On Nov 24, 2021, at 4:30 PM, Jeff Davis wrote: > > > > We need to do permission checking for WITH CHECK OPTION and RLS. The > > patch right now allows the subscription to write data that an RLS > > policy forbids. > > Thanks for review

Re: Implementing Incremental View Maintenance

2021-11-28 Thread Yugo NAGATA
Hi hackers, This is a response to a comment in "Commitfest 2021-11 Patch Triage - Part 1" [1]. > 2138: Incremental Materialized View Maintenance > === > There seems to be concensus on the thread that this is a feature that we want, > and after initial

Re: Synchronizing slots from primary to standby

2021-11-28 Thread Dilip Kumar
On Mon, Nov 29, 2021 at 9:40 AM Bharath Rupireddy wrote: > > On Mon, Nov 29, 2021 at 1:48 AM SATYANARAYANA NARLAPURAM > wrote: > > > >> 3) Instead of the subscriber pulling the slot info, why can't the > >> publisher (via the walsender or a new bg worker maybe?) push the > >> latest slot info? I'

Re: pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-28 Thread Michael Paquier
On Sun, Nov 28, 2021 at 12:13:08PM +0530, Bharath Rupireddy wrote: > Thanks. Here's the v5. By the way, one thing that I completely forgot here is that SIGINT is not handled on Windows. If we want to make that work for a WIN32 terminal, we would need to do something similar to src/fe_utils/cancel

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-28 Thread vignesh C
On Fri, Nov 26, 2021 at 8:06 PM osumi.takami...@fujitsu.com wrote: > > On Monday, November 22, 2021 3:53 PM vignesh C wrote: > > Few comments: > Thank you so much for your review ! > > > 1) Changes to handle pg_dump are missing. It should be done in > > dumpSubscription and getSubscriptions > Fix

tweak to a few index tests to hits ambuildempty() routine.

2021-11-28 Thread Amul Sul
Hi, Attached patch is doing small changes to brin, gin & gist index tests to use an unlogged table without changing the original intention of those tests and that is able to hit ambuildempty() routing which is otherwise not reachable by the current tests. -- Regards, Amul Sul EDB: http://www.ent

Re: Deduplicate code updating ControleFile's DBState.

2021-11-28 Thread Amul Sul
On Mon, Nov 29, 2021 at 10:12 AM Michael Paquier wrote: > > On Mon, Nov 29, 2021 at 09:28:23AM +0530, Bharath Rupireddy wrote: > > In that case, why can't we inline UpdateControlFile to avoid the > > function call cost? Do you see any issues with it? > > This routine is IMO not something worth bot

Re: row filtering for logical replication

2021-11-28 Thread Peter Smith
On Mon, Nov 29, 2021 at 1:54 PM houzj.f...@fujitsu.com wrote: > > On Sun, Nov 28, 2021 3:18 PM Peter Smith wrote: > > On Fri, Nov 26, 2021 at 1:16 PM houzj.f...@fujitsu.com > > wrote: > > > > > ... > > > Based on this direction, I tried to write a top up POC patch(0005) which > > > I'd > > > l

Re: Deduplicate code updating ControleFile's DBState.

2021-11-28 Thread Michael Paquier
On Mon, Nov 29, 2021 at 09:28:23AM +0530, Bharath Rupireddy wrote: > In that case, why can't we inline UpdateControlFile to avoid the > function call cost? Do you see any issues with it? This routine is IMO not something worth bothering about. > BTW, the v6 patch proposed by Amul at [1], looks go

Re: Non-superuser subscription owners

2021-11-28 Thread Amit Kapila
On Fri, Nov 26, 2021 at 1:36 AM Jeff Davis wrote: > > > as soon as possible instead of at the transaction > > boundary. > > I don't understand why it's important to detect a loss of privileges > faster than a transaction boundary. Can you elaborate? > The first reason is that way it would be con

Re: Synchronizing slots from primary to standby

2021-11-28 Thread Bharath Rupireddy
On Mon, Nov 29, 2021 at 1:48 AM SATYANARAYANA NARLAPURAM wrote: > >> 3) Instead of the subscriber pulling the slot info, why can't the >> publisher (via the walsender or a new bg worker maybe?) push the >> latest slot info? I'm not sure we want to add more functionality to >> the walsender, if yes

Re: Deduplicate code updating ControleFile's DBState.

2021-11-28 Thread Bharath Rupireddy
On Sun, Nov 28, 2021 at 10:03 AM Michael Paquier wrote: > We want to update the > control file's timestamp when it is written, before calculating its > CRC. Okay. > > Why do we even need UpdateControlFile which just calls another > > function? It may be there for usability and readability, but c

Re: How to get started with contribution for GSoc 2022

2021-11-28 Thread Bharath Rupireddy
On Sun, Nov 28, 2021 at 11:37 PM raunak parmar wrote: > > Respected madam/sir > > I am Raunak Parmar and I am pursuing Btech in computer science and > engineering with AIML. I have just entered my first year of engineering and I > am new to open source but I have sound knowledge in C++ and pytho

Re: Skipping logical replication transactions on subscriber side

2021-11-28 Thread Amit Kapila
On Mon, Nov 29, 2021 at 7:12 AM Masahiko Sawada wrote: > > On Sat, Nov 27, 2021 at 7:56 PM Amit Kapila wrote: > > > > Thank you for updating the patch! > > > I have made a number of changes in the attached patch which includes > > (a) the patch was trying to register multiple array entries for th

Re: parallel vacuum comments

2021-11-28 Thread Masahiko Sawada
On Wed, Nov 24, 2021 at 5:54 PM Amit Kapila wrote: > > On Wed, Nov 24, 2021 at 12:16 PM Masahiko Sawada > wrote: > > > > On Wed, Nov 24, 2021 at 1:34 PM Amit Kapila wrote: > > > > > > On Wed, Nov 24, 2021 at 7:43 AM Masahiko Sawada > > > wrote: > > > > > > > > On Mon, Nov 22, 2021 at 1:48 PM

Re: pg_upgrade and publication/subscription problem

2021-11-28 Thread Amit Kapila
On Fri, Nov 26, 2021 at 5:47 PM Marcos Pegoraro wrote: >> >> So, in short, I think what we need to solve is to get the data from >> new tables and newly performed writes on old tables. I could think of >> the following two approaches: >> >> Approach-1: >> 1. Drop subscription and Truncate all tabl

GUC flags

2021-11-28 Thread Justin Pryzby
This isn't flagged with GUC_EXPLAIN: enable_incremental_sort Here's some more candidates: shared_buffers - it seems unfortunate this is not included; actually, it seems like maybe this should be always included - not just if it's set to a non-default, but especially if it's left at

Re: [PATCH] ALTER tab completion

2021-11-28 Thread Michael Paquier
On Fri, Nov 26, 2021 at 05:00:44PM +0900, Ken Kato wrote: > I made following updates: I have made one small modification for ALTER VIEW, and applied what you have. Thanks, Kato-san. -- Michael signature.asc Description: PGP signature

RE: row filtering for logical replication

2021-11-28 Thread houzj.f...@fujitsu.com
On Sun, Nov 28, 2021 3:18 PM Peter Smith wrote: > On Fri, Nov 26, 2021 at 1:16 PM houzj.f...@fujitsu.com > wrote: > > > ... > > Based on this direction, I tried to write a top up POC patch(0005) which I'd > > like to share. > > > > The top up patch mainly did the following things. > > > > * Move

Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

2021-11-28 Thread vignesh C
On Sun, Nov 28, 2021 at 7:15 PM Bharath Rupireddy wrote: > > On Sun, Nov 28, 2021 at 5:21 PM vignesh C wrote: > > Thanks for the updated patch, one comment: > > 1) The function can be indented similar to other functions in the same > > file: > > +CREATE FUNCTION memcxt_get_proc_pid(text) > > +R

Re: rand48 replacement

2021-11-28 Thread Tom Lane
Fabien COELHO writes: >> Good, but then you shouldn't write associated code as if that's still >> a problem, because you'll cause other people to think it's still a >> problem and write equally contorted code elsewhere. "v & 1" is a >> transparent way of producing a bool, while this code isn't.

Re: row filtering for logical replication

2021-11-28 Thread Peter Smith
On Sun, Nov 28, 2021 at 6:17 PM Peter Smith wrote: > > On Fri, Nov 26, 2021 at 1:16 PM houzj.f...@fujitsu.com > wrote: > > > ... > > Based on this direction, I tried to write a top up POC patch(0005) which > > I'd like to share. > > > > The top up patch mainly did the following things. > > > > *

Re: Skipping logical replication transactions on subscriber side

2021-11-28 Thread Masahiko Sawada
On Sat, Nov 27, 2021 at 7:56 PM Amit Kapila wrote: > > On Fri, Nov 26, 2021 at 6:00 AM Masahiko Sawada wrote: > > > > Indeed. Attached an updated patch. Thanks! > > > Thank you for updating the patch! > I have made a number of changes in the attached patch which includes > (a) the patch was try

Re: Contributing

2021-11-28 Thread Laurenz Albe
On Fri, 2021-11-26 at 23:13 +0530, Parth Shah wrote: > I am Parth Shah, currently a second-year computer engineering student at > Mumbai University. > I only recently started learning Postgres and I was quite fascinated by it. I > recently read > about Konstantina Skovola's project about creating

Re: Synchronizing slots from primary to standby

2021-11-28 Thread SATYANARAYANA NARLAPURAM
> 3) Instead of the subscriber pulling the slot info, why can't the > publisher (via the walsender or a new bg worker maybe?) push the > latest slot info? I'm not sure we want to add more functionality to > the walsender, if yes, isn't it going to be much simpler? > Standby pulling the information

Switching XLog source from archive to streaming when primary available

2021-11-28 Thread SATYANARAYANA NARLAPURAM
Hi Hackers, When the standby couldn't connect to the primary it switches the XLog source from streaming to archive and continues in that state until it can get the WAL from the archive location. On a server with high WAL activity, typically getting the WAL from the archive is slower than streaming

Contributing

2021-11-28 Thread Parth Shah
Dear Sir/Ma'am, I am Parth Shah, currently a second-year computer engineering student at Mumbai University. I only recently started learning Postgres and I was quite fascinated by it. I recently read about Konstantina Skovola's project about creating an extension for the Julia progr

How to get started with contribution for GSoc 2022

2021-11-28 Thread raunak parmar
Respected madam/sirI am Raunak Parmar and I am pursuing Btech in computer science and engineering with AIML. I have just entered my first year of engineering and I am new to open source but I have sound knowledge in C++ and python, so I would like to contribute to your organization. Please guide me

Re: [Proposal] Add foreign-server health checks infrastructure

2021-11-28 Thread Zhihong Yu
On Tue, Nov 23, 2021 at 8:57 PM kuroda.hay...@fujitsu.com < kuroda.hay...@fujitsu.com> wrote: > Dear Kato-san, > > Thank you for reviewing! > > > Thank you for sending the patches! > > I confirmed that they can be compiled and tested successfully on CentOS > > 8. > > Thanks! > > > + { > > +

[PATCH] buffile: ensure start offset is aligned with BLCKSZ

2021-11-28 Thread Sasasu
Hi hackers, there are a very long discuss about TDE, and we agreed on that if the temporary file I/O can be aligned to some fixed size, it will be easier to use some kind of encryption algorithm. discuss: https://www.postgresql.org/message-id/20211025155814.GD20998%40tamriel.snowman.net This pat

Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

2021-11-28 Thread Bharath Rupireddy
On Sun, Nov 28, 2021 at 5:21 PM vignesh C wrote: > Thanks for the updated patch, one comment: > 1) The function can be indented similar to other functions in the same file: > +CREATE FUNCTION memcxt_get_proc_pid(text) > +RETURNS int > +LANGUAGE SQL > +AS 'SELECT pid FROM pg_stat_activity WHERE ba

Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

2021-11-28 Thread vignesh C
On Sun, Nov 28, 2021 at 12:25 PM Bharath Rupireddy wrote: > > On Sun, Nov 28, 2021 at 12:22 PM vignesh C wrote: > > > Attaching v4 patch, please review it further. > > > > One small comment: > > 1) There should be a space in between "LOGmessage level" > > +it can) for memory contexts. The

Re: rand48 replacement

2021-11-28 Thread Fabien COELHO
Hello, They are not more nor less relevant than any other "random" constant. The state needs a default initialization. The point of using DK's is that it is somehow cannot be some specially crafted value which would have some special property only know to the purveyor of the constant and could