Truncate in synchronous logical replication failed

2021-04-06 Thread tanghy.f...@fujitsu.com
Hi I met a problem in synchronous logical replication. The client hangs when TRUNCATE TABLE at publisher. Example of the procedure: --publisher-- create table test (a int primary key); create publication pub for table test; --subscriber-- create table test (a int primary key); c

RE: Disable WAL logging to speed up data loading

2021-04-06 Thread osumi.takami...@fujitsu.com
Hi Mainly affected by a commit 9de9294, I've fixed minor things to rebase the patch. All modifications I did are cosmetic changes and a little bit of documentation updates. Please have a look at attached v09. Best Regards, Takamichi Osumi disable_WAL_logging_v09.patch Description: dis

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

2021-04-06 Thread Bharath Rupireddy
On Wed, Apr 7, 2021 at 11:44 AM Michael Paquier wrote: > > On Wed, Apr 07, 2021 at 06:31:19AM +0530, Bharath Rupireddy wrote: > > Setting p->pd_flags = 0; is unnecessary and redundant after memsetting > > the page to zeros. Also, see the existing code for pd_prune_xid, > > similarly I've done that

Re: buildfarm instance bichir stuck

2021-04-06 Thread Thomas Munro
On Wed, Apr 7, 2021 at 5:44 PM Robins Tharakan wrote: > Bichir's been stuck for the past month and is unable to run regression tests > since 6a2a70a02018d6362f9841cc2f499cc45405e86b. Hrmph. That's "Use signalfd(2) for epoll latches." I had a similar report from an illumos user (but it was inte

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

2021-04-06 Thread Michael Paquier
On Wed, Apr 07, 2021 at 06:31:19AM +0530, Bharath Rupireddy wrote: > Setting p->pd_flags = 0; is unnecessary and redundant after memsetting > the page to zeros. Also, see the existing code for pd_prune_xid, > similarly I've done that for pd_flags. I think it's okay with /* > p->pd_flags = 0;

Re: Yet another fast GiST build

2021-04-06 Thread Heikki Linnakangas
On 08/03/2021 19:06, Andrey Borodin wrote: There were numerous GiST-build-related patches in this thread. Yet uncommitted is a patch with sortsupport routines for btree_gist contrib module. Here's its version which needs review. Reviewing this now again. One thing caught my eye: +static int

Re: MultiXact\SLRU buffers configuration

2021-04-06 Thread Thomas Munro
On Sun, Apr 4, 2021 at 7:57 AM Andrey Borodin wrote: > I was toying around with big values. For example if we set different big xact_buffers we can get something like > FATAL: not enough shared memory for data structure "Notify" (72768 bytes requested) > FATAL: not enough shared memory for data

buildfarm instance bichir stuck

2021-04-06 Thread Robins Tharakan
Hi, Bichir's been stuck for the past month and is unable to run regression tests since 6a2a70a02018d6362f9841cc2f499cc45405e86b. It is interesting that that commit's a month old and probably no other client has complained since, but diving in, I can see that it's been unable to even start regress

RE: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread osumi.takami...@fujitsu.com
On Wednesday, April 7, 2021 7:50 AM Fujii Masao wrote: > On 2021/04/07 5:01, Tom Lane wrote: > > Fujii Masao writes: > >> Thanks! So I pushed the patch. > > > > Seems like the test case added by this commit is not working on > > Windows. > > Thanks for the report! My analysis is [1], and I push

Re: Any objections to implementing LogicalDecodeMessageCB for pgoutput?

2021-04-06 Thread Amit Kapila
On Mon, Apr 5, 2021 at 5:45 PM Euler Taveira wrote: > > On Mon, Apr 5, 2021, at 4:06 AM, Amit Kapila wrote: > > I have made few minor changes in the attached. (a) Initialize the > streaming message callback API, (b) update docs to reflect that XID > can be sent for streaming of in-progress transac

Re: multi-install PostgresNode fails with older postgres versions

2021-04-06 Thread Mark Dilger
> On Apr 3, 2021, at 11:01 AM, Andrew Dunstan wrote: > > I've had a look at the first of these patches. I think it's generally > ok, but: > > > -TestLib::system_or_bail('initdb', '-D', $pgdata, '-A', 'trust', '-N', > +TestLib::system_or_bail('initdb', '-D', $pgdata, '-A', 'trust', > +

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-07, yuzuko wrote: > I'm working on fixing the patch according to the comments. > I'll send it as soon as I can. Thanks, I've been giving it a look too. > I've been thinking about traditional inheritance, I realized that we > need additional > handling to support them because unlike d

Re: TRUNCATE on foreign table

2021-04-06 Thread Kazutaka Onishi
> One minor thing - I think "mixtured" is not the > correct word in "+-- partition table mixtured by table and foreign > table". How about something like "+-- partitioned table with both > local and foreign table as partitions"? Sure. I've fixed this. > The v15 patch basically looks good to me. I

Re: New IndexAM API controlling index vacuum strategies

2021-04-06 Thread Peter Geoghegan
On Tue, Apr 6, 2021 at 7:05 AM Matthias van de Meent wrote: > If you have updated patches, I'll try to check them this evening (CEST). Here is v11, which is not too different from v10 as far as the truncation stuff goes. Masahiko should take a look at the last patch again. I renamed the GUCs to

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-06 Thread yuzuko
Hello, Thank you for reviewing. I'm working on fixing the patch according to the comments. I'll send it as soon as I can. > On 2021-04-06 16:56:49 -0400, Alvaro Herrera wrote: > > I think there is a good reason to treat them the same: pgstat does not > > have a provision to keep stats both of the

Re: shared-memory based stats collector

2021-04-06 Thread Kyotaro Horiguchi
At Tue, 6 Apr 2021 09:32:16 -0700, Andres Freund wrote in > Hi, > > On 2021-04-05 02:29:14 -0700, Andres Freund wrote: .. > I'm inclined to push patches > [PATCH v60 05/17] pgstat: split bgwriter and checkpointer messages. > [PATCH v60 06/17] pgstat: Split out relation stats handling from > AtE

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-04-06 Thread Japin Li
On Tue, 06 Apr 2021 at 17:56, Peter Eisentraut wrote: > On 06.04.21 07:24, Japin Li wrote: I think this patch is about ready to commit, but please provide a final version in good time. >>> I took the liberty to address all the review comments and provide a v9 >>> patch on top of Japin

Re: policies with security definer option for allowing inline optimization

2021-04-06 Thread Noah Misch
On Tue, Apr 06, 2021 at 01:16:16PM -0700, Dan Lynch wrote: > Final question: do you think using procedures vs writing inline queries for > RLS quals/checks has a big difference in performance (assuming functions > are sql)? If the function meets the criteria for inlining (see inline_function()), t

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 10:52 PM Bharath Rupireddy wrote: > > On Tue, Apr 6, 2021 at 6:37 PM Amit Langote wrote: > > > 3) will the cmin in the output always be the same? > > > +SELECT cmin, * FROM batch_cp_upd_test3; > > > > TBH, I am not so sure. Maybe it's not a good idea to rely on cmin > > af

Re: Proposal: Save user's original authenticated identity for logging

2021-04-06 Thread Michael Paquier
On Tue, Apr 06, 2021 at 06:31:16PM +, Jacob Champion wrote: > On Tue, 2021-04-06 at 14:15 +0900, Michael Paquier wrote: > > Hmm. You are making a good point here, but is that really the best > > thing we can do? We lose the context of the authentication type being > > done with this implement

Re: TRUNCATE on foreign table

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 10:15 PM Kazutaka Onishi wrote: > I've checked v15 patch with "make check-world" and confirmed this passed. Thanks for the patch. One minor thing - I think "mixtured" is not the correct word in "+-- partition table mixtured by table and foreign table". How about something l

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

2021-04-06 Thread Bharath Rupireddy
On Wed, Apr 7, 2021 at 12:07 AM Mahendra Singh Thalor wrote: > +++ b/src/backend/storage/page/bufpage.c > @@ -51,7 +51,7 @@ PageInit(Page page, Size pageSize, Size specialSize) > /* Make sure all fields of page are zero, as well as unused space */ > MemSet(p, 0, pageSize); > > -p->pd

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-06 Thread James Coleman
On Mon, Apr 5, 2021 at 11:58 PM David Rowley wrote: > > On Sat, 20 Mar 2021 at 09:41, James Coleman wrote: > > I've attached a cleaned up patch. Last CF it was listed in is > > https://commitfest.postgresql.org/29/2542/ -- what's the appropriate > > step to take here given it's an already existin

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-04-06 Thread Andy Fan
> > However the thing becomes complex with the below 2 cases. > > 1. SELECT * FROM t INNER JOIN j on t.nullable = q.b; > We know t.b will be not null **finally**. But the current plan may > something > like this: > > QUERY PLAN > -- > Merge J

Re: ModifyTable overheads in generic plans

2021-04-06 Thread Andres Freund
Hi, On 2021-04-06 19:24:11 -0400, Tom Lane wrote: > I also could not get excited about postponing initialization of RETURNING > or WITH CHECK OPTIONS expressions. I grant that that can be helpful > when those features are used, but I doubt that RETURNING is used that > heavily, and WITH CHECK OPT

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-06 Thread Andres Freund
Hi, On 2021-04-06 16:56:49 -0400, Alvaro Herrera wrote: > I think there is a good reason to treat them the same: pgstat does not > have a provision to keep stats both of the table with children, and the > table without children. It can only have one of those. For > partitioning that doesn't matt

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-07, Thomas Munro wrote: > On Wed, Apr 7, 2021 at 11:18 AM Alvaro Herrera > wrote: > > BTRW it's funny that after these patches, "xlogreader" no longer reads > > anything. It's more an "xlog interpreter" -- the piece of code that > > splits individual WAL records from a stream of WAL

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Thomas Munro
On Wed, Apr 7, 2021 at 11:18 AM Alvaro Herrera wrote: > BTRW it's funny that after these patches, "xlogreader" no longer reads > anything. It's more an "xlog interpreter" -- the piece of code that > splits individual WAL records from a stream of WAL bytes that's caller's > responsibility to obtai

Re: ModifyTable overheads in generic plans

2021-04-06 Thread Tom Lane
Amit Langote writes: > On Mon, Apr 5, 2021 at 1:43 AM Tom Lane wrote: >> OK. Do you want to pull out the bits of the patch that we can still >> do without postponing BeginDirectModify? > I ended up with the attached, whereby ExecInitResultRelation() is now > performed for all relations before c

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-07, Thomas Munro wrote: > I wonder if it would be better to have the client code access these > values through functions (even if they just access the variables in a > static inline function), to create a bit more separation? Something > like XLogReaderGetWanted(&page_lsn, &bytes_want

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Andres Freund
Hi, On 2021-04-07 05:09:53 +1200, Thomas Munro wrote: > From 560cdfa444a3b05a0e6b8054f3cfeadf56e059fc Mon Sep 17 00:00:00 2001 > From: Kyotaro Horiguchi > Date: Thu, 5 Sep 2019 20:21:55 +0900 > Subject: [PATCH v18 1/3] Move callback-call from ReadPageInternal to > XLogReadRecord. > > The curren

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Thomas Munro
On Wed, Apr 7, 2021 at 5:09 AM Thomas Munro wrote: > 0001 + 0002 get rid of the callback interface and replace it with a > state machine, making it the client's problem to supply data when it > returns XLREAD_NEED_DATA. I found this interface nicer to work with, > for my WAL decoding buffer patch

Re: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread Fujii Masao
On 2021/04/07 5:01, Tom Lane wrote: Fujii Masao writes: Thanks! So I pushed the patch. Seems like the test case added by this commit is not working on Windows. Thanks for the report! My analysis is [1], and I pushed the proposed patch. [1] https://postgr.es/m/3cc3909d-f779-7a74-c201-f1f

Re: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread Fujii Masao
On 2021/04/07 3:03, Fujii Masao wrote: On 2021/04/06 23:01, Fujii Masao wrote: On 2021/04/06 20:44, David Steele wrote: On 4/6/21 7:13 AM, Fujii Masao wrote: On 2021/04/06 15:59, osumi.takami...@fujitsu.com wrote: I just wanted to write why the error was introduced, but it was not nec

Re: policies with security definer option for allowing inline optimization

2021-04-06 Thread Dan Lynch
> > > > I suppose if the possibility exists that this could happen, perhaps using > > RLS for selects is not quite "production ready"? > > I would not draw that conclusion. > > This is great to hear! I'm betting a lot on RLS and have been investing a lot into it. > > Or perhaps if the RLS > > qua

Re: pgbench - add pseudo-random permutation function

2021-04-06 Thread Fabien COELHO
Hello Dean, Pushed. I decided not to go with the "joke" randu64() function, but instead used getrand() directly. This at least removes any *direct* use of doubles in permute() (though of course they're still used indirectly), and means that if getrand() is improved in the future, permute() wi

Re: psql - add SHOW_ALL_RESULTS option

2021-04-06 Thread Fabien COELHO
Hello Peter, My 0.02€: I tested this updated version and do not have any comment on this version. From my point of view it could be committed. I would not bother to separate the test style ajustments. Committed. The last thing I fixed was the diff in the copy2.out regression test. The ord

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-04, Tomas Vondra wrote: > 1) I still don't understand why inheritance and declarative partitioning > are treated differently. Seems unnecessary nad surprising, but maybe > there's a good reason? I think there is a good reason to treat them the same: pgstat does not have a provision to

Re: Parallel Full Hash Join

2021-04-06 Thread Zhihong Yu
On Tue, Apr 6, 2021 at 11:59 AM Melanie Plageman wrote: > On Fri, Apr 2, 2021 at 3:06 PM Zhihong Yu wrote: > > > > Hi, > > For v6-0003-Parallel-Hash-Full-Right-Outer-Join.patch > > > > +* current_chunk_idx: index in current HashMemoryChunk > > > > The above comment seems to be better fit for

Re: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread Tom Lane
Fujii Masao writes: > Thanks! So I pushed the patch. Seems like the test case added by this commit is not working on Windows. regards, tom lane

Re: PostgreSQL log query's result size

2021-04-06 Thread Justin Pryzby
On Tue, Apr 06, 2021 at 02:03:03PM -0500, Hellmuth Vargas wrote: > Could you tell me if it is possible that as well as the configuration that > the log presents the duration of the delayed queries, it can also present > the size of the result data? especially those who want to return a lot of > inf

PostgreSQL log query's result size

2021-04-06 Thread Hellmuth Vargas
Hello Excuse me in advance for my English, I'm improving :-) Could you tell me if it is possible that as well as the configuration that the log presents the duration of the delayed queries, it can also present the size of the result data? especially those who want to return a lot of information

Re: Parallel Full Hash Join

2021-04-06 Thread Melanie Plageman
On Fri, Apr 2, 2021 at 3:06 PM Zhihong Yu wrote: > > Hi, > For v6-0003-Parallel-Hash-Full-Right-Outer-Join.patch > > +* current_chunk_idx: index in current HashMemoryChunk > > The above comment seems to be better fit for ExecScanHashTableForUnmatched(), > instead of ExecParallelPrepHashTableF

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

2021-04-06 Thread Mahendra Singh Thalor
On Tue, 6 Apr 2021 at 19:14, Bharath Rupireddy wrote: > > On Tue, Apr 6, 2021 at 6:09 PM Michael Paquier wrote: > > > > On Mon, Mar 22, 2021 at 10:58:17AM +0530, Mahendra Singh Thalor wrote: > > > Your changes look to fine me and I am also not getting any failure. I > > > think we should back-pat

Re: Proposal: Save user's original authenticated identity for logging

2021-04-06 Thread Jacob Champion
On Tue, 2021-04-06 at 14:15 +0900, Michael Paquier wrote: > On Mon, Apr 05, 2021 at 04:40:41PM +, Jacob Champion wrote: > > This loses the test fixes I made in my v19 [1]; some of the tests on > > HEAD aren't testing anything anymore. I've put those fixups into 0001, > > attached. > > Argh, Th

Re: GSoC 2021 - Student looking for a mentor - Magzum Assanbayev

2021-04-06 Thread Ilaria
Hello Magzum, Thank you for the email! I am really glad you are interested in working with Postgres. Have you tried looking at the project ideas? You can find them here: https://wiki.postgresql.org/wiki/GSoC_2021 If you have any preference, you are more than welcome to ask questions and clar

Re: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread Fujii Masao
On 2021/04/06 23:01, Fujii Masao wrote: On 2021/04/06 20:44, David Steele wrote: On 4/6/21 7:13 AM, Fujii Masao wrote: On 2021/04/06 15:59, osumi.takami...@fujitsu.com wrote: I just wanted to write why the error was introduced, but it was not necessary. We should remove and fix the first

Re: [GSoC 2021 Proposal] Develop Performance Farm Benchmarks and Website

2021-04-06 Thread Ilaria
Just a quick heads up - this is being followed up by myself in private messages :) If anyone has any other inputs for the proposal, feel free to share! Ilaria > Am 06.04.2021 um 19:59 schrieb YoungHwan Joo : > >  > Hello, team PostgreSQL! > > I am YoungHwan Joo, a student who is interested

Re: Minimal logical decoding on standbys

2021-04-06 Thread Andres Freund
Hi, On 2021-04-06 14:30:29 +0200, Drouvot, Bertrand wrote: > From 827295f74aff9c627ee722f541a6c7cc6d4133cf Mon Sep 17 00:00:00 2001 > From: bdrouvotAWS > Date: Tue, 6 Apr 2021 11:59:23 + > Subject: [PATCH v15 1/5] Allow logical decoding on standby. > > Allow a logical slot to be created on s

Re: Force lookahead in COPY FROM parsing

2021-04-06 Thread Heikki Linnakangas
On 02/04/2021 20:21, John Naylor wrote: I have nothing further so it's RFC. The patch is pretty simple compared to the earlier ones, but is worth running the fuzzer again as added insurance? Good idea. I did that, and indeed it revealed bugs. If the client sent just a single byte in one CopyD

Re: GSoc Applicant

2021-04-06 Thread Mark Wong
Hello, On Mon, Apr 05, 2021 at 11:46:36PM +0200, Mohamed Mansour wrote: > Greetings, > > I'm Mohamed Mansour, a Data Engineer at IBM and a Master's degree student > in the Computer Engineering Department - Faculty of Engineering - Cairo > University. > > I would like to apply to google summer of

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Fujii Masao
On 2021/04/06 22:02, Michael Paquier wrote: On Tue, Apr 06, 2021 at 02:39:54PM +0200, Matthias van de Meent wrote: On Tue, 6 Apr 2021 at 14:29, Fujii Masao wrote: Attached is the updated version of the patch. Barring any objection, I'm thinking to commit this. Sorry for the late reply. T

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Thomas Munro
On Wed, Mar 31, 2021 at 7:17 PM Kyotaro Horiguchi wrote: > At Wed, 31 Mar 2021 10:00:02 +1300, Thomas Munro > wrote in > > On Thu, Mar 4, 2021 at 3:29 PM Kyotaro Horiguchi > > wrote: > > > A recent commot about LSN_FORMAT_ARGS conflicted this. > > > Just rebased. > > > > FYI I've been looking a

Re: TRUNCATE on foreign table

2021-04-06 Thread Kazutaka Onishi
I've attached v15. > I still feel that the above bunch of code is duplicate of what > do_sql_command function already has. I would recommend that we just > make that function non-static(it's easy to do) and keep the > declaration in postgres_fdw.h and use it in the > postgresExecForeignTruncate.

Re: shared-memory based stats collector

2021-04-06 Thread Andres Freund
Hi, On 2021-04-05 02:29:14 -0700, Andres Freund wrote: > I've spent most of the last 2 1/2 weeks on this now. Unfortunately I think > that, while it has gotten a lot closer, it's still about a week's worth of > work away from being committable. > > My main concerns are: > > - Test Coverage: > >

Re: subtransaction performance regression [kind of] due to snapshot caching

2021-04-06 Thread Andres Freund
Hi, On 2021-04-05 21:35:21 -0700, Andres Freund wrote: > See the attached fix. I did include a test that verifies that the > kill_prior_tuples optimization actually prevents the index from growing, > when subtransactions are involved. I think it should be stable, even > with concurrent activity. B

GSoc Applicant

2021-04-06 Thread Mohamed Mansour
Greetings, I'm Mohamed Mansour, a Data Engineer at IBM and a Master's degree student in the Computer Engineering Department - Faculty of Engineering - Cairo University. I would like to apply to google summer of code to work on the following project: Database Load Stress Benchmark Kindly find my

Re: [EXTERNAL] Any objection to documenting pg_sequence_last_value()?

2021-04-06 Thread James Coleman
On Tue, Mar 30, 2021 at 4:37 AM Hanefi Onaldi wrote: > > Hi All, > > I recently used pg_sequence_last_value() when working on a feature in an > extension, and it would have been easier for me if there were some > documentation for this function. > > I'd like to help document this function if the

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-04-06 Thread Justin Pryzby
On Wed, Dec 23, 2020 at 01:17:10PM -0600, Justin Pryzby wrote: > On Mon, Nov 23, 2020 at 04:14:18PM -0500, Tom Lane wrote: > > * I noticed that you did s/stat/lstat/. That's fine on Unix systems, > > but it won't have any effect on Windows systems (cf bed90759f), > > which means that we'll have to

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-06 Thread Julien Rouhaud
On Tue, Apr 06, 2021 at 11:41:52AM -0400, Alvaro Herrera wrote: > On 2021-Apr-06, Nitin Jadhav wrote: > > > I have reviewed the code. Here are a few minor comments. > > > > 1. > > +void > > +pgstat_report_queryid(uint64 queryId, bool force) > > +{ > > + volatile PgBackendStatus *beentry = MyBEEnt

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-06, Nitin Jadhav wrote: > I have reviewed the code. Here are a few minor comments. > > 1. > +void > +pgstat_report_queryid(uint64 queryId, bool force) > +{ > + volatile PgBackendStatus *beentry = MyBEEntry; > + > + if (!beentry) > + return; > + > + /* > + * if track_activities is disa

Re: psql - add SHOW_ALL_RESULTS option

2021-04-06 Thread Peter Eisentraut
On 14.03.21 10:54, Fabien COELHO wrote: Hello Peter, This reply was two months ago, and nothing has happened, so I have marked the patch as RwF. Given the ongoing work on returning multiple result sets from stored procedures[0], I went to dust off this patch. Based on the feedback, I put

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-06 Thread Julien Rouhaud
On Tue, Apr 06, 2021 at 08:05:19PM +0530, Nitin Jadhav wrote: > > 1. > +void > +pgstat_report_queryid(uint64 queryId, bool force) > +{ > + volatile PgBackendStatus *beentry = MyBEEntry; > + > + if (!beentry) > + return; > + > + /* > + * if track_activities is disabled, st_queryid should already ha

Re: ALTER TABLE ADD COLUMN fast default

2021-04-06 Thread Tom Lane
Andrew Gierth writes: > This gitlab ticket refers to the same incident: > https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6047 > (which actually contains a new relevant fact that hadn't been mentioned > in the IRC discussion, which is that the problem affected multiple > tables, not just one

[PATCH] PREPARE TRANSACTION unexpected behavior with TEMP TABLE

2021-04-06 Thread Himanshu Upadhyaya
Hi, While working on one of the issue, I have noticed below unexpected behavior with "PREPARE TRANSACTION". We are getting this unexpected behavior with PREPARE TRANSACTION when it is mixed with Temporary Objects. Please consider the below setup and SQL block. set max_prepared_transactions to 1

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-06 Thread Nitin Jadhav
I have reviewed the code. Here are a few minor comments. 1. +void +pgstat_report_queryid(uint64 queryId, bool force) +{ + volatile PgBackendStatus *beentry = MyBEEntry; + + if (!beentry) + return; + + /* + * if track_activities is disabled, st_queryid should already have been + * reset + */ + if (

Re: TRUNCATE on foreign table

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 5:36 PM Kazutaka Onishi wrote: > > Thank you for checking v13, and here is v14 patch. cfbot failure on v14 - https://cirrus-ci.com/task/4772360931770368. Looks like it is not related to this patch, please re-confirm it. With Regards, Bharath Rupireddy. EnterpriseDB: http:/

Re: TRUNCATE on foreign table

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 5:36 PM Kazutaka Onishi wrote: > > Thank you for checking v13, and here is v14 patch. > > > 1) Are we using all of these macros? I see that we are setting them > > but we only use TRUNCATE_REL_CONTEXT_ONLY. If not used, can we remove > > them? > > These may be needed for the

Re: New IndexAM API controlling index vacuum strategies

2021-04-06 Thread Matthias van de Meent
On Tue, 6 Apr 2021 at 05:13, Peter Geoghegan wrote: > > On Mon, Apr 5, 2021 at 2:44 PM Matthias van de Meent > wrote: > > This assertion is false, and should be a guarding if-statement. HOT > > redirect pointers are not updated if the tuple they're pointing to is > > vacuumed (i.e. when it was ne

Re: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread Fujii Masao
On 2021/04/06 20:44, David Steele wrote: On 4/6/21 7:13 AM, Fujii Masao wrote: On 2021/04/06 15:59, osumi.takami...@fujitsu.com wrote: I just wanted to write why the error was introduced, but it was not necessary. We should remove and fix the first part of the sentence. So the consensus i

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 6:37 PM Amit Langote wrote: > > 3) will the cmin in the output always be the same? > > +SELECT cmin, * FROM batch_cp_upd_test3; > > TBH, I am not so sure. Maybe it's not a good idea to rely on cmin > after all. I've rewritten the tests to use a different method of > determ

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 6:09 PM Michael Paquier wrote: > > On Mon, Mar 22, 2021 at 10:58:17AM +0530, Mahendra Singh Thalor wrote: > > Your changes look to fine me and I am also not getting any failure. I > > think we should back-patch all the branches. > > > > Patch is applying to all the branches(

Re: [PATCH] New default role allowing to change per-role/database settings

2021-04-06 Thread Michael Banck
Hi, Am Montag, den 05.04.2021, 14:33 -0400 schrieb Stephen Frost: > * Michael Banck (michael.ba...@credativ.de) wrote: > > Am Montag, den 08.03.2021, 20:54 +0500 schrieb Ibrar Ahmed: > > > On Thu, Dec 31, 2020 at 6:16 PM Michael Banck > > > wrote: > > > > in today's world, some DBAs have no supe

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 6:49 PM Bharath Rupireddy wrote: > On Tue, Apr 6, 2021 at 3:08 PM Amit Langote wrote: > > Updated patch attached. I had to adjust the test case a little bit to > > account for the changes of 86dc90056d, something I failed to notice > > yesterday. Also, I expanded the test

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Michael Paquier
On Tue, Apr 06, 2021 at 02:39:54PM +0200, Matthias van de Meent wrote: > On Tue, 6 Apr 2021 at 14:29, Fujii Masao wrote: >> Attached is the updated version of the patch. Barring any objection, >> I'm thinking to commit this. Sorry for the late reply. The approach to use LIMIT TO for this purpose

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Matthias van de Meent
On Tue, 6 Apr 2021 at 14:29, Fujii Masao wrote: > > Thanks! So I applied all the changes that I suggested upthread to the patch. > I also updated the comment as follows. > > * Import table data for partitions only when they are > explicitly > -* specified in LIMIT

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

2021-04-06 Thread Michael Paquier
On Mon, Mar 22, 2021 at 10:58:17AM +0530, Mahendra Singh Thalor wrote: > Your changes look to fine me and I am also not getting any failure. I > think we should back-patch all the branches. > > Patch is applying to all the branches(till v95) and there is no failure. Er, no. This is just some dup

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Fujii Masao
On 2021/04/06 16:05, Amit Langote wrote: On Tue, Apr 6, 2021 at 8:34 AM Fujii Masao wrote: For now I have no objection to this feature. -IMPORT FOREIGN SCHEMA import_source EXCEPT (t1, "x 4", nonesuch) +IMPORT FOREIGN SCHEMA import_source EXCEPT (t1, "x 4", nonesuch, t4_part) Isn't it bette

Re: Reference Leak with type

2021-04-06 Thread Michael Paquier
On Tue, Apr 06, 2021 at 11:09:13AM +0530, Rohit Bhogate wrote: > I found the below reference leak on master. Thanks for the report. This is indeed a new problem as of HEAD, coming from c9d52984 as far as I can see, and 13 does not support this grammar. From what I can see, there seems to be an

Re: TRUNCATE on foreign table

2021-04-06 Thread Kazutaka Onishi
Thank you for checking v13, and here is v14 patch. > 1) Are we using all of these macros? I see that we are setting them > but we only use TRUNCATE_REL_CONTEXT_ONLY. If not used, can we remove > them? These may be needed for the foreign data handler other than postgres_fdw. > 2) Why is this chan

Re: Replication slot stats misgivings

2021-04-06 Thread vignesh C
On Tue, Apr 6, 2021 at 12:19 PM Amit Kapila wrote: > > On Mon, Apr 5, 2021 at 8:51 PM vignesh C wrote: > > > > Few comments on the latest patches: > Comments on 0001 > > 1. > @@ -659,6 +661,8 @@ ReorderBufferTXNByXid(ReorderBuffer *rb, > TransactionId xid, bool cr

RE: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread osumi.takami...@fujitsu.com
On Tuesday, April 6, 2021 8:44 PM David Steele wrote: > On 4/6/21 7:13 AM, Fujii Masao wrote: > > > > On 2021/04/06 15:59, osumi.takami...@fujitsu.com wrote: > >> I just wanted to write why the error was introduced, but it was not > >> necessary. > >> We should remove and fix the first part of the

Re: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread David Steele
On 4/6/21 7:13 AM, Fujii Masao wrote: On 2021/04/06 15:59, osumi.takami...@fujitsu.com wrote: I just wanted to write why the error was introduced, but it was not necessary. We should remove and fix the first part of the sentence. So the consensus is almost the same as the latest patch? If the

Re: pgbench - add pseudo-random permutation function

2021-04-06 Thread Dean Rasheed
On Mon, 5 Apr 2021 at 13:07, Fabien COELHO wrote: > > Attached a v28 which I hope fixes the many above issues and stays with > ints. The randu64 is still some kind of a joke, I artificially reduced the > cost by calling jrand48 once and extending it to 64 bits, so it could give > an idea of the co

Re: Stronger safeguard for archive recovery not to miss data

2021-04-06 Thread Fujii Masao
On 2021/04/06 15:59, osumi.takami...@fujitsu.com wrote: I just wanted to write why the error was introduced, but it was not necessary. We should remove and fix the first part of the sentence. So the consensus is almost the same as the latest patch? If they are not so different, I'm thinking

Re: UniqueKey on Partitioned table.

2021-04-06 Thread David Rowley
On Tue, 6 Apr 2021 at 22:31, Andy Fan wrote: > On Wed, Mar 31, 2021 at 9:12 PM Ashutosh Bapat > wrote: >> I think the reason we add ECs for sort expressions is to use >> transitive relationship. The EC may start with a single member but >> later in the planning that member might find partners wh

Re: UniqueKey on Partitioned table.

2021-04-06 Thread Andy Fan
On Wed, Mar 31, 2021 at 9:12 PM Ashutosh Bapat wrote: > > b). How to present the element > > in UniqueKey. Prue EquivalenceClasses or Mix of Expr and > EquivalenceClass as > > we just talked about. > I think the reason we add ECs for sort expressions is to use > transitive relationship. The EC

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-06 Thread Etsuro Fujita
On Tue, Apr 6, 2021 at 5:45 PM Etsuro Fujita wrote: > Also, we should improve the code to avoid > the consistency mentioned above, Sorry, s/consistency/inconsistency/. > I'll apply the patch. Done. Let's see if this works. Best regards, Etsuro Fujita

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-04-06 Thread Peter Eisentraut
On 06.04.21 07:24, Japin Li wrote: I think this patch is about ready to commit, but please provide a final version in good time. I took the liberty to address all the review comments and provide a v9 patch on top of Japin's v8 patch-set. (Also, please combine your patches into a single patch.)

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 3:08 PM Amit Langote wrote: > > On Mon, Apr 5, 2021 at 1:16 AM Zhihong Yu wrote: > > > > Hi, > > In the description: > > > > cross-partition update of partitioned tables can't use batching > > because ExecInitRoutingInfo() which initializes the insert target > > > > 'which'

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Mon, Apr 5, 2021 at 1:16 AM Zhihong Yu wrote: > > Hi, > In the description: > > cross-partition update of partitioned tables can't use batching > because ExecInitRoutingInfo() which initializes the insert target > > 'which' should be dropped since 'because' should start a sentence. > > +-- Chec

Re: Confusing behavior of psql's \e

2021-04-06 Thread Laurenz Albe
On Sat, 2021-04-03 at 17:43 -0400, Tom Lane wrote: > Laurenz Albe writes: > > Attached is version 6. > > Pushed with some mostly-cosmetic fiddling. > > One thing I changed that wasn't cosmetic is that as you had it, > the behavior of "\e file" varied depending on whether the query > buffer had b

Re: Key management with tests

2021-04-06 Thread Neil Chen
Hi Bruce, I went through these patches and executed the test script you added for the KMS section, which looks all good. This is a point that looks like a bug - in patch 10, you changed the location and use of *RelFileNodeSkippingWAL()*, but the modified code logic seems different from the origin

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-06 Thread Etsuro Fujita
On Tue, Apr 6, 2021 at 12:01 PM Kyotaro Horiguchi wrote: > At Mon, 5 Apr 2021 17:15:47 +0900, Etsuro Fujita > wrote in > > On Fri, Apr 2, 2021 at 12:09 AM Tom Lane wrote: > > > The buildfarm points out that this fails under valgrind. > > > I easily reproduced it here: > > > > > > ==00:00:03:42.

Re: Idea: Avoid JOINs by using path expressions to follow FKs

2021-04-06 Thread Joel Jacobson
In a Hacker News discussion [2] on using foreign keys for joins, the author of PostgREST, Steve Chavez, mentioned they are actually already using this idea in PostgREST: Steve Chavez wrote: >The idea about using FK as a JOIN target is interesting. >While developing a syntax for PostgREST resource

Re: A reloption for partitioned tables - parallel_workers

2021-04-06 Thread Amit Langote
On Fri, Apr 2, 2021 at 11:36 PM Laurenz Albe wrote: > On Wed, 2021-03-24 at 14:14 +1300, David Rowley wrote: > > On Fri, 19 Mar 2021 at 02:07, Amit Langote wrote: > > > Attached a new version rebased over c8f78b616, with the grouping > > > relation partitioning enhancements as a separate patch 00

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 8:34 AM Fujii Masao wrote: > For now I have no objection to this feature. > > -IMPORT FOREIGN SCHEMA import_source EXCEPT (t1, "x 4", nonesuch) > +IMPORT FOREIGN SCHEMA import_source EXCEPT (t1, "x 4", nonesuch, t4_part) > > Isn't it better to create also another partition l

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Amit Langote
Hi Matthias, On Wed, Mar 24, 2021 at 9:23 PM Matthias van de Meent wrote: > On Mon, 22 Mar 2021 at 21:16, Bernd Helmle wrote: > > The patch changes IMPORT FOREIGN SCHEMA to explicitely allow partition > > child tables in the LIMIT TO clause of the IMPORT FOREIGN SCHEMA > > command by relaxing th