EXEC_BACKEND vs bgworkers without BGWORKER_SHMEM_ACCESS

2021-08-01 Thread Andres Freund
Hi, While looking at a patch I noticed that SubPostmasterMain() for bgworkers unconditionally does /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */ InitProcess(); which presents a problem, because StartBackgroundWorker() then does /* *

RE: Parallel Inserts (WAS: [bug?] Missed parallel safety checks..)

2021-08-01 Thread houzj.f...@fujitsu.com
On August 2, 2021 2:04 PM Greg Nancarrow wrote: > On Mon, Aug 2, 2021 at 2:52 PM Amit Kapila wrote: > > > > On Fri, Jul 30, 2021 at 6:53 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Friday, July 30, 2021 2:52 PM Greg Nancarrow > wrote: > > > > On Fri, Jul 30, 2021 at 4:02 PM Amit Kapila

Re: Skip partition tuple routing with constant partition key

2021-08-01 Thread Amit Langote
I noticed that there is no CF entry for this, so created one in the next CF: https://commitfest.postgresql.org/34/3270/ Rebased patch attached. v8-0001-Teach-get_partition_for_tuple-to-cache-bound-offs.patch Description: Binary data

Re: Parallel Inserts (WAS: [bug?] Missed parallel safety checks..)

2021-08-01 Thread Greg Nancarrow
On Mon, Aug 2, 2021 at 2:52 PM Amit Kapila wrote: > > On Fri, Jul 30, 2021 at 6:53 PM houzj.f...@fujitsu.com > wrote: > > > > On Friday, July 30, 2021 2:52 PM Greg Nancarrow wrote: > > > On Fri, Jul 30, 2021 at 4:02 PM Amit Kapila > > > wrote: > > > > > > > > > Besides, I think we need a new d

RE: Failed transaction statistics to measure the logical replication progress

2021-08-01 Thread osumi.takami...@fujitsu.com
On Thursday, July 29, 2021 10:50 AM Masahiko Sawada wrote: > On Thu, Jul 8, 2021 at 3:55 PM osumi.takami...@fujitsu.com > wrote: > > When the current HEAD fails during logical decoding, the failure > > increments txns count in pg_stat_replication_slots - [1] and adds > > the transaction size

Re: Background writer and checkpointer in crash recovery

2021-08-01 Thread Thomas Munro
On Sat, Jul 31, 2021 at 2:16 AM Robert Haas wrote: > On Fri, Jul 30, 2021 at 4:42 AM Aleksander Alekseev > wrote: > > v2-0001 and v2-0002 look fine, but I don't like much the idea of > > introducing a new GUC in v2-0003. It's for very specific needs, which most > > of the users, I believe, don'

Re: Parallel Inserts (WAS: [bug?] Missed parallel safety checks..)

2021-08-01 Thread Amit Kapila
On Fri, Jul 30, 2021 at 6:53 PM houzj.f...@fujitsu.com wrote: > > On Friday, July 30, 2021 2:52 PM Greg Nancarrow wrote: > > On Fri, Jul 30, 2021 at 4:02 PM Amit Kapila wrote: > > > > > > > Besides, I think we need a new default value about parallel dml > > > > safety. Maybe 'auto' or 'null'(dif

Re: Speeding up GIST index creation for tsvectors

2021-08-01 Thread Amit Khandekar
On Sat, 20 Mar 2021 at 02:19, John Naylor wrote: > On Fri, Mar 19, 2021 at 8:57 AM Amit Khandekar wrote: > > Regarding the alignment changes... I have removed the code that > > handled the leading identically unaligned bytes, for lack of evidence > > that percentage of such cases is significant.

Re: Skipping logical replication transactions on subscriber side

2021-08-01 Thread Amit Kapila
On Mon, Aug 2, 2021 at 7:45 AM Masahiko Sawada wrote: > > On Fri, Jul 30, 2021 at 12:52 PM Amit Kapila wrote: > > > > On Thu, Jul 29, 2021 at 11:18 AM Masahiko Sawada > > wrote: > > > > Setting up logical rep error context in a generic function looks a bit > > odd to me. Do we really need to se

Re: Record a Bitmapset of non-pruned partitions

2021-08-01 Thread Amit Langote
On Sun, Aug 1, 2021 at 9:31 PM David Rowley wrote: > On Fri, 30 Jul 2021 at 19:10, Amit Langote wrote: > > interleaved_parts idea looks clever. I wonder if you decided that > > it's maybe not worth setting that field in the joinrel's > > PartitionBoundInfos? For example, adding the code that yo

Re: Skipping logical replication transactions on subscriber side

2021-08-01 Thread Masahiko Sawada
On Fri, Jul 30, 2021 at 3:47 PM houzj.f...@fujitsu.com wrote: > > On July 29, 2021 1:48 PM Masahiko Sawada wrote: > > > > Sorry I've attached wrong ones. Reattached the correct version patches. > > Hi, > > I had some comments on the new version patches. Thank you for the comments! > > 1) > > -

Re: Skipping logical replication transactions on subscriber side

2021-08-01 Thread Masahiko Sawada
On Fri, Jul 30, 2021 at 12:52 PM Amit Kapila wrote: > > On Thu, Jul 29, 2021 at 11:18 AM Masahiko Sawada > wrote: > > > > On Thu, Jul 29, 2021 at 2:04 PM Masahiko Sawada > > wrote: > > > > > > > Yeah, it seems to be introduced by commit 0926e96c493. I've attached > > > > the patch for that. >

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-01 Thread Tom Lane
I wrote: > ... aside from the question of whether > a too-large subexpression number should be an error or not. Oh ... poking around some more, I noticed a very nearby precedent. regexp_replace's replacement string can include \1 to \9 to insert the substring matching the N'th parenthesized subexp

Re: amcheck verification for GiST and GIN

2021-08-01 Thread Nikolay Samokhvalov
Thank you, v5 didn't find any issues at all. One thing: for my 29 indexes, the tool generated output 3.5 GiB. I guess many INFO messages should be downgraded to something like DEBUG1? On Fri, Jul 30, 2021 at 2:35 AM Heikki Linnakangas wrote: > On 29/07/2021 21:34, Nikolay Samokhvalov wrote: > >

Re: A qsort template

2021-08-01 Thread Thomas Munro
On Mon, Aug 2, 2021 at 12:40 PM Thomas Munro wrote: > Great! I saw similar sorts of numbers. It's really just a few > crumbs, nothing compared to the gains David just found over in the > thread "Use generation context to speed up tuplesorts", but on the > bright side, these crumbs will be magnif

Re: A qsort template

2021-08-01 Thread Thomas Munro
On Fri, Jul 30, 2021 at 12:34 PM John Naylor wrote: > I got around to getting a benchmark together to serve as a starting point. I > based it off something I got from the archives, but don't remember where (I > seem to remember Tomas Vondra wrote the original, but not sure). To start I > just u

Re: A qsort template

2021-08-01 Thread Thomas Munro
On Fri, Jul 30, 2021 at 7:11 PM Peter Geoghegan wrote: > If you're going to specialize the sort routine for unsigned integer > style abbreviated keys then you might as well cover all relevant > opclasses/types. Almost all abbreviated key schemes produce > conditioned datums that are designed to us

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-01 Thread Tom Lane
Gilles Darold writes: > [ v5-0001-regexp-foo-functions.patch ] I've gone through this whole patch now, and found quite a lot that I did not like. In no particular order: * Wrapping parentheses around the user's regexp doesn't work. It can turn an invalid regexp into a valid one: for example 'a

Re: Corrected documentation of data type for the logical replication message formats.

2021-08-01 Thread Peter Smith
On Mon, Aug 2, 2021 at 1:32 AM vignesh C wrote: > > On Sun, Aug 1, 2021 at 4:11 PM Peter Smith wrote: > > > > On Sat, Jul 31, 2021 at 7:00 AM Tom Lane wrote: > > > > > > vignesh C writes: > > > [ v6-0001-Included-the-actual-datatype-used-in-logical-repl.patch ] > > > > > > I see what you want t

Re: Record a Bitmapset of non-pruned partitions

2021-08-01 Thread David Rowley
Thanks for having a look at this. On Mon, 2 Aug 2021 at 02:33, Zhihong Yu wrote: > Here we look for partitions which > +* might be interleaved with other partitions and set the > +* interleaved_parts field with the partition indexes of any partitions > +* which may be interleaved with

Re: slab allocator performance issues

2021-08-01 Thread Tomas Vondra
On 8/1/21 11:07 PM, Andres Freund wrote: Hi, On 2021-08-01 19:59:18 +0200, Tomas Vondra wrote: In the attached .ods file with results, the "comparison" sheets are the interesting ones - the last couple columns compare the main metrics for the two patches (labeled patch-1 and patch-2) to maste

Re: Replace l337sp34k in comments.

2021-08-01 Thread Andrew Dunstan
On 8/1/21 5:10 PM, Andres Freund wrote: > Hi, > > On 2021-07-31 12:15:34 +0300, Peter Geoghegan wrote: >> On Sat, Jul 31, 2021 at 11:22 AM Andrey Borodin wrote: >>> FWIW, my 2 cents. >>> I do not see much difference between up2date, up-to-date, up to date, >>> current, recent, actual, last, new

Re: Replace l337sp34k in comments.

2021-08-01 Thread Andres Freund
Hi, On 2021-07-31 12:15:34 +0300, Peter Geoghegan wrote: > On Sat, Jul 31, 2021 at 11:22 AM Andrey Borodin wrote: > > FWIW, my 2 cents. > > I do not see much difference between up2date, up-to-date, up to date, > > current, recent, actual, last, newest, correct, fresh etc. > > +1. > To me it se

Re: slab allocator performance issues

2021-08-01 Thread Andres Freund
Hi, On 2021-08-01 19:59:18 +0200, Tomas Vondra wrote: > In the attached .ods file with results, the "comparison" sheets are the > interesting ones - the last couple columns compare the main metrics for the > two patches (labeled patch-1 and patch-2) to master. I assume with patch-1/2 you mean the

pgstat_send_connstats() introduces unnecessary timestamp and UDP overhead

2021-08-01 Thread Andres Freund
Hi, Since commit 960869da0803427d14335bba24393f414b476e2c Author: Magnus Hagander Date: 2021-01-17 13:34:09 +0100 Add pg_stat_database counters for sessions and session time pgstat_report_stat() does another timestamp computation via pgstat_send_connstats(), despite typically having comp

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-01 Thread Gilles Darold
Le 01/08/2021 à 19:23, Tom Lane a écrit : > I've been working through this patch, and trying to verify > compatibility against Oracle and DB2, and I see some points that need > discussion or at least recording for the archives. > > * In Oracle, while the documentation for regexp_instr says that > r

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-01 Thread Gilles Darold
Le 30/07/2021 à 23:38, Tom Lane a écrit : > Gilles Darold writes: >> Le 26/07/2021 à 21:56, Tom Lane a écrit : >>> I'm inclined to just drop the regexp_replace additions. I don't think >>> that the extra parameters Oracle provides here are especially useful. >>> They're definitely not useful enou

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-01 Thread Tom Lane
I've been working through this patch, and trying to verify compatibility against Oracle and DB2, and I see some points that need discussion or at least recording for the archives. * In Oracle, while the documentation for regexp_instr says that return_option should only be 0 or 1, experimentation w

Re: Corrected documentation of data type for the logical replication message formats.

2021-08-01 Thread vignesh C
On Sun, Aug 1, 2021 at 4:11 PM Peter Smith wrote: > > On Sat, Jul 31, 2021 at 7:00 AM Tom Lane wrote: > > > > vignesh C writes: > > [ v6-0001-Included-the-actual-datatype-used-in-logical-repl.patch ] > > > > I see what you want to do here, but the way you did it seems quite > > detrimental to th

Re: Corrected documentation of data type for the logical replication message formats.

2021-08-01 Thread vignesh C
On Sat, Jul 31, 2021 at 2:30 AM Tom Lane wrote: > > vignesh C writes: > [ v6-0001-Included-the-actual-datatype-used-in-logical-repl.patch ] > > I see what you want to do here, but the way you did it seems quite > detrimental to the readability of the field descriptions. > Parenthesized interjecti

Re: Record a Bitmapset of non-pruned partitions

2021-08-01 Thread Zhihong Yu
On Sun, Aug 1, 2021 at 5:31 AM David Rowley wrote: > On Fri, 30 Jul 2021 at 19:10, Amit Langote > wrote: > > > > 0001 looks mostly fine, except I thought the following could be worded > > to say that the bitmap members are offsets into the part_rels array. > > To avoid someone confusing them wit

Re: Record a Bitmapset of non-pruned partitions

2021-08-01 Thread David Rowley
On Fri, 30 Jul 2021 at 19:10, Amit Langote wrote: > > 0001 looks mostly fine, except I thought the following could be worded > to say that the bitmap members are offsets into the part_rels array. > To avoid someone confusing them with RT indexes, for example. > > + Bitmapset *live_parts; /*

Re: Corrected documentation of data type for the logical replication message formats.

2021-08-01 Thread Peter Smith
On Sat, Jul 31, 2021 at 7:00 AM Tom Lane wrote: > > vignesh C writes: > [ v6-0001-Included-the-actual-datatype-used-in-logical-repl.patch ] > > I see what you want to do here, but the way you did it seems quite > detrimental to the readability of the field descriptions. > Parenthesized interjecti

Re: [HACKERS] logical decoding of two-phase transactions

2021-08-01 Thread Peter Smith
On Sun, Aug 1, 2021 at 3:05 AM vignesh C wrote: > > On Sat, Jul 31, 2021 at 11:12 AM Ajin Cherian wrote: > > > > On Sat, Jul 31, 2021 at 2:39 PM Amit Kapila wrote: > > > > > Here, the test is expecting 2 prepared transactions corresponding to > > > two subscriptions but it waits for just one sub