use SSE2 for is_valid_ascii

new thread [was: WIP Patch: Add a function that returns binary JSONB as a bytea] > I wrote: > > We can also shave a > > few percent by having pg_utf8_verifystr use SSE2 for the ascii path. I > > can look into this. > > Here's a patch for that. If the input is mostly ascii, I'd expect that > part o

Re: Reducing the chunk header sizes on all memory context types

On Wed, 10 Aug 2022 at 06:44, Andres Freund wrote: > I think it's fine, given that we can change this at any time, but it's > probably worth to explicitly agree that this will for now restrict us to 8 > context methods? I know there was some discussion about this elsewhere in this thread about 8

createuser doesn't tell default settings for some options

(I suppose this is a pg15 issue) createuser --help shows the following help text. > --bypassrls role can bypass row-level security (RLS) policy > --no-bypassrlsrole cannot bypass row-level security (RLS) policy > --replication role can initiate replication

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

On Wed, Aug 10, 2022 at 5:35 PM Thomas Munro wrote: > 2021 Or, rather, 14 days into 2022 :-)

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

On Wed, Aug 10, 2022 at 5:28 PM Andres Freund wrote: > I don't think it's a safe assumption that nobody would hold a pin on such a > page during recovery. While not the case here, somebody else could have used > pg_prewarm to read it in. > > But also, the checkpointer or bgwriter could have it tem

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

Hi, On 2022-08-09 20:21:19 -0700, Mark Dilger wrote: > > On Aug 9, 2022, at 7:26 PM, Andres Freund wrote: > > > > The relevant code triggering it: > > > > newbuf = XLogInitBufferForRedo(record, 1); > > _hash_initbuf(newbuf, xlrec->new_bucket, xlrec->new_bucket, > >

Blocking the use of TRIGGER privilege

The separate TRIGGER privilege is considered obsolescent. It is not heavily used and exists mainly to facilitate trigger-based replication in a multi-user system. i.e. GRANT TRIGGER ON foo TO bob; Since logical replication recommends "Limit ownership and TRIGGER privilege on such tables to trusted

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Sun, Aug 7, 2022 at 9:47 AM Andres Freund wrote: > > Hi, > > On 2022-08-07 09:24:40 +0530, Dilip Kumar wrote: > > On Sat, Aug 6, 2022 at 9:36 PM Tom Lane wrote: > > > > > > Dilip Kumar writes: > > > > On Fri, Aug 5, 2022 at 10:43 AM Dilip Kumar > > > > wrote: > > > >> Yeah maybe it is not n

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

On Wed, Aug 10, 2022 at 3:21 PM Mark Dilger wrote: > > On Aug 9, 2022, at 7:26 PM, Andres Freund wrote: > > The relevant code triggering it: > > > > newbuf = XLogInitBufferForRedo(record, 1); > > _hash_initbuf(newbuf, xlrec->new_bucket, xlrec->new_bucket, > >

Re: dropping datumSort field

On Tue, Aug 9, 2022 at 9:04 PM David Rowley wrote: > On Wed, 10 Aug 2022 at 03:16, Zhihong Yu wrote: > > On Tue, Aug 9, 2022 at 8:01 AM Robert Haas > wrote: > >> > >> One problem with this patch is that, if I apply it, PostgreSQL does not > compile: > >> > >> nodeSort.c:197:6: error: use of und

Re: dropping datumSort field

On Wed, 10 Aug 2022 at 03:16, Zhihong Yu wrote: > On Tue, Aug 9, 2022 at 8:01 AM Robert Haas wrote: >> >> One problem with this patch is that, if I apply it, PostgreSQL does not >> compile: >> >> nodeSort.c:197:6: error: use of undeclared identifier 'tupDesc' >> if (tupDesc->natts == 1)

Re: optimize lookups in snapshot [sub]xip arrays

On Wed, Aug 10, 2022 at 7:13 AM Nathan Bossart wrote: > > On Tue, Aug 09, 2022 at 01:00:37PM -0700, Nathan Bossart wrote: > > Your adjustments in 0002 seem reasonable to me. I think it makes sense to > > ensure there is test coverage for pg_lfind32(), but I don't know if that > > syscache code is

Re: Perform streaming logical transactions by background workers and parallel apply

On Tue, Aug 9, 2022 at 5:39 PM Amit Kapila wrote: > > On Tue, Aug 9, 2022 at 11:09 AM Dilip Kumar wrote: > > > > Some more comments > > > > +/* > > + * Exit if any relation is not in the READY state and if any worker is > > + * handling the streaming transaction at the same time. Beca

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

> On Aug 9, 2022, at 7:26 PM, Andres Freund wrote: > > The relevant code triggering it: > > newbuf = XLogInitBufferForRedo(record, 1); > _hash_initbuf(newbuf, xlrec->new_bucket, xlrec->new_bucket, > xlrec->new_bucket_flag, true); > if (!IsBuff

Re: shared-memory based stats collector - v70

At Tue, 9 Aug 2022 09:53:19 -0700, Andres Freund wrote in > Hi, > > On 2022-08-09 17:24:35 +0900, Kyotaro Horiguchi wrote: > > If I'm not missing something, it's strange that pgstat_lock_entry() > > only takes LW_EXCLUSIVE. > > I think it makes some sense, given that there's a larger number of

hash_xlog_split_allocate_page: failed to acquire cleanup lock

Hi, One CI run for the meson branch just failed in a way I hadn't seen before on windows, when nothing had changed on windows https://cirrus-ci.com/task/6111743586861056 027_stream_regress.pl ended up failing due to a timeout. Which in turn was caused by the standby crashing. 2022-08-10 01:46:2

Re: optimize lookups in snapshot [sub]xip arrays

On Wed, Aug 10, 2022 at 5:00 AM Nathan Bossart wrote: > > On Tue, Aug 09, 2022 at 01:21:41PM +0700, John Naylor wrote: > > I decided I wasn't quite comfortable changing snapshot handling > > without further guarantees. To this end, 0002 in the attached v11 is > > an addendum that adds assert chec

RE: Perform streaming logical transactions by background workers and parallel apply

Hi Wang, > 6.a > > It seems that the upper line represents the apply background worker, but I > think > last_msg_send_time and last_msg_receipt_time should be null. > Is it like initialization mistake? I checked again about the issue. Attributes worker->last_send_time, worker->last_recv_time,

Re: Windows now has fdatasync()

David kindly ran some tests of this thing on real hardware. The results were mostly in line with expectations, but we learned some new things. TL;DR We probably should consider this as a safer default, but it'd be good for someone more hands-on with this OS and knowledgeable about storage to inv

Re: pg_get_constraintdef: Schema qualify foreign tables unless pretty printing is enabled

On Tue, Aug 9, 2022 at 5:33 PM Tom Lane wrote: > I would say that pg_get_indexdef is the one that's out of step. > I count 11 calls of generate_relation_name in ruleutils.c, > of which only three have this business of being overridden > when not-pretty. What is the rationale for that, and why >

Re: pg_get_constraintdef: Schema qualify foreign tables unless pretty printing is enabled

Lukas Fittl writes: > Whilst debugging an issue with the output of pg_get_constraintdef, we've > discovered that pg_get_constraintdef doesn't schema qualify foreign tables > mentioned in the REFERENCES clause, even if pretty printing > (PRETTYFLAG_SCHEMA) is turned off. > This is a problem becaus

Re: optimize lookups in snapshot [sub]xip arrays

On Tue, Aug 09, 2022 at 01:00:37PM -0700, Nathan Bossart wrote: > Your adjustments in 0002 seem reasonable to me. I think it makes sense to > ensure there is test coverage for pg_lfind32(), but I don't know if that > syscache code is the right choice. For non-USE_SSE2 builds, it might make > thes

pg_get_constraintdef: Schema qualify foreign tables unless pretty printing is enabled

Hi hackers, Whilst debugging an issue with the output of pg_get_constraintdef, we've discovered that pg_get_constraintdef doesn't schema qualify foreign tables mentioned in the REFERENCES clause, even if pretty printing (PRETTYFLAG_SCHEMA) is turned off. This is a problem because it means there i

Re: Data is copied twice when specifying both child and parent table in publication

Here are some more review comments for the HEAD_v8 patch: == 1. Commit message If there are two publications, one of them publish a parent table with (publish_via_partition_root = true) and another publish child table, subscribing to both publications from one subscription results in two ini

Re: Reducing the chunk header sizes on all memory context types

Thanks for giving this a look. On Wed, 10 Aug 2022 at 02:37, Robert Haas wrote: > # We also add a restriction that block sizes for all 3 of the memory > # allocators cannot be 1GB or larger. We would be unable to store the > # number of bytes that the block is offset from the chunk stored beyond

Re: Reducing the chunk header sizes on all memory context types

David Rowley writes: > On Wed, 10 Aug 2022 at 09:23, Tom Lane wrote: >> Hmm, I suppose you mean we could reduce 4) if we needed to. Yeah, that >> seems like a reasonable place to buy more bits later if we run out of >> MemoryContextMethodIDs. Should be fine then. > I think he means 3). If 4)

Re: Reducing the chunk header sizes on all memory context types

On Wed, 10 Aug 2022 at 09:23, Tom Lane wrote: > > Andres Freund writes: > > On 2022-08-09 15:21:57 -0400, Tom Lane wrote: > >> Do we really need it to be that tight? I know we only have 3 methods > >> today, > >> but 8 doesn't seem that far away. If there were six bits reserved for > >> this I

Re: Reducing the chunk header sizes on all memory context types

Andres Freund writes: > On 2022-08-09 15:21:57 -0400, Tom Lane wrote: >> Do we really need it to be that tight? I know we only have 3 methods today, >> but 8 doesn't seem that far away. If there were six bits reserved for >> this I'd be happier. > We only have so many bits available, so that'd

Re: Reducing the chunk header sizes on all memory context types

Hi, On 2022-08-09 15:21:57 -0400, Tom Lane wrote: > Andres Freund writes: > > I think it's fine, given that we can change this at any time, but it's > > probably worth to explicitly agree that this will for now restrict us to 8 > > context methods? > > Do we really need it to be that tight? I k

Re: SQL/JSON features for v15

On 8/9/22 4:58 PM, Jonathan S. Katz wrote: We're looking for additional input on what makes sense as a best course of action, given what is presented in[3]. Missed adding Amit on the CC. Jonathan OpenPGP_signature Description: OpenPGP digital signature

SQL/JSON features for v15

Hi, (Personal hat, not RMT hat unless otherwise noted). This thread[1] raised some concerns around the implementation of the SQL/JSON features that are slated for v15, which includes an outstanding open item[2]. Given the current state of the discussion, when the RMT met on Aug 8, they severa

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 2022-08-09 Tu 16:19, Jonathan S. Katz wrote: > On 8/9/22 4:15 PM, Andrew Dunstan wrote: >> >> On 2022-08-09 Tu 15:50, Jonathan S. Katz wrote: >>> On 8/9/22 3:22 PM, Andres Freund wrote: > >>> OTOH, it's not a great sign  this is around json again... >>> >>> Yeah, I was thinking about that

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 8/9/22 4:15 PM, Andrew Dunstan wrote: On 2022-08-09 Tu 15:50, Jonathan S. Katz wrote: On 8/9/22 3:22 PM, Andres Freund wrote: OTOH, it's not a great sign  this is around json again... Yeah, I was thinking about that too. Ouch :-( I think after 10 years of being involved with our J

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 2022-08-09 Tu 15:50, Jonathan S. Katz wrote: > On 8/9/22 3:22 PM, Andres Freund wrote: >> Hi, >> >> On 2022-08-09 15:17:44 -0400, Tom Lane wrote: >>> We have delayed releases for $COOL_FEATURE in the past, and I think >>> our batting average on that is still .000: not once has it worked out >>

Re: optimize lookups in snapshot [sub]xip arrays

On Tue, Aug 09, 2022 at 01:21:41PM +0700, John Naylor wrote: > I decided I wasn't quite comfortable changing snapshot handling > without further guarantees. To this end, 0002 in the attached v11 is > an addendum that adds assert checking (also pgindent and some > comment-smithing). As I suspected,

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 8/9/22 2:57 PM, Andres Freund wrote: Hi, On 2022-08-09 14:04:48 -0400, Jonathan S. Katz wrote:   2. Recommend holding up the v15 release to allow for the code to be redesigned and fixed (as based on Andres' estimates, this would push the release out several months). Obviously that's a

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 8/9/22 3:22 PM, Andres Freund wrote: Hi, On 2022-08-09 15:17:44 -0400, Tom Lane wrote: We have delayed releases for $COOL_FEATURE in the past, and I think our batting average on that is still .000: not once has it worked out well. I think it semi worked when jsonb (?) first went in - it to

Re: Reducing the chunk header sizes on all memory context types

On 2022-Aug-09, Andres Freund wrote: > Mildly wondering whether we ought to use designated initializers instead, > given we're whacking it around already. Too easy to get the order wrong when > adding new members, and we might want to have optional callbacks too. Strong +1. It makes code much ea

Re: moving basebackup code to its own directory

On Tue, Aug 9, 2022 at 2:40 PM Justin Pryzby wrote: > It looks like this updates the header comments in the .h files but not the .c > files. > > Personally, I find these to be silly boilerplate .. Here is a version with some updates to the silly boilerplate. -- Robert Haas EDB: http://www.enter

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

Hi, On 2022-08-09 15:17:44 -0400, Tom Lane wrote: > We have delayed releases for $COOL_FEATURE in the past, and I think > our batting average on that is still .000: not once has it worked out > well. I think it semi worked when jsonb (?) first went in - it took a while and a lot of effort from a

Re: Reducing the chunk header sizes on all memory context types

Andres Freund writes: > I think it's fine, given that we can change this at any time, but it's > probably worth to explicitly agree that this will for now restrict us to 8 > context methods? Do we really need it to be that tight? I know we only have 3 methods today, but 8 doesn't seem that far a

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

"Jonathan S. Katz" writes: > Speaking personally, I would like to see what we could do to include > support for this batch of the SQL/JSON features in v15. What is included > looks like it closes most of the gap on what we've been missing > syntactically since the standard was adopted, and the

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

Hi, On 2022-08-09 14:04:48 -0400, Jonathan S. Katz wrote: > On 8/9/22 11:03 AM, Andrew Dunstan wrote: > > > > On 2022-08-09 Tu 09:59, Jonathan S. Katz wrote: > > > > The RMT met today to discuss the state of this open item surrounding > > > the SQL/JSON feature set. We discussed the specific con

Re: moving basebackup code to its own directory

On 8/9/22 14:40, Justin Pryzby wrote: On Tue, Aug 09, 2022 at 01:32:49PM -0400, Robert Haas wrote: On Tue, Aug 9, 2022 at 12:43 PM Magnus Hagander wrote: So maybe src/backend/backup? Or is that too grandiose for the amount of stuff we have here? +1 for src/backend/backup. I'd also be happy t

Re: Reducing the chunk header sizes on all memory context types

Hi, On 2022-08-09 10:36:57 -0400, Robert Haas wrote: > On Tue, Aug 9, 2022 at 8:53 AM David Rowley wrote: > > I think the patch is now starting to take shape. I've added it to the > > September commitfest [1]. > > This is extremely cool. The memory savings are really nice. +1 > And I also li

Re: moving basebackup code to its own directory

On Tue, Aug 09, 2022 at 01:32:49PM -0400, Robert Haas wrote: > On Tue, Aug 9, 2022 at 12:43 PM Magnus Hagander wrote: > >> > So maybe src/backend/backup? Or is that too grandiose for the amount > >> > of stuff we have here? > >> > >> +1 for src/backend/backup. I'd also be happy to see the start/st

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

On Tue, Aug 09, 2022 at 05:26:39PM +1200, Thomas Munro wrote: > The changes were not from elog-LOG to elog-ERROR, they were changes > from silently eating errors, but I take your (plural) general point. I think this was in reference to 0002. My comment was, at least. > OK, so there aren't many p

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 8/9/22 11:03 AM, Andrew Dunstan wrote: On 2022-08-09 Tu 09:59, Jonathan S. Katz wrote: The RMT met today to discuss the state of this open item surrounding the SQL/JSON feature set. We discussed the specific concerns raised about the code and debated four different options:   1. Do nothi

Re: moving basebackup code to its own directory

On 8/9/22 13:32, Robert Haas wrote: On Tue, Aug 9, 2022 at 12:43 PM Magnus Hagander wrote: So maybe src/backend/backup? Or is that too grandiose for the amount of stuff we have here? +1 for src/backend/backup. I'd also be happy to see the start/stop code move here at some point. Yeah, sound

Re: dropping datumSort field

On Tue, Aug 9, 2022 at 11:42 AM Zhihong Yu wrote: > Though the datumSort field may be harmless for now, in the future, the (auto) > alignment padding may not work if more fields are added to the struct. > I think we should always leave some room in struct's for future expansion. I doubt the size

Re: moving basebackup code to its own directory

On Tue, Aug 9, 2022 at 12:43 PM Magnus Hagander wrote: >> > So maybe src/backend/backup? Or is that too grandiose for the amount >> > of stuff we have here? >> >> +1 for src/backend/backup. I'd also be happy to see the start/stop code >> move here at some point. > > Yeah, sounds reasonable. There'

Re: shared-memory based stats collector - v70

Hi, On 2022-08-09 17:24:35 +0900, Kyotaro Horiguchi wrote: > At Mon, 8 Aug 2022 11:53:19 -0400, Greg Stark wrote in > > I'm trying to wrap my head around the shared memory stats collector > > infrastructure from > > <20220406030008.2qxipjxo776dw...@alap3.anarazel.de> committed in > > 5891c7a8ed8f

Re: shared-memory based stats collector - v70

Hi, On 2022-08-09 12:00:46 -0400, Greg Stark wrote: > I was more aiming at a C function that extensions could use directly > rather than an SQL function -- though I suppose having the former it > would be simple enough to implement the latter using it. (though it > would have to be one for each st

Re: moving basebackup code to its own directory

On Tue, Aug 9, 2022 at 6:41 PM David Steele wrote: > On 8/9/22 12:34, Robert Haas wrote: > > On Tue, Aug 9, 2022 at 12:12 PM Magnus Hagander > wrote: > > > >> Anyway, I have no objection. If there'd been that many files, or plans > to have it, in the beginning we probably would've put them in >

Re: moving basebackup code to its own directory

On 8/9/22 12:34, Robert Haas wrote: On Tue, Aug 9, 2022 at 12:12 PM Magnus Hagander wrote: Anyway, I have no objection. If there'd been that many files, or plans to have it, in the beginning we probably would've put them in replication/basebackup or something like that from the beginning. I'

Re: shared-memory based stats collector - v70

Hi, On 2022-08-09 12:18:47 +0200, Drouvot, Bertrand wrote: > What do you think about adding a function in core PG to provide such > functionality? (means being able to retrieve all the stats (+ eventually add > some filtering) without the need to connect to each database). I'm not that convinced

Re: moving basebackup code to its own directory

On 8/9/22 12:12, Magnus Hagander wrote: On Tue, Aug 9, 2022 at 6:08 PM Robert Haas > wrote: Hi, I was thinking that it might make sense, to reduce clutter, to move *backup*.c from src/backend/replication to a new directory, perhaps src/backend/repli

Re: moving basebackup code to its own directory

On Tue, Aug 9, 2022 at 12:12 PM Magnus Hagander wrote: > Those 11 files are mostly your fault, of course ;) They are. I tend to prefer smaller source files than many developers, because I find them easier to understand and maintain. If you only include in basebackup_gzip.c, then you can be prett

Re: remove useless comments

Fair enough, the rephased version of the comments is in the attachment, please take a look. --- a/src/backend/postmaster/postmaster.c +++ b/src/backend/postmaster/postmaster.c @@ -1583,8 +1583,8 @@ getInstallationPaths(const char *argv0) FreeDir(pdir); /* - * XXX is it worth similarly checkin

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

On Tue, 9 Aug 2022 at 12:39, Dilip Kumar wrote: > > This short patchset compiles and passes make check-world, with lengthy > > comments. > > Does this patch set work independently or it has dependency on the > patches on the other thread "Smoothing the subtrans performance > catastrophe"? Patch

Re: moving basebackup code to its own directory

On Tue, Aug 9, 2022 at 6:08 PM Robert Haas wrote: > Hi, > > I was thinking that it might make sense, to reduce clutter, to move > *backup*.c from src/backend/replication to a new directory, perhaps > src/backend/replication/backup or src/backend/backup. > > There's no particular reason we *have*

moving basebackup code to its own directory

Hi, I was thinking that it might make sense, to reduce clutter, to move *backup*.c from src/backend/replication to a new directory, perhaps src/backend/replication/backup or src/backend/backup. There's no particular reason we *have* to do this, but there are 21 C files in that directory and 11 of

Re: Fast COPY FROM based on batch insert

On Tue, Aug 9, 2022 at 4:45 AM Etsuro Fujita wrote: > On Tue, Jul 26, 2022 at 7:19 PM Etsuro Fujita > wrote: > > Yeah, I think the patch is getting better, but I noticed some issues, > > so I'm working on them. I think I can post a new version in the next > > few days. > > * When running AFTER

Re: shared-memory based stats collector - v70

On Tue, 9 Aug 2022 at 06:19, Drouvot, Bertrand wrote: > > > What do you think about adding a function in core PG to provide such > functionality? (means being able to retrieve all the stats (+ eventually > add some filtering) without the need to connect to each database). I'm working on it myself

Re: Generalize ereport_startup_progress infrastructure

On Tue, Aug 9, 2022 at 6:05 PM Robert Haas wrote: > > On Mon, Aug 8, 2022 at 12:29 AM Bharath Rupireddy > wrote: > > Here's v2 patch, passing progress report interval as an input to > > begin_progress_report_phase() so that the processes can use their own > > intervals(hard-coded or GUC) if they

Re: remove useless comments

Junwang Zhao writes: > The comments considering checking share/ directory was there for > almost 13 years, yet nobody ever trying to add the checking, and > there seems never any trouble for not checking it, then I think > we should remove those comments. I think that comment is valuable. It sho

remove useless comments

The comments considering checking share/ directory was there for almost 13 years, yet nobody ever trying to add the checking, and there seems never any trouble for not checking it, then I think we should remove those comments. diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmast

Re: dropping datumSort field

On Tue, Aug 9, 2022 at 8:24 AM Robert Haas wrote: > On Tue, Aug 9, 2022 at 11:16 AM Zhihong Yu wrote: > > tupDesc is declared inside `if (!node->sort_Done)` block whereas the > last reference to tupDesc is outside the if block. > > Yep. > > > I take your review comment and will go back to do mor

Re: dropping datumSort field

On Tue, Aug 9, 2022 at 11:16 AM Zhihong Yu wrote: > tupDesc is declared inside `if (!node->sort_Done)` block whereas the last > reference to tupDesc is outside the if block. Yep. > I take your review comment and will go back to do more homework. The real point for me here is you haven't offere

Re: making relfilenodes 56 bits

On Fri, Aug 5, 2022 at 3:25 AM Dilip Kumar wrote: > I think even if we start the range from the 4 billion we can not avoid > keeping two separate ranges for system and user tables otherwise the > next upgrade where old and new clusters both have 56 bits > relfilenumber will get conflicting files.

Re: dropping datumSort field

On Tue, Aug 9, 2022 at 8:01 AM Robert Haas wrote: > On Mon, Aug 8, 2022 at 5:51 PM Zhihong Yu wrote: > > Hi, > > I was looking at ExecSort() w.r.t. datum sort. > > > > I wonder if the datumSort field can be dropped. > > Here is a patch illustrating the potential simplification. > > > > Please ta

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 2022-08-09 Tu 09:59, Jonathan S. Katz wrote: > On 8/5/22 4:36 PM, Andres Freund wrote: >> Hi, >> >> I tried to look into some of the questions from Amit, but I have e.g. >> no idea >> what exactly the use of subtransactions tries to achieve - afaics >> 1a36bc9dba8 >> is the first patch to intr

Re: dropping datumSort field

On Mon, Aug 8, 2022 at 5:51 PM Zhihong Yu wrote: > Hi, > I was looking at ExecSort() w.r.t. datum sort. > > I wonder if the datumSort field can be dropped. > Here is a patch illustrating the potential simplification. > > Please take a look. One problem with this patch is that, if I apply it, Post

Re: Reducing the chunk header sizes on all memory context types

On Tue, Aug 9, 2022 at 8:53 AM David Rowley wrote: > I think the patch is now starting to take shape. I've added it to the > September commitfest [1]. This is extremely cool. The memory savings are really nice. And I also like this: # Additionally, this commit completely changes the rule that p

Re: Materialized view rewrite is broken when there is an event trigger

Michael Paquier writes: > Please note that we should not add an event in create_am.sql even if > it is empty, as it gets run in parallel of other tests so there could > be interferences. I think that this had better live in > sql/event_trigger.sql, even if it requires an extra table AM to check >

Re: Materialized view rewrite is broken when there is an event trigger

Hi, I should be able to grab a little bit of time next week to look at > what you have. > Thanks! > > Please note that we should not add an event in create_am.sql even if > it is empty, as it gets run in parallel of other tests so there could > be interferences. I think that this had better li

Re: Temporary file access API

On Mon, Aug 8, 2022 at 2:26 PM Antonin Houska wrote: > > I don't think that (3) is a separate advantage from (1) and (2), so I > > don't have anything separate to say about it. > > I thought that the uncontrollable buffer size is one of the things you > complaint about in > > https://www.postgresq

Re: Asking for feedback on Pgperffarm

Hi, Mark, really thank you for your feedback. On Mon, Aug 8, 2022 at 7:06 PM Mark Wong wrote: > Hi Yedil, > > On Mon, Aug 08, 2022 at 02:50:17PM +0200, Yedil Serzhan wrote: > > Dear hackers, > > > > I'm Yedil. I'm working on the project "Postgres Performance Farm" during > > Gsoc. Pgperffarm is

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

On 8/5/22 4:36 PM, Andres Freund wrote: Hi, I tried to look into some of the questions from Amit, but I have e.g. no idea what exactly the use of subtransactions tries to achieve - afaics 1a36bc9dba8 is the first patch to introduce needing to evaluate parts expressions in a subtransaction - but

Re: automatically generating node support functions

On Tue, Aug 9, 2022 at 12:14 AM Tom Lane wrote: > > I wrote: > > Ah. It'd help if that complaint said what the command input actually > > is :-(. But on looking closer, I missed stripping the empty strings > > that "split" will produce at the ends of the array. I think the > > attached will do

Re: Materialized view rewrite is broken when there is an event trigger

On Tue, Aug 09, 2022 at 02:55:06PM +0200, Önder Kalacı wrote: > Attached a patch to fix as well. If the patch looks good to you, can you > consider getting this to PG 15? Thanks, this one is on me so I have added an open item. I will unfortunately not be able to address that this week because of

Materialized view rewrite is broken when there is an event trigger

Hi, Attached a patch to fix as well. If the patch looks good to you, can you consider getting this to PG 15? Steps to repro: -- some basic examples from src/test/regress/sql/create_am.sql CREATE TABLE heaptable USING heap AS SELECT a, repeat(a::text, 100) FROM generate_series(1,9) AS a; CREATE AC

Re: Reducing the chunk header sizes on all memory context types

On Wed, 13 Jul 2022 at 17:20, David Rowley wrote: > I did consider that in all cases where the allocations are above > allocChunkLimit that the chunk is put on a dedicated block and in > fact, the blockoffset is always the same for those. I wondered if we > could use the full 60 bits for the chun

Re: [RFC] building postgres with meson

On 2022-08-09 Tu 03:10, Andres Freund wrote: > Hi, > > I was looking at re-unifying gendef2.pl that the meson patchset had introduced > for temporary ease during hacking with gendef.pl. Testing that I noticed that > either I and my machine is very confused, or gendef.pl's check whether it can > s

Re: Generalize ereport_startup_progress infrastructure

On Mon, Aug 8, 2022 at 12:29 AM Bharath Rupireddy wrote: > Here's v2 patch, passing progress report interval as an input to > begin_progress_report_phase() so that the processes can use their own > intervals(hard-coded or GUC) if they wish to not use the generic GUC > log_progress_report_interval.

Re: support for MERGE

On Tue, Aug 09, 2022 at 11:48:23AM +0200, Álvaro Herrera wrote: > On 2022-Aug-01, Álvaro Herrera wrote: > > > > > If MERGE attempts an INSERT > > > > and a unique index is present and a duplicate row is concurrently > > > > +inserted, then a uniqueness violation error is raised; > >

Re: Perform streaming logical transactions by background workers and parallel apply

On Tue, Aug 9, 2022 at 11:09 AM Dilip Kumar wrote: > > Some more comments > > +/* > + * Exit if any relation is not in the READY state and if any worker is > + * handling the streaming transaction at the same time. Because for > + * streaming transactions that is being applied in a

Re: Fast COPY FROM based on batch insert

On Tue, Jul 26, 2022 at 7:19 PM Etsuro Fujita wrote: > Yeah, I think the patch is getting better, but I noticed some issues, > so I'm working on them. I think I can post a new version in the next > few days. * When running AFTER ROW triggers in CopyMultiInsertBufferFlush(), the patch uses the sl

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

On Mon, Aug 8, 2022 at 6:41 PM Simon Riggs wrote: > > On Thu, 4 Aug 2022 at 13:11, Simon Riggs wrote: > > > > On Wed, 3 Aug 2022 at 20:18, Andres Freund wrote: > > > > > I think we should consider redesigning subtrans more substantially - even > > > with > > > the changes you propose here, ther

Re: Fix inconsistencies GUC categories

On Sat, Aug 06, 2022 at 09:54:36PM +0900, Michael Paquier wrote: > Yep. This change sounds right as well. Done as of 0b039e3. Thanks, Kato-san. -- Michael signature.asc Description: PGP signature

Re: Perform streaming logical transactions by background workers and parallel apply

On Thu, Aug 4, 2022 at 12:10 PM wangw.f...@fujitsu.com wrote: > > On Mon, Jul 25, 2022 at 21:50 PM Amit Kapila wrote: > > Few comments on 0001: > > == > > Thanks for your comments. > Review comments on v20-0001-Perform-streaming-logical-transactions-by-backgr

Re: Checking pgwin32_is_junction() errors

On 2022-08-09 05:44, Thomas Munro wrote: On Tue, Aug 9, 2022 at 8:30 AM Thomas Munro wrote: On Mon, Aug 8, 2022 at 8:23 PM wrote: > "C:/HOME" is the junction point to the second volume on my hard drive - > "\??\Volume{GUID}\" which name pgreadlink() erroneously strips here: > https://github.c

Re: Using each rel as both outer and inner for JOIN_ANTI

Just for kicks, I ran query in your original post under EXPLAIN ANALYZE in both patched and unpatched with this last version. I got this (best of three): Unpatched: 55432 16devel 437532=# explain (analyze, buffers) select * from foo left join bar on foo.a = bar.c where bar.c is null;

Re: Checking pgwin32_is_junction() errors

On 2022-08-09 03:30, Thomas Munro wrote: Then our readlink() emulation removes it again, but in the case of your \??\Volume{GUID} path, created by you, not our symlink() emulation, removing "\??\" apparently makes it unopenable with CreateFile() (I guess that's what fails? What's the error?).

Re: support for MERGE

On 2022-Aug-01, Álvaro Herrera wrote: > > > If MERGE attempts an INSERT > > > and a unique index is present and a duplicate row is concurrently > > > +inserted, then a uniqueness violation error is raised; > > > +MERGE does not attempt to avoid such > > > +errors by evaluatin

Re: [PATCH] Expose port->authn_id to extensions and triggers

On Mon, Aug 08, 2022 at 12:43:14PM +0200, Drouvot, Bertrand wrote: > but I'm not sure we should do it as a first step (given the fact that this > is not Port->authn_id that is passed down to the parallel workers in the > SYSTEM_USER patch). > > What do you think about working on both (aka a) v11-0

RE: Perform streaming logical transactions by background workers and parallel apply

Dear Wang, Thanks for updating patch sets! Followings are comments about v20-0001. 1. config.sgml ``` Specifies maximum number of logical replication workers. This includes both apply workers and table synchronization workers. ``` I think you can add a descriptio

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

On Tue, Aug 9, 2022 at 12:42 PM Kyotaro Horiguchi wrote: > > > Can you please explain more about your idea, I may be missing something? > > (I'm not sure I understand the requirements here..) I've explained the problem with the current HA setup with synchronous replication upthread at [1]. Let me

Re: shared-memory based stats collector - v70

At Mon, 8 Aug 2022 11:53:19 -0400, Greg Stark wrote in > I'm trying to wrap my head around the shared memory stats collector > infrastructure from > <20220406030008.2qxipjxo776dw...@alap3.anarazel.de> committed in > 5891c7a8ed8f2d3d577e7eea34dacff12d7b6bbd. > > I have one question about locking

Re: Using each rel as both outer and inner for JOIN_ANTI

On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: > >> I took a quick look through this. The executor changes are indeed >> impressively short, but that's largely because you've paid zero >> attention to updating obsoleted comments. For exampl

  1   2   >