Re: Memory leak from ExecutorState context?

2023-05-22 Thread Jehan-Guillaume de Rorthais
On Fri, 19 May 2023 17:23:56 +0200 Tomas Vondra wrote: > On 5/19/23 00:27, Melanie Plageman wrote: > > v10 LGTM. > > Thanks! > > I've pushed 0002 and 0003, after some general bikeshedding and minor > rewording (a bit audacious, admittedly). Thank you Melanie et Tomas for your help, review et

Re: Naming of gss_accept_deleg

2023-05-22 Thread Alvaro Herrera
I noticed that the value that enables this feature at libpq client side is 'enable'. However, for other Boolean settings like sslsni, keepalives, requiressl, sslcompression, the value that enables feature is '1' -- we use strings only for "enum" type of settings. Also, it looks like connectOption

Re: Allow pg_archivecleanup to remove backup history files

2023-05-22 Thread torikoshia
On 2023-05-15 15:22, Michael Paquier wrote: On Mon, May 15, 2023 at 03:16:46PM +0900, torikoshia wrote: On 2023-05-15 09:18, Michael Paquier wrote: How about plugging in some long options, and use something more explicit like --clean-backup-history? Agreed. If you begin to implement that, i

Re: Adding SHOW CREATE TABLE

2023-05-22 Thread Pavel Stehule
po 22. 5. 2023 v 7:19 odesílatel Kirk Wolak napsal: > On Fri, May 19, 2023 at 1:08 PM Andrew Dunstan > wrote: > >> I think the ONLY place we should have this is in server side functions. >> More than ten years ago I did some work in this area (see below), but it's >> one of those things that hav

unnecessary #include "pg_getopt.h"?

2023-05-22 Thread torikoshia
Hi, While working on [1], I thought there seems to be unnecessary #include "pg_getopt.h". getopt_long.h has already included pg_getopt.h, but some files include both getopt.h and getopt_long.h. [1] https://www.postgresql.org/message-id/flat/d660ef741ce3d82f3b4283f1cafd5...@oss.nttdata.com

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-05-22 Thread Hayato Kuroda (Fujitsu)
Dear Wang, Thank you for reviewing! PSA new version. > For patches 0001 > > 1. The latest patch set fails to apply because the new commit (0245f8d) in > HEAD. I didn't notice that. Thanks, fixed. > 2. In file pg_dump.h. > ``` > +/* > + * The LogicalReplicationSlotInfo struct is used to repres

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-05-22 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Thank you for giving suggestions. > > Dear hackers, > > > > I rebased and refined my PoC. Followings are the changes: > > > > 1. Is my understanding correct that this patch creates the delay files > for each transaction? If so, did you consider other approaches such as > using one fil

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2023-05-22 Thread Alvaro Herrera
On 2023-May-20, Alexander Lakhin wrote: > Starting from 038f586d5, the following script: > echo " > \startpipeline > \endpipeline > " >test.sql > pgbench -n -M prepared -f test.sql > > leads to the pgbench's segfault: Hah, yeah, that's because an empty pipeline never calls the code to allocate t

Re: Adding SHOW CREATE TABLE

2023-05-22 Thread Andrew Dunstan
On 2023-05-22 Mo 05:24, Pavel Stehule wrote: po 22. 5. 2023 v 7:19 odesílatel Kirk Wolak napsal: On Fri, May 19, 2023 at 1:08 PM Andrew Dunstan wrote: I think the ONLY place we should have this is in server side functions. More than ten years ago I did some work in

Re: createuser --memeber and PG 16

2023-05-22 Thread Nathan Bossart
On Mon, May 22, 2023 at 08:42:28AM +0200, Peter Eisentraut wrote: > On 21.05.23 19:07, Nathan Bossart wrote: >> How do folks feel about keeping --role undocumented? Should we give it a >> mention in the docs for --member-of? > > We made a point in this release to document deprecated options consi

RE: walsender performance regression due to logical decoding on standby changes

2023-05-22 Thread Zhijie Hou (Fujitsu)
Hi, On Monday, May 22, 2023 12:11 AM Andres Freund wrote: > On 2023-05-19 12:07:56 +0900, Kyotaro Horiguchi wrote: > > At Thu, 18 May 2023 20:11:11 +0530, Bharath Rupireddy > wrote in > > > > > + > ConditionVariableInit(&WalSndCtl->physicalWALSndCV); > > > > > + ConditionVariableInit

Re: Order changes in PG16 since ICU introduction

2023-05-22 Thread Peter Eisentraut
On 18.05.23 19:55, Jeff Davis wrote: On Wed, 2023-05-17 at 19:59 -0400, Jonathan S. Katz wrote: I did a quicker read through this time. LGTM overall. I like what you did with the explanations around sensitivity (now it makes sense). Committed, thank you. There are a few things I don't underst

Re: Order changes in PG16 since ICU introduction

2023-05-22 Thread Peter Eisentraut
On 18.05.23 00:59, Jeff Davis wrote: On Tue, 2023-05-16 at 20:23 -0700, Jeff Davis wrote: Other than that, and I took your suggestions almost verbatim. Patch attached. Thank you! Attached new patch with a typo fix and a few other edits. I plan to commit soon. Some small follow-up on this pat

Re: running logical replication as the subscription owner

2023-05-22 Thread Masahiko Sawada
On Wed, May 17, 2023 at 10:10 AM Ajin Cherian wrote: > > On Mon, May 15, 2023 at 10:47 PM Masahiko Sawada > wrote: > > > > On Mon, May 15, 2023 at 5:44 PM Ajin Cherian wrote: > > > > > > On Fri, May 12, 2023 at 9:55 PM Ajin Cherian wrote: > > > > > > > > If nobody else is working on this, I ca

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-05-22 Thread reid . thompson
On Wed, 2023-05-17 at 23:07 -0400, reid.thomp...@crunchydata.com wrote: > Thanks for the feedback. > > I'm plannning to look at this. > > Is your benchmark something that I could utilize? I.E. is it a set of > scripts or a standard test from somewhere that I can duplicate? > > Thanks, > Reid >

Re: RFI: Extending the TOAST Pointer

2023-05-22 Thread Matthias van de Meent
On Sun, 21 May 2023, 15:39 Aleksander Alekseev, wrote: > > Hi, > > > We'd need to stop using the va_tag as length indicator, but I don't > > think it's currently assumed to be a length indicator anyway (see > > VARSIZE_EXTERNAL(ptr)). By not using the varatt_external struct > > currently in use, w

Re: "38.10.10. Shared Memory and LWLocks" may require a clarification

2023-05-22 Thread Aleksander Alekseev
Hi, > Unless I missed something, I suggest updating the documentation and > pg_stat_statements.c accordingly. Since no one seems to object so far I prepared the patch. -- Best regards, Aleksander Alekseev v1-0001-Clarify-the-usage-of-shmem_startup_hook.patch Description: Binary data

Re: Naming of gss_accept_deleg

2023-05-22 Thread Tom Lane
Alvaro Herrera writes: > I noticed that the value that enables this feature at libpq client side > is 'enable'. However, for other Boolean settings like sslsni, > keepalives, requiressl, sslcompression, the value that enables feature > is '1' -- we use strings only for "enum" type of settings. >

Re: Naming of gss_accept_deleg

2023-05-22 Thread Abhijit Menon-Sen
At 2023-05-22 09:42:44 -0400, t...@sss.pgh.pa.us wrote: > > Alvaro Herrera writes: > > I noticed that the value that enables this feature at libpq client side > > is 'enable'. However, for other Boolean settings like sslsni, > > keepalives, requiressl, sslcompression, the value that enables featu

Re: RFI: Extending the TOAST Pointer

2023-05-22 Thread Aleksander Alekseev
Hi, > I see your point, but I do think we should also think about why we do > the change. Personally at the moment I care only about implementing compression dictionaries on top of this, as is discussed in the corresponding thread [1]. I'm going to need new fields in the TOAST pointer's including

Re: "38.10.10. Shared Memory and LWLocks" may require a clarification

2023-05-22 Thread Aleksander Alekseev
Hi, > Since no one seems to object so far I prepared the patch. Turned out patch v1 fails on cfbot on Windows due to extra Assert I added [1]: ``` abort() has been calledTRAP: failed Assert("!IsUnderPostmaster"), File: "../src/backend/storage/ipc/ipci.c", Line: 320, PID: 4040 abort() has been ca

Make pgbench exit on SIGINT more reliably

2023-05-22 Thread Tristan Partin
Hello, The way that pgbench handled SIGINT changed in 1d468b9ad81b9139b4a0b16b416c3597925af4b0. Unfortunately this had a couple of unintended consequences, at least from what I can tell[1]. - CTRL-C no longer stops the program unless the right point in pgbench execution is hit - pgbench no long

Re: Naming of gss_accept_deleg

2023-05-22 Thread Abhijit Menon-Sen
At 2023-05-22 09:42:44 -0400, t...@sss.pgh.pa.us wrote: > > Alvaro Herrera writes: > > I noticed that the value that enables this feature at libpq client side > > is 'enable'. However, for other Boolean settings like sslsni, > > keepalives, requiressl, sslcompression, the value that enables featu

Re: Naming of gss_accept_deleg

2023-05-22 Thread Tom Lane
Abhijit Menon-Sen writes: > Here's the diff, but the 0/1 values of settings like sslsni and > sslcompression don't seem to be validated anywhere, unlike the string > options in connectOptions2, so I didn't do anything for gssdelegation. Thanks! I'll set to work on this. I assume we want to sque

Add missing includes

2023-05-22 Thread Tristan Partin
Hi, Some files were missing information from the c.h header. -- Tristan Partin Neon (https://neon.tech) From 3c0c32077bb9626d4ebad1452b9fc9937c99518d Mon Sep 17 00:00:00 2001 From: Tristan Partin Date: Mon, 22 May 2023 10:07:45 -0500 Subject: [PATCH postgres v1] Add some missing includes Both

Re: Add missing includes

2023-05-22 Thread Alvaro Herrera
Hi, On 2023-May-22, Tristan Partin wrote: > Some files were missing information from the c.h header. Actually, these omissions are intentional, and we have bespoke handling for this in our own header-checking scripts (src/tools/pginclude). I imagine this is documented somewhere, but ATM I can't

Re: Add missing includes

2023-05-22 Thread Tristan Partin
On Mon May 22, 2023 at 10:16 AM CDT, Alvaro Herrera wrote: > > Some files were missing information from the c.h header. > > Actually, these omissions are intentional, and we have bespoke handling > for this in our own header-checking scripts (src/tools/pginclude). I > imagine this is documented so

Re: Add missing includes

2023-05-22 Thread Tom Lane
Alvaro Herrera writes: > On 2023-May-22, Tristan Partin wrote: >> Some files were missing information from the c.h header. > Actually, these omissions are intentional, and we have bespoke handling > for this in our own header-checking scripts (src/tools/pginclude). I > imagine this is documented

Re: Add missing includes

2023-05-22 Thread Tristan Partin
On Mon May 22, 2023 at 10:28 AM CDT, Tom Lane wrote: > >> Some files were missing information from the c.h header. > > > Actually, these omissions are intentional, and we have bespoke handling > > for this in our own header-checking scripts (src/tools/pginclude). I > > imagine this is documented s

Re: Add missing includes

2023-05-22 Thread Tom Lane
"Tristan Partin" writes: > Interesting. Thanks for the information. Thought I had seen in a > different email thread that header files were supposed to include all > that they needed to. Anyways, ignore the patch :). There is such a policy, but not with respect to those particular headers. You m

Re: Naming of gss_accept_deleg

2023-05-22 Thread Tom Lane
Abhijit Menon-Sen writes: > Here's the diff, Pushed, thanks. > but the 0/1 values of settings like sslsni and > sslcompression don't seem to be validated anywhere, unlike the string > options in connectOptions2, so I didn't do anything for gssdelegation. Yeah. Perhaps it's worth adding code to

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-05-22 Thread reid . thompson
On Mon, 2023-05-22 at 08:42 -0400, reid.thomp...@crunchydata.com wrote: > On Wed, 2023-05-17 at 23:07 -0400, reid.thomp...@crunchydata.com wrote: > > Thanks for the feedback. > > > > I'm plannning to look at this. > > > > Is your benchmark something that I could utilize? I.E. is it a set of > >

Re: Should CSV parsing be stricter about mid-field quotes?

2023-05-22 Thread Daniel Verite
Joel Jacobson wrote: > Is there a valid reason why \. is needed for COPY FROM filename? > It seems to me it would only be necessary for the COPY FROM STDIN case, > since files have a natural end-of-file and a known file size. Looking at CopyReadLineText() over at [1], I don't see a reason

Re: RFI: Extending the TOAST Pointer

2023-05-22 Thread Nikita Malakhov
Hi, Aleksander, I'm interested in extending TOAST pointer in various ways. 64-bit TOAST value ID allows to resolve very complex issue for production systems with large tables and heavy update rate. I agree with Matthias that there should not be processing of TOAST pointer internals outside TOAST

Re: Avoiding another needless ERROR during nbtree page deletion

2023-05-22 Thread Peter Geoghegan
On Sun, May 21, 2023 at 11:51 PM Heikki Linnakangas wrote: > Any idea what might cause this corruption? Not really, no. As far as I know the specific case that was brought to my attention (that put me on the path to writing this patch) was just an isolated incident. The interesting detail (if any

Re: Should CSV parsing be stricter about mid-field quotes?

2023-05-22 Thread Kirk Wolak
On Mon, May 22, 2023 at 12:13 PM Daniel Verite wrote: > Joel Jacobson wrote: > > > Is there a valid reason why \. is needed for COPY FROM filename? > > It seems to me it would only be necessary for the COPY FROM STDIN case, > > since files have a natural end-of-file and a known file size.

Re: Order changes in PG16 since ICU introduction

2023-05-22 Thread Jeff Davis
On Thu, 2023-05-11 at 13:09 +0200, Peter Eisentraut wrote: > There is also the deterministic flag and the icurules setting. > Depending on what level of detail you imagine the user needs, you > really > do need to look at the whole picture, not some subset of it. (Nit: all database default colla

Re: RFI: Extending the TOAST Pointer

2023-05-22 Thread Matthias van de Meent
On Mon, 22 May 2023 at 18:13, Nikita Malakhov wrote: > > Hi, Could you please not top-post. > Aleksander, I'm interested in extending TOAST pointer in various ways. > 64-bit TOAST value ID allows to resolve very complex issue for production > systems with large tables and heavy update rate. Co

Handle SIGTERM in fe_utils/cancel.c

2023-05-22 Thread Tristan Partin
Hello, This is a way that would solve bug #17698[1]. It just reuses the same handler as SIGINT (with a function rename). This patch works best if it is combined with my previous submission[2]. I can rebase that submission if and when this patch is pulled in. [1]: https://www.postgresql.org/mess

Re: Make pgbench exit on SIGINT more reliably

2023-05-22 Thread Tristan Partin
Here is a v2 that handles the Windows case that I seemingly missed in my first readthrough of this code. -- Tristan Partin Neon (https://neon.tech) From 3cff6aec8f154eb8a47524efc15f16a6b7e95f37 Mon Sep 17 00:00:00 2001 From: Tristan Partin Date: Mon, 22 May 2023 08:07:37 -0500 Subject: [PATCH po

Re: Order changes in PG16 since ICU introduction

2023-05-22 Thread Jeff Davis
On Thu, 2023-05-11 at 13:07 +0200, Peter Eisentraut wrote: > Here is my proposed patch for this. The commit message makes it sound like lc_collate/ctype are completely obsolete, and I don't think that's quite right: they still represent the server environment, which does still matter in some cases

Re: walsender performance regression due to logical decoding on standby changes

2023-05-22 Thread Andres Freund
Hi, On 2023-05-22 12:15:07 +, Zhijie Hou (Fujitsu) wrote: > About "a backend doing logical decoding", do you mean the case when a user > start a backend and invoke pg_logical_slot_get_changes() to do the logical > decoding ? If so, it seems the logical decoding in a backend won't be waked up >

Re: PG 16 draft release notes ready

2023-05-22 Thread Andres Freund
Hi, On 2023-05-21 22:46:58 -0400, Bruce Momjian wrote: > > Looking through the release notes, I didn't see an entry for > > > > commit c6e0fe1f2a08505544c410f613839664eea9eb21 > > Author: David Rowley > > Date: 2022-08-29 17:15:00 +1200 > > > > Improve performance of and reduce overheads of

Re: Avoiding another needless ERROR during nbtree page deletion

2023-05-22 Thread Peter Geoghegan
On Mon, May 22, 2023 at 9:22 AM Peter Geoghegan wrote: > > This comment notes that this is similar to what we did with the left > > sibling, but there isn't really any mention at the left sibling code > > about avoiding hard ERRORs. Feels a bit backwards. Maybe move the > > comment about avoiding

Re: PG 16 draft release notes ready

2023-05-22 Thread Bruce Momjian
I have added the major features to the release notes with the attached patch. --- On Sun, May 21, 2023 at 07:53:38PM -0400, Jonathan Katz wrote: > On 5/21/23 3:04 PM, Jonathan S. Katz wrote: > > On 5/18/23 4:49 PM, Bruce Mom

Re: PG 16 draft release notes ready

2023-05-22 Thread Andres Freund
Hi, On 2023-05-21 22:52:09 -0400, Bruce Momjian wrote: > On Sun, May 21, 2023 at 10:13:41AM -0700, Andres Freund wrote: > > Hi, > > > > Thanks for the release notes! > > > > > > > > > > > > > > > > > Allow more efficient addition of multiple heap and index pages (Andres > > > Freund) > > >

Re: PG 16 draft release notes ready

2023-05-22 Thread Bruce Momjian
On Mon, May 22, 2023 at 10:59:36AM -0700, Andres Freund wrote: > On 2023-05-21 22:46:58 -0400, Bruce Momjian wrote: > > > For the above two items, I mention items that would change user > > > like new features or changes that are significant enough that they would > > > change user behavior. For e

Re: PG 16 draft release notes ready

2023-05-22 Thread Bruce Momjian
On Mon, May 22, 2023 at 11:03:31AM -0700, Andres Freund wrote: > On 2023-05-21 22:52:09 -0400, Bruce Momjian wrote: > > Do average users know heap and index files are both relations? That > > seems too abstract so I spelled out heap and index pages. > > I don't know about average users - but I th

Re: Order changes in PG16 since ICU introduction

2023-05-22 Thread Jeff Davis
On Mon, 2023-05-22 at 14:27 +0200, Peter Eisentraut wrote: > The rules are for setting whatever sort order you like.  Maybe you > want > to sort + before - or whatever.  It's like, if you don't like it, > build > your own. A build-your-own feature is fine, but it's not completely zero cost. The

Re: PostgreSQL 16 Beta 1 release announcement draft

2023-05-22 Thread Erik Rijkers
Op 5/21/23 om 19:07 schreef Jonathan S. Katz: On 5/19/23 12:17 AM, Jonathan S. Katz wrote: Hi, Attached is a draft of the release announcement for PostgreSQL 16 Beta Please provide feedback no later than May 24, 0:00 AoE. This will give Thanks everyone for your feedback. Here is the updated t

Re: benchmark results comparing versions 15.2 and 16

2023-05-22 Thread MARK CALLAGHAN
I ran sysbench on Postgres 15.2, 15.3 and 16 prebeta at git sha 1c006c0 (built on May19). The workload was in-memory on a small server (8 cores, 16G RAM) and the workload had 1 connection (no concurrency). For some details on past benchmarks like this see: http://smalldatum.blogspot.com/2023/03/sea

Re: Order changes in PG16 since ICU introduction

2023-05-22 Thread Daniel Verite
Jeff Davis wrote: > If we special case locale=C, but do nothing for locale=fr_FR, then I'm > not sure we've solved the problem. Andrew Gierth raised the issue here, > which he called "maximally confusing": > > https://postgr.es/m/874jp9f5jo@news-spur.riddles.org.uk > > That's why I f

Re: PG 16 draft release notes ready

2023-05-22 Thread Robert Haas
On Sun, May 21, 2023 at 3:05 PM Jonathan S. Katz wrote: > * Support for regular expressions for matching usernames and databases > names in `pg_hba.conf`, and user names in `pg_ident.conf` I suggest that this is not a major feature. Perhaps the work that I did to improve CREATEROLE could be cons

Re: Should CSV parsing be stricter about mid-field quotes?

2023-05-22 Thread Daniel Verite
Kirk Wolak wrote: > We do NOT do "CSV", we mimic pg_dump. pg_dump uses the text format (as opposed to csv), where \. on a line by itself cannot appear in the data, so there's no problem. The problem is limited to the csv format. Best regards, -- Daniel Vérité https://postgresql.verite.

Re: RFI: Extending the TOAST Pointer

2023-05-22 Thread Jacob Champion
On Mon, May 22, 2023 at 6:47 AM Aleksander Alekseev wrote: > Last but not least I remember somebody on the mailing list suggested > adding ZSTD compression support for TOAST, besides LZ4. Assuming I > didn't dream it, the proposal was rejected due to the limited amount > of free bits in ToastCompr

Re: createuser --memeber and PG 16

2023-05-22 Thread Michael Paquier
On Mon, May 22, 2023 at 05:11:14AM -0700, Nathan Bossart wrote: > On Mon, May 22, 2023 at 08:42:28AM +0200, Peter Eisentraut wrote: > > On 21.05.23 19:07, Nathan Bossart wrote: > >> How do folks feel about keeping --role undocumented? Should we give it a > >> mention in the docs for --member-of? >

Re: Add missing includes

2023-05-22 Thread Michael Paquier
On Mon, May 22, 2023 at 11:34:14AM -0400, Tom Lane wrote: > There is such a policy, but not with respect to those particular > headers. You might find src/tools/pginclude/headerscheck and > src/tools/pginclude/cpluspluscheck to be interesting reading, > as those are the tests we run to verify this

Re: Add missing includes

2023-05-22 Thread Tom Lane
Michael Paquier writes: > The standalone compile policy is documented in > src/tools/pginclude/README, AFAIK, so close enough :p Hmm, that needs an update --- the scripts are slightly smarter now than that text gives them credit for. I'll see to it after the release freeze lifts.

Re: PG 16 draft release notes ready

2023-05-22 Thread John Naylor
Hi Bruce, > Add support for SSE2 (Streaming SIMD Extensions 2) vector operations on x86-64 architectures (John Naylor) > Add support for Advanced SIMD (Single Instruction Multiple Data) (NEON) instructions on ARM architectures (Nathan Bossart) Nit: It's a bit odd that SIMD is spelled out in only

Re: PG 16 draft release notes ready

2023-05-22 Thread Bruce Momjian
On Tue, May 23, 2023 at 09:58:30AM +0700, John Naylor wrote: > Hi Bruce, > > > Add support for SSE2 (Streaming SIMD Extensions 2) vector operations on > x86-64 architectures (John Naylor) > > > Add support for Advanced SIMD (Single Instruction Multiple Data) (NEON) > instructions on ARM architect

Re: PG 16 draft release notes ready

2023-05-22 Thread John Naylor
On Tue, May 23, 2023 at 11:26 AM Bruce Momjian wrote: > > On Tue, May 23, 2023 at 09:58:30AM +0700, John Naylor wrote: > > > Allow ASCII string detection to use vector operations on x86-64 architectures > > (John Naylor) > > > Allow JSON string processing to use vector operations on x86-64 archite

Re: Initial Schema Sync for Logical Replication

2023-05-22 Thread Amit Kapila
On Mon, May 22, 2023 at 6:37 AM Masahiko Sawada wrote: > > On Fri, Apr 28, 2023 at 4:16 PM Masahiko Sawada wrote: > > Yes, in this approach, we need to dump/restore objects while > > specifying with fine granularity. Ideally, the table sync worker dumps > > and restores the table schema, does cop

ERROR: no relation entry for relid 6

2023-05-22 Thread Richard Guo
I came across $subject on master and here is the query I'm running. create table t (a int unique, b int); explain (costs off) select 1 from t t1 left join t t2 on true inner join t t3 on true left join t t4 on t2.a = t4.a and t2.a = t3.a; ERROR: no relation entry for relid 6 I looked