Re: optimize several list functions with SIMD intrinsics

2023-04-20 Thread John Naylor
On Wed, Mar 8, 2023 at 7:25 AM Nathan Bossart wrote: > > was mostly a fun weekend project, and I don't presently have any concrete > examples of workloads where this might help. It seems like that should be demonstrated before seriously considering this, like a profile where the relevant list fun

Doc limitation update proposal: include out-of-line OID usage per TOAST-ed columns

2023-04-20 Thread Jakub Wartak
Hi -hackers, I would like to ask if it wouldn't be good idea to copy the https://wiki.postgresql.org/wiki/TOAST#Total_table_size_limit discussion (out-of-line OID usage per TOAST-ed columns / potential limitation) to the official "Appendix K. PostgreSQL Limits" with also little bonus mentioning th

Re: Reorder connection markers in libpq TAP tests

2023-04-20 Thread Gurjeet Singh
On Thu, Apr 20, 2023 at 11:00 PM Gurjeet Singh wrote: > > Commit 7f5b198 introduced TAP tests that use string literals to mark > the presence of a query in server logs. For no explicable reason, the > tests with the marker 'connect2' occur before the tests that use > 'connect1' marker. > > The att

Reorder connection markers in libpq TAP tests

2023-04-20 Thread Gurjeet Singh
Commit 7f5b198 introduced TAP tests that use string literals to mark the presence of a query in server logs. For no explicable reason, the tests with the marker 'connect2' occur before the tests that use 'connect1' marker. The attached patch swaps the connection marker strings so that a reader doe

Re: Support logical replication of DDLs

2023-04-20 Thread Amit Kapila
On Thu, Apr 20, 2023 at 2:28 PM shveta malik wrote: > > Few comments for ddl_json.c in the patch dated April17: > ... > > 3) expand_jsonb_array() > arrayelem is allocated as below, but not freed. > initStringInfo(&arrayelem) > > 4) expand_jsonb_array(), > we initialize iterator as below which int

RE: Test failures of 100_bugs.pl

2023-04-20 Thread Yu Shi (Fujitsu)
On Tue, Jan 24, 2023 7:41 PM Amit Kapila wrote: > > On Tue, Jan 24, 2023 at 8:53 AM Andres Freund wrote: > > > > cfbot, the buildfarm and locally I have seen 100_bugs.pl fail > > occasionally. Just rarely enough that I never got around to looking into it > > for real. > > > ... > > > > We see t2

Minor code de-duplication in fe-connect.c

2023-04-20 Thread Gurjeet Singh
Commit [1] implements Fisher-Yates shuffling algorithm to shuffle connection addresses, in two places. The attached patch moves the duplicated code to a function, and calls it in those 2 places. [1]: Support connection load balancing in libpq 7f5b19817eaf38e70ad1153db4e644ee9456853e Best regards

Re: Make message strings in fe-connect.c consistent

2023-04-20 Thread Gurjeet Singh
On Thu, Apr 20, 2023 at 9:31 PM Tom Lane wrote: > > Gurjeet Singh writes: > > When reviewing a recently committed patch [1] I noticed the odd usage > > of a format specifier: > > > + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", > > + "load_

Re: Make message strings in fe-connect.c consistent

2023-04-20 Thread Tom Lane
Gurjeet Singh writes: > When reviewing a recently committed patch [1] I noticed the odd usage > of a format specifier: > + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", > + "load_balance_hosts", > + conn->lo

Make message strings in fe-connect.c consistent

2023-04-20 Thread Gurjeet Singh
When reviewing a recently committed patch [1] I noticed the odd usage of a format specifier: + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", + "load_balance_hosts", + conn->load_balance_hosts); The oddity is

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Ashutosh Bapat
Greate news! Hearty congratulations to Nathan, Amit and Masahiko. On Thu, Apr 20, 2023 at 11:10 PM Tom Lane wrote: > > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres commi

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Thomas Munro
On Fri, Apr 21, 2023 at 5:40 AM Tom Lane wrote: > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and few reverts

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Amit Kapila
On Thu, Apr 20, 2023 at 11:10 PM Tom Lane wrote: > > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and few reve

Re: Testing autovacuum wraparound (including failsafe)

2023-04-20 Thread John Naylor
I agree having the new functions in the tree is useful. I also tried running the TAP tests in v2, but 001 and 002 fail to run: Odd number of elements in hash assignment at [...]/Cluster.pm line 2010. Can't locate object method "pump_nb" via package "PostgreSQL::Test::BackgroundPsql" at [...] It s

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Kyotaro Horiguchi
At Thu, 20 Apr 2023 13:40:49 -0400, Tom Lane wrote in > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and few

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread vignesh C
On Thu, 20 Apr 2023 at 23:10, Tom Lane wrote: > > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and few reverts

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread John Naylor
On Fri, Apr 21, 2023 at 12:40 AM Tom Lane wrote: > > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and few reve

Re: Support logical replication of DDLs

2023-04-20 Thread Peter Smith
Here are some more review comments for the patch 0002-2023_04_07-2 This was a WIP review in parts because the patch was quite large: WIP part 1 [1] was posted 17/4. WIP part 2 [2] was posted 17/4. WIP part 3 [3] was posted 19/4. WIP part 4 is this post. (This is my final WIP part for this 0002 pa

Re: [PATCH] Extend the length of BackgroundWorker.bgw_library_name

2023-04-20 Thread Nathan Bossart
On Wed, Mar 15, 2023 at 10:38:34AM +0100, Daniel Gustafsson wrote: > While here, I wonder if we should document what BGW_MAXLEN is defined as in > bgworker.sgml? I am -0.5 for this. If you are writing a new background worker, it's probably reasonable to expect that you can locate the definition o

Re: Should we remove vacuum_defer_cleanup_age?

2023-04-20 Thread Nathan Bossart
On Fri, Apr 14, 2023 at 03:07:37PM -0400, Jonathan S. Katz wrote: > +1. +1. I agree with the upthread discussion and support removing vacuum_defer_cleanup_age. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Memory leak from ExecutorState context?

2023-04-20 Thread Melanie Plageman
On Thu, Apr 20, 2023 at 12:42 PM Konstantin Knizhnik wrote: > On 11.04.2023 8:14 PM, Jehan-Guillaume de Rorthais wrote: > > On Sat, 8 Apr 2023 02:01:19 +0200 > > Jehan-Guillaume de Rorthais wrote: > > > >> On Fri, 31 Mar 2023 14:06:11 +0200 > >> Jehan-Guillaume de Rorthais wrote: > >> > >> [..

Re: Fix typos and inconsistencies for v16

2023-04-20 Thread David Rowley
On Wed, 19 Apr 2023 at 07:00, Alexander Lakhin wrote: > please look at the similar list for v15+ (596b5af1d..HEAD). I've now pushed most of these but didn't include the following ones: > 3. BufFileOpenShared -> BufFileOpenFileSet // see dcac5e7ac Maybe I need to spend longer, but I just didn't

Re: Remove io prefix from pg_stat_io columns

2023-04-20 Thread Michael Paquier
On Thu, Apr 20, 2023 at 11:38:42AM +0900, Michael Paquier wrote: > No worries, committers should take care of that. Done as of 0ecb87e, as I can keep an eye on the buildfarm today, with a catversion bump. -- Michael signature.asc Description: PGP signature

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread David Rowley
On Fri, 21 Apr 2023 at 05:40, Tom Lane wrote: > > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and few reverts

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Michael Paquier
On Thu, Apr 20, 2023 at 01:40:49PM -0400, Tom Lane wrote: > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and f

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-20 Thread David Rowley
On Thu, 20 Apr 2023 at 18:46, Richard Guo wrote: > > > On Thu, Apr 20, 2023 at 6:38 AM David Rowley wrote: >> >> That function is pretty new and was exactly added so we didn't have to >> write list_truncate(list_copy(...), n) anymore. That gets pretty >> wasteful when the input List is long and

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-20 Thread Ranier Vilela
>I searched the codes and found some other places where the manipulation >of lists can be improved in a similar way. >* lappend(list_copy(list), datum) as in get_required_extension(). >This is not very efficient as after list_copy it would need to enlarge >the list immediately. It can be improved

Re: LLVM strip -x fails

2023-04-20 Thread Tom Lane
Andres Freund writes: > On 2023-04-20 12:43:48 -0400, Tom Lane wrote: >> The previous complaint about this [1] suggested we use --strip-unneeded >> for all cases with GNU strip, same as we've long done for shared libs. >> It's an easy enough change, but I wonder if anyone will complain. > I doubt

Re: Improve logging when using Huge Pages

2023-04-20 Thread Nathan Bossart
On Thu, Mar 23, 2023 at 07:23:28AM +0900, Michael Paquier wrote: > On Wed, Mar 22, 2023 at 05:18:28PM -0500, Justin Pryzby wrote: >> Wow, good point. I think to make it work we'd need put >> huge_pages_active into BackendParameters and handle it in >> save_backend_variables(). If so, that'd be st

Re: Non-superuser subscription owners

2023-04-20 Thread Robert Haas
On Thu, Apr 20, 2023 at 1:08 AM Amit Kapila wrote: > Pushed. I noticed that we didn't display this new subscription option > 'password_required' in \dRs+: > > postgres=# \dRs+ > > List of subscriptions > Name | Owner | Enabled | Publication | Binary | Streaming | > Two-phase commit | Dis

Re: Tab completion for GRANT MAINTAIN

2023-04-20 Thread Nathan Bossart
On Wed, Apr 19, 2023 at 10:54:05AM -0400, Tom Lane wrote: > (One could wish that it didn't take touching three or so places in > tab-complete.c to add a privilege, especially when a naive hacker might > think he was done after touching Privilege_options_of_grant_and_revoke. > I didn't see any easy

Re: Request for comment on setting binary format output per session

2023-04-20 Thread Dave Cramer
On Tue, 18 Apr 2023 at 12:31, Dave Cramer wrote: > > > On Tue, 18 Apr 2023 at 12:24, Robert Haas wrote: > >> On Tue, Apr 18, 2023 at 11:51 AM Tom Lane wrote: >> > Robert Haas writes: >> > > One thing I think we should do in this area is introduce #defines for >> > > all the message type codes

Re: LLVM strip -x fails

2023-04-20 Thread Andres Freund
Hi, On 2023-04-20 12:43:48 -0400, Tom Lane wrote: > Andres Freund writes: > > Afaict the only safe thing to use when stripping static libs is > > -g/--strip-debug. > > The previous complaint about this [1] suggested we use --strip-unneeded > for all cases with GNU strip, same as we've long done f

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Andrew Dunstan
On 2023-04-20 Th 14:12, Peter Geoghegan wrote: On Thu, Apr 20, 2023 at 10:56 AM Pavel Borisov wrote: It's much deserved! Congratulations, Nathan, Amit and Masahiko! Congratulations to all three! +3 cheers andrew -- Andrew Dunstan EDB:https://www.enterprisedb.com

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-20 Thread Miroslav Bendik
> I've just pushed a fix to master for this. See [1]. If you base your > patch atop of that you should be able to list list_copy_head() without > any issues. Thanks for this fix. Now the version am_orderbyop_incremental_sort_v3.1.patch [1] works without issues using the master branch. [1] https

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Peter Geoghegan
On Thu, Apr 20, 2023 at 10:56 AM Pavel Borisov wrote: > It's much deserved! Congratulations, Nathan, Amit and Masahiko! Congratulations to all three! -- Peter Geoghegan

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Pavel Borisov
On Thu, 20 Apr 2023 at 21:40, Tom Lane wrote: > > The Core Team would like to extend our congratulations to > Nathan Bossart, Amit Langote, and Masahiko Sawada, who have > accepted invitations to become our newest Postgres committers. > > Please join me in wishing them much success and few reverts

New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-20 Thread Tom Lane
The Core Team would like to extend our congratulations to Nathan Bossart, Amit Langote, and Masahiko Sawada, who have accepted invitations to become our newest Postgres committers. Please join me in wishing them much success and few reverts. regards, tom lane

Re: LLVM strip -x fails

2023-04-20 Thread Tom Lane
Andres Freund writes: > Afaict the only safe thing to use when stripping static libs is > -g/--strip-debug. The previous complaint about this [1] suggested we use --strip-unneeded for all cases with GNU strip, same as we've long done for shared libs. It's an easy enough change, but I wonder if an

Re: Memory leak from ExecutorState context?

2023-04-20 Thread Konstantin Knizhnik
On 11.04.2023 8:14 PM, Jehan-Guillaume de Rorthais wrote: On Sat, 8 Apr 2023 02:01:19 +0200 Jehan-Guillaume de Rorthais wrote: On Fri, 31 Mar 2023 14:06:11 +0200 Jehan-Guillaume de Rorthais wrote: [...] After rebasing Tomas' memory balancing patch, I did some memory measures to answer

Re: Wrong results from Parallel Hash Full Join

2023-04-20 Thread Justin Pryzby
On Wed, Apr 19, 2023 at 08:47:07PM -0400, Melanie Plageman wrote: > On Wed, Apr 19, 2023 at 8:41 PM Justin Pryzby wrote: > > > > On Wed, Apr 19, 2023 at 12:20:51PM -0700, Andres Freund wrote: > > > On 2023-04-19 12:16:24 -0500, Justin Pryzby wrote: > > > > On Wed, Apr 19, 2023 at 11:17:04AM -0400,

Re: Wrong results from Parallel Hash Full Join

2023-04-20 Thread Melanie Plageman
On Wed, Apr 19, 2023 at 8:43 PM Melanie Plageman wrote: > On Wed, Apr 19, 2023 at 3:20 PM Andres Freund wrote: >> On 2023-04-19 12:16:24 -0500, Justin Pryzby wrote: >> > On Wed, Apr 19, 2023 at 11:17:04AM -0400, Melanie Plageman wrote: >> > > Ultimately this is probably fine. If we wanted to modi

Re: LLVM strip -x fails

2023-04-20 Thread Andres Freund
Hi, Moving this to -hackers. On 2023-04-20 11:49:23 +0200, Palle Girgensohn wrote: > I was recently made aware of a problem building postgresql using LLVM > binutils. > > A summary: > > -- > > pgsql's build has requested to strip all non-global symbols (strip -x), but > there is at least one

Re: check_strxfrm_bug()

2023-04-20 Thread Jeff Davis
On Thu, 2023-04-20 at 13:34 +1200, Thomas Munro wrote: > I could write a patch to remove the libc strxfrm support, but since > Jeff recently wrote new code in 16 to abstract that stuff, he might > prefer to look at it? +1 to removing it. As far as how it's removed, we could directly check: if

memory context mismatch in LockMethodLocalHash entries.

2023-04-20 Thread Ashutosh Bapat
Hi All, LockMethodLocalHash hash table is allocated in memory context ""LOCALLOCK hash". LockAcquireExtended() fetches an entry from this hash table and allocates memory for locallock->lockOwners in TopMemoryContext. Thus the entry locallock and an array pointed from this entry are allocated in two

Re: Should we put command options in alphabetical order in the doc?

2023-04-20 Thread Daniel Gustafsson
> On 20 Apr 2023, at 14:40, David Rowley wrote: > I see "man grep" categorises the command line options and then sorts > alphabetically within the category. On FreeBSD and macOS "man grep" lists all options alphabetically. > FWIW, vacuumdb --help has its options in alphabetical order using the

[BUG] FK broken after DETACHing referencing part

2023-04-20 Thread Jehan-Guillaume de Rorthais
Hi, Considering two partitionned tables with a FK between them: DROP TABLE IF EXISTS p, c, c_1 CASCADE; -- -- Parent table + partition + data CREATE TABLE p ( id bigint PRIMARY KEY ) PARTITION BY list (id); CREATE TABLE p_1 PARTITION OF p FOR VA

Re: Should we put command options in alphabetical order in the doc?

2023-04-20 Thread David Rowley
On Wed, 19 Apr 2023 at 22:04, Alvaro Herrera wrote: > > On 2023-Apr-18, Peter Geoghegan wrote: > > > While I'm certain that nobody will agree with me on every little > > detail, I have to imagine that most would find my preferred ordering > > quite understandable and unsurprising, at a high level

Re: Initial Schema Sync for Logical Replication

2023-04-20 Thread Kumar, Sachin
I am working on a prototype with above discussed idea, I think I will send it for initial review by Monday. Regards Sachin

Re: Support logical replication of DDLs

2023-04-20 Thread shveta malik
On Thu, Apr 20, 2023 at 2:28 PM shveta malik wrote: > > On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: >> >> On Mon, Apr 17, 2023 at 5:32 PM Zhijie Hou (Fujitsu) >> wrote: >> > >> > Attach the new version patch set which include the following changes: >> Few comments for ddl_deparse.c in pa

Re: Should we put command options in alphabetical order in the doc?

2023-04-20 Thread Melanie Plageman
On Wed, Apr 19, 2023 at 05:33:47PM -0400, Melanie Plageman wrote: > On Wed, Apr 19, 2023 at 2:39 PM Peter Geoghegan wrote: > > > > On Wed, Apr 19, 2023 at 3:04 AM Alvaro Herrera > > wrote: > > > > While I'm certain that nobody will agree with me on every little > > > > detail, I have to imagine

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-20 Thread Yurii Rashkovskii
Aleksander, On Thu, Apr 20, 2023 at 1:22 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi, > > > Also, I don't think there's a case for distributed systems here because > we're only managing a single computer's resource: the allocation of local > ports. > > I don't suggest building

Re: Note new NULLS NOT DISTINCT on unique index tutorial page

2023-04-20 Thread David Rowley
On Thu, 20 Apr 2023 at 12:04, David Gilman wrote: > The revised patch is good. Please go ahead and commit whatever > phrasing you or the other committers find acceptable. I don't really > have any preferences in how this is exactly phrased, I just think it > should be mentioned in the docs. Thank

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-20 Thread vignesh C
On Thu, 20 Apr 2023 at 11:01, Hayato Kuroda (Fujitsu) wrote: > > Dear Vignesh, > > Thank you for reviewing! PSA new patchset. > > > > Additionally, I added a checking functions in 0003 > > > According to pg_resetwal and other functions, the length of > > CHECKPOINT_SHUTDOWN > > > record seems (Siz

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-20 Thread Aleksander Alekseev
Hi, > Also, I don't think there's a case for distributed systems here because we're > only managing a single computer's resource: the allocation of local ports. I don't suggest building a distributed system but rather using well-known solutions from this area. For the described case the "resourc

Re: Initial Schema Sync for Logical Replication

2023-04-20 Thread Amit Kapila
On Mon, Apr 17, 2023 at 9:12 AM Masahiko Sawada wrote: > > On Fri, Apr 7, 2023 at 6:37 PM Amit Kapila wrote: > > > > On Thu, Apr 6, 2023 at 6:57 PM Masahiko Sawada > > wrote: > > > > > > > > > While writing a PoC patch, I found some difficulties in this idea. > > > First, I tried to add scheman

Re: Support logical replication of DDLs

2023-04-20 Thread Amit Kapila
On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > > > Few comments for ddl_deparse.c in patch dated April17: > > 1) append_format_string() > I think we need to have 'Assert(sub_fmt)' here like we have it in all > other similar functions (append_bool_object, append_object_object, > ...) > +1.

RE: Initial Schema Sync for Logical Replication

2023-04-20 Thread Kumar, Sachin
I am working on a prototype with above Idea , and will send it for review by Sunday/Monday Regards Sachin

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-20 Thread Denis Laxalde
Hi, Yurii Rashkovskii a écrit : On Wed, Apr 19, 2023 at 11:44 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: I would like to suggest a patch against master (although it may be worth backporting it) that makes it possible to listen on any unused port. [...] A bullet-proof approach

Re: Support logical replication of DDLs

2023-04-20 Thread shveta malik
On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > On Mon, Apr 17, 2023 at 5:32 PM Zhijie Hou (Fujitsu) > wrote: > > > > Attach the new version patch set which include the following changes: > > > > Few comments for ddl_deparse.c in patch dated April17: > > Few comments for ddl_json.c in the