Re: walsender performance regression due to logical decoding on standby changes

2023-05-09 Thread Drouvot, Bertrand
Hi, On 5/9/23 11:00 PM, Andres Freund wrote: Hi, On 2023-05-09 13:38:24 -0700, Jeff Davis wrote: On Tue, 2023-05-09 at 12:02 -0700, Andres Freund wrote: I don't think the approach of not having any sort of "registry" of whether anybody is waiting for the replay position to be updated is feasi

Re: walsender performance regression due to logical decoding on standby changes

2023-05-09 Thread Bharath Rupireddy
On Wed, May 10, 2023 at 12:33 AM Andres Freund wrote: > > Unfortunately I have found the following commit to have caused a performance > regression: > > commit e101dfac3a53c20bfbf1ca85d30a368c2954facf > > The problem is that, on a standby, after the change - as needed to for the > approach to work

Re: psql tests hangs

2023-05-09 Thread Pavel Stehule
Hi When I remove this test, then all tests passed diff --git a/src/bin/psql/t/001_basic.pl b/src/bin/psql/t/001_basic.pl index 596746de17..631a1a7335 100644 --- a/src/bin/psql/t/001_basic.pl +++ b/src/bin/psql/t/001_basic.pl @@ -353,11 +353,6 @@ psql_like( # Check \watch # Note: the interval v

Re: enhancing plpgsql debug api - hooks on statements errors and function errors

2023-05-09 Thread Kirk Wolak
On Tue, Apr 25, 2023 at 11:33 AM Pavel Stehule wrote: > Hi > út 25. 4. 2023 v 10:27 odesílatel Pavel Stehule > napsal: > >> Hi >> >> When I implemented profiler and coverage check to plpgsql_check I had to >> write a lot of hard maintaining code related to corect finishing some >> operations (co

Unlinking Parallel Hash Join inner batch files sooner

2023-05-09 Thread Thomas Munro
Hi, One complaint about PHJ is that it can, in rare cases, use a surprising amount of temporary disk space where non-parallel HJ would not. When it decides that it needs to double the number of batches to try to fit each inner batch into memory, and then again and again depending on your level of

Re: [PATCH] Use RelationClose rather than table_close in heap_create_with_catalog

2023-05-09 Thread tender wang
Xiaoran Wang 于2023年3月18日周六 15:04写道: > Hi hackers, > > In heap_create_with_catalog, the Relation new_rel_desc is created > by RelationBuildLocalRelation, not table_open. So it's better to > call RelationClose to release it. > Why it's better to call RelationClose? Is there a problem if using tabl

Re: Add LZ4 compression in pg_dump

2023-05-09 Thread Michael Paquier
On Tue, May 09, 2023 at 02:12:44PM +, gkokola...@pm.me wrote: > Thank you both for looking. A small consolation is that now there are > tests for this case. +1, noticing that was pure luck ;) Worth noting that the patch posted in [1] has these tests, not the version posted in [2]. +creat

Re: fix stats_fetch_consistency value in postgresql.conf.sample

2023-05-09 Thread Justin Pryzby
On Wed, Mar 29, 2023 at 11:03:59PM -0500, Justin Pryzby wrote: > On Wed, Jul 13, 2022 at 04:49:00PM -0700, Andres Freund wrote: > > On 2022-07-14 08:46:02 +0900, Michael Paquier wrote: > > > On Wed, Jul 13, 2022 at 12:30:00PM -0500, Justin Pryzby wrote: > > > > How did you make this list ? Was it

Re: Large files for relations

2023-05-09 Thread Stephen Frost
Greetings, * Corey Huinker (corey.huin...@gmail.com) wrote: > On Wed, May 3, 2023 at 1:37 AM Thomas Munro wrote: > > On Wed, May 3, 2023 at 5:21 PM Thomas Munro > > wrote: > > > rsync --link-dest ... rsync isn't really a safe tool to use for PG backups by itself unless you're using it with arch

Re: walsender performance regression due to logical decoding on standby changes

2023-05-09 Thread Andres Freund
Hi, On 2023-05-09 13:38:24 -0700, Jeff Davis wrote: > On Tue, 2023-05-09 at 12:02 -0700, Andres Freund wrote: > > I don't think the approach of not having any sort of "registry" of > > whether > > anybody is waiting for the replay position to be updated is > > feasible. Iterating over all walsende

Re: Feature: Add reloption support for table access method

2023-05-09 Thread Andres Freund
Hi, On 2023-05-05 16:44:39 +0800, 吴昊 wrote: > When I wrote an extension to implement a new storage by table access method. > I found some issues > that the existing code has strong assumptions for heap tables now. Here are 3 > issues that I currently have: > > > 1. Index access method has a ca

Re: Large files for relations

2023-05-09 Thread Corey Huinker
On Wed, May 3, 2023 at 1:37 AM Thomas Munro wrote: > On Wed, May 3, 2023 at 5:21 PM Thomas Munro > wrote: > > rsync --link-dest > > I wonder if rsync will grow a mode that can use copy_file_range() to > share blocks with a reference file (= previous backup). Something > like --copy-range-dest.

Re: walsender performance regression due to logical decoding on standby changes

2023-05-09 Thread Jeff Davis
On Tue, 2023-05-09 at 12:02 -0700, Andres Freund wrote: > I don't think the approach of not having any sort of "registry" of > whether > anybody is waiting for the replay position to be updated is > feasible. Iterating over all walsenders slots is just too expensive - Would it work to use a shared

walsender performance regression due to logical decoding on standby changes

2023-05-09 Thread Andres Freund
Hi, Unfortunately I have found the following commit to have caused a performance regression: commit e101dfac3a53c20bfbf1ca85d30a368c2954facf Author: Andres Freund Date: 2023-04-08 00:24:24 -0700 For cascading replication, wake physical and logical walsenders separately Physical walse

Re: psql tests hangs

2023-05-09 Thread Pavel Stehule
Hi út 9. 5. 2023 v 13:53 odesílatel Pavel Stehule napsal: > > > út 9. 5. 2023 v 11:07 odesílatel Pavel Stehule > napsal: > >> >> >> út 9. 5. 2023 v 10:48 odesílatel Daniel Gustafsson >> napsal: >> >>> > On 9 May 2023, at 08:52, Pavel Stehule >>> wrote: >>> > >>> > Hi >>> > >>> > I try run ma

Re: benchmark results comparing versions 15.2 and 16

2023-05-09 Thread MARK CALLAGHAN
On Fri, May 5, 2023 at 10:01 PM MARK CALLAGHAN wrote: > I have two more runs of the benchmark in progress so we will have 3 > results for each of the test cases to confirm that the small regressions > are repeatable. > They get similar results. Then I tried Linux perf but the hierarchical call s

Re: benchmark results comparing versions 15.2 and 16

2023-05-09 Thread Andres Freund
Hi, On 2023-05-08 12:11:17 -0700, Andres Freund wrote: > Hi, > > On 2023-05-08 16:00:01 +0300, Alexander Lakhin wrote: > > This difference is confirmed by multiple test runs. `git bisect` for this > > regression pointed at f193883fc. > > I can reproduce a significant regression due to f193883fc

Re: psql \watch 2nd argument: iteration count

2023-05-09 Thread Tom Lane
Peter Eisentraut writes: > On 13.03.23 02:17, Michael Paquier wrote: >> I am not sure that this will be the last option we'll ever add to >> \watch, so I'd rather have us choose a design more flexible than >> what's proposed here, in a way similar to \g or \gx. > On the other hand, we also have o

Re: psql \watch 2nd argument: iteration count

2023-05-09 Thread Peter Eisentraut
On 13.03.23 02:17, Michael Paquier wrote: On Sun, Mar 12, 2023 at 01:05:39PM -0700, Andrey Borodin wrote: In the review above Kyotaro-san suggested that message should contain information on what it expects... So, maybe then pg_log_error("\\watch interval must be non-negative number, but argumen

Re: Order changes in PG16 since ICU introduction

2023-05-09 Thread Jeff Davis
On Tue, 2023-05-09 at 10:25 +0200, Alvaro Herrera wrote: > I agree with removing these in v16, since they are going to become > more > meaningless and confusing. Agreed, but it would be nice to have an alternative that does the right thing. It's awkward for a user to read pg_database.datlocprovid

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-05-09 Thread torikoshia
On 2023-05-07 05:05, Alena Rybakina wrote: Thanks for your reviewing and comments! I noticed that you used _ignore_datatype_errors_specified_ variable in _copy.c_ , but guc has a short name _ignore_datatype_errors_. Also you used the short variable name in _CopyFormatOptions_ structure. You ma

PostgreSQL 16 Beta 1 release date

2023-05-09 Thread Jonathan S. Katz
Hi, The release date for PostgreSQL 16 Beta 1 is scheduled for May 25, 2023. Please ensure you have committed any work for Beta 1 released committed by May 21, 2023 AoE. Thank you for your efforts with resolving open items[2] as we work to stabilize PostgreSQL 16 for GA! Thanks, Jonathan [

Re: Add LZ4 compression in pg_dump

2023-05-09 Thread gkokolatos
--- Original Message --- On Tuesday, May 9th, 2023 at 2:54 PM, Tomas Vondra wrote: > > > On 5/9/23 00:10, Michael Paquier wrote: > > > On Mon, May 08, 2023 at 08:00:39PM +0200, Tomas Vondra wrote: > > > > > The LZ4Stream_write() forgot to move the pointer to the next chunk, so

Re: Proposal for Prototype Implementation to Enhance C/C++ Interoperability in PostgreSQL

2023-05-09 Thread 盏一
I apologize for my previous hasty conclusion. I have conducted further testing on different platforms and would like to share my findings. > FreeBSD Based on my tests, it appears that FreeBSD follows the Itanium C++ ABI specification. The previous test failed because the C++ compiler was not us

Re: Allow pg_archivecleanup to remove backup history files

2023-05-09 Thread torikoshia
Horiguchi-san, Michael-san Thanks for your comments and information! Attached a patch with documentation and regression tests. On 2023-04-26 06:39, Michael Paquier wrote: On Tue, Apr 25, 2023 at 05:29:48PM +0900, Kyotaro Horiguchi wrote: I thought that we have decided not to do that, but I c

Re: Add LZ4 compression in pg_dump

2023-05-09 Thread Tomas Vondra
On 5/9/23 00:10, Michael Paquier wrote: > On Mon, May 08, 2023 at 08:00:39PM +0200, Tomas Vondra wrote: >> The LZ4Stream_write() forgot to move the pointer to the next chunk, so >> it was happily decompressing the initial chunk over and over. A bit >> embarrassing oversight :-( >> >> The custom for

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-05-09 Thread Denis Laxalde
The documentation fails to build for me: $ ninja docs [1/2] Generating doc/src/sgml/postgres-full.xml with a custom command FAILED: doc/src/sgml/postgres-full.xml /usr/bin/python3 ../postgresql/doc/src/sgml/xmltools_dep_wrapper --targetname doc/src/sgml/postgres-full.xml --depfile doc/src/sgml/

Re: drop table in transaction

2023-05-09 Thread Euler Taveira
On Tue, May 9, 2023, at 7:42 AM, Fabrice Chapuis wrote: > Where in the code is written the mechanism used for isolation when drop table > is executed in a transaction RemoveRelations() in src/backend/commands/tablecmds.c If you are looking for a previous layer, check ExecDropStmt(). -- Euler

Re: psql tests hangs

2023-05-09 Thread Pavel Stehule
út 9. 5. 2023 v 11:07 odesílatel Pavel Stehule napsal: > > > út 9. 5. 2023 v 10:48 odesílatel Daniel Gustafsson > napsal: > >> > On 9 May 2023, at 08:52, Pavel Stehule wrote: >> > >> > Hi >> > >> > I try run make check-world. Now I have problems with tests of psql >> > >> > I had to cancel test

Re: Tables getting stuck at 's' state during logical replication

2023-05-09 Thread Amit Kapila
On Fri, May 5, 2023 at 7:27 PM Padmavathi G wrote: > > Some background on the setup on which I am trying to carry out the upgrade: > > We have a pod in a kubernetes cluster which contains the postgres 11 image. > We are following the logical replication process for upgrade > > Steps followed for

Re: Cleaning up array_in()

2023-05-09 Thread Alexander Lakhin
09.05.2023 06:06, Tom Lane wrote: Alexander Lakhin writes: The only thing that confused me, is the error message (it's not new, too): select '{{1}}'::int[]; or even: select '{{'::int[]; ERROR:  number of array dimensions (7) exceeds the maximum allowed (6) Yeah, I didn'

drop table in transaction

2023-05-09 Thread Fabrice Chapuis
Where in the code is written the mechanism used for isolation when drop table is executed in a transaction Thanks for your help Fabrice

Re: Perform streaming logical transactions by background workers and parallel apply

2023-05-09 Thread Amit Kapila
On Tue, May 9, 2023 at 7:50 AM Masahiko Sawada wrote: > > On Mon, May 8, 2023 at 8:09 PM Amit Kapila wrote: > > > > > > I think it is only possible for the leader apply can worker to try to > > receive the error message from an error queue after your 0002 patch. > > Because another place already

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-05-09 Thread Hayato Kuroda (Fujitsu)
Dear Peter, Thank you for reviewing! PSA new version. > > General. > > 1. pg_dump option is documented to the user. > > I'm not sure about exposing the new pg_dump > --logical-replication-slots-only option to the user. > > I thought this pg_dump option was intended only to be called > *intern

Re: base backup vs. concurrent truncation

2023-05-09 Thread Aleksander Alekseev
Hi Robert, > I admit I haven't done the legwork to nail down a test > case where everything comes together just right to show user-visible > breakage, but your success in finding one where it doesn't is no proof > of anything. Respectfully, what made you think this was my intention? Quite the op

Re: psql tests hangs

2023-05-09 Thread Pavel Stehule
út 9. 5. 2023 v 10:48 odesílatel Daniel Gustafsson napsal: > > On 9 May 2023, at 08:52, Pavel Stehule wrote: > > > > Hi > > > > I try run make check-world. Now I have problems with tests of psql > > > > I had to cancel tests > > > > log: > > > > [08:46:49.828](0.038s) ok 63 - no ON_ERROR_STOP, -

Re: psql tests hangs

2023-05-09 Thread Daniel Gustafsson
> On 9 May 2023, at 08:52, Pavel Stehule wrote: > > Hi > > I try run make check-world. Now I have problems with tests of psql > > I had to cancel tests > > log: > > [08:46:49.828](0.038s) ok 63 - no ON_ERROR_STOP, --single-transaction and > multiple -c switches > [08:46:49.860](0.033s) ok 64

Re: Order changes in PG16 since ICU introduction

2023-05-09 Thread Alvaro Herrera
On 2023-Apr-24, Peter Eisentraut wrote: > The GUC settings lc_collate and lc_ctype are from a time when those locale > settings were cluster-global. When we made those locale settings > per-database (PG 8.4), we kept them as read-only. As of PG 15, you can use > ICU as the per-database locale pr

Re: [DOC] Update ALTER SUBSCRIPTION documentation

2023-05-09 Thread Masahiko Sawada
On Tue, May 9, 2023 at 3:40 PM Amit Kapila wrote: > > On Mon, May 8, 2023 at 1:51 PM Masahiko Sawada wrote: > > > > Apart from the documentation change, given that setting slot_name = > > NONE always requires for the subscription to be disabled beforehand, > > does it make sense to change ALTER S

Re: Improve list manipulation in several places

2023-05-09 Thread Alvaro Herrera
On 2023-May-08, Ranier Vilela wrote: > Em seg., 8 de mai. de 2023 às 14:26, Alvaro Herrera > escreveu: > > > The problem I see is that each of these new functions has a single > > caller, and the only one that looks like it could have a performance > > advantage is list_copy_move_nth_to_head() (

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-05-09 Thread Drouvot, Bertrand
Hi, On 5/9/23 8:02 AM, Amit Kapila wrote: On Mon, May 8, 2023 at 1:45 PM Drouvot, Bertrand wrote: Why not initialize the cascading standby node just before the standby promotion test: "Test standby promotion and logical decoding behavior after the standby gets promoted."? That way we will a