Random pg_upgrade test failure on drongo

2023-11-08 Thread Hayato Kuroda (Fujitsu)
Dear hackers, While tracking a buildfarm, I found that drongo failed the test pg_upgrade/003_logical_slots [1]. A strange point is that the test passed in the next iteration. Currently I'm not sure the reason, but I will keep my eye for it and will investigate if it happens again. I think this f

Re: Fix use of openssl.path() if openssl isn't found

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 12:07:49AM -0600, Tristan Partin wrote: >'with_ssl': ssl_library, > - 'OPENSSL': openssl.path(), > + 'OPENSSL': openssl.found() ? openssl.path : '', Except that this was incorrect. I've fixed the grammar and applied that down to 16. -- Michael signature

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-11-08 Thread Amit Kapila
On Wed, Nov 8, 2023 at 8:44 AM vignesh C wrote: > > While verifying upgrade of subscriber patch, I found one issue with > upgrade in verbose mode. > I was able to reproduce this issue by performing a upgrade with a > verbose option. > > The trace for the same is given below: > Program received sig

Re: Remove MSVC scripts from the tree

2023-11-08 Thread Peter Eisentraut
On 12.10.23 07:23, Michael Paquier wrote: - Is src/backend/utils/README.Gen_dummy_probes still correct after this? AFAICT, the Perl-based MSVC build system uses Gen_dummy_probes.pl, but the meson build uses Gen_dummy_probes.sed even on Windows. Is that correct, intended? Interesting point. Thi

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Bharath Rupireddy
On Wed, Nov 8, 2023 at 9:43 AM Andres Freund wrote: > > > 2. > > +{ oid => '8000', > > + descr => 'statistics: reset collected statistics shared across the > > cluster', > > + proname => 'pg_stat_reset_shared', provolatile => 'v', prorettype => > > 'void', > > + proargtypes => '', prosrc => '

Re: Synchronizing slots from primary to standby

2023-11-08 Thread Amit Kapila
On Wed, Nov 8, 2023 at 12:32 PM Drouvot, Bertrand wrote: > > Hi, > > On 11/8/23 4:50 AM, Amit Kapila wrote: > > > I think if we want to follow > > this approach then we need to also monitor these slots for any change > > in the consecutive cycles and if we are able to sync them then > > accordingl

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-08 Thread Ants Aasma
On Sat, 4 Nov 2023 at 22:08, Andrey M. Borodin wrote: > On 30 Oct 2023, at 09:20, Dilip Kumar wrote: > > changed the logic of SlruAdjustNSlots() in 0002, such that now it > starts with the next power of 2 value of the configured slots and > keeps doubling the number of banks until we reach the n

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-08 Thread Andrey M. Borodin
> On 8 Nov 2023, at 14:17, Ants Aasma wrote: > > Is there a particular reason why lock partitions need to be bigger? We have > one lock per buffer anyway, bankwise locks will increase the number of locks > < 10%. The problem was not attracting much attention for some years. So my reasoning

Re: Synchronizing slots from primary to standby

2023-11-08 Thread Drouvot, Bertrand
Hi, On 11/8/23 9:57 AM, Amit Kapila wrote: On Wed, Nov 8, 2023 at 12:32 PM Drouvot, Bertrand wrote: Hi, On 11/8/23 4:50 AM, Amit Kapila wrote: I think if we want to follow this approach then we need to also monitor these slots for any change in the consecutive cycles and if we are able to

Re: Protocol question regarding Portal vs Cursor

2023-11-08 Thread Dave Cramer
Dave Cramer On Tue, 7 Nov 2023 at 10:26, Tom Lane wrote: > Dave Cramer writes: > > If we use a Portal it is possible to open the portal and do a describe > and > > then Fetch N records. > > > Using a Cursor we open the cursor. Is there a corresponding describe and > a > > way to fetch N record

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-08 Thread Tomas Vondra
Hi, here's an updated patch, addressing the review comments, and reworking how the work is divided between the workers & leader etc. 0001 is just v2, rebased to current master 0002 and 0003 address most of the issues, in particular it - removes the unnecessary spool - fixes bs_reltuples typ

Re: meson documentation build open issues

2023-11-08 Thread Peter Eisentraut
On 07.11.23 17:08, Andres Freund wrote: make world make check-world make install-world that encompasses everything. Now with meson to handle the documentation one needs to remember a variety of additional targets. (There is a risk that once this gets more widespread, more people will submit

Re: meson documentation build open issues

2023-11-08 Thread Peter Eisentraut
On 07.11.23 17:40, Alvaro Herrera wrote: If the problem is broken doc patches, then maybe a solution is to include the `xmllint --noout --valid` target in whatever the check-world equivalent is for meson. Looking at doc/src/sgml/meson.build, we don't seem to do that anywhere. Doing the no-outpu

Re: ResourceOwner refactoring

2023-11-08 Thread Heikki Linnakangas
On 07/11/2023 16:00, Alexander Lakhin wrote: 07.11.2023 14:28, Heikki Linnakangas wrote: I feel pretty good about this overall. Barring objections or new cfbot failures, I will commit this in the next few days. A script, that I published in [1], detects several typos in the patches: betwen t

Re: pgsql: Clean up role created in new subscription test.

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-08, Peter Eisentraut wrote: > I think the earlier idea of just counting roles, tablespaces, etc. before > and after would be sufficient. Maybe record global objects in a permanent table in test_setup.sql create table global_objs as select 'role', rolname from pg_roles union all selec

Re: Commitfest: older Waiting on Author entries

2023-11-08 Thread Tommy Pavlicek
On Tue, Nov 7, 2023 at 8:28 AM John Naylor wrote: > > The following entries are WoA but have not had an email update since > July. If there is no actionable update soon, I plan to mark these > Returned with Feedback before end of CF: > > ltree hash functions I should be able to update this with a

Re: Synchronizing slots from primary to standby

2023-11-08 Thread shveta malik
On Wed, Nov 8, 2023 at 3:19 PM Drouvot, Bertrand wrote: > > Hi, > > On 11/8/23 9:57 AM, Amit Kapila wrote: > > On Wed, Nov 8, 2023 at 12:32 PM Drouvot, Bertrand > > wrote: > >> > >> Hi, > >> > >> On 11/8/23 4:50 AM, Amit Kapila wrote: > >> > >>> I think if we want to follow > >>> this approach th

64-bit integer subtraction bug on some platforms

2023-11-08 Thread Dean Rasheed
One of the new tests in the infinite interval patch has revealed a bug in our 64-bit integer subtraction code. Consider the following: select 0::int8 - '-9223372036854775808'::int8; This should overflow, since the correct result (+9223372036854775808) is out of range. However, on platforms withou

Re: Infinite Interval

2023-11-08 Thread Dean Rasheed
On Wed, 8 Nov 2023 at 06:56, jian he wrote: > > > On Tue, 7 Nov 2023 at 14:33, Dean Rasheed wrote: > > > > Ah, Windows Server didn't like that. Trying again with "INT64CONST(0)" > > instead of just "0" in interval_um(). > > > I found this: > https://developercommunity.visualstudio.com/t/please-im

Re: Version 14/15 documentation Section "Alter Default Privileges"

2023-11-08 Thread Laurenz Albe
On Tue, 2023-11-07 at 17:30 -0500, Bruce Momjian wrote: > You didn't seem to like my SET ROLE suggestion so I removed it. I thought that the information that you can use SET ROLE to assume the identity of another role is correct, but leads a bit too far in the manual page of ALTER DEFAULT PRIVILEG

Re: pgsql: Clean up role created in new subscription test.

2023-11-08 Thread Daniel Gustafsson
> On 8 Nov 2023, at 08:55, Peter Eisentraut wrote: > > On 06.07.23 00:00, Daniel Gustafsson wrote: >>> On 16 May 2023, at 11:17, Daniel Gustafsson wrote: >>> Parked in the July CF for now. >> Rebased to fix a trivial conflict highlighted by the CFBot. > > I think the problem with this approach

Re: 64-bit integer subtraction bug on some platforms

2023-11-08 Thread Laurenz Albe
On Wed, 2023-11-08 at 11:58 +, Dean Rasheed wrote: > One of the new tests in the infinite interval patch has revealed a bug > in our 64-bit integer subtraction code. Consider the following: > > select 0::int8 - '-9223372036854775808'::int8; > > This should overflow, since the correct result (

Re: Fix output of zero privileges in psql

2023-11-08 Thread Laurenz Albe
On Wed, 2023-11-08 at 10:56 +0530, Shubham Khanna wrote: > I tested the Patch for the modified changes and it is working fine. Thanks for the review! I wonder how to proceed with this patch. The main disagreement is whether default privileges should be displayed as NULL (less invasive, but more

Re: pgsql: Clean up role created in new subscription test.

2023-11-08 Thread Daniel Gustafsson
> On 8 Nov 2023, at 12:42, Alvaro Herrera wrote: > On 2023-Nov-08, Peter Eisentraut wrote: >> I think the earlier idea of just counting roles, tablespaces, etc. before >> and after would be sufficient. > > Maybe record global objects in a permanent table in test_setup.sql Since test_setup.sql i

Re: pgsql: Clean up role created in new subscription test.

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-08, Daniel Gustafsson wrote: > Since test_setup.sql is part of the regress schedule and not pg_regress we > would have to implement this for each test run (regress, contribs etc), which > is what Peter didn't like about the original suggestion. Oh, somehow that aspect of his reply fai

Re: Force the old transactions logs cleanup even if checkpoint is skipped

2023-11-08 Thread Zakhlystov, Daniil (Nebius)
Hi! Thanks for your review. > I am not sure to understand your last sentence here. Once the > archiver is back up, you mean that the WAL segments that were not > previously archived still are still not archived? Or do you mean that > because of a succession of checkpoint skipped we are just ena

Re: speed up a logical replica setup

2023-11-08 Thread Euler Taveira
On Tue, Nov 7, 2023, at 8:12 PM, Michael Paquier wrote: > On Tue, Nov 07, 2023 at 10:00:39PM +0100, Peter Eisentraut wrote: > > Speaking of which, would it make sense to put this tool (whatever the name) > > into the pg_basebackup directory? It's sort of related, and it also shares > > some code.

Re: meson documentation build open issues

2023-11-08 Thread Christoph Berg
Re: Peter Eisentraut > > If the problem is broken doc patches, then maybe a solution is to > > include the `xmllint --noout --valid` target in whatever the check-world > > equivalent is for meson. Looking at doc/src/sgml/meson.build, we don't > > seem to do that anywhere. Doing the no-output lint

Re: Syncrep and improving latency due to WAL throttling

2023-11-08 Thread Tomas Vondra
On 11/8/23 07:40, Andres Freund wrote: > Hi, > > On 2023-11-04 20:00:46 +0100, Tomas Vondra wrote: >> scope >> - >> Now, let's talk about scope - what the patch does not aim to do. The >> patch is explicitly intended for syncrep clusters, not async. There have >> been proposals to also support

Re: pgsql: Clean up role created in new subscription test.

2023-11-08 Thread Daniel Gustafsson
> On 8 Nov 2023, at 13:32, Alvaro Herrera wrote: > > On 2023-Nov-08, Daniel Gustafsson wrote: > >> Since test_setup.sql is part of the regress schedule and not pg_regress we >> would have to implement this for each test run (regress, contribs etc), which >> is what Peter didn't like about the or

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Matthias van de Meent
On Wed, 8 Nov 2023 at 05:13, Andres Freund wrote: > > Hi, > > On 2023-11-06 14:00:13 +0530, Bharath Rupireddy wrote: > > Well, that's a total of ~17 LWLocks this new function takes to make > > the stats reset atomic. I'm not sure if this atomicity is worth the > > effort which can easily be misuse

Re: [PATCHES] Post-special page storage TDE support

2023-11-08 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2023-05-09 17:08:26 -0500, David Christensen wrote: > > From 965309ea3517fa734c4bc89c144e2031cdf6c0c3 Mon Sep 17 00:00:00 2001 > > From: David Christensen > > Date: Tue, 9 May 2023 16:56:15 -0500 > > Subject: [PATCH v4 1/3] Add reserved_

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-11-08 Thread Aleksander Alekseev
Maxim, I see both of us accounted for Alexanders feedback and submitted v59. Your newer version seems to have issues on cfbot, so resubmitting the previous patchset that passes the tests. Please feel free to add changes. > See SlruCorrectSegmentFilenameLength(): > > ``` > if (ctl->long_segmen

Re: Synchronizing slots from primary to standby

2023-11-08 Thread Drouvot, Bertrand
Hi, On 11/8/23 12:50 PM, shveta malik wrote: On Wed, Nov 8, 2023 at 3:19 PM Drouvot, Bertrand wrote: Hi, On 11/8/23 9:57 AM, Amit Kapila wrote: On Wed, Nov 8, 2023 at 12:32 PM Drouvot, Bertrand wrote: Hi, On 11/8/23 4:50 AM, Amit Kapila wrote: I think if we want to follow this approac

Re: Remove MSVC scripts from the tree

2023-11-08 Thread Andrew Dunstan
On 2023-11-08 We 03:41, Peter Eisentraut wrote: On 12.10.23 07:23, Michael Paquier wrote: - Is src/backend/utils/README.Gen_dummy_probes still correct after this? AFAICT, the Perl-based MSVC build system uses Gen_dummy_probes.pl, but the meson build uses Gen_dummy_probes.sed even on Windows. 

Re: Cleaning up array_in()

2023-11-08 Thread Alexander Lakhin
Hello Tom, 08.11.2023 02:52, Tom Lane wrote: Comments? Thank you for the update! I haven't looked into the code, just did manual testing and rechecked commands given in the arrays documentation ([1]). Everything works correctly, except for one minor difference: INSERT INTO sal_emp     VALUES (

XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Justin Pryzby
I found this in our logs, and reproduced it under v11-v16. CREATE TABLE t(a int, b int); INSERT INTO t SELECT generate_series(1,999); CREATE STATISTICS t_stats ON a,b FROM t; while :; do psql postgres -qtxc "ANALYZE t"; done & while :; do psql postgres -qtxc "begin; DROP STATISTICS t_stats"; done

Re: meson documentation build open issues

2023-11-08 Thread Peter Eisentraut
On 08.11.23 13:55, Christoph Berg wrote: Re: Peter Eisentraut If the problem is broken doc patches, then maybe a solution is to include the `xmllint --noout --valid` target in whatever the check-world equivalent is for meson. Looking at doc/src/sgml/meson.build, we don't seem to do that anywher

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tomas Vondra
On 11/8/23 16:10, Justin Pryzby wrote: > I found this in our logs, and reproduced it under v11-v16. > > CREATE TABLE t(a int, b int); > INSERT INTO t SELECT generate_series(1,999); > CREATE STATISTICS t_stats ON a,b FROM t; > > while :; do psql postgres -qtxc "ANALYZE t"; done & > while :; do psq

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tom Lane
Tomas Vondra writes: > On 11/8/23 16:10, Justin Pryzby wrote: >> I found this in our logs, and reproduced it under v11-v16. >> >> CREATE TABLE t(a int, b int); >> INSERT INTO t SELECT generate_series(1,999); >> CREATE STATISTICS t_stats ON a,b FROM t; >> >> while :; do psql postgres -qtxc "ANALY

Re: meson documentation build open issues

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-08, Peter Eisentraut wrote: > I think we could build doc/src/sgml/postgres-full.xml by default. That > takes less than 0.5 seconds here and it's an intermediate target for html > and man. If that detects problems like the id attributes you mentioned, apart from the other checks in th

Re: meson documentation build open issues

2023-11-08 Thread Tristan Partin
Looks good to me. Thanks for finding this. -- Tristan Partin Neon (https://neon.tech)

Re: Cleaning up array_in()

2023-11-08 Thread Tom Lane
Alexander Lakhin writes: > Thank you for the update! I haven't looked into the code, just did manual > testing and rechecked commands given in the arrays documentation ([1]). > Everything works correctly, except for one minor difference: > INSERT INTO sal_emp >     VALUES ('Bill', >     '{1,

Re: Fix use of openssl.path() if openssl isn't found

2023-11-08 Thread Tristan Partin
On Wed Nov 8, 2023 at 2:31 AM CST, Michael Paquier wrote: On Wed, Nov 08, 2023 at 12:07:49AM -0600, Tristan Partin wrote: >'with_ssl': ssl_library, > - 'OPENSSL': openssl.path(), > + 'OPENSSL': openssl.found() ? openssl.path : '', Except that this was incorrect. I've fixed the

Re: 64-bit integer subtraction bug on some platforms

2023-11-08 Thread Tom Lane
Laurenz Albe writes: > On Wed, 2023-11-08 at 11:58 +, Dean Rasheed wrote: >> This should overflow, since the correct result (+9223372036854775808) >> is out of range. However, on platforms without integer overflow >> builtins or 128-bit integers, pg_sub_s64_overflow() does the >> following: >>

Re: Call pqPipelineFlush from PQsendFlushRequest

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-07, Michael Paquier wrote: > On Tue, Nov 07, 2023 at 10:38:04AM +0100, Jelte Fennema-Nio wrote: > > In pipeline mode after queuing a message to be sent we would flush the > > buffer if the size of the buffer passed some threshold. The only message > > type that we didn't do that for wa

Re: Add PQsendSyncMessage() to libpq

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-07, Jelte Fennema-Nio wrote: > I think this function should be named something with the "PQsend" > prefix since that's the way we name all our public async message > sending functions in libpq. The "Put" word we only use in internal > libpq functions, so I feel it has no place in the e

Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tristan Partin
clang and gcc both now support -fsanitize=address,undefined. These are really useful to me personally when trying to debug issues. Unfortunately ecpg code has a ton of memory leaks, which makes builds really painful. It would be great to fix all of them, but I don't have the patience to try to

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tom Lane
"Tristan Partin" writes: > clang and gcc both now support -fsanitize=address,undefined. These are > really useful to me personally when trying to debug issues. > Unfortunately ecpg code has a ton of memory leaks, which makes builds > really painful. It would be great to fix all of them, but I d

Re: Syncrep and improving latency due to WAL throttling

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 13:59:55 +0100, Tomas Vondra wrote: > > I used netperf's tcp_rr between my workstation and my laptop on a local > > 10Gbit > > network (albeit with a crappy external card for my laptop), to put some > > numbers to this. I used -r $s,100 to test sending a variable sized data to

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Michael Meskes
Am Mittwoch, dem 08.11.2023 um 12:07 -0500 schrieb Tom Lane: > "Tristan Partin" writes: > > clang and gcc both now support -fsanitize=address,undefined. These > > are > > really useful to me personally when trying to debug issues. > > Unfortunately ecpg code has a ton of memory leaks, which make

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-11-08 Thread Andres Freund
Hi, On 2023-11-07 15:55:48 -0500, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2023-11-06 13:02:50 -0500, Stephen Frost wrote: > > > > > The max_total_memory limit is checked whenever the global counters are > > > > > updated. There is no special error handling if a me

Re: pg_upgrade and logical replication

2023-11-08 Thread vignesh C
On Thu, 2 Nov 2023 at 17:01, Amit Kapila wrote: > > On Thu, Nov 2, 2023 at 3:41 PM vignesh C wrote: > > > > I have slightly modified it now and also made it consistent with the > > replication slot upgrade, but I was not sure if we need to add > > anything more. Let me know if anything else needs

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 12:04:30 +0100, Peter Eisentraut wrote: > Ok, I didn't know about ninja install-world. That works for me. Maybe a > "world" target would also be good. Yea, I thought so as well. I'll send out a patch shortly. Kinda wondering if its worth backpatching to 16. Uniformity seems us

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-11-08 Thread vignesh C
On Wed, 8 Nov 2023 at 08:43, vignesh C wrote: > > On Tue, 7 Nov 2023 at 13:25, Amit Kapila wrote: > > > > On Tue, Nov 7, 2023 at 10:01 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Tuesday, November 7, 2023 12:14 PM Kuroda, Hayato/黒田 隼人 > > > wrote: > > > > > > > > Dear hackers, > > > > >

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tristan Partin
On Wed Nov 8, 2023 at 11:18 AM CST, Michael Meskes wrote: Am Mittwoch, dem 08.11.2023 um 12:07 -0500 schrieb Tom Lane: > "Tristan Partin" writes: > > clang and gcc both now support -fsanitize=address,undefined. These > > are > > really useful to me personally when trying to debug issues. > > U

Re: Version 14/15 documentation Section "Alter Default Privileges"

2023-11-08 Thread Bruce Momjian
On Wed, Nov 8, 2023 at 01:12:24PM +0100, Laurenz Albe wrote: > On Tue, 2023-11-07 at 17:30 -0500, Bruce Momjian wrote: > > You didn't seem to like my SET ROLE suggestion so I removed it. > > I thought that the information that you can use SET ROLE to assume > the identity of another role is corre

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tom Lane
"Tristan Partin" writes: > On Wed Nov 8, 2023 at 11:18 AM CST, Michael Meskes wrote: >> Agreed, it's not exactly uncommon for tools like ecpg to not worry >> about memory. After all it gets freed when the program ends. > In the default configuration of AddressSanitizer, I can't even complete > a

Re: cataloguing NOT NULL constraints

2023-11-08 Thread Alvaro Herrera
On 2023-Oct-12, Alexander Lakhin wrote: Hello, > I've discovered that that commit added several recursive functions, and > some of them are not protected from stack overflow. True. I reproduced the first two, but didn't attempt to reproduce the third one -- patching all these to check for stack

Re: ResourceOwner refactoring

2023-11-08 Thread Alexander Lakhin
Hello Heikki, 08.11.2023 14:37, Heikki Linnakangas wrote: Fixed these, and pushed. Thanks everyone for reviewing! Please look at a new assertion failure, I've managed to trigger with this script: CREATE TABLE t( i01 int, i02 int, i03 int, i04 int, i05 int, i06 int, i07 int, i08 int, i09 in

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-08 Thread Robert Haas
On Tue, Oct 31, 2023 at 7:39 PM Michael Paquier wrote: > Point 7. of what you quote says to use one? True that this needs a > refresh, and perhaps a bit fat warning about the fact that these are > required if you want to fetch WAL from other sources than the local > pg_wal/. Perhaps there may be

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Tom Lane
Damir writes: > [ v7-0002-Add-new-COPY-option-IGNORE_DATATYPE_ERRORS.patch ] Sorry for being so late to the party, but ... I don't think this is a well-designed feature as it stands. Simply dropping failed rows seems like an unusable definition for any application that has pretensions of robustn

Re: Syncrep and improving latency due to WAL throttling

2023-11-08 Thread Tomas Vondra
On 11/8/23 18:11, Andres Freund wrote: > Hi, > > On 2023-11-08 13:59:55 +0100, Tomas Vondra wrote: >>> I used netperf's tcp_rr between my workstation and my laptop on a local >>> 10Gbit >>> network (albeit with a crappy external card for my laptop), to put some >>> numbers to this. I used -r $

Re: XID-wraparound hazards in LISTEN/NOTIFY

2023-11-08 Thread Bruce Momjian
On Sat, Nov 23, 2019 at 12:10:56PM -0500, Tom Lane wrote: > Mark Dilger writes: > > On 11/23/19 8:34 AM, Tom Lane wrote: > >> It suddenly strikes me to worry that we have an XID wraparound hazard > >> for entries in the notify queue. > > > Is it worth checking for this condition in autovacuum? >

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Alexander Lakhin
Hello Tristan, 08.11.2023 20:37, Tristan Partin wrote: Are people using some suppression file or setting ASAN_OPTIONS to something? I use the following: ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:print_stacktrace=1:\ disable_coredump=0:strict_string_checks=1:check_initialization_order=1:\ s

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tomas Vondra
On 11/8/23 16:52, Tom Lane wrote: > Tomas Vondra writes: >> On 11/8/23 16:10, Justin Pryzby wrote: >>> I found this in our logs, and reproduced it under v11-v16. >>> >>> CREATE TABLE t(a int, b int); >>> INSERT INTO t SELECT generate_series(1,999); >>> CREATE STATISTICS t_stats ON a,b FROM t; >>>

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Daniel Gustafsson
> On 8 Nov 2023, at 19:18, Tom Lane wrote: > I think an actually usable feature of this sort would involve > copying all the failed lines to some alternate output medium, > perhaps a second table with a TEXT column to receive the original > data line. (Or maybe an array of text that could receiv

Re: XID-wraparound hazards in LISTEN/NOTIFY

2023-11-08 Thread Tom Lane
Bruce Momjian writes: > On Sat, Nov 23, 2019 at 12:10:56PM -0500, Tom Lane wrote: >>> It suddenly strikes me to worry that we have an XID wraparound hazard >>> for entries in the notify queue. > Is this still an open issue? Should it be a TODO item? I don't think anyone's done anything about it

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tom Lane
Tomas Vondra writes: > On 11/8/23 16:52, Tom Lane wrote: >> Shouldn't DROP STATISTICS be taking a lock on the associated table >> that is strong enough to lock out ANALYZE? > Yes, I think that's the correct thing to do. I recall having a > discussion about this with someone while working on the p

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Tom Lane
Daniel Gustafsson writes: >> On 8 Nov 2023, at 19:18, Tom Lane wrote: >> I think an actually usable feature of this sort would involve >> copying all the failed lines to some alternate output medium, >> perhaps a second table with a TEXT column to receive the original >> data line. (Or maybe an

Re: XID-wraparound hazards in LISTEN/NOTIFY

2023-11-08 Thread Bruce Momjian
On Wed, Nov 8, 2023 at 02:52:16PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Sat, Nov 23, 2019 at 12:10:56PM -0500, Tom Lane wrote: > >>> It suddenly strikes me to worry that we have an XID wraparound hazard > >>> for entries in the notify queue. > > > Is this still an open issue? Sh

Re: max_parallel_workers question

2023-11-08 Thread Bruce Momjian
On Sat, Sep 28, 2019 at 12:10:53AM -0400, Robert Haas wrote: > On Fri, Sep 27, 2019 at 8:07 PM Jeff Davis wrote: > > The current docs for max_parallel_workers start out: > > > > "Sets the maximum number of workers that the system can support for > > parallel operations..." > > > > In my interpreta

Re: Syncrep and improving latency due to WAL throttling

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 19:29:38 +0100, Tomas Vondra wrote: > >>> I haven't checked, but I'd assume that 100bytes back and forth should > >>> easily > >>> fit a new message to update LSNs and the existing feedback response. Even > >>> just > >>> the difference between sending 100 bytes and sending 10k

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tomas Vondra
On 11/8/23 20:58, Tom Lane wrote: > Tomas Vondra writes: >> On 11/8/23 16:52, Tom Lane wrote: >>> Shouldn't DROP STATISTICS be taking a lock on the associated table >>> that is strong enough to lock out ANALYZE? > >> Yes, I think that's the correct thing to do. I recall having a >> discussion abo

Re: Wrong sentence in the README?

2023-11-08 Thread Bruce Momjian
On Sun, Sep 22, 2019 at 12:28:02PM -0400, Tom Lane wrote: > "Daniel Westermann (DWE)" writes: > > in the README, top level, there is this: > > > PostgreSQL has many language interfaces, many of which are listed here: > > https://www.postgresql.org/download > > > I don't think the download page l

Re: Wrong rows estimations with joins of CTEs slows queries by more than factor 500

2023-11-08 Thread Tom Lane
Jian Guo writes: > I found a new approach to fix this issue, which seems better, so I would like > to post another version of the patch here. The origin patch made the > assumption of the values of Vars from CTE must be unique, which could be very > wrong. This patch examines variables for Vars

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 6:47 PM Andres Freund wrote: > Hi, > > On 2023-11-06 11:26:44 +0100, Matthias van de Meent wrote: > > On Sat, 4 Nov 2023 at 03:38, Andres Freund wrote: > > > On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > > > > I'm quite surprised at the significant number of

Re: GUC names in messages

2023-11-08 Thread Peter Smith
On Wed, Nov 8, 2023 at 7:40 AM Peter Smith wrote: > > FWIW, I am halfway through doing regex checking of the PG16 source for > all GUC names in messages to see what current styles are in use today. > > Not sure if those numbers will influence the decision. > > I hope I can post my findings today o

Re: pg_upgrade and logical replication

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 10:52:29PM +0530, vignesh C wrote: > Upgrading logical replication nodes requires multiple steps to be > performed. Because not all operations are transactional, the user is > advised to take backups. > Backups can be taken as described in > https://www.postgresql.org/docs/c

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 5:49 PM Andres Freund wrote: > Hi, > > On 2023-11-06 09:56:37 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > I still am quite quite unconvinced that using the LSN as a nonce is a > good > > > design decision. > > > > This is a really impor

Re: Remove MSVC scripts from the tree

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 09:41:19AM +0100, Peter Eisentraut wrote: > I don't think we should rely on sed being there on Windows. Maybe it's true > now on the handful of buildfarm/CI machines and early adopters, but do we > have any indication that that is systematic or just an accident? Or both?

Re: speed up a logical replica setup

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 09:50:47AM -0300, Euler Taveira wrote: > On Tue, Nov 7, 2023, at 8:12 PM, Michael Paquier wrote: > I used the CreateReplicationSlot() from streamutil.h but decided to use the > CREATE_REPLICATION_SLOT command directly because it needs the LSN as output. > As > you noticed a

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, I really like the idea of an 'help' target that prints the targets. It seemed annoying to document such targets in both the sgml docs and the input for a the help target. Particularly due to the redundancies between id attributes, the target name etc. First I generated the list of targets fro

Re: ResourceOwner refactoring

2023-11-08 Thread Heikki Linnakangas
On 08/11/2023 20:00, Alexander Lakhin wrote: Please look at a new assertion failure, I've managed to trigger with this script: CREATE TABLE t( i01 int, i02 int, i03 int, i04 int, i05 int, i06 int, i07 int, i08 int, i09 int, i10 int, i11 int, i12 int, i13 int, i14 int, i15 int, i16 int, i17 int,

Some deleted GUCs are still referred to

2023-11-08 Thread Peter Smith
Hi, I happened to notice that some GUC names "max_fsm_pages" and "max_fsm_relations" are still mentioned in these translation files (from the REL_16_1 source zip) src\backend\po\fr.po src\backend\po\tr.po ~~ Should those be removed? There was a commit [1] that said these all traces of those GU

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 13:18:39 -0500, Tom Lane wrote: > Damir writes: > > [ v7-0002-Add-new-COPY-option-IGNORE_DATATYPE_ERRORS.patch ] > > Sorry for being so late to the party, but ... I don't think this > is a well-designed feature as it stands. Simply dropping failed rows > seems like an unusable

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 02:15:24PM +0530, Bharath Rupireddy wrote: > On Wed, Nov 8, 2023 at 9:43 AM Andres Freund wrote: >> It's not like oids are a precious resource. It's a more confusing API to have >> to have to specify a NULL as an argument than not having to do so. If we >> really want to av

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Tom Lane
Andres Freund writes: > On 2023-11-08 13:18:39 -0500, Tom Lane wrote: >> I think an actually usable feature of this sort would involve >> copying all the failed lines to some alternate output medium, >> perhaps a second table with a TEXT column to receive the original >> data line. > If we go in

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 19:00:01 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-11-08 13:18:39 -0500, Tom Lane wrote: > >> I think an actually usable feature of this sort would involve > >> copying all the failed lines to some alternate output medium, > >> perhaps a second table with a TEXT

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Damir Belyalov
Hello everyone! Thanks for turning back to this patch. I had already thought about storing errors in the table / separate file / logfile and it seems to me that the best way is to output errors in logfile. As for user it is more convenient to look for errors in the place where they are usually ge

Re: Force the old transactions logs cleanup even if checkpoint is skipped

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 12:44:09PM +, Zakhlystov, Daniil (Nebius) wrote: >> I am not sure to understand your last sentence here. Once the >> archiver is back up, you mean that the WAL segments that were not >> previously archived still are still not archived? Or do you mean that >> because of

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 13:55:02 +0100, Christoph Berg wrote: > Re: Peter Eisentraut > > > If the problem is broken doc patches, then maybe a solution is to > > > include the `xmllint --noout --valid` target in whatever the check-world > > > equivalent is for meson. Looking at doc/src/sgml/meson.build,

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 16:19:51 +0100, Peter Eisentraut wrote: > On 08.11.23 13:55, Christoph Berg wrote: > > Re: Peter Eisentraut > > > > If the problem is broken doc patches, then maybe a solution is to > > > > include the `xmllint --noout --valid` target in whatever the check-world > > > > equivalen

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread torikoshia
On 2023-11-09 08:58, Michael Paquier wrote: On Wed, Nov 08, 2023 at 02:15:24PM +0530, Bharath Rupireddy wrote: On Wed, Nov 8, 2023 at 9:43 AM Andres Freund wrote: It's not like oids are a precious resource. It's a more confusing API to have to have to specify a NULL as an argument than not hav

Re: ensure, not insure

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 08:31:28PM +1300, David Rowley wrote: > On Wed, 8 Nov 2023 at 19:56, Peter Smith wrote: >> In passing I found/fixed a bunch of similar misuses in comments. > > Those all look fine to me too. +1. -- Michael signature.asc Description: PGP signature

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 10:10:39AM +0900, torikoshia wrote: > I am a little concerned about that the reset time is not the same and that > GetCurrentTimestamp() is called multiple times, but I think it would be > acceptable because the function is probably not used that often and the > reset time i

Re: make pg_ctl more friendly

2023-11-08 Thread Crisp Lee
How to judge from 'DB_SHUTDOWNED' that PITR ends normally? 'DB_SHUTDOWNED' is just a state, it could not give more meaning, so I reuse the recovery.done. On Sat, Nov 4, 2023 at 9:56 AM Andres Freund wrote: > Hi, > > On 2023-11-02 14:50:14 +0800, Crisp Lee wrote: > > I got a basebackup using pg_b

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Andres Freund
Hi, On 2023-11-09 10:25:18 +0900, Michael Paquier wrote: > On Thu, Nov 09, 2023 at 10:10:39AM +0900, torikoshia wrote: > > I am a little concerned about that the reset time is not the same and that > > GetCurrentTimestamp() is called multiple times, but I think it would be > > acceptable because t

Re: make pg_ctl more friendly

2023-11-08 Thread Andres Freund
Hi, On 2023-11-09 09:29:32 +0800, Crisp Lee wrote: > How to judge from 'DB_SHUTDOWNED' that PITR ends normally? 'DB_SHUTDOWNED' > is just a state, it could not give more meaning, so I reuse the > recovery.done. DB_SHUTDOWNED cannot be encountered while recovery is ongoing. If there was a hard cra

Re: [PATCHES] Post-special page storage TDE support

2023-11-08 Thread David Christensen
On Wed, Nov 8, 2023 at 8:04 AM Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > On 2023-05-09 17:08:26 -0500, David Christensen wrote: > > > From 965309ea3517fa734c4bc89c144e2031cdf6c0c3 Mon Sep 17 00:00:00 2001 > > > From: David Christensen > > > Date: Tue,

  1   2   >