Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Richard Guo
On Wed, Jun 28, 2023 at 6:28 AM Tom Lane wrote: > For a real fix, I'm inclined to extend the loop that calculates > param_source_rels (in add_paths_to_joinrel) so that it also tracks > a set of incompatible relids that *must not* be present in the > parameterization of a proposed path. This woul

Re: Synchronizing slots from primary to standby

2023-06-27 Thread Drouvot, Bertrand
Hi, On 6/26/23 12:34 PM, Amit Kapila wrote: On Mon, Jun 26, 2023 at 11:15 AM Drouvot, Bertrand wrote: On 6/20/23 12:22 PM, Amit Kapila wrote: On Mon, Jun 19, 2023 at 9:56 PM Drouvot, Bertrand wrote: In such a case (slot valid on the primary but invalidated on the standby) then I think w

Re: Do we want a hashset type?

2023-06-27 Thread jian he
On Tue, Jun 27, 2023 at 4:27 PM Joel Jacobson wrote: > > On Tue, Jun 27, 2023, at 04:35, jian he wrote: > > in SQLMultiSets.pdf(previously thread) I found a related explanation > > on page 45, 46. > > /home/jian/hashset/0001-make-int4hashset_contains-strict-and-header-file-change.patch > > (CASE W

Re: sslinfo extension - add notbefore and notafter timestamps

2023-06-27 Thread Daniel Gustafsson
> On 23 Jun 2023, at 22:10, Cary Huang wrote: >> Off the cuff that doesn't seem like a bad idea, but I wonder if we should add >> them to pg_stat_ssl (or both) instead if we deem them valuable? > > I think the same information should be available to pg_stat_ssl as well. > pg_stat_ssl can show

Re: Assistance Needed: Issue with pg_upgrade and --link option

2023-06-27 Thread Laurenz Albe
On Wed, 2023-06-28 at 11:49 +0530, Pradeep Kumar wrote: > I was under the impression that the --link option would create hard links > between the > old and new cluster's data files, but it appears that the entire old cluster > data was > copied to the new cluster, resulting in a significant incre

Assistance Needed: Issue with pg_upgrade and --link option

2023-06-27 Thread Pradeep Kumar
Dear Postgres Hackers, I hope this email finds you well. I am currently facing an issue while performing an upgrade using the pg_upgrade utility with the --link option. I was under the impression that the --link option would create hard links between the old and new cluster's data files, but it ap

Re: pg_waldump: add test for coverage

2023-06-27 Thread Peter Eisentraut
On 14.06.23 09:16, Peter Eisentraut wrote: On 06.09.22 07:57, Peter Eisentraut wrote: I wrote a test for coverage. Unfortunately, it seems to take quite a while to run the test. I want to improve these execution times, but I don't know exactly what to do. Therefore, I want to hear feedback fro

Re: Add GUC to tune glibc's malloc implementation.

2023-06-27 Thread Ronan Dunklau
Le mardi 27 juin 2023, 20:17:46 CEST Andres Freund a écrit : > > Yes this is probably much more appropriate, but a much larger change with > > greater risks of regression. Especially as we have to make sure we're not > > overfitting our own code for a specific malloc implementation, to the > > detr

harmonize password reuse in vacuumdb, clusterdb, and reindexdb

2023-06-27 Thread Nathan Bossart
While looking at something unrelated, I noticed that the vacuumdb docs mention the following: vacuumdb might need to connect several times to the PostgreSQL server, asking for a password each time. IIUC this has been fixed since 83dec5a from 2015 (which was superceded by ff402ae),

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-06-27 Thread Amit Kapila
On Tue, Jun 27, 2023 at 1:12 PM Hayato Kuroda (Fujitsu) wrote: > > > This actually makes sense. I quickly try to do that without adding any > > new replication message. As you would expect, it did not work. > > I don't really know what's needed to make a connection to last for > > more than one it

Re: Another incorrect comment for pg_stat_statements

2023-06-27 Thread Japin Li
On Wed, 28 Jun 2023 at 11:22, Richard Guo wrote: > On Wed, Jun 28, 2023 at 10:53 AM Japin Li wrote: > >> >> Hi, hackers >> >> There has $subject that introduced by commit 6b4d23feef6. When we reset >> the entries >> if all parameters are avaiable, non-top-level entries removed first, then >> to

Re: Another incorrect comment for pg_stat_statements

2023-06-27 Thread Richard Guo
On Wed, Jun 28, 2023 at 10:53 AM Japin Li wrote: > > Hi, hackers > > There has $subject that introduced by commit 6b4d23feef6. When we reset > the entries > if all parameters are avaiable, non-top-level entries removed first, then > top-level > entries. I did not see the diffs. Maybe uploaded

Re: pg_upgrade and logical replication

2023-06-27 Thread vignesh C
On Mon, 24 Apr 2023 at 12:52, Julien Rouhaud wrote: > > Hi, > > On Tue, Apr 18, 2023 at 01:40:51AM +, Hayato Kuroda (Fujitsu) wrote: > > > > I found a cfbot failure on macOS [1]. According to the log, > > "SELECT count(*) FROM t2" was executed before synchronization was done. > > > > ``` > > [

Re: PG 16 draft release notes ready

2023-06-27 Thread Bruce Momjian
On Tue, Jun 27, 2023 at 03:49:44PM -0600, Roberto Mello wrote: > Adding to this thread as suggested by jkatz for consideration of > adding to release notes... > > In [1] I mention the omission of ldap_password_hook and a suggested paragraph. > > Roberto > > [1] > https://www.postgresql.org/mess

Another incorrect comment for pg_stat_statements

2023-06-27 Thread Japin Li
Hi, hackers There has $subject that introduced by commit 6b4d23feef6. When we reset the entries if all parameters are avaiable, non-top-level entries removed first, then top-level entries. On branch REL_15_STABLE Your branch is up to date with 'origin/REL_15_STABLE'. Changes not staged for c

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Richard Guo
On Tue, Jun 27, 2023 at 10:12 PM Tom Lane wrote: > Richard Guo writes: > > That's right. This issue has something to do with the > > outer-join-aware-Var changes. I reduced the repro to the query below. > > Thanks for the simplified test case. > > > When joining s1/t3 to t4, the relid of outer

RE: Assert while autovacuum was executing

2023-06-27 Thread Zhijie Hou (Fujitsu)
On Monday, June 26, 2023 12:18 PM Amit Kapila wrote: > > On Fri, Jun 23, 2023 at 10:07 PM Andres Freund wrote: > > > > On 2023-06-23 14:04:15 +0530, Amit Kapila wrote: > > > OTOH, if the above theory is wrong or people are not convinced, I am > > > okay with removing all the changes in commits 7

Re: check_strxfrm_bug()

2023-06-27 Thread Thomas Munro
On Wed, Jun 28, 2023 at 11:03 AM Thomas Munro wrote: > The GCC build farm has just received some SPARC hardware new enough to > run modern Solaris (hostname gcc106), so if wrasse were moved over > there we could finally assume all systems have POSIX 2008 (AKA > SUSv4)'s locale_t. That would look

Re: Row pattern recognition

2023-06-27 Thread Tatsuo Ishii
> Okay, I see the problem now, and why you need the rpr() function. > > You are doing this as something that happens over a window frame, but > it is actually something that *reduces* the window frame. The pattern > matching needs to be done when the frame is calculated and not when > any particu

Re: Add TLI number to name of files generated by pg_waldump --save-fullpage

2023-06-27 Thread Kyotaro Horiguchi
At Tue, 27 Jun 2023 18:58:39 -0500, David Christensen wrote in > > Adjusted as per the v2 attached. > > +1 +1 regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Add error context during loading of libraries

2023-06-27 Thread Andres Freund
Hi, This discussion started at https://postgr.es/m/20230627212423.p56zxuqeh5kya...@awork3.anarazel.de but isn't really related to the bug. On 2023-06-27 17:44:57 -0400, Tom Lane wrote: > Andres Freund writes: > > That's not going to help you / the reporter, but to make issues like this > > eas

Re: Add TLI number to name of files generated by pg_waldump --save-fullpage

2023-06-27 Thread David Christensen
> Adjusted as per the v2 attached. +1

Re: Add TLI number to name of files generated by pg_waldump --save-fullpage

2023-06-27 Thread Michael Paquier
On Tue, Jun 27, 2023 at 04:39:52PM +0900, Kyotaro Horiguchi wrote: > I meant that the name is structured as > TLIh-TLIl._, which > appears to be inconsistent with the comment. (And I'm not sure what > "TLOID" is..) Well, to be clear, it should not be TLIh-TLIl but LSNh-LSNl :) I'm OK with the

Re: Add TLI number to name of files generated by pg_waldump --save-fullpage

2023-06-27 Thread Michael Paquier
On Tue, Jun 27, 2023 at 11:53:10AM -0500, David Christensen wrote: > Patch looks good, but agreed that that comment should also be fixed. Okay, thanks for checking! -- Michael signature.asc Description: PGP signature

Re: check_strxfrm_bug()

2023-06-27 Thread Thomas Munro
The GCC build farm has just received some SPARC hardware new enough to run modern Solaris (hostname gcc106), so if wrasse were moved over there we could finally assume all systems have POSIX 2008 (AKA SUSv4)'s locale_t. It's slightly annoying that Windows has locale_t but doesn't have uselocale().

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Tom Lane
I wrote: > So it looks to me like something further up should have rejected this > path as not being usable here. Not sure what's dropping the ball. After further digging, I've concluded that what usually stops us from generating this bogus path for the t3/t4 join is the "param_source_rels" heuri

Re: PG 16 draft release notes ready

2023-06-27 Thread Roberto Mello
Adding to this thread as suggested by jkatz for consideration of adding to release notes... In [1] I mention the omission of ldap_password_hook and a suggested paragraph. Roberto [1] https://www.postgresql.org/message-id/CAKz%3D%3DbLzGb-9O294AoZHqEWpAi2Ki58yCr4gaqg1HnZyh3L1uA%40mail.gmail.com

Re: PostgreSQL 16 Beta 2 release announcement draft

2023-06-27 Thread Roberto Mello
On Tue, Jun 27, 2023 at 11:40 AM Jonathan S. Katz wrote: > > Was this discussed on the release notes thread?[1]. It can always be > added to the release notes -- those aren't finalized until GA. > > After Beta 1, the announcements are either about new feature > additions/removals since the last be

Re: Do we want a hashset type?

2023-06-27 Thread Joel Jacobson
On Tue, Jun 27, 2023, at 10:26, Joel Jacobson wrote: > Attachments: > * hashset-0.0.1-b7e5614-full.patch > * hashset-0.0.1-b7e5614-incremental.patch To help verify that the semantics, I thought it might be helpful to provide a comprehensive set of examples that tries to cover all different ways of

Re: POC, WIP: OR-clause support for indexes

2023-06-27 Thread Peter Geoghegan
On Tue, Jun 27, 2023 at 6:19 AM Alena Rybakina wrote: > I learned something new from your letter, thank you very much for that! Cool. The MDAM paper is also worth a read: https://vldb.org/conf/1995/P710.PDF Some of the techniques it describes are already in Postgres. With varying degrees of mat

Re: POC, WIP: OR-clause support for indexes

2023-06-27 Thread Ranier Vilela
Hi, >I finished writing the code patch for transformation "Or" expressions to >"Any" expressions. I didn't see any problems in regression tests, even >when I changed the constant at which the minimum or expression is >replaced by any at 0. I ran my patch on sqlancer and so far the code has >never

Re: [PATCH] Honor PG_TEST_NOCLEAN for tempdirs

2023-06-27 Thread Andrew Dunstan
On 2023-06-27 Tu 11:54, Dagfinn Ilmari Mannsåker wrote: Andrew Dunstan writes: On 2023-06-26 Mo 19:55, Jacob Champion wrote: Hello, I was running the test_pg_dump extension suite, and I got annoyed that I couldn't keep it from deleting its dump artifacts after a successful run. Here's a pat

Re: Add GUC to tune glibc's malloc implementation.

2023-06-27 Thread Andres Freund
Hi, On 2023-06-27 08:35:28 +0200, Ronan Dunklau wrote: > Le lundi 26 juin 2023, 23:03:48 CEST Andres Freund a écrit : > > Hi, > > > > On 2023-06-26 08:38:35 +0200, Ronan Dunklau wrote: > > > I hope what I'm trying to achieve is clearer that way. Maybe this patch is > > > not the best way to go abo

Re: ReadRecentBuffer() doesn't scale well

2023-06-27 Thread Andres Freund
Hi, On 2023-06-27 01:10:25 -0700, Peter Geoghegan wrote: > On Mon, Jun 26, 2023 at 11:27 PM Andres Freund wrote: > > On 2023-06-26 21:53:12 -0700, Peter Geoghegan wrote: > > > It should be safe to allow searchers to see a version of the root page > > > that is out of date. The Lehman & Yao design

Re: PostgreSQL 16 Beta 2 release announcement draft

2023-06-27 Thread Jonathan S. Katz
On 6/27/23 12:54 PM, Roberto Mello wrote: On Tue, Jun 27, 2023 at 8:32 AM Jonathan S. Katz wrote: I used the open items list[1] to build the draft. If there are any notable please omissions, please let me know. I noticed that ldap_password_hook [1] was omitted from the release notes. I belie

Re: ReadRecentBuffer() doesn't scale well

2023-06-27 Thread Andres Freund
Hi, On 2023-06-27 19:04:31 +0300, Ants Aasma wrote: > On Tue, 27 Jun 2023 at 18:40, Andres Freund wrote: > > On 2023-06-27 14:49:48 +0300, Ants Aasma wrote: > > > If you want to experiment, here is a rebased version of something I > > > hacked up a couple of years back on the way to Fosdem Pgday.

Re: PostgreSQL 16 Beta 2 release announcement draft

2023-06-27 Thread Roberto Mello
On Tue, Jun 27, 2023 at 8:32 AM Jonathan S. Katz wrote: > > I used the open items list[1] to build the draft. If there are any > notable please omissions, please let me know. I noticed that ldap_password_hook [1] was omitted from the release notes. I believe it should be included if nothing else

Re: Add TLI number to name of files generated by pg_waldump --save-fullpage

2023-06-27 Thread David Christensen
On Tue, Jun 27, 2023 at 1:12 AM Michael Paquier wrote: > > Hi all, > (Fujii-san and David in CC.) > > Fujii-san has reported on Twitter that we had better add the TLI > number to what pg_waldump --save-fullpage generates for the file names > of the blocks, as it could be possible that we overwrite

Re: [PATCH] Honor PG_TEST_NOCLEAN for tempdirs

2023-06-27 Thread Jacob Champion
On 6/26/23 22:47, Michael Paquier wrote: > src/test/perl/README and regress.sgml both describe what > PG_TEST_NOCLEAN does, and it seems to me that these should be updated > to tell that temporary files are not removed on top of the data > folders? I've added a couple of quick lines to the docs in

Re: ReadRecentBuffer() doesn't scale well

2023-06-27 Thread Ants Aasma
On Tue, 27 Jun 2023 at 18:40, Andres Freund wrote: > On 2023-06-27 14:49:48 +0300, Ants Aasma wrote: > > If you want to experiment, here is a rebased version of something I > > hacked up a couple of years back on the way to Fosdem Pgday. I didn't > > pursue it further because I didn't have a use c

Re: Incremental View Maintenance, take 2

2023-06-27 Thread jian he
On Thu, Jun 1, 2023 at 2:47 AM Yugo NAGATA wrote: > > On Thu, 1 Jun 2023 23:59:09 +0900 > Yugo NAGATA wrote: > > > Hello hackers, > > > > Here's a rebased version of the patch-set adding Incremental View > > Maintenance support for PostgreSQL. That was discussed in [1]. > > > [1] > > https://www

Re: [PATCH] Honor PG_TEST_NOCLEAN for tempdirs

2023-06-27 Thread Dagfinn Ilmari Mannsåker
Andrew Dunstan writes: > On 2023-06-26 Mo 19:55, Jacob Champion wrote: >> Hello, >> >> I was running the test_pg_dump extension suite, and I got annoyed that >> I couldn't keep it from deleting its dump artifacts after a successful >> run. Here's a patch to make use of PG_TEST_NOCLEAN (which curr

Re: ReadRecentBuffer() doesn't scale well

2023-06-27 Thread Andres Freund
Hi, On 2023-06-27 14:49:48 +0300, Ants Aasma wrote: > If you want to experiment, here is a rebased version of something I > hacked up a couple of years back on the way to Fosdem Pgday. I didn't > pursue it further because I didn't have a use case where it showed a > significant difference. Thanks

Re: [PATCH] Honor PG_TEST_NOCLEAN for tempdirs

2023-06-27 Thread Andrew Dunstan
On 2023-06-26 Mo 19:55, Jacob Champion wrote: Hello, I was running the test_pg_dump extension suite, and I got annoyed that I couldn't keep it from deleting its dump artifacts after a successful run. Here's a patch to make use of PG_TEST_NOCLEAN (which currently covers the test cluster's base d

Make uselocale protection more consistent

2023-06-27 Thread Tristan Partin
This is a patch which implements an issue discussed in bug #17946[0]. It doesn't fix the overarching issue of the bug, but merely a consistency issue which was found while analyzing code by Heikki. I had originally submitted the patch within that thread, but for visibility and the purposes of the c

RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-06-27 Thread Hayato Kuroda (Fujitsu)
Dear Melih, Thanks for updating the patch. Followings are my comments. Note that some lines exceeds 80 characters and some other lines seem too short. And comments about coding conventions were skipped. 0001 01. logicalrep_worker_launch() ``` if (is_parallel_apply_worker) + {

Re: Make pgbench exit on SIGINT more reliably

2023-06-27 Thread Tristan Partin
On Thu Jun 22, 2023 at 6:19 PM CDT, Michael Paquier wrote: > On Thu, Jun 22, 2023 at 02:58:14PM +0900, Yugo NAGATA wrote: > > On Mon, 19 Jun 2023 16:49:05 -0700 > > "Tristan Partin" wrote: > >> On Mon Jun 19, 2023 at 6:39 AM PDT, Yugo NAGATA wrote: > >>> [1] > >>> https://www.postgresql.org/messa

PostgreSQL 16 Beta 2 release announcement draft

2023-06-27 Thread Jonathan S. Katz
Hi, Please see the attached draft of the PostgreSQL 16 Beta 2 release announcement. I used the open items list[1] to build the draft. If there are any notable please omissions, please let me know. Please leave all feedback by June 29, 0:00 AoE. Thanks, Jonathan [1] https://wiki.postgresq

Re: Analyze on table creation?

2023-06-27 Thread James Coleman
On Mon, Jun 26, 2023 at 4:16 PM James Coleman wrote: > > On Mon, Jun 26, 2023 at 4:00 PM Andres Freund wrote: > > > > Hi, > > > > On 2023-06-26 13:40:49 -0400, James Coleman wrote: > > > Have we ever discussed running an analyze immediately after creating a > > > table? > > > > That doesn't make

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Tom Lane
Richard Guo writes: > That's right. This issue has something to do with the > outer-join-aware-Var changes. I reduced the repro to the query below. Thanks for the simplified test case. > When joining s1/t3 to t4, the relid of outer join t3/t4 appears both in > the joinrel's relids and in the j

initial pruning in parallel append

2023-06-27 Thread Amit Langote
Hi, In an off-list chat, Robert suggested that it might be a good idea to look more closely into $subject, especially in the context of the project of moving the locking of child tables / partitions to the ExecInitNode() phase when executing cached generic plans [1]. Robert's point is that a work

Re: ReadRecentBuffer() doesn't scale well

2023-06-27 Thread Ants Aasma
On Tue, 27 Jun 2023 at 07:09, Andres Freund wrote: > On 2023-06-27 15:33:57 +1200, Thomas Munro wrote: > > On Tue, Jun 27, 2023 at 2:05 PM Andres Freund wrote: > > > Unfortunately it scaled way worse at first. This is not an inherent > > > issue, but > > > due to an implementation choice in Read

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Richard Guo
On Tue, Jun 27, 2023 at 1:35 PM Michael Paquier wrote: > On Mon, Jun 26, 2023 at 11:05:43PM -0500, Jaime Casanova wrote: > > The attached query makes beta2 crash with attached backtrace. > > Interestingly the index on ref_6 is needed to make it crash, without > > it the query works fine. > > Issu

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-06-27 Thread Amit Kapila
On Fri, Jun 23, 2023 at 7:03 PM Melih Mutlu wrote: > > You can find the updated patchset attached. > I worked to address the reviews and made some additional changes. > > Let me first explain the new patchset. > 0001: Refactors the logical replication code, mostly worker.c and > tablesync.c. Altho

Re: RFC: Logging plan of the running query

2023-06-27 Thread torikoshia
On 2023-06-16 01:34, James Coleman wrote: On Thu, Jun 15, 2023 at 9:00 AM torikoshia wrote: On 2023-06-15 01:48, James Coleman wrote: > On Tue, Jun 13, 2023 at 11:53 AM James Coleman > wrote: >> >> ... >> I'm going to re-run tests with my patch version + resetting the flag >> on SIGINT (and a

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-06-27 Thread Masahiko Sawada
On Fri, Jun 23, 2023 at 6:54 PM John Naylor wrote: > > > I wrote: > > I cleaned up a few things and attached v34 so you can do that if you like. > > Of course, "clean" is a relative term. While making a small bit of progress > working in tidbitmap.c earlier this week, I thought it useful to proto

Re: ReadRecentBuffer() doesn't scale well

2023-06-27 Thread Peter Geoghegan
On Mon, Jun 26, 2023 at 11:27 PM Andres Freund wrote: > On 2023-06-26 21:53:12 -0700, Peter Geoghegan wrote: > > It should be safe to allow searchers to see a version of the root page > > that is out of date. The Lehman & Yao design is very permissive about > > these things. There aren't any speci

Re: pgbnech: allow to cancel queries during benchmark

2023-06-27 Thread Yugo NAGATA
On Mon, 26 Jun 2023 12:59:21 -0400 Kirk Wolak wrote: > On Mon, Jun 26, 2023 at 9:46 AM Yugo NAGATA wrote: > > > Hello, > > > > This attached patch enables pgbench to cancel queries during benchmark. > > > > Formerly, Ctrl+C during benchmark killed pgbench immediately, but backend > > processes

Re: Add TLI number to name of files generated by pg_waldump --save-fullpage

2023-06-27 Thread Kyotaro Horiguchi
Of course, it's wrong. At Tue, 27 Jun 2023 16:39:52 +0900 (JST), Kyotaro Horiguchi wrote in > I meant that the name is structured as - TLIh-TLIl._, which + LSNh-LSNl._, which > appears to be inconsistent with the comment. (And I'm not sure what > "TLOID" is..) > > regards. > > -- > K

Re: Add TLI number to name of files generated by pg_waldump --save-fullpage

2023-06-27 Thread Kyotaro Horiguchi
At Tue, 27 Jun 2023 15:58:38 +0900, Michael Paquier wrote in > On Tue, Jun 27, 2023 at 03:44:04PM +0900, Kyotaro Horiguchi wrote: > > The file name was "LSNh-LSNl.spcOid.dbOid.relNumber.blk_forkname", but > > the comment in the TAP script read as: > > > > -# -.DBOID.TLOID.NODEOI

Re: Incorrect comment for memset() on pgssHashKey?

2023-06-27 Thread Heikki Linnakangas
On 27/06/2023 07:32, Japin Li wrote: Hi, Commit 6b4d23feef introduces a toplevel field in pgssHashKey, which leads padding. In pgss_store(), it comments that memset() is required when pgssHashKey is without padding only. @@ -1224,9 +1227,14 @@ pgss_store(const char *query, uint64 queryId,