Re: pg_walinspect memory leaks

2023-02-16 Thread Michael Paquier
On Thu, Feb 16, 2023 at 06:00:00PM +0530, Bharath Rupireddy wrote: > The memory usage goes up with many WAL records in GetWALRecordsInfo(). > The affected functions are pg_get_wal_records_info() and > pg_get_wal_records_info_till_end_of_wal(). I think the best way to fix > this is to use a temporar

pg_upgrade and logical replication

2023-02-16 Thread Julien Rouhaud
Hi, I was working on testing a major upgrade scenario using a mix of physical and logical replication when I faced some unexpected problem leading to missing rows. Note that my motivation is to rely on physical replication / physical backup to avoid recreating a node from scratch using logical re

Re: ICU locale validation / canonicalization

2023-02-16 Thread Jeff Davis
On Thu, 2023-02-09 at 14:09 -0800, Jeff Davis wrote: > It feels like BCP 47 is the right catalog representation. We are > already using it for the import of initial collations, and it's a > standard, and there seems to be good support in ICU. Patch attached. We should have been canonicalizing all

Re: wrong query result due to wang plan

2023-02-16 Thread Richard Guo
On Thu, Feb 16, 2023 at 5:50 PM Richard Guo wrote: > It seems we still need to check whether a variable-free qual comes from > somewhere that is below the nullable side of an outer join before we > decide that it can be evaluated at join domain level, just like we did > before. So I wonder if we

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-02-16 Thread Michael Paquier
On Thu, Feb 16, 2023 at 11:00:20AM -0800, Andres Freund wrote: > I don't really understand this bit? As of this message, I saw this quote: https://www.postgresql.org/message-id/fcalj2acxebwy_bm3kmzekypcxsm+ygitpyhi4fdt6msk6yrt...@mail.gmail.com "However, it increases the number of iovec initializa

Re: [PATCH] Align GSS and TLS error handling in PQconnectPoll()

2023-02-16 Thread Michael Paquier
On Thu, Feb 16, 2023 at 09:59:54AM -0800, Jacob Champion wrote: > On Thu, Feb 16, 2023 at 3:31 AM Jelte Fennema wrote: >> Patch looks good to me. Definitely an improvement over the status quo. > > Thanks for the review! I was looking at that a second time, and with fresh eyes I can see that we w

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-02-16 Thread Peter Smith
FYI, I accidentally left this (v23) patch's TAP test t/032_subscribe_use_index.pl still lurking even after removing all other parts of this patch. In this scenario, the t/032 test gets stuck (build of latest HEAD) IIUC the patch is only meant to affect performance, so I expected this 032 test to

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-16 Thread Hayato Kuroda (Fujitsu)
Dear Andres, Thank you for giving comments! I understood that you have agreed the approach that publisher delays to send data. > > I'm not sure why output plugin is involved in the delay mechanism. > > +many > > The output plugin absolutely never should be involved in something like > this. It

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-16 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > > > Perhaps what I understand > > > > correctly is that we could delay right before only sending commit > > > > records in this case. If we delay at publisher end, all changes will > > > > be sent at once if !streaming, and otherwise, all changes in a > > > > transaction will be spo

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-16 Thread Hayato Kuroda (Fujitsu)
Dear Horiguchi-san, Thank you for replying! This direction seems OK, so I started to revise the patch. PSA new version. > > > As Amit-K mentioned, we may need to change the name of the option in > > > this version, since the delay mechanism in this version causes a delay > > > in sending from pu

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Andres Freund
Hi, On February 16, 2023 9:40:17 PM PST, Michael Paquier wrote: >On Fri, Feb 17, 2023 at 10:44:05AM +0530, Robert Haas wrote: >> Uh, I had the contrary impression from the discussion upthread, but it >> sounds like I might be misunderstanding the situation? > >IMO, it would be nice to be able t

shoud be get_extension_schema visible?

2023-02-16 Thread Pavel Stehule
Hi more times I needed to get the extension's assigned namespace. There is already a cooked function get_extension_schema, but it is static. I need to find a function with a known name, but possibly an unknown schema from a known extension. Regards Pavel

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Michael Paquier
On Fri, Feb 17, 2023 at 10:44:05AM +0530, Robert Haas wrote: > Uh, I had the contrary impression from the discussion upthread, but it > sounds like I might be misunderstanding the situation? IMO, it would be nice to be able to have the automatic detection of meson work in the CFbot to see how this

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Andres Freund
Hi, On February 16, 2023 9:14:05 PM PST, Robert Haas wrote: >On Thu, Feb 16, 2023 at 9:45 PM Andres Freund wrote: >> On 2023-02-16 15:05:10 +0530, Robert Haas wrote: >> > The fact that we can't use ICU on Windows, though, weakens this >> > argument a lot. In my experience, we have a lot of Wind

Re: Support logical replication of global object commands

2023-02-16 Thread Zheng Li
> > > Actually, I intend something for global objects. But the main thing > > > that is worrying me about this is that we don't have a clean way to > > > untie global object replication from database-specific object > > > replication. > > > > I think ultimately we need a clean and efficient way to

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Robert Haas
On Thu, Feb 16, 2023 at 9:45 PM Andres Freund wrote: > On 2023-02-16 15:05:10 +0530, Robert Haas wrote: > > The fact that we can't use ICU on Windows, though, weakens this > > argument a lot. In my experience, we have a lot of Windows users, and > > they're not any happier with the operating syste

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-16 Thread David Rowley
On Fri, 17 Feb 2023 at 17:40, Jonah H. Harris wrote: > Yeah. There’s definitely a smarter and more reusable approach than I was > proposing. A lot of that code is fairly mature and I figured more people > wouldn’t want to alter it in such ways - but I’m up for it if an approach > like this is t

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-16 Thread Jonah H. Harris
On Thu, Feb 16, 2023 at 11:26 PM David Rowley wrote: > I didn't hear it mentioned explicitly here, but I suspect it's faster > when increasing the initial size due to the memory context caching > code that reuses aset MemoryContexts (see context_freelists[] in > aset.c). Since we reset the contex

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-16 Thread David Rowley
On Fri, 17 Feb 2023 at 16:40, Andres Freund wrote: > I'd like a workload that hits a perf issue with this, because I think there > likely are some general performance improvements that we could make, without > changing the initial size or the "growth rate". I didn't hear it mentioned explicitly h

Re: Support logical replication of DDLs

2023-02-16 Thread Zheng Li
> > I've implemented a prototype to allow replicated objects to have the > > same owner from the publisher in > > v69-0008-Allow-replicated-objects-to-have-the-same-owner-from.patch. > > > > I also think it would be a helpful addition for users.A few points Thanks for supporting this addition. > t

Re: Change xl_hash_vacuum_one_page.ntuples from int to uint16

2023-02-16 Thread Andres Freund
Hi On 2023-02-17 08:30:09 +0530, Amit Kapila wrote: > Thanks, I was not completely sure about whether we need to bump > XLOG_PAGE_MAGIC for this patch as this makes the additional space just > by changing the datatype of one of the members of the existing WAL > record. We normally change it for th

Re: proposal: psql: psql variable BACKEND_PID

2023-02-16 Thread Pavel Stehule
čt 16. 2. 2023 v 12:49 odesílatel Jelte Fennema napsal: > On Thu, 16 Feb 2023 at 12:44, Pavel Stehule > wrote: > > To find and use pg_backend_pid is not rocket science. But use > :BACKEND_PID is simpler. > > I wanted to call out that if there's a connection pooler (e.g. > PgBouncer) in the middl

Should CreateExprContext() be using ALLOCSET_DEFAULT_SIZES?

2023-02-16 Thread Andres Freund
Hi, The thread around https://postgr.es/m/caduqk8uqw5qauqldd-0sbcvzvncre3jmjb9+ydwo_umv_ht...@mail.gmail.com reminded me of the following: ISTM that we really shouldn't use ALLOCSET_DEFAULT_SIZES for expression contexts, as they most commonly see only a few small, or no, allocations. That's tru

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 21:34:18 -0500, Jonah H. Harris wrote: > On Thu, Feb 16, 2023 at 7:32 PM Andres Freund wrote: > Given not much changed regarding that allocation context IIRC, I’d think > all recents. It was observed in 13, 14, and 15. We did have a fair bit of changes in related code in the la

Re: Introduce list_reverse() to make lcons() usage less inefficient

2023-02-16 Thread David Rowley
On Fri, 17 Feb 2023 at 13:23, Andres Freund wrote: > But wouldn't an even cheaper way here be to iterate over the children in > reverse order when match_partition_order_desc? We can do that efficiently > now. Looks like we don't have a readymade helper for it, but it'd be easy > enough to add or o

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-16 Thread Amit Kapila
On Fri, Feb 17, 2023 at 2:59 AM Andres Freund wrote: > > On 2023-02-16 12:37:57 +0530, Robert Haas wrote: > > The patch creates 100_bugs.pl > > What's the story behind 100_bugs.pl? This name clearly is copied from > src/test/subscription/t/100_bugs.pl - but I've never understood why that is > outs

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-02-16 Thread Thomas Munro
On Tue, Feb 14, 2023 at 4:38 PM Anton A. Melnikov wrote: > First of all it seemed to me that is not a problem at all since msdn > guarantees sector-by-sector atomicity. > "Physical Sector: The unit for which read and write operations to the device > are completed in a single operation. This is the

Re: Change xl_hash_vacuum_one_page.ntuples from int to uint16

2023-02-16 Thread Amit Kapila
On Thu, Feb 16, 2023 at 8:39 PM Drouvot, Bertrand wrote: > > On 2/16/23 1:26 PM, Drouvot, Bertrand wrote: > > Hi, > > > > On 2/16/23 12:00 PM, Amit Kapila wrote: > >> I think this would require XLOG_PAGE_MAGIC as it changes the WAL record. > >> > > > > Oh, I Was not aware about it, thanks! Will do

Re: Missing free_var() at end of accum_sum_final()?

2023-02-16 Thread Michael Paquier
On Thu, Feb 16, 2023 at 01:35:54PM -0800, Andres Freund wrote: > But why do we need it? Most SQL callable functions don't need to be careful > about not leaking O(1) memory, the exception being functions backing btree > opclasses. > > In fact, the detailed memory management often is *more* expensi

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-16 Thread Jonah H. Harris
On Thu, Feb 16, 2023 at 7:32 PM Andres Freund wrote: > What PG version? > Hey, Andres. Thanks for the reply. Given not much changed regarding that allocation context IIRC, I’d think all recents. It was observed in 13, 14, and 15. Do you have a way to reproduce this with core code, > e.g. postg

Re: REASSIGN OWNED vs ALTER TABLE OWNER TO permission inconsistencies

2023-02-16 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Feb 15, 2023 at 9:01 AM Stephen Frost wrote: > > I'm not really a fan of just dropping the CREATE check. If we go with > > "recipient needs CREATE rights" then at least without superuser > > intervention and excluding cases where

Re: Dead code in ps_status.c

2023-02-16 Thread Thomas Munro
On Fri, Feb 17, 2023 at 3:38 AM Tom Lane wrote: > My account still works, and what I see on wrasse's host is > > tgl@gcc-solaris11:~$ gcc -x c /dev/null -dM -E | grep -i svr > #define __SVR4 1 > #define __svr4__ 1 > tgl@gcc-solaris11:~$ gcc -x c /dev/null -dM -E | grep -i sun > #define __sun 1 > #

Re: Introduce list_reverse() to make lcons() usage less inefficient

2023-02-16 Thread Tom Lane
Andres Freund writes: > On 2023-02-17 11:36:40 +1300, David Rowley wrote: >> One of those places is in generate_orderedappend_paths() when we find >> that the required sort order is the same as the reverse of the >> partition order. In this case, we build a list of paths for each >> partition usi

Re: Make set_ps_display faster and easier to use

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 14:19:24 +1300, David Rowley wrote: > After fixing up the set_ps_display()s to use set_ps_display_with_len() > where possible, I discovered some not so nice code which appends " > waiting" onto the process title. Basically, there's a bunch of code > that looks like this: > > con

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 16:49:07 -0500, Jonah H. Harris wrote: > I've been working on a federated database project that heavily relies on > foreign data wrappers. During benchmarking, we noticed high system CPU > usage in OLTP-related cases, which we traced back to multiple brk calls > resulting from bl

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-16 Thread Justin Pryzby
On Wed, Feb 08, 2023 at 04:40:49PM -0600, Justin Pryzby wrote: > This squishes together 001/2 as the main patch. > I believe it's ready. Update to address a compiler warning in the supplementary patches adding assertions. >From 71427bf7cd9927af04513ba3fe99e481a8ba1f61 Mon Sep 17 00:00:00 2001 From

Re: Introduce list_reverse() to make lcons() usage less inefficient

2023-02-16 Thread Andres Freund
Hi, On 2023-02-17 11:36:40 +1300, David Rowley wrote: > While working on [1] to make improvements in the query planner around > the speed to find EquivalenceMembers in an EquivalenceClass, because > that patch does have a large impact in terms of performance > improvements, some performance tests

Re: [PATCH] Add pretty-printed XML output option

2023-02-16 Thread Andrey Borodin
On Thu, Feb 16, 2023 at 2:12 PM Jim Jones wrote: > > I'm squashing v12-0001 and v12-0002 (v13 attached). I've looked into the patch. The code looks to conform to usual expectations. One nit: this comment should have just one asterisk. + /** And I have a dumb question: is this function protected fr

psql \watch 2nd argument: iteration count

2023-02-16 Thread Andrey Borodin
Hi hackers! >From time to time I want to collect some stats from locks, activity and other stat views into one table from different time points. In this case the \watch psql command is very handy. However, it's not currently possible to specify the number of times a query is performed. Also, if we

Re: ATTACH PARTITION seems to ignore column generation status

2023-02-16 Thread Tom Lane
Alvaro Herrera writes: > On 2023-Feb-16, Alexander Lakhin wrote: >> I've encountered a query that triggers an assert added in that commit: >> CREATE TABLE t(a int, b int GENERATED ALWAYS AS (a) STORED) PARTITION BY >> RANGE (a); >> CREATE TABLE tp PARTITION OF t(b DEFAULT 1) FOR VALUES FROM (0) to

Introduce list_reverse() to make lcons() usage less inefficient

2023-02-16 Thread David Rowley
While working on [1] to make improvements in the query planner around the speed to find EquivalenceMembers in an EquivalenceClass, because that patch does have a large impact in terms of performance improvements, some performance tests with that patch started to highlight some other places that bot

Re: [PATCH] Add pretty-printed XML output option

2023-02-16 Thread Jim Jones
On 16.02.23 00:13, Peter Smith wrote: Today I fetched and tried the latest v11. It is failing too, but only just. - see attached file pretty-v11-results It looks only due to a whitespace EOF issue in the xml_2.out @@ -1679,4 +1679,4 @@ -- XML format: empty string SELECT xmlformat(''); ERR

Re: recovery modules

2023-02-16 Thread Nathan Bossart
On Thu, Feb 16, 2023 at 01:17:54PM -0800, Andres Freund wrote: > On 2023-02-16 12:15:12 -0800, Nathan Bossart wrote: >> On Thu, Feb 16, 2023 at 11:29:56AM -0800, Andres Freund wrote: >> > On 2023-02-15 10:44:07 -0800, Nathan Bossart wrote: >> >> @@ -144,10 +170,12 @@ basic_archive_configured(void)

Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-16 Thread Jonah H. Harris
Hi everyone, I've been working on a federated database project that heavily relies on foreign data wrappers. During benchmarking, we noticed high system CPU usage in OLTP-related cases, which we traced back to multiple brk calls resulting from block frees in AllocSetReset upon ExecutorEnd's FreeEx

Re: Add connection active, idle time to pg_stat_activity

2023-02-16 Thread Andrey Borodin
On Wed, Feb 1, 2023 at 12:46 PM Sergey Dudoladov wrote: > > I've sketched the first version of a patch to add pg_stat_session. > Please review this early version. Hi Sergey! I've taken a look into the patch and got some notes. 1. It is hard to understand what fastpath backend state is. What do f

Re: Missing free_var() at end of accum_sum_final()?

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 15:26:26 +0900, Michael Paquier wrote: > On Thu, Feb 16, 2023 at 06:59:13AM +0100, Joel Jacobson wrote: > > I noticed the NumericVar's pos_var and neg_var are not free_var()'d > > at the end of accum_sum_final(). > > > > The potential memory leak seems small, since the function

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-16 Thread Andres Freund
On 2023-02-16 12:37:57 +0530, Robert Haas wrote: > The patch creates 100_bugs.pl What's the story behind 100_bugs.pl? This name clearly is copied from src/test/subscription/t/100_bugs.pl - but I've never understood why that is outside of the normal numbering space.

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

2023-02-16 Thread Andres Freund
Hi, On 2023-02-15 09:21:48 +0100, Drouvot, Bertrand wrote: > diff --git a/src/backend/utils/activity/pgstat_relation.c > b/src/backend/utils/activity/pgstat_relation.c > index f793ac1516..b26e2a5a7a 100644 > --- a/src/backend/utils/activity/pgstat_relation.c > +++ b/src/backend/utils/activity/pgs

Re: recovery modules

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 12:15:12 -0800, Nathan Bossart wrote: > Thanks for reviewing. > > On Thu, Feb 16, 2023 at 11:29:56AM -0800, Andres Freund wrote: > > On 2023-02-15 10:44:07 -0800, Nathan Bossart wrote: > >> @@ -144,10 +170,12 @@ basic_archive_configured(void) > >> * Archives one file. > >> *

Re: Support logical replication of DDLs

2023-02-16 Thread Jonathan S. Katz
On 2/16/23 2:43 PM, Jonathan S. Katz wrote: On 2/16/23 2:38 PM, Alvaro Herrera wrote: On 2023-Feb-16, Jonathan S. Katz wrote: On 2/16/23 12:53 PM, Alvaro Herrera wrote: I don't think this is the fault of logical replication.  Consider that for the backend server, the function source code is

Re: recovery modules

2023-02-16 Thread Nathan Bossart
Thanks for reviewing. On Thu, Feb 16, 2023 at 11:29:56AM -0800, Andres Freund wrote: > On 2023-02-15 10:44:07 -0800, Nathan Bossart wrote: >> @@ -144,10 +170,12 @@ basic_archive_configured(void) >> * Archives one file. >> */ >> static bool >> -basic_archive_file(const char *file, const char *

Re: Support logical replication of DDLs

2023-02-16 Thread Jonathan S. Katz
On 2/16/23 2:38 PM, Alvaro Herrera wrote: On 2023-Feb-16, Jonathan S. Katz wrote: On 2/16/23 12:53 PM, Alvaro Herrera wrote: I don't think this is the fault of logical replication. Consider that for the backend server, the function source code is just an opaque string that is given to the p

Re: Support logical replication of DDLs

2023-02-16 Thread Alvaro Herrera
On 2023-Feb-16, Jonathan S. Katz wrote: > On 2/16/23 12:53 PM, Alvaro Herrera wrote: > > I don't think this is the fault of logical replication. Consider that > > for the backend server, the function source code is just an opaque > > string that is given to the plpgsql engine to interpret. So t

Re: Make ON_ERROR_STOP stop on shell script failure

2023-02-16 Thread Andreas 'ads' Scherbaum
On 16/02/2023 20:33, Andreas 'ads' Scherbaum wrote: On 23/11/2022 00:16, Matheus Alcantara wrote: --- Original Message --- On Tuesday, November 22nd, 2022 at 20:10, bt22nakamorit wrote: There was a mistake in the error message for \! so I updated the patch. Best, Tatsuhiro Nakamor

Re: Make ON_ERROR_STOP stop on shell script failure

2023-02-16 Thread Andreas 'ads' Scherbaum
On 23/11/2022 00:16, Matheus Alcantara wrote: --- Original Message --- On Tuesday, November 22nd, 2022 at 20:10, bt22nakamorit wrote: There was a mistake in the error message for \! so I updated the patch. Best, Tatsuhiro Nakamori Hi I was checking your patch and seems that it fai

Re: recovery modules

2023-02-16 Thread Andres Freund
Hi, On 2023-02-15 10:44:07 -0800, Nathan Bossart wrote: > On Wed, Feb 15, 2023 at 03:38:21PM +0900, Michael Paquier wrote: > > I may tweak a bit the comments, but nothing more. And I don't think I > > have more to add. Andres, do you have anything you would like to > > mention? Just some minor

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 14:21:01 +0900, Kyotaro Horiguchi wrote: > I'm not sure why output plugin is involved in the delay mechanism. +many The output plugin absolutely never should be involved in something like this. It was a grave mistake that OutputPluginUpdateProgress() calls were added to the com

Re: Add WAL read stats to pg_stat_wal

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 23:39:00 +0530, Bharath Rupireddy wrote: > While working on [1], I was in need to know WAL read stats (number of > times and amount of WAL data read from disk, time taken to read) to > measure the benefit. I had to write a developer patch to capture WAL > read stats as pg_stat_wa

Re: Support logical replication of DDLs

2023-02-16 Thread Jonathan S. Katz
On 2/16/23 12:53 PM, Alvaro Herrera wrote: On 2023-Feb-16, Jonathan S. Katz wrote: [replication tries to execute this command] 2023-02-16 16:11:10.570 UTC [25207] STATEMENT: CREATE OR REPLACE FUNCTION public.availability_rule_bulk_insert ( IN availability_rule public.availability_rule, IN day

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 16:58:23 +0900, Michael Paquier wrote: > On Wed, Feb 15, 2023 at 01:00:00PM +0530, Bharath Rupireddy wrote: > > The v3 patch reduces initialization of iovec array elements which is a > > clear win when pg_pwrite_zeros is called for sizes less than BLCKSZ > > many times (I assume

Re: [PoC] Let libpq reject unexpected authentication requests

2023-02-16 Thread Jacob Champion
v14 rebases over the test and solution conflicts from 9244c11afe2. Thanks, --Jacob 1: 542d330310 ! 1: eec891c519 libpq: let client reject unexpected auth methods @@ src/test/ssl/t/002_scram.pl: $node->connect_ok( +qr/channel binding is required, but server did not offer an

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-02-16 Thread Jacob Champion
On Thu, Feb 16, 2023 at 1:35 AM Jelte Fennema wrote: > > FYI the last patch does not apply cleanly anymore. So a rebase is needed. Thanks for the nudge, v7 rebases over the configure conflict from 9244c11afe. --Jacob 1: e7e2d43b18 ! 1: b07af1c564 libpq: add sslrootcert=system to use default CA

Add WAL read stats to pg_stat_wal

2023-02-16 Thread Bharath Rupireddy
Hi, While working on [1], I was in need to know WAL read stats (number of times and amount of WAL data read from disk, time taken to read) to measure the benefit. I had to write a developer patch to capture WAL read stats as pg_stat_wal currently emits WAL write stats. With recent works on pg_stat

Re: ATTACH PARTITION seems to ignore column generation status

2023-02-16 Thread Alvaro Herrera
On 2023-Feb-16, Alexander Lakhin wrote: > I've encountered a query that triggers an assert added in that commit: > CREATE TABLE t(a int, b int GENERATED ALWAYS AS (a) STORED) PARTITION BY > RANGE (a); > CREATE TABLE tp PARTITION OF t(b DEFAULT 1) FOR VALUES FROM (0) to (1); It seems wrong that th

Re: psql: Add role's membership options to the \du+ command

2023-02-16 Thread Pavel Luzanov
On 16.02.2023 00:37, David G. Johnston wrote: I mean, either you accept the change in how this meta-command presents its information or you don't. Yes, that's the main issue of this patch. On the one hand, it would be nice to see the membership options with the psql command. On the other ha

Re: Support logical replication of DDLs

2023-02-16 Thread Alvaro Herrera
On 2023-Feb-16, houzj.f...@fujitsu.com wrote: > I did some research for this. > > The provider seems not a database object, user needs to register a provider > via > C ode via register_label_provider. And ObjectAddress only have three > fields(classId, objectId, objectSubId), so it seems hard to

Re: [PATCH] Align GSS and TLS error handling in PQconnectPoll()

2023-02-16 Thread Jacob Champion
On Thu, Feb 16, 2023 at 3:31 AM Jelte Fennema wrote: > > Patch looks good to me. Definitely an improvement over the status quo. Thanks for the review! > Looking at the TLS error handling though I see these two lines: > > && conn->allow_ssl_try/* redundant? */ > && !conn->wait_ssl_try) /* red

Re: ATTACH PARTITION seems to ignore column generation status

2023-02-16 Thread Alexander Lakhin
Hello, 11.01.2023 23:58, Tom Lane wrote: Amit Langote writes: I've updated your disallow-generated-child-columns-2.patch to do this, and have also merged the delta post that I had attached with my last email, whose contents you sound to agree with. Pushed with some further work to improve the

Re: Support logical replication of DDLs

2023-02-16 Thread Alvaro Herrera
On 2023-Feb-16, Jonathan S. Katz wrote: [replication tries to execute this command] > 2023-02-16 16:11:10.570 UTC [25207] STATEMENT: CREATE OR REPLACE FUNCTION > public.availability_rule_bulk_insert ( IN availability_rule > public.availability_rule, IN day_of_week pg_catalog.int4 ) RETURNS > pg_

Re: Weird failure with latches in curculio on v15

2023-02-16 Thread Nathan Bossart
On Thu, Feb 16, 2023 at 03:08:14PM +0530, Robert Haas wrote: > On Thu, Feb 9, 2023 at 10:53 PM Nathan Bossart > wrote: >> I've been thinking about this, actually. I'm wondering if we could provide >> a list of files to the archiving callback (configurable via a variable in >> ArchiveModuleState)

Re: Support logical replication of DDLs

2023-02-16 Thread Jonathan S. Katz
Hi, On 2/14/23 10:01 PM, houzj.f...@fujitsu.com wrote: Here is the new version patch which addressed above comments. I also fixed a bug for the deparsing of CREATE RULE that it didn't add parentheses for rule action list. I started testing this change set from this patch. I'm doing a mix of

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 16:22:56 +0700, John Naylor wrote: > On Thu, Feb 16, 2023 at 10:24 AM Masahiko Sawada > > Right. TidStore is implemented not only for heap, so loading > > out-of-order TIDs might be important in the future. > > That's what I was probably thinking about some weeks ago, but I'm h

Re: run pgindent on a regular basis / scripted manner

2023-02-16 Thread Andrew Dunstan
On 2023-02-16 Th 03:26, shiy.f...@fujitsu.com wrote: On Thu, Feb 9, 2023 6:10 AM Andrew Dunstan wrote: Thanks, I have committed this. Still looking at Robert's other request. Hi, Commit #068a243b7 supported directories to be non-option arguments of pgindent. But the help text doesn't mentio

Re: Weird failure with latches in curculio on v15

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 15:18:57 +0530, Robert Haas wrote: > On Fri, Feb 10, 2023 at 12:59 AM Andres Freund wrote: > > I don't think it's that hard to imagine problems. To be reasonably fast, a > > decent restore implementation will have to 'restore ahead'. Which also > > provides ample things to go wr

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 15:05:10 +0530, Robert Haas wrote: > The fact that we can't use ICU on Windows, though, weakens this > argument a lot. In my experience, we have a lot of Windows users, and > they're not any happier with the operating system collations than > Linux users. Possibly less so. Why c

Re: Refactor calculations to use instr_time

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 16:19:02 +0300, Nazir Bilal Yavuz wrote: > What do you think? Here's a small review: > +#define WALSTAT_ACC(fld, var_to_add) \ > + (stats_shmem->stats.fld += var_to_add.fld) > +#define WALLSTAT_ACC_INSTR_TIME_TYPE(fld) \ > + (stats_shmem->stats.fld += INSTR_TIME_GET_MIC

Re: PATCH: Using BRIN indexes for sorted output

2023-02-16 Thread Justin Pryzby
On Thu, Feb 16, 2023 at 03:07:59PM +0100, Tomas Vondra wrote: > Rebased version of the patches, fixing only minor conflicts. Per cfbot, the patch fails on 32 bit builds. +ERROR: count mismatch: 0 != 1000 And causes warnings in mingw cross-compile. On Sun, Oct 23, 2022 at 11:32:37PM -0500, Justi

Re: Change xl_hash_vacuum_one_page.ntuples from int to uint16

2023-02-16 Thread Drouvot, Bertrand
Hi, On 2/16/23 1:26 PM, Drouvot, Bertrand wrote: Hi, On 2/16/23 12:00 PM, Amit Kapila wrote: I think this would require XLOG_PAGE_MAGIC as it changes the WAL record. Oh, I Was not aware about it, thanks! Will do in V2 (and in the logical decoding on standby patch as it adds the new field me

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Jonathan S. Katz
On 2/16/23 4:35 AM, Robert Haas wrote: On Thu, Feb 16, 2023 at 1:01 AM Jeff Davis wrote: It feels very wrong to me to explain that sort order is defined by the operating system on which Postgres happens to run. Saying that it's defined by ICU, which is part of the Unicode consotium, is much bet

Re: Dead code in ps_status.c

2023-02-16 Thread Tom Lane
Thomas Munro writes: > On Thu, Feb 16, 2023 at 6:34 PM Tom Lane wrote: >> Hm, is "defined(sun)" true on any live systems at all? > My GCC compile farm account seems to have expired, or something, so I > couldn't check on wrasse's host (though whether wrasse is "live" is > debatable: Solaris 11.3

Refactor calculations to use instr_time

2023-02-16 Thread Nazir Bilal Yavuz
Hi, In 'instr_time.h' it is stated that: * When summing multiple measurements, it's recommended to leave the * running sum in instr_time form (ie, use INSTR_TIME_ADD or * INSTR_TIME_ACCUM_DIFF) and convert to a result format only at the end. So, I refactored 'PendingWalStats' to use 'instr_ti

RE: Support logical replication of DDLs

2023-02-16 Thread houzj.f...@fujitsu.com
On Wednesday, February 15, 2023 5:51 PM Amit Kapila wrote: > > On Wed, Feb 15, 2023 at 2:02 PM Alvaro Herrera > wrote: > > > > On 2023-Feb-15, Peter Smith wrote: > > > > > On Thu, Feb 9, 2023 at 8:55 PM Ajin Cherian wrote: > > > > > > > > On Fri, Feb 3, 2023 at 11:41 AM Peter Smith > wrote: >

Re: Silent overflow of interval type

2023-02-16 Thread Nick Babadzhanian
On Thu, Feb 16, 2023 at 1:12 AM Tom Lane wrote: > Yeah, I don't think this would create a performance problem, at least not > if you're using a compiler that implements pg_sub_s64_overflow reasonably. > (And if you're not, and this bugs you, the answer is to get a better Please find attached the

Re: Allow logical replication to copy tables in binary format

2023-02-16 Thread Amit Kapila
On Mon, Jan 30, 2023 at 4:19 PM Melih Mutlu wrote: > > Hi Bharath, > > Thanks for reviewing. > > Bharath Rupireddy , 18 Oca 2023 Çar, > 10:17 tarihinde şunu yazdı: >> >> On Thu, Jan 12, 2023 at 1:53 PM Melih Mutlu wrote: >> 1. The performance numbers posted upthread [1] look impressive for the >

Re: pg_walinspect memory leaks

2023-02-16 Thread Bharath Rupireddy
On Tue, Feb 14, 2023 at 4:07 PM Bharath Rupireddy wrote: > > On Tue, Feb 14, 2023 at 6:25 AM Andres Freund wrote: > > > > Hi, > > > > On 2023-02-13 15:22:02 -0800, Peter Geoghegan wrote: > > > More concretely, it looks like GetWALRecordInfo() calls > > > CStringGetTextDatum/cstring_to_text in a w

Re: Change xl_hash_vacuum_one_page.ntuples from int to uint16

2023-02-16 Thread Drouvot, Bertrand
Hi, On 2/16/23 12:00 PM, Amit Kapila wrote: On Wed, Feb 15, 2023 at 3:35 AM Nathan Bossart wrote: On Sat, Jan 21, 2023 at 06:42:08AM +0100, Drouvot, Bertrand wrote: On 1/20/23 9:01 PM, Nathan Bossart wrote: Should we also change the related variables (e.g., ndeletable in _hash_vacuum_one_pa

Re: Some revises in adding sorting path

2023-02-16 Thread Etsuro Fujita
Hi Richard, On Tue, Jan 10, 2023 at 8:06 PM Richard Guo wrote: > In add_paths_with_pathkeys_for_rel, we do not try incremental sort atop > of the epq_path, which I think we can do. I'm not sure how useful this > is in real world since the epq_path is used only for EPQ checks, but it > seems doin

Re: proposal: psql: psql variable BACKEND_PID

2023-02-16 Thread Jelte Fennema
On Thu, 16 Feb 2023 at 12:44, Pavel Stehule wrote: > To find and use pg_backend_pid is not rocket science. But use :BACKEND_PID is > simpler. I wanted to call out that if there's a connection pooler (e.g. PgBouncer) in the middle, then BACKEND_PID (and %p) are incorrect, but pg_backend_pid() wou

Re: Considering additional sort specialisation functions for PG16

2023-02-16 Thread John Naylor
I wrote: > Have two memtuple arrays, one for first sortkey null and one for first sortkey non-null: Hacking on this has gotten only as far as the "compiles but segfaults" stage, but I wanted to note an idea that occurred to me: Internal qsort doesn't need the srctape member, and removing both tha

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-16 Thread Melih Mutlu
Hi Shveta and Shi, Thanks for your investigations. shveta malik , 8 Şub 2023 Çar, 16:49 tarihinde şunu yazdı: > On Tue, Feb 7, 2023 at 8:18 AM shiy.f...@fujitsu.com > wrote: > > I reproduced the problem I reported before with latest patch (v7-0001, > > v10-0002), and looked into this problem. I

Re: Missing default value of createrole_self_grant in document

2023-02-16 Thread Daniel Gustafsson
> On 16 Feb 2023, at 10:47, shiy.f...@fujitsu.com wrote: > I noticed that the document of GUC createrole_self_grant doesn't mention its > default value. The attached patch adds that. Agreed, showing the default value in the documentation is a good pattern IMO. Unless objected to I'll go apply thi

Re: [PATCH] Align GSS and TLS error handling in PQconnectPoll()

2023-02-16 Thread Jelte Fennema
Patch looks good to me. Definitely an improvement over the status quo. Looking at the TLS error handling though I see these two lines: && conn->allow_ssl_try/* redundant? */ && !conn->wait_ssl_try) /* redundant? */ Are they actually redundant like the comment suggests? If so, we should proba

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-02-16 Thread John Naylor
On Thu, Feb 16, 2023 at 10:03 AM David Rowley wrote: > I suspect it's slower because the final sort must sort the entire > array still without knowledge that portions of it are pre-sorted. It > would be very interesting to improve this and do some additional work > and keep track of the "memtups

Re: Change xl_hash_vacuum_one_page.ntuples from int to uint16

2023-02-16 Thread Amit Kapila
On Wed, Feb 15, 2023 at 3:35 AM Nathan Bossart wrote: > > On Sat, Jan 21, 2023 at 06:42:08AM +0100, Drouvot, Bertrand wrote: > > On 1/20/23 9:01 PM, Nathan Bossart wrote: > >> Should we also change the related > >> variables (e.g., ndeletable in _hash_vacuum_one_page()) to uint16? > > > > Yeah, I

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Laurenz Albe
On Thu, 2023-02-16 at 15:05 +0530, Robert Haas wrote: > On Thu, Feb 16, 2023 at 1:01 AM Jeff Davis wrote: > > It feels very wrong to me to explain that sort order is defined by the > > operating system on which Postgres happens to run. Saying that it's > > defined by ICU, which is part of the Unic

Re: wrong query result due to wang plan

2023-02-16 Thread Richard Guo
On Thu, Feb 16, 2023 at 5:50 PM Richard Guo wrote: > It seems we still need to check whether a variable-free qual comes from > somewhere that is below the nullable side of an outer join before we > decide that it can be evaluated at join domain level, just like we did > before. So I wonder if we

Re: Normalization of utility queries in pg_stat_statements

2023-02-16 Thread Drouvot, Bertrand
Hi, On 2/16/23 1:34 AM, Michael Paquier wrote: While wondering about this stuff about the last few days and discussing with bertrand, I have changed my mind on the point that there is no need to be that aggressive yet with the normalization of the A_Const nodes, because the query string normaliz

Re: wrong query result due to wang plan

2023-02-16 Thread Richard Guo
On Thu, Feb 16, 2023 at 3:16 PM tender wang wrote: > select ref_1.r_comment as c0, subq_0.c1 as c1 from public.region as > sample_0 right join public.partsupp as sample_1 right join public.lineitem > as sample_2 on (cast(null as path) = cast(null as path)) on (cast(null as > "timestamp") < cast(n

Re: Weird failure with latches in curculio on v15

2023-02-16 Thread Robert Haas
On Fri, Feb 10, 2023 at 12:59 AM Andres Freund wrote: > I'm somewhat concerned about that too, but perhaps from a different > angle. First, I think we don't do our users a service by defaulting the > in-core implementation to something that doesn't scale to even a moderately > busy server. +1. >

Missing default value of createrole_self_grant in document

2023-02-16 Thread shiy.f...@fujitsu.com
Hi hackers, I noticed that the document of GUC createrole_self_grant doesn't mention its default value. The attached patch adds that. Regards, Shi Yu v1-0001-Add-default-value-of-createrole_self_grant-in-doc.patch Description: v1-0001-Add-default-value-of-createrole_self_grant-in-doc.patch

  1   2   >