Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Masahiko Sawada
On Tue, Jul 19, 2022 at 1:47 PM Amit Kapila wrote: > > On Tue, Jul 19, 2022 at 6:34 AM Masahiko Sawada wrote: > > > > On Mon, Jul 18, 2022 at 1:12 PM Amit Kapila wrote: > > > > > > On Fri, Jul 15, 2022 at 8:09 PM Masahiko Sawada > > > wrote: > > > > > > > > This patch should have the fix for t

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread osumi.takami...@fujitsu.com
On Sunday, July 17, 2022 9:59 PM Masahiko Sawada wrote: > I've attached patches for all supported branches including the master. Hi, Minor comments for REL14. (1) There are some foreign characters in the patches (in the commit message) When I had a look at your patch for back branches with som

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Martin Kalcher
Am 19.07.22 um 00:52 schrieb Martin Kalcher: On the contrary! I am pretty sure there are people out there wanting sampling-without-shuffling. I will think about that. I gave it some thought. Even though there might be use cases, where a stable order is desired, i would consider them edge cas

Re: NAMEDATALEN increase because of non-latin languages

2022-07-19 Thread John Naylor
I wrote: > On Mon, Jul 18, 2022 at 9:58 AM Andres Freund wrote: > > Hm. Wouldn't it make sense to just use the normal tuple deforming routines and > > then map the results to the structs? > > I wasn't sure if they'd be suitable for this, but if they are, that'd make this easier and more maintaina

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Kyotaro Horiguchi
At Tue, 19 Jul 2022 10:17:15 +0530, Amit Kapila wrote in > Good work. I wonder without comments this may create a problem in the > future. OTOH, I don't see adding a check "catchange.xcnt > 0" before > freeing the memory any less robust. Also, for consistency, we can use > a similar check based

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Masahiko Sawada
On Mon, Jul 18, 2022 at 8:49 PM Amit Kapila wrote: > > On Sun, Jul 17, 2022 at 6:29 PM Masahiko Sawada wrote: > > > > On Fri, Jul 15, 2022 at 3:32 PM shiy.f...@fujitsu.com > > wrote: > > > > > > > I've attached patches for all supported branches including the master. > > > > For back branch patc

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Masahiko Sawada
On Tue, Jul 19, 2022 at 4:28 PM osumi.takami...@fujitsu.com wrote: > > On Sunday, July 17, 2022 9:59 PM Masahiko Sawada > wrote: > > I've attached patches for all supported branches including the master. > Hi, > > > Minor comments for REL14. > > (1) There are some foreign characters in the patch

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Kyotaro Horiguchi
At Tue, 19 Jul 2022 16:02:26 +0900, Masahiko Sawada wrote in > On Tue, Jul 19, 2022 at 1:47 PM Amit Kapila wrote: > > Good work. I wonder without comments this may create a problem in the > > future. OTOH, I don't see adding a check "catchange.xcnt > 0" before > > freeing the memory any less ro

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Kyotaro Horiguchi
At Tue, 19 Jul 2022 16:57:14 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 19 Jul 2022 16:02:26 +0900, Masahiko Sawada > wrote in > > On Tue, Jul 19, 2022 at 1:47 PM Amit Kapila wrote: > > > Good work. I wonder without comments this may create a problem in the > > > future. OTOH, I don't

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-07-19 Thread Önder Kalacı
Hi, thanks for your reply. Amit Kapila , 18 Tem 2022 Pzt, 08:29 tarihinde şunu yazdı: > On Tue, Jul 12, 2022 at 7:07 PM Önder Kalacı > wrote: > > > > Hi hackers, > > > > > > It is often not feasible to use `REPLICA IDENTITY FULL` on the > publication, because it leads to full table scan > > > >

Re: Costing elided SubqueryScans more nearly correctly

2022-07-19 Thread Alvaro Herrera
On 2022-Jul-19, Richard Guo wrote: > On Tue, Jul 19, 2022 at 1:30 AM Tom Lane wrote: > > WFM. (I'd fixed the comment typo in my patch, but I don't mind if > > you get there first.) Ah, I see now you had other grammatical fixes and even more content there. > +1 The fix looks good to me. Thank

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Masahiko Sawada
On Tue, Jul 19, 2022 at 4:35 PM Kyotaro Horiguchi wrote: Thank you for the comments! > > At Tue, 19 Jul 2022 10:17:15 +0530, Amit Kapila > wrote in > > Good work. I wonder without comments this may create a problem in the > > future. OTOH, I don't see adding a check "catchange.xcnt > 0" before

Re: System column support for partitioned tables using heap

2022-07-19 Thread Morris de Oryx
> What is motivating you to want to see the xmax value here? It's not an > unreasonable thing to want to do, IMHO, but it's a little bit niche so > I'm just curious what the motivation is. Yeah, I figured it was niche when I saw so little mention of the issue. My reason for xmax() in the result i

Re: System column support for partitioned tables using heap

2022-07-19 Thread Morris de Oryx
> The column list for a partitioned table must be fixed at the time it is > created, but we do not know what partitions might be added in the > future, and thus we don't know whether they will have an xmax column. Right, seeing what you're meaning there. It's fantastic that a partition might be an

Expose last replayed timeline ID along with last replayed LSN

2022-07-19 Thread Bharath Rupireddy
Hi, At times it's useful to know the last replayed WAL record's timeline ID (especially on the standbys that are lagging in applying WAL while failing over - for reporting, logging and debugging purposes). AFICS, there's no function that exposes the last replayed TLI. We can either change the exis

Memory leak fix in psql

2022-07-19 Thread tanghy.f...@fujitsu.com
Hi I think there is a newly introduced memory leak in your patch d2d3547. Try to fix it in the attached patch. Kindly to have a check. Regards, Tang v1-0001-fix-the-memory-leak-in-validateSQLNamePattern.patch Description: v1-0001-fix-the-memory-leak-in-validateSQLNamePattern.patch

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-07-19 Thread Bharath Rupireddy
On Tue, Jul 19, 2022 at 12:00 AM Nathan Bossart wrote: > > Overall, these patches look reasonable. > > On Mon, Jul 18, 2022 at 04:24:12PM +0530, Bharath Rupireddy wrote: > > record. Because the entire content of data pages is saved in the > > - log on the first page modification after a che

Re: Fast COPY FROM based on batch insert

2022-07-19 Thread Andrey Lepikhov
On 18/7/2022 13:22, Etsuro Fujita wrote: On Thu, Mar 24, 2022 at 3:43 PM Andrey V. Lepikhov wrote: On 3/22/22 06:54, Etsuro Fujita wrote: * To allow foreign multi insert, the patch made an invasive change to the existing logic to determine whether to use multi insert for the target relation, a

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Aleksander Alekseev
Hi Martin, I didn't look at the code yet but I very much like the idea. Many thanks for working on this! It's a pity your patch was too late for the July commitfest. In September, please keep an eye on cfbot [1] to make sure your patch applies properly. > As Tom's investigation showed, there is

Re: Memory leak fix in psql

2022-07-19 Thread Japin Li
On Tue, 19 Jul 2022 at 17:02, tanghy.f...@fujitsu.com wrote: > Hi > > I think there is a newly introduced memory leak in your patch d2d3547. > Try to fix it in the attached patch. > Kindly to have a check. > Yeah, it leaks, and the patch can fix it. After looking around, I found psql/describe

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Martin Kalcher
Am 18.07.22 um 23:48 schrieb Martin Kalcher: If we go with (1) array_shuffle() and array_sample() should shuffle each element individually and always return a one-dimensional array.   select array_shuffle('{{1,2},{3,4},{5,6}}');   ---    {1,4,3,5,6,2}   select array_sample('{{1,2

Re: Proposal: allow database-specific role memberships

2022-07-19 Thread Antonin Houska
Kenaniah Cerny wrote: > Rebased yet again... > > On Mon, Jul 4, 2022 at 1:17 PM Kenaniah Cerny wrote: > The "unsafe_tests" directory is where the pre-existing role tests were > located. According to the readme of the "unsafe_tests" directory, the tests > contained within are not run during

Re: Handle infinite recursion in logical replication setup

2022-07-19 Thread Amit Kapila
On Mon, Jul 18, 2022 at 9:46 PM vignesh C wrote: > > I have updated the patch to handle the origin value case > insensitively. The attached patch has the changes for the same. > Thanks, the patch looks mostly good to me. I have made a few changes in 0001 patch which are as follows: (a) make a com

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-07-19 Thread Amit Langote
Hi, On Tue, Jul 19, 2022 at 4:09 AM Andrew Dunstan wrote: > On 2022-07-15 Fr 17:07, Andres Freund wrote: > > Perhaps you could post your current state? I might be able to help resolving > > some of the problems. > > Ok. Here is the state of things. This has proved to be rather more > intractable

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Amit Kapila
On Tue, Jul 19, 2022 at 1:43 PM Kyotaro Horiguchi wrote: > > At Tue, 19 Jul 2022 16:57:14 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Tue, 19 Jul 2022 16:02:26 +0900, Masahiko Sawada > > wrote in > > > On Tue, Jul 19, 2022 at 1:47 PM Amit Kapila > > > wrote: > > > > Good work. I wonder

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-07-19 Thread Bharath Rupireddy
On Tue, Apr 26, 2022 at 6:31 AM Michael Paquier wrote: > > On Mon, Apr 25, 2022 at 01:34:38PM -0700, Nathan Bossart wrote: > > I took another look at the example output, and I think I agree that logging > > the total time for logical decoding operations is probably the best path > > forward. This

errdetail/errhint style

2022-07-19 Thread Justin Pryzby
Most of these are new in v15. In any case, I'm not sure if the others ought to be backpatched. There may be additional fixes to make with the same grepping. >From a33bd79c2f36046463489fbd37c76d7f0c3f7a8e Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 18 Jul 2022 13:54:38 -0500 Subject: [P

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Amit Kapila
On Tue, Jul 19, 2022 at 1:10 PM Masahiko Sawada wrote: > > On Mon, Jul 18, 2022 at 8:49 PM Amit Kapila wrote: > > > > On Sun, Jul 17, 2022 at 6:29 PM Masahiko Sawada > > wrote: > > > > > > On Fri, Jul 15, 2022 at 3:32 PM shiy.f...@fujitsu.com > > > wrote: > > > > > > > > > > I've attached patc

Re: errdetail/errhint style

2022-07-19 Thread Alvaro Herrera
On 2022-Jul-19, Justin Pryzby wrote: > https://www.postgresql.org/docs/current/error-style-guide.html#id-1.10.6.4.7 > > git grep 'errdetail("[[:lower:]]' > git grep 'errdetail(".*".*;' |grep -v '\."' Hmm, +1, though a few of these are still missing ending periods after your patch. -- Álvaro He

Re: Memory leak fix in psql

2022-07-19 Thread Michael Paquier
On Tue, Jul 19, 2022 at 06:41:13PM +0800, Japin Li wrote: > After looking around, I found psql/describe.c also has some memory leaks, > attached a patch to fix these leaks. Indeed. There are quite a bit of them, so let's fix all that. You have missed a couple of code paths in objectDescription()

Re: System column support for partitioned tables using heap

2022-07-19 Thread Robert Haas
On Tue, Jul 19, 2022 at 4:44 AM Morris de Oryx wrote: > My reason for xmax() in the result is to break down the affected rows count > into an insert count, and a modified estimate. Not super critical, but > helpful. I've built out some simple custom logging table in out system for > this kind o

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2022-07-19 Thread Damir Belyalov
Hi! Improved my patch by adding block subtransactions. The block size is determined by the REPLAY_BUFFER_SIZE parameter. I used the idea of a buffer for accumulating tuples in it. If we read REPLAY_BUFFER_SIZE rows without errors, the subtransaction will be committed. If we find an error, the subt

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Amit Kapila
On Tue, Jul 19, 2022 at 2:01 PM Masahiko Sawada wrote: > > On Tue, Jul 19, 2022 at 4:35 PM Kyotaro Horiguchi > wrote: > > > > > > > + Assert((xcnt > 0) && (xcnt == rb->catchange_ntxns)); > > > > (xcnt > 0) is obvious here (otherwise means dlist_foreach is broken..). > > (xcnt == rb->catchan

Re: errdetail/errhint style

2022-07-19 Thread Michael Paquier
On Tue, Jul 19, 2022 at 02:31:28PM +0200, Alvaro Herrera wrote: > Hmm, +1, though a few of these are still missing ending periods after > your patch. +1. -- Michael signature.asc Description: PGP signature

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Robert Haas
On Mon, Jul 18, 2022 at 6:43 PM Tom Lane wrote: > Um ... why is "the order in which the elements were chosen" a concept > we want to expose? ISTM sample() is a black box in which notionally > the decisions could all be made at once. I agree with that. But I also think it's fine for the elements

Re: Memory leak fix in psql

2022-07-19 Thread Japin Li
On Tue, 19 Jul 2022 at 20:32, Michael Paquier wrote: > On Tue, Jul 19, 2022 at 06:41:13PM +0800, Japin Li wrote: >> After looking around, I found psql/describe.c also has some memory leaks, >> attached a patch to fix these leaks. > > Indeed. There are quite a bit of them, so let's fix all that.

Re: Windows now has fdatasync()

2022-07-19 Thread Tom Lane
Thomas Munro writes: > The reason we need it for macOS is that they have had fdatasync > function for many years now, and configure detects it, but they > haven't ever declared it in a header, so we (accidentally?) do it in > c.h. We didn't set that up for Apple! The commit that added it was > 3

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Andrew Dunstan
On 2022-07-19 Tu 07:15, Martin Kalcher wrote: > Am 18.07.22 um 23:48 schrieb Martin Kalcher: >> >> If we go with (1) array_shuffle() and array_sample() should shuffle >> each element individually and always return a one-dimensional array. >> >>    select array_shuffle('{{1,2},{3,4},{5,6}}'); >>  

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Robert Haas
On Tue, Jul 19, 2022 at 9:53 AM Andrew Dunstan wrote: > > Having thought about it, i would go with (2). It gives the user the > > ability to decide wether or not array-of-arrays behavior is desired. > > If he wants the behavior of (1) he can flatten the array before > > applying array_shuffle(). U

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Masahiko Sawada
On Tue, Jul 19, 2022 at 9:25 PM Amit Kapila wrote: > > On Tue, Jul 19, 2022 at 1:10 PM Masahiko Sawada wrote: > > > > On Mon, Jul 18, 2022 at 8:49 PM Amit Kapila wrote: > > > > > > On Sun, Jul 17, 2022 at 6:29 PM Masahiko Sawada > > > wrote: > > > > > > > > On Fri, Jul 15, 2022 at 3:32 PM shiy

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 19, 2022 at 9:53 AM Andrew Dunstan wrote: >> Why not have an optional second parameter for array_shuffle that >> indicates whether or not to flatten? e.g. array_shuffle(my_array, >> flatten => true) > IMHO, if we think that's something many people are going to w

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Alvaro Herrera
[ Redirecting thread to -hackers from -committers ] On 2022-Jul-19, Tom Lane wrote: > Alvaro Herrera writes: > > Do you just need to send a patch to add an exports.txt file to > > src/pl/plpgsql/src/ for these functions? > > The precedent of plpython says that PGDLLEXPORT markers are sufficien

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Julien Rouhaud
On Tue, Jul 19, 2022 at 04:28:07PM +0200, Alvaro Herrera wrote: > ... oh, and: > > $ postmaster -c shared_preload_libraries=plugin_debugger > 2022-07-19 16:27:24.006 CEST [742142] FATAL: cannot request additional > shared memory outside shmem_request_hook This has been reported multiple times (i

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Pavel Stehule
Hi út 19. 7. 2022 v 16:28 odesílatel Alvaro Herrera napsal: > [ Redirecting thread to -hackers from -committers ] > > On 2022-Jul-19, Tom Lane wrote: > > > Alvaro Herrera writes: > > > > Do you just need to send a patch to add an exports.txt file to > > > src/pl/plpgsql/src/ for these functions

Re: Costing elided SubqueryScans more nearly correctly

2022-07-19 Thread Tom Lane
Alvaro Herrera writes: > Thanks, pushed. Pushed the original patch now too. regards, tom lane

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote: > Anyway, the minimal patch that makes plpgsql_check tests pass is > attached. This seems a bit random. Maybe it'd be better to have a > plpgsql_internal.h with functions that are exported only for plpgsql > itself, and keep plpgsql.h with

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 20:40:11 +0900, Amit Langote wrote: > About that, I was wondering if the blocks in llvm_compile_expr() need > to be hand-coded to match what's added in ExecInterpExpr() or if I've > missed some tool that can be used instead? The easiest way is to just call an external function f

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Pavel Stehule
Hi út 19. 7. 2022 v 17:31 odesílatel Andres Freund napsal: > Hi, > > On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote: > > Anyway, the minimal patch that makes plpgsql_check tests pass is > > attached. This seems a bit random. Maybe it'd be better to have a > > plpgsql_internal.h with functi

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 17:37:11 +0200, Pavel Stehule wrote: > út 19. 7. 2022 v 17:31 odesílatel Andres Freund napsal: > > > Hi, > > > > On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote: > > > Anyway, the minimal patch that makes plpgsql_check tests pass is > > > attached. This seems a bit random.

Autovacuum worker spawning strategy

2022-07-19 Thread Rafael Thofehrn Castro
Hello all, While investigating a problem in a PG14 instance I noticed that autovacuum workers stop processing other databases when a database has a temporary table with age older than `autovacuum_freeze_max_age`. To test that I added a custom logline showing which database the about to spawned aut

Re: Convert planner's AggInfo and AggTransInfo to Nodes

2022-07-19 Thread Peter Eisentraut
On 18.07.22 18:08, Tom Lane wrote: I'm kind of tempted to mount an effort to get rid of as many of pathnodes.h's "read_write_ignore" annotations as possible. Some are necessary to prevent infinite recursion, and others represent considered judgments that they'd bloat node dumps more than they're

Re: NAMEDATALEN increase because of non-latin languages

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 14:30:34 +0700, John Naylor wrote: > I wrote: > > > On Mon, Jul 18, 2022 at 9:58 AM Andres Freund wrote: > > > Hm. Wouldn't it make sense to just use the normal tuple deforming > routines and > > > then map the results to the structs? > > > > I wasn't sure if they'd be suitable

Re: [PATCH] Log details for client certificate failures

2022-07-19 Thread Jacob Champion
On Fri, Jul 15, 2022 at 4:45 PM Andres Freund wrote: > On 2022-07-15 14:51:38 -0700, Jacob Champion wrote: > > That seems much worse than escaping for this particular patch; if your > > cert's Common Name is in (non-ASCII) UTF-8 then all you'll see is > > "CN=?" in the log lines that were

Re: [PATCH] Log details for client certificate failures

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 09:07:31 -0700, Jacob Champion wrote: > On Fri, Jul 15, 2022 at 4:45 PM Andres Freund wrote: > > On 2022-07-15 14:51:38 -0700, Jacob Champion wrote: > > > That seems much worse than escaping for this particular patch; if your > > > cert's Common Name is in (non-ASCII) UTF-8 then

Re: Memory leak fix in psql

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 21:08:53 +0800, Japin Li wrote: > From b2bcc3a1bac67b8b414f2025607f8dd35e096289 Mon Sep 17 00:00:00 2001 > From: Japin Li > Date: Tue, 19 Jul 2022 18:27:25 +0800 > Subject: [PATCH v2 1/1] Fix the memory leak in psql describe > > --- > src/bin/psql/describe.c | 168

Re: [PATCH] Log details for client certificate failures

2022-07-19 Thread Jacob Champion
[resending to list] On 7/19/22 09:14, Andres Freund wrote: > IMO this should replace the existing ascii escape function instead. That will affect the existing behavior of application_name and cluster_name; is that acceptable? --Jacob

Re: Convert planner's AggInfo and AggTransInfo to Nodes

2022-07-19 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > It seems like a reasonable idea, but I don't know enough to judge the > wider ramifications of it. But one thing that the patch should also do, > is switch to using the l*_node() functions instead of manual casting. Hm, I didn't bother with tha

Re: Memory leak fix in psql

2022-07-19 Thread Tom Lane
Andres Freund writes: > On 2022-07-19 21:08:53 +0800, Japin Li wrote: >> +{ >> +termPQExpBuffer(&buf); >> return false; >> +} > Adding copy over copy of this same block doesn't seem great. Can we instead > add a helper for it or such? The usual style in these fil

Re: [PATCH] Log details for client certificate failures

2022-07-19 Thread Tom Lane
Jacob Champion writes: > On 7/19/22 09:14, Andres Freund wrote: >> IMO this should replace the existing ascii escape function instead. > That will affect the existing behavior of application_name and > cluster_name; is that acceptable? I think Andres' point is exactly that these should all act a

Re: Memory leak fix in psql

2022-07-19 Thread Mark Dilger
> On Jul 19, 2022, at 2:02 AM, tanghy.f...@fujitsu.com wrote: > > I think there is a newly introduced memory leak in your patch d2d3547. I agree. Thanks for noticing, and for the patch! > Try to fix it in the attached patch. > Kindly to have a check. This looks ok, but comments down-thread

Re: Autovacuum worker spawning strategy

2022-07-19 Thread Nathan Bossart
On Tue, Jul 19, 2022 at 12:40:06PM -0300, Rafael Thofehrn Castro wrote: > PG prioritizes databases that need to be frozen and since a temporary table > can't > be frozen by a process other than the session that created it, that DB will > remain > a priority until the table is dropped. > > I acknow

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-07-19 Thread Nathan Bossart
On Tue, Jul 19, 2022 at 12:55:14AM -0500, Steve Chavez wrote: > Taking your options into consideration, for me the correct behaviour should > be: > > - The ALTER ROLE placeholder should always be stored with a PGC_USERSET > GucContext. It's a placeholder anyway, so it should be the less restrictiv

StrategyAM for IndexAMs

2022-07-19 Thread Simon Riggs
I'm preparing the way for a later patch that would allow unique hash indexes to be primary keys. There are various parts to the problem. I was surprised at how many times we hardcode BTREE_AM_OID and associated BT Strategy Numbers in many parts of the code, so have been looking for ways to reconcil

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-07-19 Thread Nathan Bossart
On Tue, Jul 19, 2022 at 02:43:59PM +0530, Bharath Rupireddy wrote: > Done. PSA v5 patch set. LGTM. I've marked this as ready-for-committer. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [PATCH] Log details for client certificate failures

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 12:39:43 -0400, Tom Lane wrote: > Having said that, I struggle to see why we are panicking about badly > encoded log data from this source while blithely ignoring the problems > posed by non-ASCII role names, database names, and tablespace names. I think we should fix these as w

Re: First draft of the PG 15 release notes

2022-07-19 Thread Bruce Momjian
On Mon, Jul 18, 2022 at 08:23:23PM -0500, Justin Pryzby wrote: > > Increase hash_mem_multiplier default to 2.0 (Peter Geoghegan) > > This allows query hash operations to use double the amount of work_mem > > memory as other operations. > > I wonder if it's worth pointing out that a query may end

Re: System catalog documentation chapter

2022-07-19 Thread Bruce Momjian
On Mon, Jul 18, 2022 at 09:22:24PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Sat, Jul 16, 2022 at 10:53:17AM +0200, Peter Eisentraut wrote: > >> Maybe this whole notion that "system views" is one thing is not suitable. > > > Are you thinking we should just call the chapter "System Cat

Re: First draft of the PG 15 release notes

2022-07-19 Thread Justin Pryzby
On Tue, Jul 19, 2022 at 01:24:30PM -0400, Bruce Momjian wrote: > > > Remove pg_dump's --no-synchronized-snapshots option since all supported > > > server versions support synchronized snapshots (Tom Lane) > > > > It'd be better to put that after the note about dropping support for > > upgrading

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2022-07-19 Thread Tomas Vondra
On 7/18/22 20:45, Tom Lane wrote: > Tomas Vondra writes: >> Thanks. I'll switch this to "needs review" now. > > OK, I looked through this, and attach some review suggestions in the > form of a delta patch. (0001 below is your two patches merged, 0002 > is my delta.) A lot of the delta is com

Re: Convert planner's AggInfo and AggTransInfo to Nodes

2022-07-19 Thread Tom Lane
Peter Eisentraut writes: > On 18.07.22 18:08, Tom Lane wrote: >> I'm kind of tempted to mount an effort to get rid of as many of >> pathnodes.h's "read_write_ignore" annotations as possible. Some are >> necessary to prevent infinite recursion, and others represent considered >> judgments that the

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2022-07-19 Thread Tom Lane
Tomas Vondra writes: > I we want to improve sampling for partitioned cases (where the foreign > table is just one of many partitions), I think we'd have to rework how > we determine sample size for each partition. Now we simply calculate > that from relpages, which seems quite fragile (different a

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-07-19 Thread Martin Kalcher
Am 19.07.22 um 16:20 schrieb Tom Lane: So I withdraw my original position. These functions should just shuffle or select in the array's first dimension, preserving subarrays. Or else be lazy and reject more-than-one-D arrays; but it's probably not that hard to handle them. Here is a patch w

Re: pg_parameter_aclcheck() and trusted extensions

2022-07-19 Thread Tom Lane
Nathan Bossart writes: >> I think this is because GUCArrayReset() is the only caller of >> validate_option_array_item() that sets skipIfNoPermissions to true. The >> others fall through to set_config_option(), which does a >> pg_parameter_aclcheck(). So, you are right. > Here's a small patch th

Re: First draft of the PG 15 release notes

2022-07-19 Thread Bruce Momjian
On Tue, Jul 19, 2022 at 01:13:07PM -0500, Justin Pryzby wrote: > On Tue, Jul 19, 2022 at 01:24:30PM -0400, Bruce Momjian wrote: > > Well, I put the --no-synchronized-snapshots item in incompatibilities > > since it is a user-visible change that might require script adjustments. > > However, I put t

Re: Slow standby snapshot

2022-07-19 Thread Michail Nikolaev
Hello, Andrey. > I've looked into v5. Thanks! Patch is updated accordingly your remarks. Best regards, Michail. From 1301a262dea7f541c11092851e82f10932150ee3 Mon Sep 17 00:00:00 2001 From: Michail Nikolaev Date: Tue, 19 Jul 2022 23:50:53 +0300 Subject: [PATCH v6] Currently KnownAssignedXidsGetA

Re: pg_parameter_aclcheck() and trusted extensions

2022-07-19 Thread Nathan Bossart
On Tue, Jul 19, 2022 at 04:27:08PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> However, I wonder if a >> better way to fix this is to provide a way to stop set_config_option() from >> throwing errors (e.g., setting elevel to -1). That way, we could remove >> the manual permissions checks i

Re: [PATCH] Log details for client certificate failures

2022-07-19 Thread Jacob Champion
On Tue, Jul 19, 2022 at 10:09 AM Andres Freund wrote: > On 2022-07-19 12:39:43 -0400, Tom Lane wrote: > > Having said that, I struggle to see why we are panicking about badly > > encoded log data from this source while blithely ignoring the problems > > posed by non-ASCII role names, database name

Re: [PATCH] Log details for client certificate failures

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 15:08:38 -0700, Jacob Champion wrote: > v2 adds escaping to pg_clean_ascii(). My original attempt used > StringInfo allocation, but that didn't play well with guc_malloc(), so > I switched to a two-pass API where the caller allocates. Let me know > if I'm missing something obviou

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote: > Anyway, the minimal patch that makes plpgsql_check tests pass is > attached. Do you want to commit that or should I? Greetings, Andres Freund

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-19 Thread Andres Freund
Hi, On 2022-07-19 08:31:49 -0700, Andres Freund wrote: > But I think we probably should err on the side of adding more > declarations, rather than the oposite. To see if there's other declarations that should be added, I used https://codesearch.debian.net/search?q=load_external_function&literal=1

Re: Convert planner's AggInfo and AggTransInfo to Nodes

2022-07-19 Thread Tom Lane
I wrote: > * WRITE_OID_ARRAY and WRITE_BOOL_ARRAY needed extension to handle a null > array pointer. I think we should make all the WRITE_FOO_ARRAY macros > work alike, so I added that to all of them. I first tried to make the > rest work like WRITE_INDEX_ARRAY, but that failed because readfuncs.

Re: [Commitfest 2022-07] Begins Now

2022-07-19 Thread Joe Conway
On 7/18/22 02:53, Alvaro Herrera wrote: On 2022-Jul-18, Aleksander Alekseev wrote: Hi hackers, If someone put a lot of review into a patchset a few months ago, they absolutely deserve credit. But if that entry has been sitting with no feedback this month, why is it useful to keep that Reviewe

Re: Memory leak fix in psql

2022-07-19 Thread Michael Paquier
On Tue, Jul 19, 2022 at 12:36:31PM -0400, Tom Lane wrote: > Yeah, it's old school, but please let's not have a few functions that > do it randomly differently from all their neighbors. True enough. And it is not like we should free the PQExpBuffer given by the caller in validateSQLNamePattern().

Re: Memory leak fix in psql

2022-07-19 Thread Michael Paquier
On Tue, Jul 19, 2022 at 09:43:21AM -0700, Mark Dilger wrote: > This looks ok, but comments down-thread seem reasonable, so I > suspect a new patch will be needed. Would you like to author it, or > would you prefer that I, as the guilty party, do so? If any of you could update the patch, that wou

Re: Add a test for "cannot truncate foreign table"

2022-07-19 Thread Fujii Masao
On 2022/07/15 16:52, Fujii Masao wrote: On 2022/07/08 17:03, Etsuro Fujita wrote: Rather than doing so, I'd vote for adding a test case to file_fdw, as proposed in the patch, because that would be much simpler and much less expensive. So ISTM that most agreed to push Nagata-san's patch ad

Re: Add a test for "cannot truncate foreign table"

2022-07-19 Thread Yugo NAGATA
On Wed, 20 Jul 2022 09:38:17 +0900 Fujii Masao wrote: > > > On 2022/07/15 16:52, Fujii Masao wrote: > > > > > > On 2022/07/08 17:03, Etsuro Fujita wrote: > >> Rather than doing so, I'd vote for adding a test case to file_fdw, as > >> proposed in the patch, because that would be much simpler a

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Kyotaro Horiguchi
At Tue, 19 Jul 2022 17:31:07 +0900, Masahiko Sawada wrote in > On Tue, Jul 19, 2022 at 4:35 PM Kyotaro Horiguchi > wrote: > > At Tue, 19 Jul 2022 10:17:15 +0530, Amit Kapila > > wrote in > > > Good work. I wonder without comments this may create a problem in the > > > future. OTOH, I don't se

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-07-19 Thread Kyotaro Horiguchi
At Tue, 19 Jul 2022 09:58:28 -0700, Nathan Bossart wrote in > On Tue, Jul 19, 2022 at 02:43:59PM +0530, Bharath Rupireddy wrote: > > Done. PSA v5 patch set. > > LGTM. I've marked this as ready-for-committer. I find the following sentense in config.sgml. "Specifies the minimum size of past log

Re: errdetail/errhint style

2022-07-19 Thread Michael Paquier
On Tue, Jul 19, 2022 at 09:51:13PM +0900, Michael Paquier wrote: > +1. I have looked at that and added the extra periods, and applied it. Thanks, Justin. -- Michael signature.asc Description: PGP signature

Re: Expose last replayed timeline ID along with last replayed LSN

2022-07-19 Thread Kyotaro Horiguchi
At Tue, 19 Jul 2022 14:28:40 +0530, Bharath Rupireddy wrote in > Hi, > > At times it's useful to know the last replayed WAL record's timeline > ID (especially on the standbys that are lagging in applying WAL while > failing over - for reporting, logging and debugging purposes). AFICS, > there's

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Masahiko Sawada
On Wed, Jul 20, 2022 at 9:58 AM Kyotaro Horiguchi wrote: > > At Tue, 19 Jul 2022 17:31:07 +0900, Masahiko Sawada > wrote in > > On Tue, Jul 19, 2022 at 4:35 PM Kyotaro Horiguchi > > wrote: > > > At Tue, 19 Jul 2022 10:17:15 +0530, Amit Kapila > > > wrote in > > > > Good work. I wonder without

Is select_outer_pathkeys_for_merge() too strict now we have Incremental Sorts?

2022-07-19 Thread David Rowley
Hackers, Currently, if we have a query such as: SELECT a,b, COUNT(*) FROM a INNER JOIN b on a.a = b.x GROUP BY a,b ORDER BY a DESC, b; With enable_hashagg = off, we get the following plan: QUERY PLAN --- GroupAggregate Group Key: a.a, a.b

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Amit Kapila
On Tue, Jul 19, 2022 at 7:28 PM Masahiko Sawada wrote: > > On Tue, Jul 19, 2022 at 9:25 PM Amit Kapila wrote: > > > > On Tue, Jul 19, 2022 at 1:10 PM Masahiko Sawada > > wrote: > > > > > > > > BTW on backbranches, I think that the reason why we add > > > initial_running_xacts stuff to ReorderBu

RE: Memory leak fix in psql

2022-07-19 Thread tanghy.f...@fujitsu.com
On Tuesday, July 19, 2022 7:41 PM, Japin Li wrote: > After looking around, I found psql/describe.c also has some memory > leaks, > attached a patch to fix these leaks. Thanks for your check and improvement. Your fix LGTM so please allow me to merge it in the attached patch. Based on your rebased

Re: System column support for partitioned tables using heap

2022-07-19 Thread Morris de Oryx
On Tue, Jul 19, 2022 at 10:38 PM Robert Haas wrote: > For MERGE itself, I wonder if some information about this should be > included in the command tag. It looks like MERGE already includes some > sort of row count in the command tag, but I guess perhaps it doesn't > distinguish between inserts

Re: Windows now has fdatasync()

2022-07-19 Thread Thomas Munro
Ok, I've pushed the Windows patch. I'll watch the build farm to see if I've broken any of the frankentoolchain Windows animals. Mikael kindly upgraded conchuela, so that leaves just prairiedog without fdatasync. I've attached a patch to drop the configure probe for that once prairiedog's host is

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-19 Thread Masahiko Sawada
On Wed, Jul 20, 2022 at 12:11 PM Amit Kapila wrote: > > On Tue, Jul 19, 2022 at 7:28 PM Masahiko Sawada wrote: > > > > On Tue, Jul 19, 2022 at 9:25 PM Amit Kapila wrote: > > > > > > On Tue, Jul 19, 2022 at 1:10 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > BTW on backbranches, I think

Re: Memory leak fix in psql

2022-07-19 Thread Michael Paquier
On Wed, Jul 20, 2022 at 03:14:35AM +, tanghy.f...@fujitsu.com wrote: > Your fix LGTM so please allow me to merge it in the attached patch. > Based on your rebased version, now this new patch version is V3. What about the argument of upthread where we could use a goto in functions where there

Re: Is select_outer_pathkeys_for_merge() too strict now we have Incremental Sorts?

2022-07-19 Thread David Rowley
On Wed, 20 Jul 2022 at 15:02, David Rowley wrote: > I've attached a patch for this and it changes the plan for the above query to: Looks like I based that patch on the wrong branch. Here's another version of the patch that's based on master. David diff --git a/src/backend/optimizer/path/pathkey

Re: Memory leak fix in psql

2022-07-19 Thread Japin Li
On Wed, 20 Jul 2022 at 11:51, Michael Paquier wrote: > On Wed, Jul 20, 2022 at 03:14:35AM +, tanghy.f...@fujitsu.com wrote: >> Your fix LGTM so please allow me to merge it in the attached patch. >> Based on your rebased version, now this new patch version is V3. > > What about the argument

  1   2   >