Re: New strategies for freezing, advancing relfrozenxid early

2022-08-31 Thread Peter Geoghegan
On Tue, Aug 30, 2022 at 11:28 PM Jeff Davis wrote: > That clarifies your point. It's still a challenge for me to reason > about which of these potential new problems really need to be solved in > v1, though. I don't claim to understand it that well myself -- not just yet. I feel like I have the r

Re: pg_stat_have_stats() returns true for dropped indexes (or for index creation transaction rolled back)

2022-08-31 Thread Kyotaro Horiguchi
At Thu, 25 Aug 2022 11:44:34 +0200, "Drouvot, Bertrand" wrote in > Looking closer at it, I think we are already good for the drop case on > the tables (by making direct use of the pg_stat_get_* functions on the > before dropped oid). > > So I think we can remove all the "table" new checks: new

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Michael Paquier
On Sun, Aug 28, 2022 at 02:52:43PM -0700, Nathan Bossart wrote: > On Sat, Aug 27, 2022 at 02:06:32PM +0530, Bharath Rupireddy wrote: >> PSA v17 patch with reorderbuffer.c changes reverted. > > LGTM Yeah, what you have here looks pretty fine to me, so this is IMO committable. Let's wait a bit to

Re: SQL/JSON features for v15

2022-08-31 Thread Amit Langote
On Wed, Aug 31, 2022 at 3:51 PM Amit Langote wrote: > On Wed, Aug 31, 2022 at 6:25 AM Nikita Glukhov > wrote: > > v10 patches > > Finally, I get this warning: > > execExprInterp.c: In function ‘ExecJsonCoerceCStringToText’: > execExprInterp.c:4765:3: warning: missing braces around initializer >

Re: [RFC] building postgres with meson - v12

2022-08-31 Thread Peter Eisentraut
I found that the perl test modules are not installed. See attached patch to correct this. To the patches: 4e15ee0e24 Don't hardcode tmp_check/ as test directory for tap tests 1a3169bc3f Split TESTDIR into TESTLOGDIR and TESTDATADIR It's a bit weird that the first patch changes the meaning of

Re: [RFC] building postgres with meson - v11

2022-08-31 Thread Peter Eisentraut
On 24.08.22 17:30, Andres Freund wrote: 0545eec895 meson: Add docs We should think more about how to arrange the documentation. We probably don't want to copy-and-paste all the introductory and requirements information. I think we can make this initially much briefer, like the Windows installa

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Peter Eisentraut
On 30.08.22 15:16, Christoph Berg wrote: I found the list of TG_ variables on https://www.postgresql.org/docs/current/plpgsql-trigger.html#PLPGSQL-DML-TRIGGER hard to read for several reasons: too much whitespace, all the lines start with "Data type", and even after that, the actual content is hi

Re: First draft of the PG 15 release notes

2022-08-31 Thread Peter Eisentraut
On 30.08.22 22:42, Bruce Momjian wrote: Good question --- the full text is: Record and check the collation of each database (Peter Eisentraut) This is designed to detect collation mismatches to avoid data corruption. Function pg_da

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Daniel Gustafsson
> On 31 Aug 2022, at 11:35, Peter Eisentraut > wrote: > > On 30.08.22 15:16, Christoph Berg wrote: >> I found the list of TG_ variables on >> https://www.postgresql.org/docs/current/plpgsql-trigger.html#PLPGSQL-DML-TRIGGER >> hard to read for several reasons: too much whitespace, all the lines >

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Christoph Berg
Re: Peter Eisentraut > I find the new version even harder to read. The catalog_table_entry stuff > doesn't really make sense here, since what you have before is already a > definition list, and afterwards you have the same, just marked up > "incorrectly". Fair enough. For comparison, this is what

Re: [PATCH] Fix alter subscription concurrency errors

2022-08-31 Thread Asif Rehman
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested The patch applies with few "Hunk succeeded, offset -3 lines" warnings

Re: Handle infinite recursion in logical replication setup

2022-08-31 Thread Amit Kapila
On Wed, Aug 31, 2022 at 11:35 AM Peter Smith wrote: > > Here are my review comments for v43-0001. > > == > > 1. Commit message > > The initial copy phase has no way to know the origin of the row data, > so if 'copy_data = true' in the step 4 below, log a warning to notify user > that potential

Re: Small cleanups to tuplesort.c and a bonus small performance improvement

2022-08-31 Thread David Rowley
On Fri, 26 Aug 2022 at 16:48, David Rowley wrote: > 0003: Changes writetuple to tell it what it should do in regards to > freeing and adjusting the memory accounting. > > Probably 0003 could be done differently. I'm certainly not set on the > bool args. I understand that I'm never calling it with

Re: SQL/JSON features for v15

2022-08-31 Thread Amit Langote
On Wed, Aug 31, 2022 at 3:51 PM Amit Langote wrote: > SELECT JSON_VALUE(jsonb '"aaa"', '$' RETURNING int DEFAULT 111 ON ERROR); > - json_value > - > -111 > -(1 row) > - > +ERROR: syntax error at or near "DEFAULT" > +LINE 1: ...ELECT JSON_VALUE(jsonb '"aaa"', '$' RETURNING int

Avoid overhead open-close indexes (catalog updates)

2022-08-31 Thread Ranier Vilela
Hi, The commit https://github.com/postgres/postgres/commit/b17ff07aa3eb142d2cde2ea00e4a4e8f63686f96 Introduced the CopyStatistics function. To do the work, CopyStatistics uses a less efficient function to update/insert tuples at catalog systems. The comment at indexing.c says: "Avoid using it fo

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Christoph Berg
Re: To Peter Eisentraut > The new version of the patch just moves up the data types, and removes > the extra clutter from the beginnings of each description: The last version had the brackets in TG_ARGV[] (text[]) duplicated. Christoph >From 1355794d56919a015bd1528f62428beaab0a681b Mon Sep 17 00:

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-31 Thread Robert Haas
On Tue, Aug 30, 2022 at 5:20 PM Nathan Bossart wrote: > On Tue, Aug 30, 2022 at 03:24:56PM -0400, Robert Haas wrote: > > - william => charles => elizabeth => uk is OK because the first two > > hops have ADMIN and the last hop has INHERIT > > Don't you mean the first two hops have INHERIT and the l

pg_upgrade generated files in subdir follow-up

2022-08-31 Thread Daniel Gustafsson
Commit 38bfae36526 moved the .txt files pg_upgrade generates to a separate subdir, but there are a few left which are written to cwd. The thread resulting in that patch doesn't discuss these files specifically so it seems they are just an oversight. Unless I'm missing something. Should something

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Daniel Gustafsson
> On 31 Aug 2022, at 13:55, Christoph Berg wrote: > > Re: To Peter Eisentraut >> The new version of the patch just moves up the data types, and removes >> the extra clutter from the beginnings of each description: > > The last version had the brackets in TG_ARGV[] (text[]) duplicated. This, and

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Christoph Berg
Re: Daniel Gustafsson > This, and the other string variables, now reads a bit strange IMO: > > - Data type text; a string of > + string > INSERT, UPDATE, > DELETE, or TRUNCATE > > Wouldn't it be better with "string containing INSERT.." or something > similar? Right, t

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-31 Thread Robert Haas
On Tue, Aug 30, 2022 at 6:10 PM Nathan Bossart wrote: > On Tue, Aug 30, 2022 at 08:33:46AM -0400, Robert Haas wrote: > > Third try. > > Now that I have this grantor stuff fresh in my mind, this patch looks good > to me. Thanks for reviewing. Committed. -- Robert Haas EDB: http://www.enterprised

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Christoph Berg
Re: To Daniel Gustafsson > "string containing" is again pretty boilerplatish, how about just > "contains"? Actually, just omitting the whole prefix works best. TG_WHEN (text) BEFORE, AFTER, or INSTEAD OF, depending on the trigger's definition. I also shortened some "name of table" to just "

Re: SQL/JSON features for v15

2022-08-31 Thread Andrew Dunstan
On 2022-08-31 We 07:01, Amit Langote wrote: > On Wed, Aug 31, 2022 at 3:51 PM Amit Langote wrote: >> SELECT JSON_VALUE(jsonb '"aaa"', '$' RETURNING int DEFAULT 111 ON ERROR); >> - json_value >> - >> -111 >> -(1 row) >> - >> +ERROR: syntax error at or near "DEFAULT" >> +LINE

allowing for control over SET ROLE

2022-08-31 Thread Robert Haas
Hi, There are two ways in which a role can exercise the privileges of some other role which has been granted to it. First, it can implicitly inherit the privileges of the granted role. Second, it can assume the identity of the granted role using the SET ROLE command. It is possible to control the

Re: Slight refactoring of state check in pg_upgrade check_ function

2022-08-31 Thread Daniel Gustafsson
> On 30 Aug 2022, at 23:08, Bruce Momjian wrote: > On Sun, Aug 28, 2022 at 03:06:09PM -0700, Nathan Bossart wrote: >> The patch looks reasonable to me. > > +1. Those checks have accumulated over time with different authors, > hence the stylistic differences. Pushed, thanks for review! -- Danie

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-08-31 Thread Imseih (AWS), Sami
>> For the benefit of anyone who may be looking at this thread in the >> archive later, I believe this commit will have fixed this issue: I can also confirm that indeed the commit https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=6672d79 does fix this issue. Thanks! --

Re: pg_upgrade generated files in subdir follow-up

2022-08-31 Thread Tom Lane
Daniel Gustafsson writes: > Commit 38bfae36526 moved the .txt files pg_upgrade generates to a separate > subdir, but there are a few left which are written to cwd. The thread > resulting in that patch doesn't discuss these files specifically so it seems > they are just an oversight. Unless I'm mi

pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
An internal VM crashed last night due to OOM. When I tried to start postgres, it failed like: < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery immediate wait < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of generated WAL; request 1201/1CAF84F0, curren

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

2022-08-31 Thread Dagfinn Ilmari Mannsåker
Christoph Berg writes: > Re: To Daniel Gustafsson >> "string containing" is again pretty boilerplatish, how about just >> "contains"? > > Actually, just omitting the whole prefix works best. > > TG_WHEN (text) > > BEFORE, AFTER, or INSTEAD OF, depending on the trigger's definition. The attac

Re: SQL/JSON features for v15

2022-08-31 Thread Jonathan S. Katz
On 8/31/22 8:38 AM, Andrew Dunstan wrote: On 2022-08-31 We 07:01, Amit Langote wrote: On Wed, Aug 31, 2022 at 3:51 PM Amit Langote wrote: SELECT JSON_VALUE(jsonb '"aaa"', '$' RETURNING int DEFAULT 111 ON ERROR); - json_value - -111 -(1 row) - +ERROR: syntax error at or

Re: pg_upgrade generated files in subdir follow-up

2022-08-31 Thread Daniel Gustafsson
> On 31 Aug 2022, at 15:59, Tom Lane wrote: > > Daniel Gustafsson writes: >> Commit 38bfae36526 moved the .txt files pg_upgrade generates to a separate >> subdir, but there are a few left which are written to cwd. The thread >> resulting in that patch doesn't discuss these files specifically so

Re: introduce bufmgr hooks

2022-08-31 Thread Andres Freund
HHi, On 2022-08-29 15:24:49 -0700, Nathan Bossart wrote: > I'd like to propose some new hooks for the buffer manager. My primary goal > is to allow users to create an additional caching mechanism between the > shared buffers and disk for evicted buffers. I'm very doubtful this is a good idea. Th

Re: SQL/JSON features for v15

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 10:20:24 -0400, Jonathan S. Katz wrote: > Andres, Robert, Tom: With this recent work, have any of your opinions > changed on including SQL/JSON in v15? I don't really know what to do here. It feels blatantly obvious that this code isn't even remotely close to being releasable. I

Re: [PATCH] Query Jumbling for CALL and SET utility statements

2022-08-31 Thread Pavel Stehule
Hi st 31. 8. 2022 v 17:34 odesílatel Drouvot, Bertrand napsal: > Hi hackers, > > While query jumbling is provided for function calls that’s currently not > the case for procedures calls. > The reason behind this is that all utility statements are currently > discarded for jumbling. > > We’ve re

Re: SQL/JSON features for v15

2022-08-31 Thread Andrew Dunstan
On 2022-08-30 Tu 17:25, Nikita Glukhov wrote: > > > Patches 0001-0006: Yeah, these add the overhead of an extra function call (typin() -> typin_opt_error()) in possibly very common paths. Other than refactoring *all* places that call typin() to use the new API, the

Re: Tracking last scan time

2022-08-31 Thread Dave Page
On Tue, 30 Aug 2022 at 19:46, Bruce Momjian wrote: > On Fri, Aug 26, 2022 at 02:05:36PM +0100, Dave Page wrote: > > On Thu, 25 Aug 2022 at 01:44, David Rowley wrote: > > I don't have a particular opinion about the patch, I'm just pointing > > out that there are other ways. Even just writ

Add tracking of backend memory allocated to pg_stat_activity

2022-08-31 Thread Reid Thompson
Hi Hackers, Attached is a patch to Add tracking of backend memory allocated to pg_stat_activity     This new field displays the current bytes of memory allocated to the backend process.  It is updated as memory for the process is malloc'd/free'd. Memory allocated to items on the freelist is incl

Re: SQL/JSON features for v15

2022-08-31 Thread Tom Lane
Andres Freund writes: > On 2022-08-31 10:20:24 -0400, Jonathan S. Katz wrote: >> Andres, Robert, Tom: With this recent work, have any of your opinions >> changed on including SQL/JSON in v15? > I don't really know what to do here. It feels blatantly obvious that this code > isn't even remotely cl

Re: [PATCH] Query Jumbling for CALL and SET utility statements

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 17:33:44 +0200, Drouvot, Bertrand wrote: > While query jumbling is provided for function calls that’s currently not the > case for procedures calls. > The reason behind this is that all utility statements are currently > discarded for jumbling. > [...] > That’s why we think it wo

Re: Tracking last scan time

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 05:02:33PM +0100, Dave Page wrote: > > > On Tue, 30 Aug 2022 at 19:46, Bruce Momjian wrote: > > On Fri, Aug 26, 2022 at 02:05:36PM +0100, Dave Page wrote: > > On Thu, 25 Aug 2022 at 01:44, David Rowley wrote: > >     I don't have a particular opinion about t

Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Reid Thompson
Hi Hackers, This patch ensures get_database_list() switches back to the memory context in use upon entry rather than returning with TopMemoryContext as the context. This will address memory allocations in autovacuum.c being associated with TopMemoryContext when they should not be. autovacuum.c d

Re: Tracking last scan time

2022-08-31 Thread Andres Freund
Hi, On 2022-08-23 10:55:09 +0100, Dave Page wrote: > Often it is beneficial to review one's schema with a view to removing > indexes (and sometimes tables) that are no longer required. It's very > difficult to understand when that is the case by looking at the number of > scans of a relation as, f

Re: SQL/JSON features for v15

2022-08-31 Thread Robert Haas
On Wed, Aug 31, 2022 at 10:20 AM Jonathan S. Katz wrote: > Andres, Robert, Tom: With this recent work, have any of your opinions > changed on including SQL/JSON in v15? No. Nothing's been committed, and there's no time to review anything in detail, and there was never going to be. Nikita said he

Re: SQL/JSON features for v15

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 12:26:29 -0400, Robert Haas wrote: > On Wed, Aug 31, 2022 at 10:20 AM Jonathan S. Katz > wrote: > > Andres, Robert, Tom: With this recent work, have any of your opinions > > changed on including SQL/JSON in v15? > > No. Nothing's been committed, and there's no time to review a

Re: SQL/JSON features for v15

2022-08-31 Thread Jonathan S. Katz
On 8/31/22 12:26 PM, Robert Haas wrote: On Wed, Aug 31, 2022 at 10:20 AM Jonathan S. Katz wrote: Andres, Robert, Tom: With this recent work, have any of your opinions changed on including SQL/JSON in v15? No. Nothing's been committed, and there's no time to review anything in detail, and ther

Add the ability to limit the amount of memory that can be allocated to backends.

2022-08-31 Thread Reid Thompson
Hi Hackers, Add the ability to limit the amount of memory that can be allocated to backends. This builds on the work that adds backend memory allocated to pg_stat_activity https://www.postgresql.org/message-id/67bb5c15c0489cb499723b0340f16e10c22485ec.camel%40crunchydata.com Both patches are attac

Re: [PATCH] Query Jumbling for CALL and SET utility statements

2022-08-31 Thread Pavel Stehule
st 31. 8. 2022 v 17:50 odesílatel Pavel Stehule napsal: > Hi > > > st 31. 8. 2022 v 17:34 odesílatel Drouvot, Bertrand > napsal: > >> Hi hackers, >> >> While query jumbling is provided for function calls that’s currently not >> the case for procedures calls. >> The reason behind this is that all

Re: Add tracking of backend memory allocated to pg_stat_activity

2022-08-31 Thread Justin Pryzby
On Wed, Aug 31, 2022 at 12:03:06PM -0400, Reid Thompson wrote: > Hi Hackers, > > Attached is a patch to > Add tracking of backend memory allocated to pg_stat_activity > + proargmodes => > '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', In the past, there was concern about

Re: SQL/JSON features for v15

2022-08-31 Thread Tom Lane
I wrote: > Andres Freund writes: >> From my POV the only real discussion is whether we'd want to revert this in >> 15 >> and HEAD or just 15. There's imo a decent point to be made to just revert in >> 15 and aggressively press forward with the changes posted in this thread. > I'm not for that.

Re: SQL/JSON features for v15

2022-08-31 Thread Robert Haas
On Wed, Aug 31, 2022 at 1:06 PM Tom Lane wrote: > The currently proposed patchset hacks up a relatively small number > of core datatypes to be able to do that. But it's just a hack > and there's no prospect of extension types being able to join > in the fun. I think where we need to start, for v

Re: SQL/JSON features for v15

2022-08-31 Thread Tom Lane
Robert Haas writes: > On Wed, Aug 31, 2022 at 1:06 PM Tom Lane wrote: >> (I don't say that we'd convert every datatype to do so right away; >> in the long run we should, but I'm content to start with just the >> same core types touched here.) > I would be in favor of making more of an effort tha

Re: [PATCH] Add native windows on arm64 support

2022-08-31 Thread Tom Lane
Michael Paquier writes: > At the end, I'd like to think that it would be wiser to just remove > entirely the test for node() or reduce the contents of the string to > be able to strictly control the output order (say a simple > 'prepost' would do the trick). I cannot think > about a better idea n

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2022-08-31 Thread Justin Pryzby
On Wed, Aug 31, 2022 at 12:50:19PM -0400, Reid Thompson wrote: > Hi Hackers, > > Add the ability to limit the amount of memory that can be allocated to > backends. > > This builds on the work that adds backend memory allocated to > pg_stat_activity > https://www.postgresql.org/message-id/67bb5c15

Re: PostgreSQL 15 release announcement draft

2022-08-31 Thread Michael Banck
Hi, On Tue, Aug 30, 2022 at 03:58:48PM -0400, Jonathan S. Katz wrote: > ### Other Notable Changes > > PostgreSQL server-level statistics are now collected in shared memory, > eliminating the statistics collector process and writing these stats to disk. > PostgreSQL 15 also revokes the `CREATE` pe

Re: Tracking last scan time

2022-08-31 Thread Matthias van de Meent
On Wed, 31 Aug 2022 at 18:21, Andres Freund wrote: > > Hi, > > On 2022-08-23 10:55:09 +0100, Dave Page wrote: > > Often it is beneficial to review one's schema with a view to removing > > indexes (and sometimes tables) that are no longer required. It's very > > difficult to understand when that is

Re: Tracking last scan time

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 07:52:49PM +0200, Matthias van de Meent wrote: > As for having a lower granularity and preventing the > one-syscall-per-Relation issue, can't we reuse the query_start or > state_change timestamps that appear in pg_stat_activity (potentially Yeah, query start should be fine,

Re: [RFC] building postgres with meson - v12

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 10:28:05 +0200, Peter Eisentraut wrote: > I found that the perl test modules are not installed. See attached patch to > correct this. > > To the patches: > > 4e15ee0e24 Don't hardcode tmp_check/ as test directory for tap tests > 1a3169bc3f Split TESTDIR into TESTLOGDIR and TESTD

Re: SQL/JSON features for v15

2022-08-31 Thread Andrew Dunstan
On 2022-08-31 We 12:48, Jonathan S. Katz wrote: > > > With RMT hat on -- Andrew can you please revert the patchset? :-( Yes, I'll do it, starting with the v15 branch. Might take a day or so. cheers (kinda) andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com

Re: SQL/JSON features for v15

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 12:26:29PM -0400, Robert Haas wrote: > someone else to make the decision. But that's not how it works. The > RMT concept was invented precisely to solve problems like this one, > where the patch authors don't really want to revert it but other > people think it's pretty bust

Re: SQL/JSON features for v15

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 12:04:44PM -0400, Tom Lane wrote: > Andres Freund writes: > > From my POV the only real discussion is whether we'd want to revert this in > > 15 > > and HEAD or just 15. There's imo a decent point to be made to just revert in > > 15 and aggressively press forward with the

Re: effective_multixact_freeze_max_age issue

2022-08-31 Thread Peter Geoghegan
On Tue, Aug 30, 2022 at 8:56 PM Nathan Bossart wrote: > LGTM Pushed, thanks. -- Peter Geoghegan

Re: SQL/JSON features for v15

2022-08-31 Thread Tom Lane
Bruce Momjian writes: > I guess you are saying that setting a cut-off was a bad idea, or that > the cut-off was too close to the final release date. For me, I think > there were three questions: > 1. Were subtransactions acceptable, consensus no > 2. Could trapping errors work for PG 15, conse

Re: Tracking last scan time

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 19:52:49 +0200, Matthias van de Meent wrote: > As for having a lower granularity and preventing the > one-syscall-per-Relation issue, can't we reuse the query_start or > state_change timestamps that appear in pg_stat_activity (potentially > updated immediately before this stat fl

Re: [PATCH] Query Jumbling for CALL and SET utility statements

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 11:00:05 -0700, Jeremy Schneider wrote: > On 8/31/22 9:08 AM, Andres Freund wrote: > > > > I suspect we should carve out things like CALL, PREPARE, EXECUTE from > > track_utility - it's more or less an architectural accident that they're > > utility statements. It's a bit less

Re: SQL/JSON features for v15

2022-08-31 Thread Andrew Dunstan
On 2022-08-31 We 14:45, Tom Lane wrote: > To the extent that there was a management failure here, it was that > we didn't press for a resolution sooner. Given the scale of the > concerns raised in June, I kind of agree with Andres' opinion that > fixing them post-freeze was doomed to failure. I

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Andres Freund
Hi, On 2022-08-27 14:06:32 +0530, Bharath Rupireddy wrote: > @@ -50,7 +51,7 @@ copydir(char *fromdir, char *todir, bool recurse) > > while ((xlde = ReadDir(xldir, fromdir)) != NULL) > { > - struct stat fst; > + PGFileType xlde_type; > > /*

Re: [PATCH] Add sortsupport for range types and btree_gist

2022-08-31 Thread Christoph Heiss
Hi! Sorry for the long delay. This fixes the crashes, now all tests run fine w/ and w/o debug configuration. That shouldn't even have slipped through as such. Notable changes from v1: - gbt_enum_sortsupport() now passes on fcinfo->flinfo enum_cmp_internal() needs a place to cache the typcac

Re: Tracking last scan time

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 11:56:29AM -0700, Andres Freund wrote: > Hi, > > On 2022-08-31 19:52:49 +0200, Matthias van de Meent wrote: > > As for having a lower granularity and preventing the > > one-syscall-per-Relation issue, can't we reuse the query_start or > > state_change timestamps that appear

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Nathan Bossart
On Wed, Aug 31, 2022 at 12:14:33PM -0700, Andres Freund wrote: > On 2022-08-27 14:06:32 +0530, Bharath Rupireddy wrote: >> -if (lstat(fromfile, &fst) < 0) >> -ereport(ERROR, >> -(errcode_for_file_access(), >> -

Re: pg_stat_have_stats() returns true for dropped indexes (or for index creation transaction rolled back)

2022-08-31 Thread Andres Freund
Hiu, On 2022-08-25 11:44:34 +0200, Drouvot, Bertrand wrote: > For REPLSLOT, I agree that we can add one test: I added it in > contrib/test_decoding/sql/stats.sql. It relies on pg_stat_have_stats() (as > relying on pg_stat_replication_slots and/or pg_stat_get_replication_slot() > would not help tha

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 12:24:28 -0700, Nathan Bossart wrote: > On Wed, Aug 31, 2022 at 12:14:33PM -0700, Andres Freund wrote: > > On 2022-08-27 14:06:32 +0530, Bharath Rupireddy wrote: > > It continues to make no sense to me to add behaviour changes around > > error-handling as part of a conversion to

Re: First draft of the PG 15 release notes

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 11:38:33AM +0200, Peter Eisentraut wrote: > On 30.08.22 22:42, Bruce Momjian wrote: > > Good question --- the full text is: > > > > > > > > Record and check the collation of each > linkend="sql-createdatabase">database (Peter Eisentraut) > >

Re: Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Tom Lane
Reid Thompson writes: > This patch ensures get_database_list() switches back to the memory > context in use upon entry rather than returning with TopMemoryContext > as the context. > This will address memory allocations in autovacuum.c being associated > with TopMemoryContext when they should not

Re: Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 16:05:03 -0400, Tom Lane wrote: > Even if there's only a visible leak in v15/HEAD, I'm inclined > to back-patch this all the way, because it's certainly buggy > on its own terms. It's just the merest accident that neither > caller is leaking other stuff into TopMemoryContext, be

Re: SQL/JSON features for v15

2022-08-31 Thread Jonathan S. Katz
On 8/31/22 3:08 PM, Andrew Dunstan wrote: On 2022-08-31 We 14:45, Tom Lane wrote: To the extent that there was a management failure here, it was that we didn't press for a resolution sooner. Given the scale of the concerns raised in June, I kind of agree with Andres' opinion that fixing them p

Re: SQL/JSON features for v15

2022-08-31 Thread Nikita Glukhov
On 31.08.2022 20:14, Tom Lane wrote: Robert Haas writes: On Wed, Aug 31, 2022 at 1:06 PM Tom Lane wrote: The currently proposed patchset hacks up a relatively small number of core datatypes to be able to do that. But it's just a hack and there's no prospect of extension types being able to

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tomas Vondra
On 8/31/22 00:40, David Rowley wrote: > On Wed, 31 Aug 2022 at 02:17, Tom Lane wrote: >> >> I wrote: >>> So maybe we should revisit the question. It'd be worth collecting some >>> stats about how much extra space would be needed if we force there >>> to be room for a sentinel. >> >> Actually, a

Re: Trivial doc patch

2022-08-31 Thread Bruce Momjian
On Fri, Aug 19, 2022 at 10:42:45AM -0400, Bruce Momjian wrote: > > Given that 'optional, optional' has no independent meaning from > > 'optional'; it requires one to scan the entire set looking for > > the non-optional embedded in the option. So no gain. > > I originally had the same reaction Mi

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread David Rowley
On Thu, 1 Sept 2022 at 08:53, Tomas Vondra wrote: > So the raw size (what we asked for) is ~23.5GB, but in practice we > allocate ~28.8GB because of the pow-of-2 logic. And by adding the extra > 1B we end up allocating 31.5GB. That doesn't seem like a huge increase, > and it's far from the +60% yo

Re: New strategies for freezing, advancing relfrozenxid early

2022-08-31 Thread Peter Geoghegan
On Thu, Aug 25, 2022 at 2:21 PM Peter Geoghegan wrote: > Attached patch series is a completely overhauled version of earlier > work on freezing. Related work from the Postgres 15 cycle became > commits 0b018fab, f3c15cbe, and 44fa8488. Attached is v2. This is just to keep CFTester happy, since v

Solaris "sed" versus pre-v13 plpython tests

2022-08-31 Thread Tom Lane
I noticed that buildfarm member margay, which just recently started running tests on REL_12_STABLE, is failing the plpython tests in that branch [1], though it's happy in v13 and later. The failures appear due to syntax errors in Python "except" statements, and it's visible in some of the tests th

Re: cataloguing NOT NULL constraints

2022-08-31 Thread Zhihong Yu
On Wed, Aug 31, 2022 at 3:19 PM Alvaro Herrera wrote: > So I was wrong in thinking that "this case was simple to implement" as I > replied upthread. Doing that actually required me to rewrite large > parts of the patch. I think it ended up being a good thing, because in > hindsight the approach

Re: cataloguing NOT NULL constraints

2022-08-31 Thread Zhihong Yu
On Wed, Aug 31, 2022 at 4:08 PM Zhihong Yu wrote: > > > On Wed, Aug 31, 2022 at 3:19 PM Alvaro Herrera > wrote: > >> So I was wrong in thinking that "this case was simple to implement" as I >> replied upthread. Doing that actually required me to rewrite large >> parts of the patch. I think it

Re: Support tls-exporter as channel binding for TLSv1.3

2022-08-31 Thread Jacob Champion
On Sun, Aug 28, 2022 at 11:02 PM Michael Paquier wrote: > RFC9266, that has been released not so long ago, has added > tls-exporter as a new channel binding type: > https://www.rfc-editor.org/rfc/rfc5929.html Hi Michael, thank you for sending this! > Note also that tls-exporter is aimed for > TL

Re: Small cleanups to tuplesort.c and a bonus small performance improvement

2022-08-31 Thread David Rowley
On Wed, 31 Aug 2022 at 22:39, David Rowley wrote: > My current thoughts are that this is a very trivial patch and unless > there's any objections I plan to push it soon. Pushed. David

Re: Doc patch

2022-08-31 Thread Bruce Momjian
On Fri, Aug 19, 2022 at 12:04:54PM -0400, Bruce Momjian wrote: > You are right about the above to changes. The existing syntax shows > FROM/IN is only possible if a direction is specified, while > src/parser/gram.y says that FROM/IN with no direction is supported. > > I plan to apply this second

Re: Transparent column encryption

2022-08-31 Thread Jacob Champion
On Tue, Aug 30, 2022 at 4:53 AM Peter Eisentraut wrote: > I would be interested in learning more about such padding systems. I > have done a lot of reading for this development project, and I have > never come across a cryptographic approach to hide length differences by > padding. Of course, pa

Re: Inconsistent error message for varchar(n)

2022-08-31 Thread Bruce Momjian
8On Tue, Aug 16, 2022 at 09:56:17PM -0400, Bruce Momjian wrote: > On Sun, Nov 14, 2021 at 10:33:19AM +0800, Japin Li wrote: > > Oh! I didn't consider this situation. Since the max size of varchar cannot > > exceed 10485760, however, I cannot find this in documentation [1]. Is there > > something I

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > immediate wait > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of generated > WAL; request 1201/1CAF84F0, current position 1201/1CADB730 > < 2022-0

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tomas Vondra
On 8/31/22 23:46, David Rowley wrote: > On Thu, 1 Sept 2022 at 08:53, Tomas Vondra > wrote: >> So the raw size (what we asked for) is ~23.5GB, but in practice we >> allocate ~28.8GB because of the pow-of-2 logic. And by adding the extra >> 1B we end up allocating 31.5GB. That doesn't seem like a

Re: PostgreSQL 15 release announcement draft

2022-08-31 Thread Justin Pryzby
On Tue, Aug 30, 2022 at 03:58:48PM -0400, Jonathan S. Katz wrote: > In this latest release, PostgreSQL improves on its in-memory and on-disk > sorting > algorithms, with benchmarks showing speedups of 25% - 400% based on sort > types. rather than "based on": "depending on the data types being s

Re: Handle infinite recursion in logical replication setup

2022-08-31 Thread Peter Smith
On Wed, Aug 31, 2022 at 8:36 PM Amit Kapila wrote: > > On Wed, Aug 31, 2022 at 11:35 AM Peter Smith wrote: > > > > Here are my review comments for v43-0001. > > ... > > == > > > > 2. doc/src/sgml/ref/create_subscription.sgml > > > > + > > + If the subscription is created with origin = non

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tom Lane
Tomas Vondra writes: > Focusing on the aset, vast majority of allocations (60M out of 64M) is > small enough to use power-of-2 logic, and we go from 6.3GB to 8.2GB, so > ~30%. Not great, not terrible. Not sure why this escaped me before, but I remembered another argument for not forcibly adding s

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread David Rowley
On Thu, 1 Sept 2022 at 12:23, Tom Lane wrote: > Is there reason to think we can't validate headers enough to catch > clobbers? For non-sentinel chunks, the next byte after the end of the chunk will be storing the block offset for the following chunk. I think: if (block != MemoryChunkGetBlock(ch

Re: [PATCH] Add native windows on arm64 support

2022-08-31 Thread Michael Paquier
On Wed, Aug 31, 2022 at 01:33:53PM -0400, Tom Lane wrote: > I did not check the other XMLTABLE infrastructure, so what probably > is best to do is keep the two output columns but change 'node()' > to something with a more stable result; any preferences? The input object could also be reworked so a

Re: Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Michael Paquier
On Wed, Aug 31, 2022 at 01:09:22PM -0700, Andres Freund wrote: > On 2022-08-31 16:05:03 -0400, Tom Lane wrote: >> Even if there's only a visible leak in v15/HEAD, I'm inclined >> to back-patch this all the way, because it's certainly buggy >> on its own terms. It's just the merest accident that ne

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tom Lane
David Rowley writes: > Maybe we should just consider always making room for a sentinel for > chunks that are on dedicated blocks. At most that's an extra 8 bytes > in some allocation that's either over 1024 or 8192 (depending on > maxBlockSize). Agreed, if we're not doing that already then we sho

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tomas Vondra
On 9/1/22 02:23, Tom Lane wrote: > Tomas Vondra writes: >> Focusing on the aset, vast majority of allocations (60M out of 64M) is >> small enough to use power-of-2 logic, and we go from 6.3GB to 8.2GB, so >> ~30%. Not great, not terrible. > > Not sure why this escaped me before, but I remember

Re: POC: Better infrastructure for automated testing of concurrency issues

2022-08-31 Thread Alexander Korotkov
Hi! On Tue, Feb 23, 2021 at 3:09 AM Peter Geoghegan wrote: > On Tue, Dec 8, 2020 at 2:42 AM Alexander Korotkov > wrote: > > Thank you for your feedback! > > It would be nice to use this patch to test things that are important > but untested inside vacuumlazy.c, such as the rare > HEAPTUPLE_DEAD

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 12:05:36PM +1200, Thomas Munro wrote: > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > > immediate wait > > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of > > generat

  1   2   >