Re: [PATCH v1] [doc] polish the comments of reloptions

2022-08-31 Thread Junwang Zhao
thoughts? On Tue, Aug 30, 2022 at 11:56 AM Junwang Zhao wrote: > > When adding an option, we have 5 choices (bool, integer, real, enum, string), > so the comments seem stale. > > There are some sentences missing *at ShareUpdateExclusiveLock*, this > patch adds them to make the sentences complete.

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-08-31 Thread Önder Kalacı
Hi again, > == > > 1. Commit message > > 1a. > With this patch, I'm proposing the following change: If there is an > index on the subscriber, use the index as long as the planner > sub-modules picks any index over sequential scan. The index should be > a btree index, not a partital index. Fin

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-08-31 Thread Önder Kalacı
Hi Peter, Thanks for the reviews! I'll reply to both of your reviews separately. > >> 10. > >> > >> + /* we only need to allocate once */ > >> + if (eq == NULL) > >> + eq = palloc0(sizeof(*eq) * outslot->tts_tupleDescriptor->natts); > >> > >> But shouldn't you also free this 'eq' before the func

struct Trigger definition in trigger.sgml

2022-08-31 Thread Etsuro Fujita
Hi, While working on something else, I noticed that commit 487e9861d added a new field to struct Trigger, but failed to update $SUBJECT to match. Attached is a small patch for that. Best regards, Etsuro Fujita doc-trigger-def.patch Description: Binary data

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 05:35:23PM +1200, Thomas Munro wrote: > So it *looks* like it finished early (and without the expected > error?). But it also looks like it replayed that record, according to > the page LSN. So which is it? Could you recompile with WAL_DEBUG > defined in pg_config_manual.

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 5:18 PM Kyotaro Horiguchi wrote: > At Wed, 31 Aug 2022 23:47:53 -0500, Justin Pryzby > wrote in > > On Thu, Sep 01, 2022 at 04:22:20PM +1200, Thomas Munro wrote: > > > Hmm. Justin, when you built from source, which commit were you at? > > > If it's REL_15_BETA3, > > > > N

Re: FOR EACH ROW triggers, on partitioend tables, with indexes?

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 04:19:37PM +1200, David Rowley wrote: > On Sat, 20 Aug 2022 at 09:18, Justin Pryzby wrote: > > Is it somwhow possible to call CreateTrigger() to create a FOR EACH ROW > > trigger, with an index, and not internally ? > > I've been looking over this and I very much agree tha

Re: question about access custom enum type from C

2022-08-31 Thread Dmitry Markman
Hi David as you suggested create type first_type as enum ('red', 'green', 'brown', 'yellow', 'blue'); SELECT oid,typname,typlen,typtype from pg_type where typname='first_type' returns everything I was looking for thanks again, I think I’m all set dm > On Sep 1, 2022, at 12:49 AM, David Rowle

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Kyotaro Horiguchi
At Wed, 31 Aug 2022 23:47:53 -0500, Justin Pryzby wrote in > On Thu, Sep 01, 2022 at 04:22:20PM +1200, Thomas Munro wrote: > > On Thu, Sep 1, 2022 at 3:08 PM Kyotaro Horiguchi > > wrote: > > > Just for information, there was a fixed bug about > > > overwrite-aborted-contrecord feature, which ca

Re: question about access custom enum type from C

2022-08-31 Thread Dmitry Markman
Hi David, thanks a lot for your answer I got that number from PQparamtype I already see that 16387 is not a ‘constant’, if I have few custom types I got different numbers for them thanks dm > On Sep 1, 2022, at 12:49 AM, David Rowley wrote: > > (I think this is a better question for the

Re: question about access custom enum type from C

2022-08-31 Thread David Rowley
(I think this is a better question for the general mailing list) On Thu, 1 Sept 2022 at 16:28, Dmitry Markman wrote: > > Hi, when I’m trying to access values of my custom enum type I created with > > create type colors as enum ('red', 'green', 'brown', 'yellow', 'blue'); > > I’m getting oid as 16

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 04:22:20PM +1200, Thomas Munro wrote: > On Thu, Sep 1, 2022 at 3:08 PM Kyotaro Horiguchi > wrote: > > At Thu, 1 Sep 2022 12:05:36 +1200, Thomas Munro > > wrote in > > > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > > > < 2022-08-31 08:44:10.495 CDT >LOG: chec

Re: Add tracking of backend memory allocated to pg_stat_activity

2022-08-31 Thread Kyotaro Horiguchi
At Wed, 31 Aug 2022 12:03:06 -0400, Reid Thompson wrote in > Attached is a patch to > Add tracking of backend memory allocated to pg_stat_activity > @@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size) > return NULL; > > context->mem_allocat

Re: [PATCH] Add native windows on arm64 support

2022-08-31 Thread Michael Paquier
On Wed, Aug 31, 2022 at 10:22:06PM -0400, Tom Lane wrote: > Michael Paquier writes: > > The input object could also be reworked so as we would not have any > > ordering issues, say "pre&deeppost". > > Changing only the path, you could use "/e/n2" instead of "node()", so > > as we'd always get the

question about access custom enum type from C

2022-08-31 Thread Dmitry Markman
Hi, when I’m trying to access values of my custom enum type I created with create type colors as enum ('red', 'green', 'brown', 'yellow', 'blue'); I’m getting oid as 16387 and I can see it stored as a chars is number 16387 is always OID for enum type? if not how I can get information about type

Re: Solaris "sed" versus pre-v13 plpython tests

2022-08-31 Thread Noah Misch
On Wed, Aug 31, 2022 at 06:25:01PM -0400, Tom Lane wrote: > I am not completely sure why buildfarm member wrasse isn't failing > similarly wrasse disabled plpython in v12-, from day one, due to this and a crash bug that I shelved. I will be interested to see how margay reacts to your fix.

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 3:08 PM Kyotaro Horiguchi wrote: > At Thu, 1 Sep 2022 12:05:36 +1200, Thomas Munro > wrote in > > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: > > > end-of-recovery immediate wait > > > < 2022-08-31

Re: FOR EACH ROW triggers, on partitioend tables, with indexes?

2022-08-31 Thread David Rowley
On Sat, 20 Aug 2022 at 09:18, Justin Pryzby wrote: > Is it somwhow possible to call CreateTrigger() to create a FOR EACH ROW > trigger, with an index, and not internally ? I've been looking over this and I very much agree that the code looks very broken. As for whether this is dead code or not, I

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tom Lane
David Rowley writes: > On Thu, 1 Sept 2022 at 16:06, Tom Lane wrote: >> It doesn't seem to be fixing any live bug in the back branches, >> but by the same token it's harmless. > I considered that an extension might use the Slab allocator with a > non-MAXALIGNED chunksize and might run into some

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread David Rowley
On Thu, 1 Sept 2022 at 16:06, Tom Lane wrote: > > David Rowley writes: > > Does anyone have any objections to d5ee4db0e in its entirety being > > backpatched? > > It doesn't seem to be fixing any live bug in the back branches, > but by the same token it's harmless. I considered that an extensio

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tom Lane
David Rowley writes: > Does anyone have any objections to d5ee4db0e in its entirety being > backpatched? It doesn't seem to be fixing any live bug in the back branches, but by the same token it's harmless. regards, tom lane

Re: Handle infinite recursion in logical replication setup

2022-08-31 Thread Amit Kapila
On Wed, Aug 31, 2022 at 11:35 AM Peter Smith wrote: > > Here are my review comments for v43-0001. > > == > > 1. Commit message > > The initial copy phase has no way to know the origin of the row data, > so if 'copy_data = true' in the step 4 below, log a warning to notify user > that potential

Re: Instrumented pages/tuples frozen in autovacuum's server log out (and VACUUM VERBOSE)

2022-08-31 Thread Peter Geoghegan
On Wed, Aug 31, 2022 at 7:49 PM Jeff Janes wrote: > I like this addition, but I would also like to see how many pages got newly > set to all frozen by the vacuum. I'd say that that's independent work. Though I'm happy to discuss it now. It would be fairly straightforward to show something about

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread David Rowley
On Tue, 30 Aug 2022 at 13:16, David Rowley wrote: > I'm also wondering if this should also be backpatched back to v10, > providing the build farm likes it well enough on master. Does anyone have any objections to d5ee4db0e in its entirety being backpatched? David

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Kyotaro Horiguchi
At Thu, 1 Sep 2022 12:05:36 +1200, Thomas Munro wrote in > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > > immediate wait > > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of > > generated

Re: Instrumented pages/tuples frozen in autovacuum's server log out (and VACUUM VERBOSE)

2022-08-31 Thread Jeff Janes
On Sat, Aug 20, 2022 at 7:29 PM Peter Geoghegan wrote: > XIDs processed: 45 pages from table (100.00% of total) had 1 tuples > frozen > I like this addition, but I would also like to see how many pages got newly set to all frozen by the vacuum. Would that be a convenient thing to also repo

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2022-08-31 Thread Kyotaro Horiguchi
At Wed, 31 Aug 2022 12:50:19 -0400, Reid Thompson wrote in > Hi Hackers, > > Add the ability to limit the amount of memory that can be allocated to > backends. The patch seems to limit both of memory-context allocations and DSM allocations happen on a specific process by the same budget. In th

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
Some more details, in case they're important: First: the server has wal_compression=zstd (I wonder if something doesn't allow/accomodate compressed FPI?) I thought to mention that after compiling pg15 locally and forgetting to use --with-zstd. I compiled it to enable your debug logging, which wr

Re: num_sa_scans in genericcostestimate

2022-08-31 Thread Jeff Janes
On Sun, Aug 21, 2022 at 2:45 PM Jeff Janes wrote: > ... > > The context for this is that I was looking at cases where btree indexes > were not using all the columns they could, but rather shoving some of the > conditions down into a Filter unnecessarily/unhelpfully. This change > doesn't fix

RE: Handle infinite recursion in logical replication setup

2022-08-31 Thread shiy.f...@fujitsu.com
On Wed, Aug 31, 2022 1:06 AM vignesh C wrote: > > The attached v43 patch has the changes for the same. > Thanks for updating the patch. Here is a comment on the 0001 patch. + if (!isnewtable) + { + pfree(nspname); + pfree

Re: [PATCH] Add native windows on arm64 support

2022-08-31 Thread Tom Lane
Michael Paquier writes: > The input object could also be reworked so as we would not have any > ordering issues, say "pre&deeppost". > Changing only the path, you could use "/e/n2" instead of "node()", so > as we'd always get the most internal member. Done that way. regar

Re: SELECT documentation

2022-08-31 Thread Bruce Momjian
On Mon, Aug 15, 2022 at 10:53:18PM -0400, Bruce Momjian wrote: > On Sat, Aug 13, 2022 at 10:21:26PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > Hi, I agree we should show the more modern JOIN sytax. However, this is > > > just an example, so one example should be sufficient. I went w

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 12:53 PM Justin Pryzby wrote: > Yes, I have a copy that reproduces the issue: That's good news. So the last record touching that page was: > rmgr: Heap2 len (rec/tot): 59/59, tx: 0, lsn: > 1201/1CAF84B0, prev 1201/1CAF8478, desc: VISIBLE cutoff xid

Re: Add tracking of backend memory allocated to pg_stat_activity

2022-08-31 Thread Kyotaro Horiguchi
At Wed, 31 Aug 2022 12:05:55 -0500, Justin Pryzby wrote in > On Wed, Aug 31, 2022 at 12:03:06PM -0400, Reid Thompson wrote: > > Hi Hackers, > > > > Attached is a patch to > > Add tracking of backend memory allocated to pg_stat_activity > > > + proargmodes => > > '{i,o,o,o,o,o,o,o,o,o,o,o,o,

Re: Avoid overhead open-close indexes (catalog updates)

2022-08-31 Thread Kyotaro Horiguchi
At Wed, 31 Aug 2022 08:16:55 -0300, Ranier Vilela wrote in > Hi, > > The commit > https://github.com/postgres/postgres/commit/b17ff07aa3eb142d2cde2ea00e4a4e8f63686f96 > Introduced the CopyStatistics function. > > To do the work, CopyStatistics uses a less efficient function > to update/insert

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tom Lane
Tomas Vondra writes: > You're probably right we'll notice the clobber cases due to corruption > of the next chunk header. The annoying thing is having a corrupted > header only tells you there's a corruption somewhere, but it may be hard > to know which part of the code caused it. Same's true of

Re: Support tls-exporter as channel binding for TLSv1.3

2022-08-31 Thread Michael Paquier
On Wed, Aug 31, 2022 at 04:16:29PM -0700, Jacob Champion wrote: > For protocols less than 1.3 we'll need to ensure that the extended > master secret is in use: > >This channel binding mechanism is defined only when the TLS handshake >results in unique master secrets. This is true of TLS v

Re: Solaris "sed" versus pre-v13 plpython tests

2022-08-31 Thread Tom Lane
I wrote: > I confirmed on the gcc farm's Solaris 11 box that the pattern > doesn't work as expected with /usr/bin/sed: > tgl@gcc-solaris11:~$ echo except foo, bar: | sed -e 's/except > \([[:alpha:]][[:alpha:].]*\), *\([[:alpha:]][[:alpha:]]*\):/except \1 as > \2:/g' > except foo, bar: > We coul

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 12:05:36PM +1200, Thomas Munro wrote: > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > > immediate wait > > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of > > generat

Re: POC: Better infrastructure for automated testing of concurrency issues

2022-08-31 Thread Alexander Korotkov
Hi! On Tue, Feb 23, 2021 at 3:09 AM Peter Geoghegan wrote: > On Tue, Dec 8, 2020 at 2:42 AM Alexander Korotkov > wrote: > > Thank you for your feedback! > > It would be nice to use this patch to test things that are important > but untested inside vacuumlazy.c, such as the rare > HEAPTUPLE_DEAD

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tomas Vondra
On 9/1/22 02:23, Tom Lane wrote: > Tomas Vondra writes: >> Focusing on the aset, vast majority of allocations (60M out of 64M) is >> small enough to use power-of-2 logic, and we go from 6.3GB to 8.2GB, so >> ~30%. Not great, not terrible. > > Not sure why this escaped me before, but I remember

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tom Lane
David Rowley writes: > Maybe we should just consider always making room for a sentinel for > chunks that are on dedicated blocks. At most that's an extra 8 bytes > in some allocation that's either over 1024 or 8192 (depending on > maxBlockSize). Agreed, if we're not doing that already then we sho

Re: Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Michael Paquier
On Wed, Aug 31, 2022 at 01:09:22PM -0700, Andres Freund wrote: > On 2022-08-31 16:05:03 -0400, Tom Lane wrote: >> Even if there's only a visible leak in v15/HEAD, I'm inclined >> to back-patch this all the way, because it's certainly buggy >> on its own terms. It's just the merest accident that ne

Re: [PATCH] Add native windows on arm64 support

2022-08-31 Thread Michael Paquier
On Wed, Aug 31, 2022 at 01:33:53PM -0400, Tom Lane wrote: > I did not check the other XMLTABLE infrastructure, so what probably > is best to do is keep the two output columns but change 'node()' > to something with a more stable result; any preferences? The input object could also be reworked so a

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread David Rowley
On Thu, 1 Sept 2022 at 12:23, Tom Lane wrote: > Is there reason to think we can't validate headers enough to catch > clobbers? For non-sentinel chunks, the next byte after the end of the chunk will be storing the block offset for the following chunk. I think: if (block != MemoryChunkGetBlock(ch

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tom Lane
Tomas Vondra writes: > Focusing on the aset, vast majority of allocations (60M out of 64M) is > small enough to use power-of-2 logic, and we go from 6.3GB to 8.2GB, so > ~30%. Not great, not terrible. Not sure why this escaped me before, but I remembered another argument for not forcibly adding s

Re: Handle infinite recursion in logical replication setup

2022-08-31 Thread Peter Smith
On Wed, Aug 31, 2022 at 8:36 PM Amit Kapila wrote: > > On Wed, Aug 31, 2022 at 11:35 AM Peter Smith wrote: > > > > Here are my review comments for v43-0001. > > ... > > == > > > > 2. doc/src/sgml/ref/create_subscription.sgml > > > > + > > + If the subscription is created with origin = non

Re: PostgreSQL 15 release announcement draft

2022-08-31 Thread Justin Pryzby
On Tue, Aug 30, 2022 at 03:58:48PM -0400, Jonathan S. Katz wrote: > In this latest release, PostgreSQL improves on its in-memory and on-disk > sorting > algorithms, with benchmarks showing speedups of 25% - 400% based on sort > types. rather than "based on": "depending on the data types being s

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tomas Vondra
On 8/31/22 23:46, David Rowley wrote: > On Thu, 1 Sept 2022 at 08:53, Tomas Vondra > wrote: >> So the raw size (what we asked for) is ~23.5GB, but in practice we >> allocate ~28.8GB because of the pow-of-2 logic. And by adding the extra >> 1B we end up allocating 31.5GB. That doesn't seem like a

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > immediate wait > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of generated > WAL; request 1201/1CAF84F0, current position 1201/1CADB730 > < 2022-0

Re: Inconsistent error message for varchar(n)

2022-08-31 Thread Bruce Momjian
8On Tue, Aug 16, 2022 at 09:56:17PM -0400, Bruce Momjian wrote: > On Sun, Nov 14, 2021 at 10:33:19AM +0800, Japin Li wrote: > > Oh! I didn't consider this situation. Since the max size of varchar cannot > > exceed 10485760, however, I cannot find this in documentation [1]. Is there > > something I

Re: Transparent column encryption

2022-08-31 Thread Jacob Champion
On Tue, Aug 30, 2022 at 4:53 AM Peter Eisentraut wrote: > I would be interested in learning more about such padding systems. I > have done a lot of reading for this development project, and I have > never come across a cryptographic approach to hide length differences by > padding. Of course, pa

Re: Doc patch

2022-08-31 Thread Bruce Momjian
On Fri, Aug 19, 2022 at 12:04:54PM -0400, Bruce Momjian wrote: > You are right about the above to changes. The existing syntax shows > FROM/IN is only possible if a direction is specified, while > src/parser/gram.y says that FROM/IN with no direction is supported. > > I plan to apply this second

Re: Small cleanups to tuplesort.c and a bonus small performance improvement

2022-08-31 Thread David Rowley
On Wed, 31 Aug 2022 at 22:39, David Rowley wrote: > My current thoughts are that this is a very trivial patch and unless > there's any objections I plan to push it soon. Pushed. David

Re: Support tls-exporter as channel binding for TLSv1.3

2022-08-31 Thread Jacob Champion
On Sun, Aug 28, 2022 at 11:02 PM Michael Paquier wrote: > RFC9266, that has been released not so long ago, has added > tls-exporter as a new channel binding type: > https://www.rfc-editor.org/rfc/rfc5929.html Hi Michael, thank you for sending this! > Note also that tls-exporter is aimed for > TL

Re: cataloguing NOT NULL constraints

2022-08-31 Thread Zhihong Yu
On Wed, Aug 31, 2022 at 4:08 PM Zhihong Yu wrote: > > > On Wed, Aug 31, 2022 at 3:19 PM Alvaro Herrera > wrote: > >> So I was wrong in thinking that "this case was simple to implement" as I >> replied upthread. Doing that actually required me to rewrite large >> parts of the patch. I think it

Re: cataloguing NOT NULL constraints

2022-08-31 Thread Zhihong Yu
On Wed, Aug 31, 2022 at 3:19 PM Alvaro Herrera wrote: > So I was wrong in thinking that "this case was simple to implement" as I > replied upthread. Doing that actually required me to rewrite large > parts of the patch. I think it ended up being a good thing, because in > hindsight the approach

Solaris "sed" versus pre-v13 plpython tests

2022-08-31 Thread Tom Lane
I noticed that buildfarm member margay, which just recently started running tests on REL_12_STABLE, is failing the plpython tests in that branch [1], though it's happy in v13 and later. The failures appear due to syntax errors in Python "except" statements, and it's visible in some of the tests th

Re: New strategies for freezing, advancing relfrozenxid early

2022-08-31 Thread Peter Geoghegan
On Thu, Aug 25, 2022 at 2:21 PM Peter Geoghegan wrote: > Attached patch series is a completely overhauled version of earlier > work on freezing. Related work from the Postgres 15 cycle became > commits 0b018fab, f3c15cbe, and 44fa8488. Attached is v2. This is just to keep CFTester happy, since v

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread David Rowley
On Thu, 1 Sept 2022 at 08:53, Tomas Vondra wrote: > So the raw size (what we asked for) is ~23.5GB, but in practice we > allocate ~28.8GB because of the pow-of-2 logic. And by adding the extra > 1B we end up allocating 31.5GB. That doesn't seem like a huge increase, > and it's far from the +60% yo

Re: Trivial doc patch

2022-08-31 Thread Bruce Momjian
On Fri, Aug 19, 2022 at 10:42:45AM -0400, Bruce Momjian wrote: > > Given that 'optional, optional' has no independent meaning from > > 'optional'; it requires one to scan the entire set looking for > > the non-optional embedded in the option. So no gain. > > I originally had the same reaction Mi

Re: Reducing the chunk header sizes on all memory context types

2022-08-31 Thread Tomas Vondra
On 8/31/22 00:40, David Rowley wrote: > On Wed, 31 Aug 2022 at 02:17, Tom Lane wrote: >> >> I wrote: >>> So maybe we should revisit the question. It'd be worth collecting some >>> stats about how much extra space would be needed if we force there >>> to be room for a sentinel. >> >> Actually, a

Re: SQL/JSON features for v15

2022-08-31 Thread Nikita Glukhov
On 31.08.2022 20:14, Tom Lane wrote: Robert Haas writes: On Wed, Aug 31, 2022 at 1:06 PM Tom Lane wrote: The currently proposed patchset hacks up a relatively small number of core datatypes to be able to do that. But it's just a hack and there's no prospect of extension types being able to

Re: SQL/JSON features for v15

2022-08-31 Thread Jonathan S. Katz
On 8/31/22 3:08 PM, Andrew Dunstan wrote: On 2022-08-31 We 14:45, Tom Lane wrote: To the extent that there was a management failure here, it was that we didn't press for a resolution sooner. Given the scale of the concerns raised in June, I kind of agree with Andres' opinion that fixing them p

Re: Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 16:05:03 -0400, Tom Lane wrote: > Even if there's only a visible leak in v15/HEAD, I'm inclined > to back-patch this all the way, because it's certainly buggy > on its own terms. It's just the merest accident that neither > caller is leaking other stuff into TopMemoryContext, be

Re: Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Tom Lane
Reid Thompson writes: > This patch ensures get_database_list() switches back to the memory > context in use upon entry rather than returning with TopMemoryContext > as the context. > This will address memory allocations in autovacuum.c being associated > with TopMemoryContext when they should not

Re: First draft of the PG 15 release notes

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 11:38:33AM +0200, Peter Eisentraut wrote: > On 30.08.22 22:42, Bruce Momjian wrote: > > Good question --- the full text is: > > > > > > > > Record and check the collation of each > linkend="sql-createdatabase">database (Peter Eisentraut) > >

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 12:24:28 -0700, Nathan Bossart wrote: > On Wed, Aug 31, 2022 at 12:14:33PM -0700, Andres Freund wrote: > > On 2022-08-27 14:06:32 +0530, Bharath Rupireddy wrote: > > It continues to make no sense to me to add behaviour changes around > > error-handling as part of a conversion to

Re: pg_stat_have_stats() returns true for dropped indexes (or for index creation transaction rolled back)

2022-08-31 Thread Andres Freund
Hiu, On 2022-08-25 11:44:34 +0200, Drouvot, Bertrand wrote: > For REPLSLOT, I agree that we can add one test: I added it in > contrib/test_decoding/sql/stats.sql. It relies on pg_stat_have_stats() (as > relying on pg_stat_replication_slots and/or pg_stat_get_replication_slot() > would not help tha

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Nathan Bossart
On Wed, Aug 31, 2022 at 12:14:33PM -0700, Andres Freund wrote: > On 2022-08-27 14:06:32 +0530, Bharath Rupireddy wrote: >> -if (lstat(fromfile, &fst) < 0) >> -ereport(ERROR, >> -(errcode_for_file_access(), >> -

Re: Tracking last scan time

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 11:56:29AM -0700, Andres Freund wrote: > Hi, > > On 2022-08-31 19:52:49 +0200, Matthias van de Meent wrote: > > As for having a lower granularity and preventing the > > one-syscall-per-Relation issue, can't we reuse the query_start or > > state_change timestamps that appear

Re: [PATCH] Add sortsupport for range types and btree_gist

2022-08-31 Thread Christoph Heiss
Hi! Sorry for the long delay. This fixes the crashes, now all tests run fine w/ and w/o debug configuration. That shouldn't even have slipped through as such. Notable changes from v1: - gbt_enum_sortsupport() now passes on fcinfo->flinfo enum_cmp_internal() needs a place to cache the typcac

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Andres Freund
Hi, On 2022-08-27 14:06:32 +0530, Bharath Rupireddy wrote: > @@ -50,7 +51,7 @@ copydir(char *fromdir, char *todir, bool recurse) > > while ((xlde = ReadDir(xldir, fromdir)) != NULL) > { > - struct stat fst; > + PGFileType xlde_type; > > /*

Re: SQL/JSON features for v15

2022-08-31 Thread Andrew Dunstan
On 2022-08-31 We 14:45, Tom Lane wrote: > To the extent that there was a management failure here, it was that > we didn't press for a resolution sooner. Given the scale of the > concerns raised in June, I kind of agree with Andres' opinion that > fixing them post-freeze was doomed to failure. I

Re: [PATCH] Query Jumbling for CALL and SET utility statements

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 11:00:05 -0700, Jeremy Schneider wrote: > On 8/31/22 9:08 AM, Andres Freund wrote: > > > > I suspect we should carve out things like CALL, PREPARE, EXECUTE from > > track_utility - it's more or less an architectural accident that they're > > utility statements. It's a bit less

Re: Tracking last scan time

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 19:52:49 +0200, Matthias van de Meent wrote: > As for having a lower granularity and preventing the > one-syscall-per-Relation issue, can't we reuse the query_start or > state_change timestamps that appear in pg_stat_activity (potentially > updated immediately before this stat fl

Re: SQL/JSON features for v15

2022-08-31 Thread Tom Lane
Bruce Momjian writes: > I guess you are saying that setting a cut-off was a bad idea, or that > the cut-off was too close to the final release date. For me, I think > there were three questions: > 1. Were subtransactions acceptable, consensus no > 2. Could trapping errors work for PG 15, conse

Re: effective_multixact_freeze_max_age issue

2022-08-31 Thread Peter Geoghegan
On Tue, Aug 30, 2022 at 8:56 PM Nathan Bossart wrote: > LGTM Pushed, thanks. -- Peter Geoghegan

Re: SQL/JSON features for v15

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 12:04:44PM -0400, Tom Lane wrote: > Andres Freund writes: > > From my POV the only real discussion is whether we'd want to revert this in > > 15 > > and HEAD or just 15. There's imo a decent point to be made to just revert in > > 15 and aggressively press forward with the

Re: SQL/JSON features for v15

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 12:26:29PM -0400, Robert Haas wrote: > someone else to make the decision. But that's not how it works. The > RMT concept was invented precisely to solve problems like this one, > where the patch authors don't really want to revert it but other > people think it's pretty bust

Re: SQL/JSON features for v15

2022-08-31 Thread Andrew Dunstan
On 2022-08-31 We 12:48, Jonathan S. Katz wrote: > > > With RMT hat on -- Andrew can you please revert the patchset? :-( Yes, I'll do it, starting with the v15 branch. Might take a day or so. cheers (kinda) andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com

Re: [RFC] building postgres with meson - v12

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 10:28:05 +0200, Peter Eisentraut wrote: > I found that the perl test modules are not installed. See attached patch to > correct this. > > To the patches: > > 4e15ee0e24 Don't hardcode tmp_check/ as test directory for tap tests > 1a3169bc3f Split TESTDIR into TESTLOGDIR and TESTD

Re: Tracking last scan time

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 07:52:49PM +0200, Matthias van de Meent wrote: > As for having a lower granularity and preventing the > one-syscall-per-Relation issue, can't we reuse the query_start or > state_change timestamps that appear in pg_stat_activity (potentially Yeah, query start should be fine,

Re: Tracking last scan time

2022-08-31 Thread Matthias van de Meent
On Wed, 31 Aug 2022 at 18:21, Andres Freund wrote: > > Hi, > > On 2022-08-23 10:55:09 +0100, Dave Page wrote: > > Often it is beneficial to review one's schema with a view to removing > > indexes (and sometimes tables) that are no longer required. It's very > > difficult to understand when that is

Re: PostgreSQL 15 release announcement draft

2022-08-31 Thread Michael Banck
Hi, On Tue, Aug 30, 2022 at 03:58:48PM -0400, Jonathan S. Katz wrote: > ### Other Notable Changes > > PostgreSQL server-level statistics are now collected in shared memory, > eliminating the statistics collector process and writing these stats to disk. > PostgreSQL 15 also revokes the `CREATE` pe

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2022-08-31 Thread Justin Pryzby
On Wed, Aug 31, 2022 at 12:50:19PM -0400, Reid Thompson wrote: > Hi Hackers, > > Add the ability to limit the amount of memory that can be allocated to > backends. > > This builds on the work that adds backend memory allocated to > pg_stat_activity > https://www.postgresql.org/message-id/67bb5c15

Re: [PATCH] Add native windows on arm64 support

2022-08-31 Thread Tom Lane
Michael Paquier writes: > At the end, I'd like to think that it would be wiser to just remove > entirely the test for node() or reduce the contents of the string to > be able to strictly control the output order (say a simple > 'prepost' would do the trick). I cannot think > about a better idea n

Re: SQL/JSON features for v15

2022-08-31 Thread Tom Lane
Robert Haas writes: > On Wed, Aug 31, 2022 at 1:06 PM Tom Lane wrote: >> (I don't say that we'd convert every datatype to do so right away; >> in the long run we should, but I'm content to start with just the >> same core types touched here.) > I would be in favor of making more of an effort tha

Re: SQL/JSON features for v15

2022-08-31 Thread Robert Haas
On Wed, Aug 31, 2022 at 1:06 PM Tom Lane wrote: > The currently proposed patchset hacks up a relatively small number > of core datatypes to be able to do that. But it's just a hack > and there's no prospect of extension types being able to join > in the fun. I think where we need to start, for v

Re: SQL/JSON features for v15

2022-08-31 Thread Tom Lane
I wrote: > Andres Freund writes: >> From my POV the only real discussion is whether we'd want to revert this in >> 15 >> and HEAD or just 15. There's imo a decent point to be made to just revert in >> 15 and aggressively press forward with the changes posted in this thread. > I'm not for that.

Re: Add tracking of backend memory allocated to pg_stat_activity

2022-08-31 Thread Justin Pryzby
On Wed, Aug 31, 2022 at 12:03:06PM -0400, Reid Thompson wrote: > Hi Hackers, > > Attached is a patch to > Add tracking of backend memory allocated to pg_stat_activity > + proargmodes => > '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', In the past, there was concern about

Re: [PATCH] Query Jumbling for CALL and SET utility statements

2022-08-31 Thread Pavel Stehule
st 31. 8. 2022 v 17:50 odesílatel Pavel Stehule napsal: > Hi > > > st 31. 8. 2022 v 17:34 odesílatel Drouvot, Bertrand > napsal: > >> Hi hackers, >> >> While query jumbling is provided for function calls that’s currently not >> the case for procedures calls. >> The reason behind this is that all

Add the ability to limit the amount of memory that can be allocated to backends.

2022-08-31 Thread Reid Thompson
Hi Hackers, Add the ability to limit the amount of memory that can be allocated to backends. This builds on the work that adds backend memory allocated to pg_stat_activity https://www.postgresql.org/message-id/67bb5c15c0489cb499723b0340f16e10c22485ec.camel%40crunchydata.com Both patches are attac

Re: SQL/JSON features for v15

2022-08-31 Thread Jonathan S. Katz
On 8/31/22 12:26 PM, Robert Haas wrote: On Wed, Aug 31, 2022 at 10:20 AM Jonathan S. Katz wrote: Andres, Robert, Tom: With this recent work, have any of your opinions changed on including SQL/JSON in v15? No. Nothing's been committed, and there's no time to review anything in detail, and ther

Re: SQL/JSON features for v15

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 12:26:29 -0400, Robert Haas wrote: > On Wed, Aug 31, 2022 at 10:20 AM Jonathan S. Katz > wrote: > > Andres, Robert, Tom: With this recent work, have any of your opinions > > changed on including SQL/JSON in v15? > > No. Nothing's been committed, and there's no time to review a

Re: SQL/JSON features for v15

2022-08-31 Thread Robert Haas
On Wed, Aug 31, 2022 at 10:20 AM Jonathan S. Katz wrote: > Andres, Robert, Tom: With this recent work, have any of your opinions > changed on including SQL/JSON in v15? No. Nothing's been committed, and there's no time to review anything in detail, and there was never going to be. Nikita said he

Re: Tracking last scan time

2022-08-31 Thread Andres Freund
Hi, On 2022-08-23 10:55:09 +0100, Dave Page wrote: > Often it is beneficial to review one's schema with a view to removing > indexes (and sometimes tables) that are no longer required. It's very > difficult to understand when that is the case by looking at the number of > scans of a relation as, f

Bug fix. autovacuum.c do_worker_start() associates memory allocations with TopMemoryContext rather than 'Autovacuum start worker (tmp)'

2022-08-31 Thread Reid Thompson
Hi Hackers, This patch ensures get_database_list() switches back to the memory context in use upon entry rather than returning with TopMemoryContext as the context. This will address memory allocations in autovacuum.c being associated with TopMemoryContext when they should not be. autovacuum.c d

Re: Tracking last scan time

2022-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2022 at 05:02:33PM +0100, Dave Page wrote: > > > On Tue, 30 Aug 2022 at 19:46, Bruce Momjian wrote: > > On Fri, Aug 26, 2022 at 02:05:36PM +0100, Dave Page wrote: > > On Thu, 25 Aug 2022 at 01:44, David Rowley wrote: > >     I don't have a particular opinion about t

Re: [PATCH] Query Jumbling for CALL and SET utility statements

2022-08-31 Thread Andres Freund
Hi, On 2022-08-31 17:33:44 +0200, Drouvot, Bertrand wrote: > While query jumbling is provided for function calls that’s currently not the > case for procedures calls. > The reason behind this is that all utility statements are currently > discarded for jumbling. > [...] > That’s why we think it wo

  1   2   >