Re: Allow an extention to be updated without a script

2023-02-01 Thread Yugo NAGATA
On Wed, 1 Feb 2023 14:37:27 +0800 Julien Rouhaud wrote: > Hi, > > On Tue, Jan 31, 2023 at 11:17:22AM +0900, Yugo NAGATA wrote: > > > > On Mon, 30 Jan 2023 16:05:52 -0500 > > Tom Lane wrote: > > > > > If you have no update script, why call it a new version? The point > > > of extension versions

Re: Exit walsender before confirming remote flush in logical replication

2023-02-01 Thread Masahiko Sawada
On Fri, Jan 20, 2023 at 7:45 PM Amit Kapila wrote: > > On Tue, Jan 17, 2023 at 2:41 PM Amit Kapila wrote: > > > > Let me try to summarize the discussion till now. The problem we are > > trying to solve here is to allow a shutdown to complete when walsender > > is not able to send the entire WAL.

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-01 Thread Kyotaro Horiguchi
At Wed, 1 Feb 2023 08:38:11 +0530, Amit Kapila wrote in > On Wed, Feb 1, 2023 at 8:13 AM Kyotaro Horiguchi > wrote: > > > > At Tue, 31 Jan 2023 15:12:14 +0530, Amit Kapila > > wrote in > > > So, shall we check if the result of parse_int is in the range 0 and > > > PG_INT32_MAX to ameliorate t

transition tables and UPDATE

2023-02-01 Thread Alvaro Herrera
Earlier today I gave a talk about MERGE and wanted to provide an example with FOR EACH STATEMENT triggers using transition tables. However, I can't find a non-ugly way to obtain the NEW row that corresponds to each OLD row ... I had to resort to an ugly trick with OFFSET n LIMIT 1. Can anyone sug

Re: [PATCH] New [relation] option engine

2023-02-01 Thread Nikolay Shaplov
В письме от среда, 1 февраля 2023 г. 12:04:26 MSK пользователь Alvaro Herrera написал: > On 2023-Jan-31, vignesh C wrote: > > On Tue, 3 Jan 2023 at 18:38, vignesh C wrote: > > > The patch does not apply on top of HEAD as in [1], please post a rebased > > > patch: === Applying patches on top of Po

Re: Exit walsender before confirming remote flush in logical replication

2023-02-01 Thread Amit Kapila
On Wed, Feb 1, 2023 at 2:09 PM Masahiko Sawada wrote: > > On Fri, Jan 20, 2023 at 7:45 PM Amit Kapila wrote: > > > > On Tue, Jan 17, 2023 at 2:41 PM Amit Kapila wrote: > > > > > > Let me try to summarize the discussion till now. The problem we are > > > trying to solve here is to allow a shutdow

Re: transition tables and UPDATE

2023-02-01 Thread Thomas Munro
On Wed, Feb 1, 2023 at 10:18 PM Alvaro Herrera wrote: > Earlier today I gave a talk about MERGE and wanted to provide an example > with FOR EACH STATEMENT triggers using transition tables. However, I > can't find a non-ugly way to obtain the NEW row that corresponds to each > OLD row ... I had t

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-01 Thread shiy.f...@fujitsu.com
On Mon, Jan 30, 2023 6:05 PM Takamichi Osumi (Fujitsu) wrote: > > On Saturday, January 28, 2023 1:28 PM I wrote: > > Attached the updated patch v24. > Hi, > > > I've conducted the rebase affected by the commit(1e8b61735c) > by renaming the GUC to logical_replication_mode accordingly, > because

Re: [DOCS] Stats views and functions not in order?

2023-02-01 Thread Alvaro Herrera
On 2023-Jan-30, Peter Smith wrote: > On Fri, Jan 27, 2023 at 10:30 PM Peter Eisentraut > wrote: > > We could change the chunking boundary to be sect2 globally. This is > > easily configurable (chunk.section.depth). > > Thinking about it now, maybe this is what we need. As the documentation >

Re: Syncrep and improving latency due to WAL throttling

2023-02-01 Thread Jakub Wartak
On Mon, Jan 30, 2023 at 9:16 AM Bharath Rupireddy wrote: Hi Bharath, thanks for reviewing. > I think measuring the number of WAL flushes with and without this > feature that the postgres generates is great to know this feature > effects on IOPS. Probably it's even better with variations in > syn

Re: Generating "Subplan Removed" in EXPLAIN

2023-02-01 Thread Yugo NAGATA
On Wed, 1 Feb 2023 16:52:07 +1300 David Rowley wrote: > On Wed, 1 Feb 2023 at 15:53, Yugo NAGATA wrote: > > Maybe, you missed to set plan_cache_mode to force_generic_plan. > > "Subplan Removed" doesn't appear when using a custom plan. > > I wouldn't say that's 100% true. The planner is only abl

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Aleksander Alekseev
Hi hackers, > Concretely, I'm thinking something like the attached. Notes: > > 1. I have not tested the meson changes. > Works here. Took me a while to figure out how to build the documentation with Meson: ``` XML_CATALOG_FILES=/usr/local/etc/xml/catalog ninja -C build alldocs ``` It works. P

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 13:05:32 +0300, Aleksander Alekseev wrote: > Took me a while to figure out how to build the documentation with Meson: > > ``` > XML_CATALOG_FILES=/usr/local/etc/xml/catalog ninja -C build alldocs > ``` > > It works. Perhaps we should add: > > ``` > ninja -C build alldocs > ```

Re: Clarify deleting comments and security labels in synopsis

2023-02-01 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: >> Tom Lane writes: >>> Agreed; as-is, the syntax summary is not just confusing but outright >>> wrong. >>> >>> I think we could go further and split the entry under Parameters >>> to match: > >> Makes sense. Something like t

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 16:21:16 +1300, Thomas Munro wrote: > My database off assertion failures has four like that, all 15 and master: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=curculio&dt=2023-02-01%2001:05:17 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=curculio&dt=202

Re: heapgettup refactoring

2023-02-01 Thread David Rowley
On Tue, 31 Jan 2023 at 12:18, Melanie Plageman wrote: > v7 attached I've been looking over the v7-0002 patch today and I did make a few adjustments to heapgettup_initial_block() as I would prefer to see the branching of each of all these helper functions follow the pattern of: if () { if ()

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread Melih Mutlu
Hi, Please see attached patches for the below changes. shveta malik , 27 Oca 2023 Cum, 13:12 tarihinde şunu yazdı: > On Thu, Jan 26, 2023 at 7:53 PM Melih Mutlu > wrote: > 1. > --Can we please add a few more points to the Summary to make it more clear. > a) something telling that reusability of

Re: Can we do something to help stop users mistakenly using force_parallel_mode?

2023-02-01 Thread David Rowley
On Thu, 30 Jun 2022 at 00:31, Justin Pryzby wrote: > > On Wed, Jun 29, 2022 at 03:23:27PM +1200, David Rowley wrote: > > Over on [1] I noticed that the user had set force_parallel_mode to > > "on" in the hope that would trick the planner into making their query > > run more quickly. Of course, th

RLS makes COPY TO process child tables

2023-02-01 Thread Antonin Houska
While working on [1] I noticed that if RLS gets enabled, the COPY TO command includes the contents of child table into the result, although the documentation says it should not: "COPY TO can be used only with plain tables, not views, and does not copy rows from child tables or chil

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread Melih Mutlu
Hi, I mistakenly attached v9 in my previous email. Please see attached v6 and v10 for the previous and below changes. shveta malik , 31 Oca 2023 Sal, 12:59 tarihinde şunu yazdı: > On Fri, Jan 27, 2023 at 3:41 PM shveta malik > wrote: > 1) > REPLICATION_SLOT_SNAPSHOT > --Do we need 'CREATE' pref

Re: Perform streaming logical transactions by background workers and parallel apply

2023-02-01 Thread Amit Kapila
On Tue, Jan 31, 2023 at 9:04 AM houzj.f...@fujitsu.com wrote: > > I think your comment makes sense, thanks. > I updated the patch for the same. > The patch looks mostly good to me. I have made a few changes in the comments and docs, see attached. -- With Regards, Amit Kapila. v91-0001-Allow-t

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread shveta malik
On Wed, Feb 1, 2023 at 5:05 PM Melih Mutlu wrote: > > Hi, > > Please see attached patches for the below changes. > > Thanks for reviewing, > -- > Melih Mutlu > Microsoft Hello Melih, Thank you for making the changes. I have few more comments: 1) src/backend/replication/logical/worker.c: (errmsg

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread Melih Mutlu
Hi, wangw.f...@fujitsu.com , 31 Oca 2023 Sal, 13:40 tarihinde şunu yazdı: > Sorry, I forgot to add the link to the email. Please refer to [1]. > > [1] - > https://www.postgresql.org/message-id/OSZPR01MB631013C833C98E826B3CFCB9FDC69%40OSZPR01MB6310.jpnprd01.prod.outlook.com Thanks for pointing o

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread Melih Mutlu
Hi Shveta, shveta malik , 1 Şub 2023 Çar, 15:01 tarihinde şunu yazdı: > On Wed, Feb 1, 2023 at 5:05 PM Melih Mutlu wrote: > 2) I found a crash in the previous patch (v9), but have not tested it > on the latest yet. Crash happens when all the replication slots are > consumed and we are trying to

Re: transition tables and UPDATE

2023-02-01 Thread Alvaro Herrera
On 2023-Feb-01, Thomas Munro wrote: > On Wed, Feb 1, 2023 at 10:18 PM Alvaro Herrera > wrote: > > Earlier today I gave a talk about MERGE and wanted to provide an example > > with FOR EACH STATEMENT triggers using transition tables. However, I > > can't find a non-ugly way to obtain the NEW row

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Alvaro Herrera
Hmm, count_leaf_partitions has to scan pg_inherits and do a syscache lookup for every single element therein ... this sounds slow. In one of the callsites, we already have the partition descriptor available. We could just scan partdesc->is_leaf[] and add one for each 'true' value we see there.

Re: Can we do something to help stop users mistakenly using force_parallel_mode?

2023-02-01 Thread John Naylor
On Wed, Feb 1, 2023 at 6:41 PM David Rowley wrote: > > I don't really share Laurenz's worry [2] about compatibility break > from renaming this GUC. I think the legitimate usages of this setting > are probably far more rare than the illegitimate ones. I'm not overly > concerned about renaming if it

Re: transition tables and UPDATE

2023-02-01 Thread Dean Rasheed
On Wed, 1 Feb 2023 at 12:12, Alvaro Herrera wrote: > > I had tried to tie these relations using WITH ORDINALITY, but the only > way I could think of (array_agg to then unnest() WITH ORDINALITY) was > even uglier than what I already had. So yeah, I think it might be > useful if we had a way to inj

Re: meson: Optionally disable installation of test modules

2023-02-01 Thread Nazir Bilal Yavuz
Hi, On 1/31/23 11:44, Peter Eisentraut wrote: On 30.01.23 18:42, Andres Freund wrote: Bilal, with a bit of help by me, worked on an alternative approach to this. It's a lot more verbose in the initial change, but wouldn't increase the amount of work/lines for new test modules. The main advant

Re: transition tables and UPDATE

2023-02-01 Thread Yugo NAGATA
On Wed, 1 Feb 2023 10:03:26 +0100 Alvaro Herrera wrote: > Earlier today I gave a talk about MERGE and wanted to provide an example > with FOR EACH STATEMENT triggers using transition tables. However, I > can't find a non-ugly way to obtain the NEW row that corresponds to each > OLD row ... I ha

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread shveta malik
On Wed, Feb 1, 2023 at 5:42 PM Melih Mutlu wrote: > > > Thanks for investigating this error. I think it's the same error as the one > Shi reported earlier. [1] > I couldn't reproduce it yet but will apply your tweaks and try again. > Looking into this. > > [1] > https://www.postgresql.org/messa

Re: [Commitfest 2023-01] has started

2023-02-01 Thread vignesh C
On Wed, 1 Feb 2023 at 11:25, Julien Rouhaud wrote: > > On Wed, Feb 1, 2023 at 10:44 AM vignesh C wrote: > > > > I don't have permissions to close the commitfest, could one of them > > help in closing the commitfest. > > It's technically 17:53 at Anywhere on Earth, so we usually wait for > the day

Re: Support for dumping extended statistics

2023-02-01 Thread Justin Pryzby
On Wed, Feb 01, 2023 at 04:38:17AM +, Hari krishna Maddileti wrote: > Hi Justin, > Thanks for the update, I have attached the updated patch with > meson compatible and addressed warnings from make file too. Thanks - I see it compiles now under both build systems. But there's

Re: Syncrep and improving latency due to WAL throttling

2023-02-01 Thread Tomas Vondra
On 2/1/23 11:04, Jakub Wartak wrote: > On Mon, Jan 30, 2023 at 9:16 AM Bharath Rupireddy > wrote: > > Hi Bharath, thanks for reviewing. > >> I think measuring the number of WAL flushes with and without this >> feature that the postgres generates is great to know this feature >> effects on IOP

Re: Show various offset arrays for heap WAL records

2023-02-01 Thread Robert Haas
On Tue, Jan 31, 2023 at 6:20 PM Peter Geoghegan wrote: > Actually the really wide output comes from COMMIT records. After I run > the regression tests, and execute some of my own custom pg_walinspect > queries, I see that some individual COMMIT records have a > length(description) of over 10,000 b

Re: recovery modules

2023-02-01 Thread Andres Freund
Hi, On 2023-01-31 15:30:13 -0800, Nathan Bossart wrote: > +/* > + * basic_archive_startup > + * > + * Creates the module's memory context. > + */ > +void * > +basic_archive_startup(void) > +{ > + return (void *) AllocSetContextCreate(TopMemoryContext, > +

Re: Support for dumping extended statistics

2023-02-01 Thread Tomas Vondra
On 1/7/23 03:39, Bruce Momjian wrote: > On Thu, Jan 5, 2023 at 06:29:03PM +, Hari krishna Maddileti wrote: >> Hi Team, >> In order to restore dumped extended statistics (stxdndistinct, >> stxddependencies, stxdmcv) we need to provide input functions to parse >> pg_distinct/pg_dependency/pg_

Re: Syncrep and improving latency due to WAL throttling

2023-02-01 Thread Jakub Wartak
On Wed, Feb 1, 2023 at 2:14 PM Tomas Vondra wrote: > > Maybe we should avoid calling fsyncs for WAL throttling? (by teaching > > HandleXLogDelayPending()->XLogFlush()->XLogWrite() to NOT to sync when > > we are flushing just because of WAL thortting ?) Would that still be > > safe? > > It's not c

Performance issues with parallelism and LIMIT

2023-02-01 Thread David Geier
Hi hackers, While migrating from PostgreSQL 14 to 15, we encountered the following performance degradation caused by commit 46846433a03dff: "shm_mq: Update mq_bytes_written less often", discussion in [1]. The batching can make queries with a LIMIT clause run significantly slower compared to

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Ilya Gladyshev
> 1 февр. 2023 г., в 16:01, Alvaro Herrera написал(а): > > Hmm, count_leaf_partitions has to scan pg_inherits and do a syscache > lookup for every single element therein ... this sounds slow. > > In one of the callsites, we already have the partition descriptor > available. We could just scan

Re: Non-superuser subscription owners

2023-02-01 Thread Robert Haas
On Tue, Jan 31, 2023 at 7:01 PM Andres Freund wrote: > I don't really understand that - the run-as approach seems like a > necessary piece of improving the security model. > > I think it's perfectly reasonable to want to replicate from one system > in another, but to not want to allow logical repl

Re: Support for dumping extended statistics

2023-02-01 Thread Tom Lane
Tomas Vondra writes: > On 1/7/23 03:39, Bruce Momjian wrote: >> There is certainly interest in allowing the optimizer statistics to be >> dumped and reloaded. This could be used by pg_restore and pg_upgrade. > Indeed, although I think it'd be better to deal with regular statistics > (which is wh

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Tom Lane
Andres Freund writes: > On 2023-02-01 16:21:16 +1300, Thomas Munro wrote: >> It's always in proc_exit() in StartupProcShutdownHandler(), a SIGTERM >> handler which is allowed to call that while in_restore_command is >> true. > Ugh, no wonder we're getting crashes. This whole business seems bogus

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Matthias van de Meent
On Wed, 1 Feb 2023 at 15:21, Ilya Gladyshev wrote: > > > 1 февр. 2023 г., в 16:01, Alvaro Herrera > > написал(а): > > > > Hmm, count_leaf_partitions has to scan pg_inherits and do a syscache > > lookup for every single element therein ... this sounds slow. > > > > In one of the callsites, we alr

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Justin Pryzby
On Wed, Feb 01, 2023 at 04:21:35PM +0100, Matthias van de Meent wrote: > On Wed, 1 Feb 2023 at 15:21, Ilya Gladyshev > wrote: > > > 1 февр. 2023 г., в 16:01, Alvaro Herrera > > > написал(а): > > > Hmm, count_leaf_partitions has to scan pg_inherits and do a syscache > > > lookup for every single

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Nathan Bossart
On Wed, Feb 01, 2023 at 10:12:26AM -0500, Tom Lane wrote: > Andres Freund writes: >> On 2023-02-01 16:21:16 +1300, Thomas Munro wrote: >>> It's always in proc_exit() in StartupProcShutdownHandler(), a SIGTERM >>> handler which is allowed to call that while in_restore_command is >>> true. > >> Ugh

Re: RLS makes COPY TO process child tables

2023-02-01 Thread Yugo NAGATA
On Wed, 01 Feb 2023 12:45:57 +0100 Antonin Houska wrote: > While working on [1] I noticed that if RLS gets enabled, the COPY TO command > includes the contents of child table into the result, although the > documentation says it should not: > > "COPY TO can be used only with plain tables,

pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Over at [1] we have a complaint that dump-and-restore fails for hash-partitioned tables if a partitioning column is an enum, because the enum values are unlikely to receive the same OIDs in the destination database as they had in the source, and the hash codes are dependent on those OIDs. So resto

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Matthias van de Meent
On Wed, 1 Feb 2023 at 16:53, Justin Pryzby wrote: > > On Wed, Feb 01, 2023 at 04:21:35PM +0100, Matthias van de Meent wrote: > > On Wed, 1 Feb 2023 at 15:21, Ilya Gladyshev > > wrote: > > > > 1 февр. 2023 г., в 16:01, Alvaro Herrera > > > > написал(а): > > > > Hmm, count_leaf_partitions has to

Re: RLS makes COPY TO process child tables

2023-02-01 Thread Tom Lane
Yugo NAGATA writes: > Antonin Houska wrote: >> While working on [1] I noticed that if RLS gets enabled, the COPY TO command >> includes the contents of child table into the result, although the >> documentation says it should not: > I think this is a bug because the current behaviour is differen

Re: Timeline ID hexadecimal format

2023-02-01 Thread Sébastien Lardière
On 31/01/2023 20:16, Greg Stark wrote: The fact that the *filename* has it encoded in hex is an implementation detail and really gets exposed here because it's giving you the underlying system error that caused the problem. It's an implementation detail, but an exposed detail, so, people refer

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 10:12:26 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-02-01 16:21:16 +1300, Thomas Munro wrote: > >> It's always in proc_exit() in StartupProcShutdownHandler(), a SIGTERM > >> handler which is allowed to call that while in_restore_command is > >> true. > > > Ugh, n

Re: About PostgreSQL Core Team

2023-02-01 Thread Pavel Borisov
Hi, Adherent! IMO "not liking" that you quote in the picture is just other words for expressing caution for the patch or for the general direction of some change. At least I never felt personal or arbitrary presumptions in relation to my patches. So if you can join a discussion with your proposals

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 11:58 AM Andres Freund wrote: > > 9a740f81e clearly made things a lot worse, but it wasn't great > > before. Can we see a way forward to removing the problem entirely? > > Yea, I think we can - we should stop relying on system(). If we instead > run the command properly as

Re: meson: pkgconfig difference

2023-02-01 Thread Peter Eisentraut
On 01.02.23 08:55, Andres Freund wrote: Hi, On January 31, 2023 11:40:52 PM PST, Peter Eisentraut wrote: I think there is a tiny typo in src/interfaces/ecpg/ecpglib/meson.build: diff --git a/src/interfaces/ecpg/ecpglib/meson.build b/src/interfaces/ecpg/ecpglib/meson.build index dba9e3c3d9..

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 12:08:24 -0500, Robert Haas wrote: > On Wed, Feb 1, 2023 at 11:58 AM Andres Freund wrote: > > > 9a740f81e clearly made things a lot worse, but it wasn't great > > > before. Can we see a way forward to removing the problem entirely? > > > > Yea, I think we can - we should stop r

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Tom Lane
Andres Freund writes: > On 2023-02-01 13:05:32 +0300, Aleksander Alekseev wrote: >> It works. Perhaps we should add: >> ninja -C build alldocs > FWIW, just 'docs' would build just the multi-page html/man pages, > alldocs takes a lot longer... Hmm ... why does 'docs' include the man pages, and no

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Tom Lane
Andres Freund writes: > On 2023-02-01 12:08:24 -0500, Robert Haas wrote: >> I like the idea of not relying on system(). In most respects, doing >> fork() + exec() ourselves seems superior. We can control where the >> output goes, what we do while waiting, etc. But system() runs the >> command thro

Re: pg_dump versus hash partitioning

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 11:18 AM Tom Lane wrote: > Over at [1] we have a complaint that dump-and-restore fails for > hash-partitioned tables if a partitioning column is an enum, > because the enum values are unlikely to receive the same OIDs > in the destination database as they had in the source,

Re: Show various offset arrays for heap WAL records

2023-02-01 Thread Peter Geoghegan
On Wed, Feb 1, 2023 at 5:20 AM Robert Haas wrote: > If we're dumping a lot of details out of each WAL record, we might > want to switch to a multi-line format of some kind. No one enjoys a > 460-character wide line, let alone 46000. I generally prefer it when I can use psql without using expanded

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 12:23:27 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-02-01 13:05:32 +0300, Aleksander Alekseev wrote: > >> It works. Perhaps we should add: > >> ninja -C build alldocs > > > FWIW, just 'docs' would build just the multi-page html/man pages, > > alldocs takes a lot

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Ilya Gladyshev
> 1 февр. 2023 г., в 20:27, Matthias van de Meent > написал(а): > > On Wed, 1 Feb 2023 at 16:53, Justin Pryzby > wrote: >> >> On Wed, Feb 01, 2023 at 04:21:35PM +0100, Matthias van de Meent wrote: >>> On Wed, 1 Feb 2023 at 15:21, Ilya Gladyshev >>> wrote: >

Re: Show various offset arrays for heap WAL records

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 12:47 PM Peter Geoghegan wrote: > On Wed, Feb 1, 2023 at 5:20 AM Robert Haas wrote: > > If we're dumping a lot of details out of each WAL record, we might > > want to switch to a multi-line format of some kind. No one enjoys a > > 460-character wide line, let alone 46000. >

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Nathan Bossart
On Wed, Feb 01, 2023 at 08:58:01AM -0800, Andres Freund wrote: > On 2023-02-01 10:12:26 -0500, Tom Lane wrote: >> The fundamental issue is that we have no good way to break out >> of system(), and I think the original idea was that >> in_restore_command would be set *only* for the duration of the >

Re: Non-superuser subscription owners

2023-02-01 Thread Mark Dilger
> On Feb 1, 2023, at 6:43 AM, Robert Haas wrote: > The thing I'm > struggling to understand is: if you only want to replicate into tables > that Alice can write, why not just make Alice own the subscription? > For a run-as user to make sense, you need a scenario where we want the > replication

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 12:27:19 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-02-01 12:08:24 -0500, Robert Haas wrote: > >> I like the idea of not relying on system(). In most respects, doing > >> fork() + exec() ourselves seems superior. We can control where the > >> output goes, what we

Re: Optimizing PostgreSQL with LLVM's PGO+LTO

2023-02-01 Thread João Paulo Labegalini de Carvalho
On Mon, Jan 30, 2023 at 10:47 AM Andres Freund wrote: > For some reason my notes for using LTO include changing RANLIB to point to > gcc/llvm-ranlib of the appropriate version. Won't even be used on HEAD, but > before that it can make a difference. > I will try that. > Depending on how you bui

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 1, 2023 at 11:18 AM Tom Lane wrote: >> Over at [1] we have a complaint that dump-and-restore fails for >> hash-partitioned tables if a partitioning column is an enum, >> because the enum values are unlikely to receive the same OIDs >> in the destination database

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Matthias van de Meent
On Wed, 1 Feb 2023 at 18:51, Ilya Gladyshev wrote: > > 1 февр. 2023 г., в 20:27, Matthias van de Meent > написал(а): > >> In HEAD we set TOTAL to whatever number partitioned table we're >> currently processing has - regardless of whether we're the top level >> statement. >> With the patch we ins

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Tom Lane
Andres Freund writes: > On 2023-02-01 12:23:27 -0500, Tom Lane wrote: >> It's unlike what "make -C doc/src/sgml all" does in the Makefile >> system, and I don't find it to be an improvement. > Well, that'd actually build the manpages too, afaics :). But I get the > point. Ah, sorry, I too had fo

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 09:49:00 -0800, Andres Freund wrote: > On 2023-02-01 12:23:27 -0500, Tom Lane wrote: > > And the minimum version appears to be newer than RHEL8's 1.8.2, which > > I find pretty unfortunate. On RHEL8, it fails with > > $ ninja > > ninja: error: build.ninja:6771: multiple outputs

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Tom Lane
Andres Freund writes: > On 2023-02-01 09:49:00 -0800, Andres Freund wrote: >> On 2023-02-01 12:23:27 -0500, Tom Lane wrote: >>> And the minimum version appears to be newer than RHEL8's 1.8.2, which >>> I find pretty unfortunate. > Unfortunately the test script accidentally pulled in ninja from ep

Re: Introduce "log_connection_stages" setting.

2023-02-01 Thread Sergey Dudoladov
Hi again, Justin, thank you for the fast review. The new version is attached. Regards, Sergey Dudoladov From 994a86e6ac3abb647d93bdaf0f42be76f42b83a8 Mon Sep 17 00:00:00 2001 From: Sergey Dudoladov Date: Tue, 8 Nov 2022 18:56:26 +0100 Subject: [PATCH] Introduce 'log_connection_messages' This pa

Re: recovery modules

2023-02-01 Thread Nathan Bossart
On Wed, Feb 01, 2023 at 03:54:26AM -0800, Andres Freund wrote: > I'd make basic_archive's private data a struct, with a member for the > context, but it's not that important. > > I'd also be inclined to do the same for the private_state you're passing > around for each module. Even if it's just to

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Robert Haas writes: > ... I like the > fact that we have --load-via-partition-root, but it is a bit of a > hack. You don't get a single copy into the partition root, you get one > per child table -- and those COPY statements are listed as data for > the partitions where the data lives now, not for

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-02-01 Thread Jacob Champion
On Mon, Jan 30, 2023 at 2:21 PM Robert Haas wrote: > On Mon, Jan 30, 2023 at 4:12 PM Jacob Champion > wrote: > > For our case, assuming that connections have side effects, one > > solution could be for the client to signal to the server that the > > connection should use in-band authentication o

Re: Non-superuser subscription owners

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 09:43:39 -0500, Robert Haas wrote: > On Tue, Jan 31, 2023 at 7:01 PM Andres Freund wrote: > > I don't really understand that - the run-as approach seems like a > > necessary piece of improving the security model. > > > > I think it's perfectly reasonable to want to replicate fro

Re: pg_dump versus hash partitioning

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 1:23 PM Tom Lane wrote: > Well, that was what I thought too to start with, but I now think that > it is far too narrow-minded a view of the problem. The real issue > is something I said that you trimmed: > > >> In general, we've never thought that hash values are > >> requi

Re: [PATCH] CF app: add "Returned: Needs more interest"

2023-02-01 Thread Jacob Champion
On Wed, Jan 4, 2023 at 9:33 AM Jacob Champion wrote: > Is there a good way to remind people that, hey, this exists as a > patchset? (Other than me pinging the list every so often.) I've withdrawn this patchset for now, but if anyone has any ideas on where and how I can better propose features for

Re: Add connection active, idle time to pg_stat_activity

2023-02-01 Thread Sergey Dudoladov
Hello hackers, I've sketched the first version of a patch to add pg_stat_session. Please review this early version. Regards, Sergey. From 31f781ecd69fc42aaadd9bcdbebaf8f72449946c Mon Sep 17 00:00:00 2001 From: Sergey Dudoladov Date: Tue, 22 Nov 2022 09:23:32 +0100 Subject: [PATCH] Add pg_stat_se

Re: Non-superuser subscription owners

2023-02-01 Thread Andres Freund
Hi, On 2023-01-30 15:32:34 -0500, Robert Haas wrote: > I had a long think about what to do with ALTER SUBSCRIPTION ... OWNER > TO in terms of permissions checks. The previous version required that > the new owner have permissions of pg_create_subscription, but there > seems to be no particular rea

Re: recovery modules

2023-02-01 Thread Andres Freund
Hi, On 2023-02-01 12:15:29 -0800, Nathan Bossart wrote: > Here's a new patch set in which I've attempted to address this feedback and > Michael's feedback. Looks better! > @@ -25,12 +34,14 @@ extern PGDLLIMPORT char *XLogArchiveLibrary; > * For more information about the purpose of each callb

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 1, 2023 at 1:23 PM Tom Lane wrote: >> In the meantime, I think we need to recognize that hash values are >> not very portable. I do not think we do our users a service by >> letting them discover the corner cases the hard way. > I think you're not really engagi

Re: heapgettup refactoring

2023-02-01 Thread Melanie Plageman
On Thu, Feb 02, 2023 at 12:21:20AM +1300, David Rowley wrote: > On Tue, 31 Jan 2023 at 12:18, Melanie Plageman > wrote: > > v7 attached > > I've been looking over the v7-0002 patch today and I did make a few > adjustments to heapgettup_initial_block() as I would prefer to see the > branching of e

Re: pg_dump versus hash partitioning

2023-02-01 Thread Peter Geoghegan
On Wed, Feb 1, 2023 at 12:34 PM Tom Lane wrote: > > Also, and I think pretty > > significantly, using --load-via-partition-root forces you to pay the > > overhead of rerouting every tuple to the target partition whether you > > need it or not, which is potentially a large unnecessary expense. > >

Re: pg_dump versus hash partitioning

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 3:34 PM Tom Lane wrote: > I spent a bit more time thinking about that, and while I agree that > it's an oddity, I don't see that it matters in the case of hash > partitioning. You would notice an issue if you tried to do a selective > restore of just one partition --- but u

Re: recovery modules

2023-02-01 Thread Nathan Bossart
On Wed, Feb 01, 2023 at 01:06:06PM -0800, Andres Freund wrote: > On 2023-02-01 12:15:29 -0800, Nathan Bossart wrote: >> Here's a new patch set in which I've attempted to address this feedback and >> Michael's feedback. > > Looks better! Thanks! >> @@ -25,12 +34,14 @@ extern PGDLLIMPORT char *XLo

Re: pg_dump versus hash partitioning

2023-02-01 Thread Peter Geoghegan
On Wed, Feb 1, 2023 at 1:14 PM Robert Haas wrote: > It seems to me that the job of pg_dump is to produce a dump that, when > reloaded on another system, recreates the same database state. That > means that we end up with all of the same objects, each defined in the > same way, and that all of the

Re: pg_dump versus hash partitioning

2023-02-01 Thread Peter Geoghegan
On Wed, Feb 1, 2023 at 12:39 PM Robert Haas wrote: > I don't think the fact that our *traditional* standard for how stable > a hash function needs to be has been XYZ carries any water. Needs > change over time, and we adapt the code to meet the new needs. Since > we have no system for type propert

Re: pg_dump versus hash partitioning

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 4:12 PM Tom Lane wrote: > > I don't think the fact that our *traditional* standard for how stable > > a hash function needs to be has been XYZ carries any water. > > Well, it wouldn't need to if we had a practical way of changing the > behavior of an existing hash function,

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Robert Haas writes: > It seems to me that the job of pg_dump is to produce a dump that, when > reloaded on another system, recreates the same database state. That > means that we end up with all of the same objects, each defined in the > same way, and that all of the tables end up with all the sam

Re: pg_dump versus hash partitioning

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 4:44 PM Peter Geoghegan wrote: > This is a misrepresentation of Tom's words. It isn't actually > self-evident what "we end up with all of the same objects, each > defined in the same way, and that all of the tables end up with all > the same contents that they had before" ac

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-02-01 Thread Aleksander Alekseev
Hi, Here are my two cents. > the minimum version appears to be newer than RHEL8's 1.8.2, > which I find pretty unfortunate. On RHEL8, it fails with > $ ninja > ninja: error: build.ninja:6771: multiple outputs aren't (yet?) supported by > depslog; bring this up on the mailing list if it affects

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Robert Haas writes: > Tom, as I understand it, is arguing that the > --load-via-partition-root behavior has negligible downsides and is > almost categorically better than the current default behavior, and > thus making that the new default in some or all situations in a minor > release is totally

Re: pg_dump versus hash partitioning

2023-02-01 Thread Peter Geoghegan
On Wed, Feb 1, 2023 at 2:12 PM Robert Haas wrote: > On Wed, Feb 1, 2023 at 4:44 PM Peter Geoghegan wrote: > > This is a misrepresentation of Tom's words. It isn't actually > > self-evident what "we end up with all of the same objects, each > > defined in the same way, and that all of the tables e

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Nathan Bossart
On Wed, Feb 01, 2023 at 09:58:06AM -0800, Nathan Bossart wrote: > On Wed, Feb 01, 2023 at 08:58:01AM -0800, Andres Freund wrote: >> On 2023-02-01 10:12:26 -0500, Tom Lane wrote: >>> The fundamental issue is that we have no good way to break out >>> of system(), and I think the original idea was tha

Re: pg_dump versus hash partitioning

2023-02-01 Thread Robert Haas
On Wed, Feb 1, 2023 at 5:08 PM Tom Lane wrote: > > Here, you'd like to argue that it's perfectly > > fine if we instead insert some of the rows into different tables than > > where they were on the original system. > > I can agree with that argument for range or list partitioning, where > the part

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Peter Geoghegan writes: > You mentioned "minor releases" here. Who said anything about that? I did: I'd like to back-patch the fix if possible. I think changing the default --load-via-partition-root choice could be back-patchable. If Robert is resistant to that but would accept it in master, I'

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 1, 2023 at 5:08 PM Tom Lane wrote: >> I can agree with that argument for range or list partitioning, where >> the partitions have some semantic meaning to the user. I don't buy it >> for hash partitioning. It was an implementation artifact to begin >> with that

Re: pg_dump versus hash partitioning

2023-02-01 Thread David G. Johnston
On Wed, Feb 1, 2023 at 3:38 PM Tom Lane wrote: > Peter Geoghegan writes: > > You mentioned "minor releases" here. Who said anything about that? > > I did: I'd like to back-patch the fix if possible. I think changing > the default --load-via-partition-root choice could be back-patchable. > > If

  1   2   >