Re: speed up a logical replica setup

2024-03-19 Thread Amit Kapila
On Mon, Mar 18, 2024 at 7:22 PM Peter Eisentraut wrote: > > In check_subscriber(): All these permissions checks seem problematic > to me. We shouldn't reimplement our own copy of the server's > permission checks. The server can check the permissions. And if the > permission checking in the serv

Re: Reducing connection overhead in pg_upgrade compat check phase

2024-03-19 Thread Peter Eisentraut
On 18.03.24 13:11, Daniel Gustafsson wrote: Attached is a fresh rebase with only minor cosmetic touch-ups which I would like to go ahead with during this CF. Peter: does this address the comments you had on translation and code duplication? Yes, this looks good.

Re: DOCS: add helpful partitioning links

2024-03-19 Thread Ashutosh Bapat
Hi Robert, On Mon, Mar 18, 2024 at 10:52 PM Robert Treat wrote: > On Thu, Mar 14, 2024 at 12:15 PM Ashutosh Bapat > wrote: > > > > Hi Robert, > > > > On Thu, Mar 7, 2024 at 10:49 PM Robert Treat wrote: > >> > >> This patch adds a link to the "attach partition" command section > >> (similar to

Re: Autogenerate some wait events code and documentation

2024-03-19 Thread Bertrand Drouvot
Hi, On Tue, Mar 19, 2024 at 09:59:35AM +0900, Michael Paquier wrote: > On Mon, Mar 18, 2024 at 05:57:02PM +, Bertrand Drouvot wrote: > > Thanks for looking at it! > > Oh right, the comment is wrong, re-worded in v2 attached. > > I've added a couple of fake events in my txt file, and this resu

Re: A problem about partitionwise join

2024-03-19 Thread Ashutosh Bapat
On Tue, Mar 19, 2024 at 8:18 AM Richard Guo wrote: > (Sorry it takes me some time to get back to this thread.) > > On Thu, Mar 7, 2024 at 7:13 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> I did a deeper review of the patch. Here are some comments >> > > Thank you for the review

What is a typical precision of gettimeofday()?

2024-03-19 Thread Peter Eisentraut
Over in the thread discussing the addition of UUIDv7 support [0], there is some uncertainty about what timestamp precision one can expect from gettimeofday(). UUIDv7 uses milliseconds since Unix epoch, but can optionally use up to 12 additional bits of timestamp precision (see [1]), but it can

Re: UUID v7

2024-03-19 Thread Peter Eisentraut
On 16.03.24 18:43, Andrey M. Borodin wrote: On 15 Mar 2024, at 14:47, Aleksander Alekseev wrote: +1 to the idea. I doubt that anyone will miss it. PFA v22. Changes: 1. Squashed all editorialisation by Jelte 2. Fixed my erroneous comments on using Method 2 (we are using method 1 instead) 3. R

hot updates and fillfactor

2024-03-19 Thread Fabrice Chapuis
Hi, I do not understand why hot_updates value is not 0 for pg_database? Given that reloptions is empty for this table that means it has a default value of 100% Regards, Fabrice SELECT relname AS table_name, seq_scan AS sequential_scans, idx_scan AS index_scans, n_tup_ins AS ins

Re: SQL:2011 application time

2024-03-19 Thread jian he
On Tue, Mar 19, 2024 at 6:49 AM Paul Jungwirth wrote: > > Rebased to 846311051e. > Hi, I just found out some minor issues. + * types matching the PERIOD element. periodprocoid is a GiST support function to + * aggregate multiple PERIOD element values into a single value + * (whose return type ne

Re: documentation structure

2024-03-19 Thread Daniel Gustafsson
> On 18 Mar 2024, at 22:40, Laurenz Albe wrote: > On Mon, 2024-03-18 at 10:11 -0400, Robert Haas wrote: >> For reasons that I don't understand, all chapters except >> for those in "VI. Reference" are numbered, but the chapters in that >> section have Roman numerals instead. > > That last fact is

Re: Statistics Import and Export

2024-03-19 Thread Corey Huinker
v11 attached. - TAP tests passing (the big glitch was that indexes that are used in constraints should have their stats dependent on the constraint, not the index, thanks Jeff) - The new range-specific statistics types are now supported. I'm not happy with the typid machinations I do to get them t

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Heikki Linnakangas
I want to remind everyone of this from Gabriele's first message that started this thread: At the moment, a possible workaround is that `ALTER SYSTEM` can be blocked by making the postgresql.auto.conf read only, but the returned message is misleading and that’s certainly bad user experience (whi

Re: doc issues in event-trigger-matrix.html

2024-03-19 Thread Daniel Gustafsson
> On 19 Mar 2024, at 02:14, Michael Paquier wrote: > > On Tue, Mar 19, 2024 at 08:00:00AM +0800, jian he wrote: >> I think the "X" and "-" mean in this matrix [0] is not very intuitive. >> mainly because "X" tends to mean negative things in most cases. >> we can write a sentence saying "X" means

Re: Table AM Interface Enhancements

2024-03-19 Thread Pavel Borisov
Hi, Alexander! On Tue, 19 Mar 2024 at 03:34, Alexander Korotkov wrote: > On Sun, Mar 3, 2024 at 1:50 PM Alexander Korotkov > wrote: > > On Mon, Nov 27, 2023 at 10:18 PM Mark Dilger > > wrote: > > > > > > > On Nov 25, 2023, at 9:47 AM, Alexander Korotkov < > aekorot...@gmail.com> wrote: > > > >

Re: What is a typical precision of gettimeofday()?

2024-03-19 Thread Aleksander Alekseev
Hi, cc: Andrey > Over in the thread discussing the addition of UUIDv7 support [0], there > is some uncertainty about what timestamp precision one can expect from > gettimeofday(). > > UUIDv7 uses milliseconds since Unix epoch, but can optionally use up to > 12 additional bits of timestamp precisi

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-19 Thread John Naylor
On Tue, Mar 19, 2024 at 10:24 AM Masahiko Sawada wrote: > > On Tue, Mar 19, 2024 at 8:35 AM John Naylor wrote: > > > > On Mon, Mar 18, 2024 at 11:12 AM Masahiko Sawada > > wrote: > > > > > > On Sun, Mar 17, 2024 at 11:46 AM John Naylor > > > wrote: > > It might also be worth reducing the num

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-19 Thread Bertrand Drouvot
Hi, On Tue, Mar 19, 2024 at 10:56:25AM +0530, Amit Kapila wrote: > On Mon, Mar 18, 2024 at 8:19 PM Bertrand Drouvot > wrote: > > Agree. While it makes sense to invalidate slots for wal removal in > > CreateCheckPoint() (because this is the place where wal is removed), I 'm > > not > > sure this

Re: Inconsistent printf placeholders

2024-03-19 Thread Peter Eisentraut
On 15.03.24 08:20, Kyotaro Horiguchi wrote: diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c @@ -1369,8 +1369,8 @@ ReadTwoPhaseFile(TransactionId xid, bool missing_ok) errmsg("could not read file \"%s\": %m", pat

Re: add AVX2 support to simd.h

2024-03-19 Thread John Naylor
On Tue, Mar 19, 2024 at 10:16 AM Nathan Bossart wrote: > > On Tue, Mar 19, 2024 at 10:03:36AM +0700, John Naylor wrote: > > I took a brief look, and 0001 isn't quite what I had in mind. I can't > > quite tell what it's doing with the additional branches and "goto > > retry", but I meant something

Re: Catalog domain not-null constraints

2024-03-19 Thread jian he
create domain connotnull integer; create table domconnotnulltest ( col1 connotnull , col2 connotnull ); alter domain connotnull add not null value; --- the above query does not work in pg16. ERROR: syntax error at or near "not". after applying the patch, now this works. thi

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-19 Thread Alvaro Herrera
On 2024-Mar-08, Michael Paquier wrote: > I have spent more time reviewing the whole and the tests (I didn't see > much value in testing the DEFAULT clause twice for the partitioned > table case and there is a test in d61a6cad6418), tweaked a few > comments and the documentation, did an indentation

Re: hot updates and fillfactor

2024-03-19 Thread Aleksander Alekseev
Hi Fabrice, > I do not understand why hot_updates value is not 0 for pg_database? Given > that reloptions is empty for this table that means it has a default value of > 100% Maybe I didn't entirely understand your question, but why would you assume they are somehow related? According to the do

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-19 Thread Alvaro Herrera
On 2024-Mar-19, Alvaro Herrera wrote: > 0001 is Michaël's patch, 0002 are my proposed changes. Doh, I sent the wrong set of attachments. But I see no reason to post again: what I attached as 0001 is what I wrote was going to be 0002, Michaël's patch is already in archives, and the CI tests with

minor tweak to catalogs.sgml pg_class.reltablespace

2024-03-19 Thread Alvaro Herrera
While reviewing the patch for SET ACCESS METHOD[1] I noticed that pg_class.relam is not documented fully for partitioned tables, so I proposed the attached. Also, I remove a comment that merely repeats what was already said a few lines above. This is intended for backpatch to 12. [1] https://pos

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-19 Thread Amit Kapila
On Tue, Mar 19, 2024 at 3:11 PM Bertrand Drouvot wrote: > > On Tue, Mar 19, 2024 at 10:56:25AM +0530, Amit Kapila wrote: > > On Mon, Mar 18, 2024 at 8:19 PM Bertrand Drouvot > > wrote: > > > Agree. While it makes sense to invalidate slots for wal removal in > > > CreateCheckPoint() (because this

Re: Recent 027_streaming_regress.pl hangs

2024-03-19 Thread Alexander Lakhin
14.03.2024 23:56, Tom Lane wrote: Thomas Munro writes: On Fri, Mar 15, 2024 at 7:00 AM Alexander Lakhin wrote: Could it be that the timeout (360 sec?) is just not enough for the test under the current (changed due to switch to meson) conditions? But you're right that under meson the test tak

Re: SQL:2011 application time

2024-03-19 Thread Peter Eisentraut
On 16.03.24 22:37, Paul A Jungwirth wrote: Here is a new patch series addressing the last few feedback emails from Peter & Jian He. It mostly focuses on the FKs patch, trying to get it really ready to commit, I have committed the test changes (range and date format etc.). The FOREIGN KEY patch

Re: Proposal to include --exclude-extension Flag in pg_dump

2024-03-19 Thread Dean Rasheed
On Sat, 16 Mar 2024 at 17:36, Ayush Vatsa wrote: > > Attached is the complete patch with all the required code changes. > Looking forward to your review and feedback. > This looks good to me. I tested it and everything worked as expected. I ran it through pgindent to fix some whitespace issues a

Re: speed up a logical replica setup

2024-03-19 Thread Shlok Kyal
Hi, > I'm attaching a new version (v30) that adds: > > * 3 new options (--publication, --subscription, --replication-slot) to assign > names to the objects. The --database option used to ignore duplicate names, > however, since these new options rely on the number of database options to > ma

RE: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-03-19 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, Thanks for giving comments! > This behavior makes sense to me. But do we want to handle the case of > using environment variables too? Yeah, v5 does not consider which libpq parameters are specified by environment variables. Such a variable should be used when the dbname is not

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Andrew Dunstan
On Tue, Mar 19, 2024 at 5:26 AM Heikki Linnakangas wrote: > I want to remind everyone of this from Gabriele's first message that > started this thread: > > > At the moment, a possible workaround is that `ALTER SYSTEM` can be > blocked > > by making the postgresql.auto.conf read only, but the retu

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-19 Thread Amit Kapila
On Mon, Mar 18, 2024 at 3:24 PM Alexander Korotkov wrote: > > On Mon, Mar 18, 2024 at 5:17 AM Amit Kapila wrote: > > > 1. First, check that it was called with non-atomic context (that is, > > > it's not called within a transaction). Trigger error if called with > > > atomic context. > > > 2. Rele

Re: Proposal to include --exclude-extension Flag in pg_dump

2024-03-19 Thread Daniel Gustafsson
> On 19 Mar 2024, at 12:19, Dean Rasheed wrote: > I'm marking this ready-for-commit (which I'll probably do myself in a > day or two, unless anyone else claims it first). LGTM too from a read through. I did notice a few mistakes in the --filter documentation portion for other keywords but that'

Re: speed up a logical replica setup

2024-03-19 Thread Peter Eisentraut
On 19.03.24 08:05, Amit Kapila wrote: On Mon, Mar 18, 2024 at 7:22 PM Peter Eisentraut wrote: In check_subscriber(): All these permissions checks seem problematic to me. We shouldn't reimplement our own copy of the server's permission checks. The server can check the permissions. And if the

Re: speed up a logical replica setup

2024-03-19 Thread Peter Eisentraut
On 19.03.24 12:26, Shlok Kyal wrote: I'm attaching a new version (v30) that adds: * 3 new options (--publication, --subscription, --replication-slot) to assign names to the objects. The --database option used to ignore duplicate names, however, since these new options rely on the number of

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-19 Thread Alexander Korotkov
On Tue, Mar 19, 2024 at 1:51 PM Amit Kapila wrote: > On Mon, Mar 18, 2024 at 3:24 PM Alexander Korotkov > wrote: > > > > On Mon, Mar 18, 2024 at 5:17 AM Amit Kapila wrote: > > > > 1. First, check that it was called with non-atomic context (that is, > > > > it's not called within a transaction).

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-19 Thread Heikki Linnakangas
On 18/03/2024 17:19, Melanie Plageman wrote: I've attached v7 rebased over this commit. Thanks! v7-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch If we delayed table_beginscan_bm() call further, after starting the TBM iterator, we could skip it altogether when the iterator is e

Re: Built-in CTYPE provider

2024-03-19 Thread Peter Eisentraut
* v25-0001-Address-more-review-comments-on-commit-2d819a08a.patch This was committed. * v25-0002-Support-C.UTF-8-locale-in-the-new-builtin-collat.patch Looks ok. * v25-0003-Inline-basic-UTF-8-functions.patch ok * v25-0004-Use-version-for-builtin-collations.patch Not sure about the version f

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-19 Thread Bertrand Drouvot
Hi, On Tue, Mar 19, 2024 at 04:20:35PM +0530, Amit Kapila wrote: > On Tue, Mar 19, 2024 at 3:11 PM Bertrand Drouvot > wrote: > > > > On Tue, Mar 19, 2024 at 10:56:25AM +0530, Amit Kapila wrote: > > > On Mon, Mar 18, 2024 at 8:19 PM Bertrand Drouvot > > > wrote: > > > > Agree. While it makes sens

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-03-19 Thread stephane tachoires
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, I have failing tap test after patches apply: ok 201 + partitio

Re: DOCS: add helpful partitioning links

2024-03-19 Thread Robert Treat
On Tue, Mar 19, 2024 at 3:08 AM Ashutosh Bapat wrote: > > Hi Robert, > > > On Mon, Mar 18, 2024 at 10:52 PM Robert Treat wrote: >> >> On Thu, Mar 14, 2024 at 12:15 PM Ashutosh Bapat >> wrote: >> > >> > Hi Robert, >> > >> > On Thu, Mar 7, 2024 at 10:49 PM Robert Treat wrote: >> >> >> >> This pat

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Jelte Fennema-Nio
On Mon, 18 Mar 2024 at 18:27, Robert Haas wrote: > I think for now we > should just file this under "Other platforms and clients," which only > has one existing setting. If the number of settings of this type > grows, we can split it out. Done. I also included a patch to rename COMPAT_OPTIONS_CLI

Re: minor tweak to catalogs.sgml pg_class.reltablespace

2024-03-19 Thread Tom Lane
Alvaro Herrera writes: > While reviewing the patch for SET ACCESS METHOD[1] I noticed that > pg_class.relam is not documented fully for partitioned tables, so I > proposed the attached. The bit about "(Not meaningful if the relation has no on-disk file.)" is not correct, and now it's adjacent to

Re: Reducing connection overhead in pg_upgrade compat check phase

2024-03-19 Thread Daniel Gustafsson
> On 19 Mar 2024, at 08:07, Peter Eisentraut wrote: > > On 18.03.24 13:11, Daniel Gustafsson wrote: >> Attached is a fresh rebase with only minor cosmetic touch-ups which I would >> like to go ahead with during this CF. >> Peter: does this address the comments you had on translation and code >> d

Re: documentation structure

2024-03-19 Thread Tom Lane
Daniel Gustafsson writes: > It's actually not very odd, the reference section is using > elements > and we had missed the arabic numerals setting on those. The attached fixes > that for me. That being said, we've had roman numerals for the reference > section since forever (all the way down to

Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
Hey, I'm trying to build a postgres export tool that reads data from table pages and exports it to an S3 bucket. I'd like to avoid manual commands like pg_dump, I need access to the raw data. Can you please point me to the postgres source header / cc files that encapsulate this functionality? -

Re: Table AM Interface Enhancements

2024-03-19 Thread Japin Li
On Tue, 19 Mar 2024 at 21:05, Alexander Korotkov wrote: > Hi, Pavel! > > On Tue, Mar 19, 2024 at 11:34 AM Pavel Borisov wrote: >> On Tue, 19 Mar 2024 at 03:34, Alexander Korotkov >> wrote: >>> >>> On Sun, Mar 3, 2024 at 1:50 PM Alexander Korotkov >>> wrote: >>> > On Mon, Nov 27, 2023 at 10:

Re: Q: Escapes in jsonpath Idents

2024-03-19 Thread David E. Wheeler
On Mar 17, 2024, at 20:09, Erik Wienhold wrote: > > On 2024-03-17 20:50 +0100, David E. Wheeler wrote: >> On Mar 17, 2024, at 15:12, Erik Wienhold wrote: >>> So I think it makes sense to reword the entire backslash part of the >>> paragraph and remove references to JSON entirely. The attached p

Re: Read data from Postgres table pages

2024-03-19 Thread Alexander Korotkov
Hi On Tue, Mar 19, 2024 at 4:23 PM Sushrut Shivaswamy wrote: > I'm trying to build a postgres export tool that reads data from table pages > and exports it to an S3 bucket. I'd like to avoid manual commands like > pg_dump, I need access to the raw data. > > Can you please point me to the postgr

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-03-19 Thread stephane tachoires
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: not tested Spec compliant: not tested Documentation:not tested Sorry, tests passed when applying all patches. I planned to check without

Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
I'd like to read individual rows from the pages as they are updated and stream them to a server to create a copy of the data. The data will be rewritten to columnar format for analytics queries. On Tue, Mar 19, 2024 at 7:58 PM Alexander Korotkov wrote: > Hi > > On Tue, Mar 19, 2024 at 4:23 PM Su

Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
The binary I"m trying to create should automatically be able to read data from a postgres instance without users having to run commands for backup / pg_dump etc. Having access to the appropriate source headers would allow me to read the data. On Tue, Mar 19, 2024 at 8:03 PM Sushrut Shivaswamy < su

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Joe Conway
On 3/19/24 07:49, Andrew Dunstan wrote: On Tue, Mar 19, 2024 at 5:26 AM Heikki Linnakangas > wrote: I want to remind everyone of this from Gabriele's first message that started this thread: > At the moment, a possible workaround is that `ALTER SYSTEM` can

Re: Read data from Postgres table pages

2024-03-19 Thread Alexander Korotkov
On Tue, Mar 19, 2024 at 4:35 PM Sushrut Shivaswamy wrote: > The binary I"m trying to create should automatically be able to read data > from a postgres instance without users having to > run commands for backup / pg_dump etc. > Having access to the appropriate source headers would allow me to rea

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-03-19 Thread Dmitry Koval
Hi! The following review has been posted through the commitfest application: make installcheck-world: tested, passed Thanks for info! I was unable to reproduce the problem and I wanted to ask for clarification. But your message was ahead of my question. -- With best regards, Dmitry Koval

Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
If we query the DB directly, is it possible to know which new rows have been added since the last query? Is there a change pump that can be latched onto? I’m assuming the page data structs are encapsulated in specific headers which can be used to list / read pages. Why would Postgres need to be

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Tom Lane
Heikki Linnakangas writes: > Perhaps we could make that even better with a GUC though. I propose a > GUC called 'configuration_managed_externally = true / false". If you set > it to true, we prevent ALTER SYSTEM and make the error message more > definitive: > postgres=# ALTER SYSTEM SET wal_le

Re: Streaming I/O, vectored I/O (WIP)

2024-03-19 Thread Heikki Linnakangas
Some quick comments: On 12/03/2024 15:02, Thomas Munro wrote: src/backend/storage/aio/streaming_read.c src/include/storage/streaming_read.h Standard file header comments missing. It would be nice to have a comment at the top of streaming_read.c, explaining at a high level how the circular bu

Re: Read data from Postgres table pages

2024-03-19 Thread Alexander Korotkov
On Tue, Mar 19, 2024 at 4:48 PM Sushrut Shivaswamy wrote: > > If we query the DB directly, is it possible to know which new rows have been > added since the last query? > Is there a change pump that can be latched onto? Please, check this. https://www.postgresql.org/docs/current/logicaldecoding.

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2024-03-19 Thread Alexander Korotkov
Hi, Pavel! On Tue, Nov 28, 2023 at 11:00 AM Pavel Borisov wrote: > > You're designing new APIs, days before the feature freeze. > On Wed, 5 Apr 2023 at 06:54, Michael Paquier wrote: > > > > On Tue, Apr 04, 2023 at 01:25:46AM +0300, Alexander Korotkov wrote: > > > Pavel, thank you for you review,

Re: pg_upgrade --copy-file-range

2024-03-19 Thread Tomas Vondra
Hi, I took a quick look at the remaining part adding copy_file_range to pg_combinebackup. The patch no longer applies, so I had to rebase it. Most of the issues were trivial, but I had to fix a couple missing prototypes - I added them to copy_file.h/c, mostly. 0001 is the minimal rebase + those f

Re: speed up a logical replica setup

2024-03-19 Thread vignesh C
On Mon, 18 Mar 2024 at 16:36, Peter Eisentraut wrote: > > On 18.03.24 08:18, vignesh C wrote: > > 1) Maximum size of the object name is 64, we can have a check so that > > we don't specify more than the maximum allowed length: > > + case 3: > > + if (!simple_string_list_member(&opt.replslot_names,

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Jelte Fennema-Nio
On Tue, 19 Mar 2024 at 15:52, Tom Lane wrote: > I like this idea. The "bonus" is not optional though, because > setting the files' ownership/permissions is the only way to be > sure that the prohibition is even a little bit bulletproof. I don't agree with this. The only "normal" way of modifying

Re: DRAFT: Pass sk_attno to consistent function

2024-03-19 Thread Michał Kłeczek
Hi All, Since it looks like there is not much interest in the patch I will try to provide some background to explain why I think it is needed. We are in the process of migration from an old db platform to PostgreSQL. Our database is around 10TB big and contains around 10 billion financial trans

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Tom Lane
Jelte Fennema-Nio writes: > On Tue, 19 Mar 2024 at 15:52, Tom Lane wrote: >> I like this idea. The "bonus" is not optional though, because >> setting the files' ownership/permissions is the only way to be >> sure that the prohibition is even a little bit bulletproof. > I don't agree with this.

Re: Reducing output size of nodeToString

2024-03-19 Thread Peter Eisentraut
On 11.03.24 21:52, Matthias van de Meent wrote: * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch This looks sensible, but maybe making Location a global type is a bit much? Maybe something more specific like ParseLocation, or ParseLoc, to keep it under 12 characters. I've gone

Re: speed up a logical replica setup

2024-03-19 Thread Peter Eisentraut
On 19.03.24 16:24, vignesh C wrote: The problem with this failure is that standby has been promoted already and we will have to re-create the physica replica again. If you are not planning to have the checks for name length, this could alternatively be fixed by including database id also while q

Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
> > > lol, thanks for the inputs Alexander :)!

Re: add AVX2 support to simd.h

2024-03-19 Thread Nathan Bossart
On Tue, Mar 19, 2024 at 04:53:04PM +0700, John Naylor wrote: > On Tue, Mar 19, 2024 at 10:16 AM Nathan Bossart > wrote: >> 0002 does the opposite of this. That is, after we've completed as many >> blocks as possible, we move the iterator variable back to "end - >> block_size" and do one final ite

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-19 Thread Masahiko Sawada
On Tue, Mar 19, 2024 at 6:40 PM John Naylor wrote: > > On Tue, Mar 19, 2024 at 10:24 AM Masahiko Sawada > wrote: > > > > On Tue, Mar 19, 2024 at 8:35 AM John Naylor wrote: > > > > > > On Mon, Mar 18, 2024 at 11:12 AM Masahiko Sawada > > > wrote: > > > > > > > > On Sun, Mar 17, 2024 at 11:46 A

Re: Improving EXPLAIN's display of SubPlan nodes

2024-03-19 Thread Tom Lane
I wrote: > I won't update the patch right now, but "(rescan SubPlan N)" > seems like a winner to me. Here's a hopefully-final version that makes that adjustment and tweaks a couple of comments. regards, tom lane diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out

Re: Reducing output size of nodeToString

2024-03-19 Thread Matthias van de Meent
On Tue, 19 Mar 2024 at 17:13, Peter Eisentraut wrote: > > On 11.03.24 21:52, Matthias van de Meent wrote: > >> * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch > >> > >> This looks sensible, but maybe making Location a global type is a bit > >> much? Maybe something more specific

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Jelte Fennema-Nio
On Tue, 19 Mar 2024 at 17:05, Tom Lane wrote: > I've said this repeatedly: it's not enough. The only reason we need > any feature whatsoever is that somebody doesn't trust their database > superusers to not try to modify the configuration. And as everyone else on this thread has said: It is enou

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Greg Sabino Mullane
On Tue, Mar 19, 2024 at 12:05 PM Tom Lane wrote: > If you aren't willing to build a solution that blocks off mods > using COPY TO FILE/PROGRAM and other readily-available-to-superusers > tools (plpythonu for instance), I think you shouldn't bother asking > for a feature at all. Just trust your s

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-19 Thread Kartyshov Ivan
Intro == The main purpose of the feature is to achieve read-your-writes-consistency, while using async replica for reads and primary for writes. In that case lsn of last modification is stored inside application. We cannot store this lsn inside database, since reads are distributed across

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Daniel Gustafsson
> On 19 Mar 2024, at 17:53, Jelte Fennema-Nio wrote: > > On Tue, 19 Mar 2024 at 17:05, Tom Lane wrote: >> I've said this repeatedly: it's not enough. The only reason we need >> any feature whatsoever is that somebody doesn't trust their database >> superusers to not try to modify the configurat

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Magnus Hagander
On Tue, Mar 19, 2024 at 3:52 PM Tom Lane wrote: > > Heikki Linnakangas writes: > > Perhaps we could make that even better with a GUC though. I propose a > > GUC called 'configuration_managed_externally = true / false". If you set > > it to true, we prevent ALTER SYSTEM and make the error message

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Daniel Gustafsson
> On 19 Mar 2024, at 15:51, Tom Lane wrote: > > Heikki Linnakangas writes: >> Perhaps we could make that even better with a GUC though. I propose a >> GUC called 'configuration_managed_externally = true / false". If you set >> it to true, we prevent ALTER SYSTEM and make the error message more

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-19 Thread Nathan Bossart
On Mon, Mar 11, 2024 at 04:59:36PM +0100, Alvaro Herrera wrote: > All in all, I support the original patch. I'll commit this in a few days if there are no objections. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: cleanup patches for incremental backup

2024-03-19 Thread Nathan Bossart
On Thu, Mar 14, 2024 at 08:52:55PM -0500, Nathan Bossart wrote: > Subject: [PATCH v1 1/2] Revert "Temporary patch to help debug pg_walsummary > test failures." > Subject: [PATCH v1 2/2] Fix possible overflow in MaybeRemoveOldWalSummaries(). Assuming there are no objections or feedback, I plan to

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-03-19 Thread Tom Lane
Richard Guo writes: > Here is the patch for HEAD. I simply re-posted v10. Nothing has > changed. I got back to this finally, and pushed it with some minor cosmetic adjustments. regards, tom lane

Re: Proposal to include --exclude-extension Flag in pg_dump

2024-03-19 Thread Ayush Vatsa
> I ran it through pgindent to fix some whitespace issues and added > another test for the filter option, based on the test case you added. Thank you for addressing those whitespaces issues and adding more tests. I appreciate your attention to detail and will certainly be more vigilant in future.

Why is parula failing?

2024-03-19 Thread Tom Lane
For the last few days, buildfarm member parula has been intermittently failing the partition_prune regression test, due to unexpected plan changes [1][2][3][4]. The symptoms can be reproduced exactly by inserting a "vacuum" of one or another of the partitions of table "ab", so we can presume that

Re: Popcount optimization using AVX512

2024-03-19 Thread Nathan Bossart
On Tue, Mar 19, 2024 at 12:30:50PM +1300, David Rowley wrote: > Looks good. Committed. Thanks for the suggestion and for reviewing! Paul, I suspect your patches will need to be rebased after commit cc4826d. Would you mind doing so? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Vectored I/O in bulk_write.c

2024-03-19 Thread Thomas Munro
On Sun, Mar 17, 2024 at 8:10 AM Andres Freund wrote: > I don't think zeroextend on the one hand and and on the other hand a normal > write or extend are really the same operation. In the former case the content > is hard-coded in the latter it's caller provided. Sure, we can deal with that > by sp

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-19 Thread Nathan Bossart
On Tue, Mar 19, 2024 at 01:47:53PM -0500, Nathan Bossart wrote: > On Mon, Mar 11, 2024 at 04:59:36PM +0100, Alvaro Herrera wrote: >> All in all, I support the original patch. > > I'll commit this in a few days if there are no objections. Actually, I just took a look at the patch and it appears to

Re: Partial aggregates pushdown

2024-03-19 Thread Bruce Momjian
On Sat, Mar 16, 2024 at 02:28:50AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > Hi. Mr.Pyhalov. > > > From: Alexander Pyhalov Sent: Wednesday, > > February 28, 2024 10:43 PM > > > 1. Transmitting state value safely between machines > > >> From: Robert Haas Sent: Wednesday, > > >> Decem

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-19 Thread Tomas Vondra
On 3/18/24 16:55, Tomas Vondra wrote: > > ... > > OK, I've restarted the tests for only 0012 and 0014 patches, and I'll > wait for these to complete - I don't want to be looking for patterns > until we have enough data to smooth this out. > > I now have results for 1M and 10M runs on the two bui

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread walther
Greg Sabino Mullane: On Tue, Mar 19, 2024 at 12:05 PM Tom Lane > wrote: If you aren't willing to build a solution that blocks off mods using COPY TO FILE/PROGRAM and other readily-available-to-superusers tools (plpythonu for instance), I think you shouldn't

Re: Improving EXPLAIN's display of SubPlan nodes

2024-03-19 Thread Dean Rasheed
On Tue, 19 Mar 2024 at 16:42, Tom Lane wrote: > > Here's a hopefully-final version that makes that adjustment and > tweaks a couple of comments. > This looks very good to me. One final case that could possibly be improved is this one from aggregates.out: explain (verbose, costs off) select arra

Re: Partial aggregates pushdown

2024-03-19 Thread Tom Lane
Bruce Momjian writes: > The current patch has: > if ((OidIsValid(aggform->aggfinalfn) || > (aggform->aggtranstype == INTERNALOID)) && > fpinfo->check_partial_aggregate_support) > { > if (fpinfo->remoteversion == 0) > { >

Re: documentation structure

2024-03-19 Thread Andrew Dunstan
On Mon, Mar 18, 2024 at 10:12 AM Robert Haas wrote: > I was looking at the documentation index this morning[1], and I can't > help feeling like there are some parts of it that are over-emphasized > and some parts that are under-emphasized. I'm not sure what we can do > about this exactly, but I t

Re: Improving EXPLAIN's display of SubPlan nodes

2024-03-19 Thread Tom Lane
Dean Rasheed writes: > One final case that could possibly be improved is this one from > aggregates.out: > explain (verbose, costs off) > select array(select sum(x+y) s > from generate_series(1,3) y group by y order by s) > from generate_series(1,3) x; >

Re: Improving EXPLAIN's display of SubPlan nodes

2024-03-19 Thread Dean Rasheed
On Tue, 19 Mar 2024 at 21:40, Tom Lane wrote: > > I'm inclined to leave that alone. The actual source sub-SELECT > could only have had one column, so no real confusion is possible. > Yeah, there's a resjunk grouping column visible in the plan as well, > but those exist in many other queries, and

Re: Improving EXPLAIN's display of SubPlan nodes

2024-03-19 Thread Tom Lane
Dean Rasheed writes: > Fair enough. I have no further comments. Pushed then. Thanks for reviewing! I gave you credit as co-author. regards, tom lane

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Andrew Dunstan
On Tue, Mar 19, 2024 at 2:28 PM Magnus Hagander wrote: > On Tue, Mar 19, 2024 at 3:52 PM Tom Lane wrote: > > > > Heikki Linnakangas writes: > > > Perhaps we could make that even better with a GUC though. I propose a > > > GUC called 'configuration_managed_externally = true / false". If you > se

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-19 Thread Kartyshov Ivan
Bharath Rupireddy, thank you for you review. But here is some points. On 2024-03-16 10:02, Bharath Rupireddy wrote: 4.1 With invalid LSN succeeds, shouldn't it error out? Or at least, add a fast path/quick exit to WaitForLSN()? BEGIN AFTER '0/0'; In postgresql '0/0' is Valid pg_lsn, but it is

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Tom Lane
Andrew Dunstan writes: > On Tue, Mar 19, 2024 at 2:28 PM Magnus Hagander wrote: >> Windows has had full ACL support since 1993. The easiest way to do >> what you're doing here is to just set a DENY permission on the >> postgres operating system user. > Yeah. See < > https://learn.microsoft.com/

RE: Popcount optimization using AVX512

2024-03-19 Thread Amonson, Paul D
> -Original Message- > From: Nathan Bossart > > Committed. Thanks for the suggestion and for reviewing! > > Paul, I suspect your patches will need to be rebased after commit cc4826d. > Would you mind doing so? Changed in this patch set. * Rebased. * Direct *slow* calls via macros as sh

Re: Partial aggregates pushdown

2024-03-19 Thread Bruce Momjian
On Tue, Mar 19, 2024 at 05:29:07PM -0400, Tom Lane wrote: > I'd like to vociferously protest both of those decisions. > > "No version check by default" means "unsafe by default", which is not > project style in general and is especially not so for postgres_fdw. > We have tried very hard for years

  1   2   >