Re: pg_dump/pg_dumpall help synopses and terminology

2025-06-10 Thread Ashutosh Bapat
e or a PostgreSQL cluster from an archive created by pg_dump or pg_dumpall respectively. [1] https://www.postgresql.org/docs/18/app-pgrestore.html -- Best Wishes, Ashutosh Bapat

Re: Add enable_groupagg GUC parameter to control GroupAggregate usage

2025-06-09 Thread Ashutosh Bapat
resolving those misestimates would ideally improve the root issue, > I'd like to keep the focus of this thread on adding the GUC parameter. > Then, I plan to report or address the estimation problem in a separate > thread. > > +1. -- Best Wishes, Ashutosh Bapat

Re: pg_get_multixact_members not documented

2025-06-05 Thread Ashutosh Bapat
s involved in multixacts. > +1. Multixacts are highly under documented in user facing as well as internal documentations. Thanks for the patch. [1] https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-TABLES -- Best Wishes, Ashutosh Bapat diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/f

Re: Proposal: Make cfbot fail on patches not created by "git format-patch"

2025-05-29 Thread Ashutosh Bapat
ts will > still run, but people would get a failing CI job which tells them that > something is wrong. We could also include a pgindent in that CI job. > > > +1. Knowing whether to use git am or patch to apply the patch itself will save reviewers' time. -- Best Wishes, Ashutosh Bapat

Re: Changing shared_buffers without restart

2025-05-09 Thread Ashutosh Bapat
d right after the resize ftruncate. > > I thought I had shared a test program upthread, but I don't find it now. Attached here. Can you please share your test program? There are concerns around portability of this approach, though. -- Best Wishes, Ashutosh Bapat #define _GNU_SOURC

Re: POC: make mxidoff 64 bits

2025-04-29 Thread Ashutosh Bapat
to document how pg_multixact/members and pg_multixact/offset are used and what is their layout. It's not the responsibility of this patchset to document it, but it will be good if we add a section about multixacts in transam/README. It will make reviews easier. -- Best Wishes, Ashutosh Bapat

Re: Does RENAME TABLE rename associated identity sequence?

2025-04-25 Thread Ashutosh Bapat
On Thu, Apr 24, 2025 at 11:10 PM Tom Lane wrote: > > I think it's up to the user to rename subsidiary objects if > they wish to do so. > I absolutely agree. -- Best Wishes, Ashutosh Bapat

Re: Does RENAME TABLE rename associated identity sequence?

2025-04-24 Thread Ashutosh Bapat
R TABLE ... RENAME ... should rename the sequence too since the identity sequences are created implicitly when the table is created, so they should be renamed implicitly. We should not require WITH SEQUENCE clause. -- Best Wishes, Ashutosh Bapat

Re: bug: virtual generated column can be partition key

2025-04-22 Thread Ashutosh Bapat
mething to the effect "partition key expression contains a whole-row reference which in turn contains a generated column." But that might be too verbose. -- Best Wishes, Ashutosh Bapat

Re: bug: virtual generated column can be partition key

2025-04-22 Thread Ashutosh Bapat
Thanks Jian for your review. On Tue, Apr 22, 2025 at 12:32 PM jian he wrote: > On Tue, Apr 22, 2025 at 11:45 AM Ashutosh Bapat > wrote: > > > > I have included your original tests, but ended up rewriting code. Please > let me know what do you think. > > > > +

Re: bug: virtual generated column can be partition key

2025-04-21 Thread Ashutosh Bapat
On Tue, Apr 22, 2025 at 11:19 AM jian he wrote: > On Tue, Apr 22, 2025 at 11:45 AM Ashutosh Bapat > wrote: > > > > > > While looking at this I realised that a generated column may end up > being part of the partition key if the partition key expression contains a >

Re: bug: virtual generated column can be partition key

2025-04-21 Thread Ashutosh Bapat
sions. I have included your original tests, but ended up rewriting code. Please let me know what do you think. -- Best Wishes, Ashutosh Bapat From 2adb796b1fdfa5e72c7302e4ecaf7d69b105d0f3 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Mon, 21 Apr 2025 18:06:58 +0530 Subject: [PATCH] Tighten

Re: Changing shared_buffers without restart

2025-04-17 Thread Ashutosh Bapat
Hi Dmitry, On Mon, Apr 14, 2025 at 12:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Mon, Apr 14, 2025 at 10:40:28AM GMT, Ashutosh Bapat wrote: > > > > However, when we put back the patches to shrink buffers, we will evict > > the extra buffers,

Re: pg_dump: Fix dangling pointer in EndCompressorZstd()

2025-04-16 Thread Ashutosh Bapat
ssionState::end() EndCompressorGzip() calls DeflateCompressorEnd() which also sets cs->private_data explicitly. So should EndCompressorZstd(). -- Best Wishes, Ashutosh Bapat

Re: Fundamental scheduling bug in parallel restore of partitioned tables

2025-04-16 Thread Ashutosh Bapat
On Wed, Apr 16, 2025 at 12:19 AM Tom Lane wrote: > > Ashutosh Bapat writes: > > On Mon, Apr 14, 2025 at 11:14 PM Tom Lane wrote: > >> This is disastrous for assorted reasons. The ALTER ADD CONSTRAINT > >> command might fail outright if we've loaded data for

Re: Built-in Raft replication

2025-04-15 Thread Ashutosh Bapat
On Wed, Apr 16, 2025 at 11:57 AM Andrey Borodin wrote: > > > > > On 16 Apr 2025, at 11:18, Ashutosh Bapat > > wrote: > > > > Having only some of the nodes act as DCS > > and others purely PostgreSQL nodes will reduce waste of resources. > > But typi

Re: Built-in Raft replication

2025-04-15 Thread Ashutosh Bapat
that had no real PGDATA, only participates in voting. But this is a > solution to problem induced by built-in autofailover. Users find it a waste of resources to deploy 3 big PostgreSQL instances just for HA where 2 suffice even if they deploy 3 lightweight DCS instances. Having only some of the nodes act as DCS and others purely PostgreSQL nodes will reduce waste of resources. -- Best Wishes, Ashutosh Bapat

Re: Built-in Raft replication

2025-04-15 Thread Ashutosh Bapat
n logical replication or FDW or both will use this ensemble to manage its shared state. The same ensemble can be shared across multiple distributed clusters if it has scaling capabilities. -- Best Wishes, Ashutosh Bapat

Re: Fundamental scheduling bug in parallel restore of partitioned tables

2025-04-15 Thread Ashutosh Bapat
ef_fkey ... snip pg_restore: finished item 2625 TABLE DATA c21 I tried applying your patch on v17 to see whether it causes the FK creation to wait, but the patch doesn't apply cleanly. -- Best Wishes, Ashutosh Bapat

Re: Changing shared_buffers without restart

2025-04-14 Thread Ashutosh Bapat
On Mon, Apr 14, 2025 at 12:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Mon, Apr 14, 2025 at 10:40:28AM GMT, Ashutosh Bapat wrote: > > > > However, when we put back the patches to shrink buffers, we will evict > > the extra buffers, and shri

Re: Changing shared_buffers without restart

2025-04-13 Thread Ashutosh Bapat
his work for shrinking buffers? What > > scheme do you have in mind? > > I didn't like this part originally, and after changes to support hugetlb > I think it's worth it just to replace mremap with munmap/mmap. That way > there will be no such question, e.g. if a segment is getting shrinked > the unmapped area will again become a part of the reserved space. > I might have not noticed it, but are we putting two mappings one reserved and one allocated in the same address space, so that when the allocated mapping shrinks or expands, the reserved mapping continues to prohibit any other mapping from appearing there? I looked at some of the previous emails, but didn't find anything that describes how the reserved mapped space is managed. -- Best Wishes, Ashutosh Bapat

Re: Fix replica identity checks for MERGE command on published table.

2025-04-13 Thread Ashutosh Bapat
d, whether MERGE should be considered a separate operation as far as publication is concerned. This topic may have been discussed either when MERGE was implemented or when operation filters were implemented. Sorry for the noise in that case. This isn't a detailed review. -- Best Wishes, Ashutosh Bapat

Re: New committer: Jacob Champion

2025-04-13 Thread Ashutosh Bapat
newest PostgreSQL > > committer. > > > > Please join us in wishing Jacob much success and few reverts! > > Congratulations Jacob! Well deserved! > > Thanks > Richard > > -- Best Wishes, Ashutosh Bapat

Re: Changing shared_buffers without restart

2025-04-11 Thread Ashutosh Bapat
/* Resize the backing anon file. */ if(ftruncate(m->segment_fd, new_size) == -1) ... /* Reclaim the space */ ptr = mmap(m->shmem, new_size, PROT_READ | PROT_WRITE, mmap_flags | MAP_FIXED, m->segment_fd, 0); How are we preventing something get mapped into the space after m->shmem + newsize? We will need to add an unallocated but reserved addressed space map after m->shmem+newsize right? -- Best Wishes, Ashutosh Bapat

Re: Changing shared_buffers without restart

2025-04-09 Thread Ashutosh Bapat
On Wed, Apr 9, 2025 at 1:15 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Wed, Apr 09, 2025 at 11:12:18AM GMT, Ashutosh Bapat wrote: > > On Mon, Apr 7, 2025 at 2:13 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > > > In the new v4 v

Re: Changing shared_buffers without restart

2025-04-08 Thread Ashutosh Bapat
| ^~~ In file included from ../../coderoot/pg/src/port/pg_numa.c:26: ../../coderoot/pg/src/include/storage/pg_shmem.h:127:13: note: declared here 127 | extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags, | ^~~ -- Best Wishes, Ashutosh Bapat

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-08 Thread Ashutosh Bapat
relevant structures seems good enough, not a change that should bubble up to the README. The only change in the EC interface is the addition of a new iterator method - maybe we could document that in README but that too seems more detail than what README is about. -- Best Wishes, Ashutosh Bapat

Re: Feature freeze

2025-04-08 Thread Ashutosh Bapat
time of day, it doesn't make > intuitive sense. +1. I think the idea is too simple to be acceptable. Any timezone based deadline might be seen as unfair to those for whom that time falls in their respective nights. AoE removes that unfairness. -- Best Wishes, Ashutosh Bapat

Re: [PATCH] clarify palloc comment on quote_literal_cstr

2025-04-08 Thread Ashutosh Bapat
On Tue, Apr 8, 2025 at 10:49 AM Michael Paquier wrote: > > On Tue, Apr 08, 2025 at 08:47:53AM +0530, Ashutosh Bapat wrote: > > Thanks for addressing the comment. > > > > In PG code, we start a multiline comment with just /* on the first > > line and end with jus

Re: [PATCH] clarify palloc comment on quote_literal_cstr

2025-04-08 Thread Ashutosh Bapat
eparate lines. -- Best Wishes, Ashutosh Bapat

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread Ashutosh Bapat
On Tue, Apr 8, 2025 at 8:30 AM David Rowley wrote: > > On Tue, 8 Apr 2025 at 04:54, Ashutosh Bapat > wrote: > > - foreach(lc2, cur_ec->ec_members) > > + setup_eclass_member_iterator(&it, cur_ec, rel); > > + while ((cur_em = eclass_member_iterator_next(&it))

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread Ashutosh Bapat
asses(EquivalenceClass *ec, int from, int to) List *new_members = NIL; List *new_sources = NIL; + /* + * We don't expect any EC child members to exist at this point. Ensure + * that's the case, otherwise we might be getting asked to do something + * this function hasn't been coded for. + */ + Assert(ec->ec_childmembers == NULL); + Similar to above. -- Best Wishes, Ashutosh Bapat

Re: [PATCH] clarify palloc comment on quote_literal_cstr

2025-04-07 Thread Ashutosh Bapat
tes, in case every character is a quote, b. two bytes for two outer quotes c. extra byte for E' d. one byte for null terminator. */ -- Best Wishes, Ashutosh Bapat

Re: Changing shared_buffers without restart

2025-04-06 Thread Ashutosh Bapat
On Fri, Feb 28, 2025 at 5:31 PM Ashutosh Bapat wrote: > > I think we should add a way to monitor the progress of resizing; at > least whether resizing is complete and whether the new GUC value is in > effect. > I further tested this approach by tracing the barrier synchroniz

Re: SQL Property Graph Queries (SQL/PGQ)

2025-04-06 Thread Ashutosh Bapat
; the variable length edge? > > 3. What if the query doesn't specify the Label of edge, and there can > be different edge labels of r, can we easily do the rewrite? > > I did some study of the apache age, they have fixed columns for node > labels(id, agtype) > and edge labels(id, source_id, end_id, agtype), agtype is kind of > json. So they can > resolve the above question easily. > > Above are just my random thoughts of the quantifier feature, I don't have a > copy > of the PGQ standard, so I'd like to hear your opinion about this. > I think the questions you have raised are valid. If we decide to discuss this in a separate thread, I will start that thread just by responding to these questions and design I have in mind. -- Best Wishes, Ashutosh Bapat

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-05 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 12:01 PM Ashutosh Bapat wrote: > > Comparing root->join_rel_hash with EC->ec_derives_hash in the context > of initial hash table size is a thinko on my part. It's less likely > that there will be 1000 subqueries (requiring 1000 PlannerInfos) each &

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-05 Thread Ashutosh Bapat
he size of ec_derives_list when creating the hash table and we are using simplehash which uses its own fillfactor and its own logic to expand the hash table, I think we should just use the length of ec_derives_list as the initial size. What do you think? -- Best Wishes, Ashutosh Bapat

Detach partition with constraint test

2025-04-05 Thread Ashutosh Bapat
nt in the same block. -- Best Wishes, Ashutosh Bapat From c0305c8572c9c416da45b9a14146be049cf91102 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Thu, 3 Apr 2025 17:29:49 +0530 Subject: [PATCH 1/2] Test detaching a partition with foreign key constraint A partition of a partitioned table with a fo

Re: Test to dump and restore objects left behind by regression

2025-04-05 Thread Ashutosh Bapat
On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera wrote: > > On 2025-Mar-24, Ashutosh Bapat wrote: > > > One concern I have with directory format is the dumped database is not > > readable. This might make investigating a but identified the test a > > bit more complex

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-05 Thread Ashutosh Bapat
e generation is not accurate, since we lookup EM = constant clauses as well which are not join clauses. 2. In the second paragraph em1 and em2 are mentioned without mentioning what are they. I have rephrased it so as to avoid mentioning names of structure member. -- Best Wishes,

Re: Test to dump and restore objects left behind by regression

2025-04-05 Thread Ashutosh Bapat
ing all formats when running test as opt-in. Alvaro and Vignesh are for just one format. We need a tie-breaker or someone amongst us needs to change their vote :D. -- Best Wishes, Ashutosh Bapat

statistics import and export: another difference in dump/restore

2025-04-05 Thread Ashutosh Bapat
m_bounds', '{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100}'::text, + 'correlation', '1'::real +); First 3 and 5th differences are new differences but we have seen something similar to the 4th one previously. I am not able to reproduce the failure even after running it 30 times on my linux laptop. [1] https://www.postgresql.org/message-id/caexhw5vvftcejh+uyznxmgsxofj_1xwi5aqhqfemqjgfmky...@mail.gmail.com [2] https://cirrus-ci.com/task/5164175841820672 -- Best Wishes, Ashutosh Bapat

Re: 002_pg_upgrade is broken for custom install

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 5:23 PM Álvaro Herrera wrote: > > On 2025-Apr-04, Ashutosh Bapat wrote: > > > connection error: 'psql: error: connection to server on socket > > "/tmp/LiPa_UJpSb/.s.PGSQL.13779" failed: FATAL: database "regression" > > do

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-04 Thread Ashutosh Bapat
the patches improve performance of simple scans with thousands of partitions. Have you tried measuring that? -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Thu, Apr 3, 2025 at 1:50 PM Alvaro Herrera wrote: > > On 2025-Apr-03, Ashutosh Bapat wrote: > > > Looks like the problem is in the test itself as pointed out by Jeff in > > [1]. PFA patch fixing the test and enabling statistics back. > > Thanks, pushed. Thanks.

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 2:04 PM Amit Langote wrote: > > On Thu, Apr 3, 2025 at 12:28 PM Amit Langote wrote: > > On Wed, Apr 2, 2025 at 9:52 PM Ashutosh Bapat > > wrote: > > > On Wed, Apr 2, 2025 at 1:00 PM Amit Langote > > > wrote: > > > > I&#x

Re: 002_pg_upgrade is broken for custom install

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 5:02 PM Álvaro Herrera wrote: > > On 2025-Apr-04, Ashutosh Bapat wrote: > > > # test failed > > stderr: > > # Tests were run but no plan was declared and done_testing() was not seen. > > # Looks like your test exited with 29 just after

002_pg_upgrade is broken for custom install

2025-04-04 Thread Ashutosh Bapat
as far as f52345995d3691de7b0a888903b6cfed1497c672, the failure still appears. IIRC, the test was working fine till October 2024. But I might be wrong. Don't have energy to try git blame right now, but might do once I get some time. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 4:41 PM Ashutosh Bapat wrote: > > On Thu, Apr 3, 2025 at 10:44 PM Alvaro Herrera > wrote: > > > > On 2025-Apr-03, Andres Freund wrote: > > > > > I've increased the timeout even further, but I can't say that I am happy &

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
e test added 9 seconds on my laptop and we gained back 3 seconds; doesn't seem bad. It will show a significant difference in valgrind run. -- Best Wishes, Ashutosh Bapat From ef7880e1f1774c6875f4b265287f70d844e20ca4 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Fri, 4 Apr 2025 14:26:48 +05

Re: two occurrences of assign print_notnull within pg_dump.c

2025-04-03 Thread Ashutosh Bapat
her. I wonder why. - probably just an oversight or define to closer to usage. But we don't do the latter. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
On Wed, Apr 2, 2025 at 3:36 PM Ashutosh Bapat wrote: > > > > No commitfest entry please. Better to add an open item on the wiki > > page. > > https://wiki.postgresql.org/wiki/Open_Items > > Posted it on the thread where I have reported the bug. Hopefully, we >

Re: statistics import and export: another difference in dump/restore

2025-04-02 Thread Ashutosh Bapat
On Wed, Apr 2, 2025 at 10:55 PM Jeff Davis wrote: > > On Wed, 2025-04-02 at 15:35 +0530, Ashutosh Bapat wrote: > > Once we fix this issue, we need to enable statistics dump and > > comparison in pg_upgrade/002_pg_upgrade using the attached patch. > > The diff a

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
mistake. If possible let's keep this entry open for a few days. I am trying something so that we could remove --no-statistics as well. -- Best Wishes, Ashutosh Bapat

Re: pg_restore documentation and --create/--single-transaction limitation

2025-04-02 Thread Ashutosh Bapat
On Mon, Mar 24, 2025 at 5:11 PM Ashutosh Bapat wrote: > > Hi All, > The pg_restore documentation usually mentions the pair of switches > which can not be used together. However, it does not mention that > --create and --single-transaction can not be used together. Here's a >

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
Hi Alvaro, On Wed, Apr 2, 2025 at 2:49 PM Alvaro Herrera wrote: > > On 2025-Apr-02, Ashutosh Bapat wrote: > > > I have closed the CF entry > > https://commitfest.postgresql.org/patch/4564/ committed. I will > > create another CF entry to park --no-statistics revers

Re: statistics import and export: another difference in dump/restore

2025-04-02 Thread Ashutosh Bapat
Hi, On Tue, Apr 1, 2025 at 12:54 PM Ashutosh Bapat wrote: > [1] > https://www.postgresql.org/message-id/caexhw5vvftcejh+uyznxmgsxofj_1xwi5aqhqfemqjgfmky...@mail.gmail.com > [2] https://cirrus-ci.com/task/5164175841820672 I have added this to PG 18 open items. It might be too earl

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
On Tue, Apr 1, 2025 at 10:31 PM Alvaro Herrera wrote: > > On 2025-Apr-01, Ashutosh Bapat wrote: > > > Just today morning, I found something which looks like another bug in > > statistics dump/restore [1]. As Daniel has expressed upthread [2], we > > should go ahead and

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-01 Thread Ashutosh Bapat
perties to create canonical hash table key. Should we move them to ec_add_derived_clause() for better coverage? I have also removed some comments repeated in the function prologue and function body. 0001, 0002 and 0003 are the same as your set. 0004 contains my changes in a separate patch so that it&

Re: Test to dump and restore objects left behind by regression

2025-04-01 Thread Ashutosh Bapat
dbkq0xzcnjth7019zm...@mail.gmail.com -- Best Wishes, Ashutosh Bapat From 28a146c7cdaf581d889cec90f541bb42046e7904 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Mon, 24 Mar 2025 11:21:12 +0530 Subject: [PATCH 2/3] Use only one format and make the test run default According to Alvaro (and I agree wit

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-31 Thread Ashutosh Bapat
On Mon, Mar 31, 2025 at 8:27 AM David Rowley wrote: > > On Sat, 29 Mar 2025 at 05:46, Ashutosh Bapat > wrote: > > PFA patches. 0001 and 0002 are the same as the previous set. 0003 > > changes the initial hash table size to the length of ec_derives. > > I'm just n

Re: Test to dump and restore objects left behind by regression

2025-03-31 Thread Ashutosh Bapat
e order in which they are run. It will require some significant effort to carefully separate the statements. 2. With the new set of files backpatching would become hard. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-03-31 Thread Ashutosh Bapat
On Mon, Mar 31, 2025 at 5:07 PM Ashutosh Bapat wrote: > The bug related to materialized views has been fixed and now the test passes even if we compare statistics from dumped and restored databases. Hence removing 0003. In the attached patchset I have also addressed Vignesh's below com

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-29 Thread Ashutosh Bapat
are indexes or not and Yuya's first three patches are related to ec_members handling. So those show some improvement with David's script. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 12:20 PM Ashutosh Bapat wrote: > > On Fri, Mar 28, 2025 at 7:07 AM Michael Paquier wrote: > > > > On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote: > > > BTW another idea to shorten this tests's runtime might be to

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 4:05 PM Alvaro Herrera wrote: > > On 2025-Mar-28, Ashutosh Bapat wrote: > > > No, that's losing some information like default installation and the > > same version. > > You don't need to preserve such information. This is just a

Re: NOT ENFORCED constraint feature

2025-03-28 Thread Ashutosh Bapat
elation and the constraint. There are tests which mask variable parts of EXPLAIN output. Can we use similar trick to mask OIDs from the trigger names? -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
Vignesh and Alvaro On Fri, Mar 28, 2025 at 12:02 PM Ashutosh Bapat wrote: > > > > Maybe > > fail("roundtrip dump/restore of the regression database") > > No, that's losing some information like default installation and the > same version. How about

Re: Statistics import and export: difference in statistics of materialized view dumped

2025-03-28 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 10:41 AM Jeff Davis wrote: > > On Thu, 2025-03-27 at 17:07 +0530, Ashutosh Bapat wrote: > > Pulled the latest sources but the test is still failing with the same > > differences. > > The attached set of patches (your 0001 and 0002, combined

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
EST_EXTRA. (Hence I have kept 0001 and 0002 separate.) But I will be disappointed if the test, which has unearthed four bugs in a year alone, does not get committed to PG 18 because of this debate. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-03-27 Thread Ashutosh Bapat
On Thu, Mar 27, 2025 at 10:45 PM Alvaro Herrera wrote: > > On 2025-Mar-27, Ashutosh Bapat wrote: > > > On Thu, Mar 27, 2025 at 6:01 PM vignesh C wrote: > > > > Couple of minor thoughts: > > > 1) I felt this error message is not conveying the error message co

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-03-27 Thread Ashutosh Bapat
s comments discussed in [1]. Since there was a change in design, I am assuming that those will be addressed once the design change is accepted. [1] https://www.postgresql.org/message-id/202503121157.3zabg6m3anwp@alvherre.pgsql -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-03-27 Thread Ashutosh Bapat
On Thu, Mar 27, 2025 at 6:01 PM vignesh C wrote: > > On Tue, 25 Mar 2025 at 16:09, Ashutosh Bapat > wrote: > > > > On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera > > wrote: > > > > > > On 2025-Mar-24, Ashutosh Bapat wrote: > > > > >

Re: Options to control remote transactions’ access/deferrable modes in postgres_fdw

2025-03-27 Thread Ashutosh Bapat
plicate it along with the transaction properties. And also we need a test which tests readonly subtransaction behaviour. -- Best Wishes, Ashutosh Bapat

Re: Statistics import and export: difference in statistics of materialized view dumped

2025-03-27 Thread Ashutosh Bapat
On Wed, Mar 12, 2025 at 4:29 PM Ashutosh Bapat wrote: > > I ran my test with this patch (we have to remove 0003 patch in my test > which uses --no-statistics option). It failed with following > differences > @@ -452068,8 +452068,8 @@ > SELECT * FROM pg_catalog.pg_rest

Re: NOT ENFORCED constraint feature

2025-03-26 Thread Ashutosh Bapat
| | + a | integer | | | +Partition of: fk_partitioned_fk FOR VALUES IN (1500, 1502) +Foreign-key constraints: +TABLE "fk_partitioned_fk" CONSTRAINT "fk_partitioned_fk_a_b_fkey" FOREIGN KEY (a, b) REFERENCES fk_notpartitioned_pk(a, b) ON UPDATE CASCADE ON DELETE CASCADE NOT ENFORCED Did it just rename an existing constraint on fk_partitioned_fk_2? Is that ok? I thought we just mark the constraint as inherited. -- Best Wishes, Ashutosh Bapat

Re: [PoC] Reducing planning time when tables have many partitions

2025-03-26 Thread Ashutosh Bapat
adds on top of it. But without PWJ, the memory consumption is high, especially at higher number of joins and higher number of partitions. [1] https://www.postgresql.org/message-id/CAExHW5vNk4f8VSTnizL-avREYhPgocEtH3E7MCEJgp=r33c...@mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-26 Thread Ashutosh Bapat
tried partitions upto 256. Previous measurements have seen stable improvements in case of higher number of partitions. -- Best Wishes, Ashutosh Bapat

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-25 Thread Ashutosh Bapat
source_databases(&opt); + extra line, not needed This looks mostly ready except the test changes. I believe when committing, we are going to squash all three into a single commit. Is that correct? On Tue, Mar 25, 2025 at 4:37 PM Shubham Khanna wrote: > > On Tue, Mar 25, 2025 at 4:3

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-25 Thread Ashutosh Bapat
On Tue, Mar 25, 2025 at 5:20 PM Amit Langote wrote: > > On Tue, Mar 25, 2025 at 6:36 PM Ashutosh Bapat > wrote: > > On Tue, Mar 25, 2025 at 12:58 PM Amit Langote > > wrote: > > > Btw, about ec_clear_derived_clauses(): > > > > > > @@ -749,7 +749,

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-25 Thread Ashutosh Bapat
target server will receive a subscription. Only those which have the same name as a database on the source server. Am I correct? I prefer the previous wording, even if it's a bit complex, over a simpler but incorrect description. -- Best Wishes, Ashutosh Bapat

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-25 Thread Ashutosh Bapat
blisher-server > + > +connstr > > Most of this is unrelated to this patch. I suggest making a top-up > patch, we can commit it separately. Isn't this documenting -a/--all option in synopsis. Why do you think it's unrelated? -- Best Wishes, Ashutosh Bapat

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-25 Thread Ashutosh Bapat
On Tue, Mar 25, 2025 at 5:15 PM Amit Kapila wrote: > > On Tue, Mar 25, 2025 at 5:08 PM Ashutosh Bapat > wrote: > > > > This looks mostly ready except the test changes. I believe when > > committing, we are going to squash all three into a single commit. Is > >

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-25 Thread Ashutosh Bapat
On Tue, Mar 25, 2025 at 4:28 PM Shubham Khanna wrote: > > The attached patches contain the suggested changes. > [1] - > https://www.postgresql.org/message-id/CAA4eK1KUDEO0t6i16_CcEpg33sgcgEddHcdVC_q8j4tVUb5FWw%40mail.gmail.com Forgot to attach patches? -- Best Wishes, Ashutosh Bapat

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-25 Thread Ashutosh Bapat
_equivalence() would reuse the memory allocated to ec2->ec_derives_list but it doesn't. I verified that by setting the threshold to 0, thus forcing the hash table always and running a regression suite. It runs without any segfaults. I don't see any change in time required to run regress

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-24 Thread Ashutosh Bapat
evious sentence as " ... faster lookup when the list grows beyond a threshold" or something similar. The next sentence anyway mentions that both the structures are maintained. + We retain the list even + * when the hash is used to simplify serialization (e.g., in + * _outEquivalenceClass()) and

Re: Allow default \watch interval in psql to be configured

2025-03-24 Thread Ashutosh Bapat
able > linkend="app-psql-variables-watch-interval"/>." > > Fixed. > > > + " number of seconds \\watch waits between executing the query buffer\n"); > > > > I am feeling that this should mention "default" somewhere - maybe just > > add it before "number of ". > > Fixed. LGTM. I think this is RFC. Updated CF entry. -- Best Wishes, Ashutosh Bapat

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-24 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 6:59 PM Shubham Khanna wrote: > > On Thu, Mar 20, 2025 at 4:53 PM Ashutosh Bapat > wrote: > > > > On Thu, Mar 20, 2025 at 10:25 AM Shubham Khanna > > wrote: > > > > > > > > The attached patches contain the suggested

Re: Allow default \watch interval in psql to be configured

2025-03-24 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 2:15 AM Daniel Gustafsson wrote: > > > On 17 Mar 2025, at 13:37, Ashutosh Bapat > > wrote: > > > 0 is an accepted value for interval, even though it might look insensible. > > > > The behaviour should be same in both cases \watch i=

pg_restore documentation and --create/--single-transaction limitation

2025-03-24 Thread Ashutosh Bapat
e sections corresponding to both the switches so that a user reading either of them knows about the limitation. -- Best Wishes, Ashutosh Bapat From e9efa27688cb0f85825fd7cda02118818a1edeae Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Mon, 24 Mar 2025 16:59:16 +0530 Subject: [PATCH] pg_restore doc

Re: Test to dump and restore objects left behind by regression

2025-03-24 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 11:38 PM Alvaro Herrera wrote: > > On 2025-Mar-21, Ashutosh Bapat wrote: > > > I used the same parallelism in pg_restore and pg_dump too. And your > > numbers seem to be similar to mine; slightly less than 20% slowdown. > > But is that slowdown a

Re: Test to dump and restore objects left behind by regression

2025-03-22 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 6:04 PM Alvaro Herrera wrote: > > On 2025-Mar-21, Ashutosh Bapat wrote: > > > On Thu, Mar 20, 2025 at 8:37 PM vignesh C wrote: > > > > Should the copyright be only 2025 in this case: > > > The patch was posted in 2024 to this mailin

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread Ashutosh Bapat
est case is about > four seconds. In all cases I used the same parallelism in pg_restore > than pg_dump; not sure if that could cause a difference. I used the same parallelism in pg_restore and pg_dump too. And your numbers seem to be similar to mine; slightly less than 20% slowdown. But is that slowdown acceptable? From the earlier discussions, it seems the answer is No. Haven't heard otherwise. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread Ashutosh Bapat
2024-2025, PostgreSQL Global Development Group The patch was posted in 2024 to this mailing list. So we better protect the copyright since then. I remember a hackers discussion where a senior member of the community mentioned that there's not harm in mentioning longer copyright periods than being stricter about it. I couldn't find the discussion though. -- Best Wishes, Ashutosh Bapat

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-20 Thread Ashutosh Bapat
it's a good idea. Other tests may be testing replication but they don't test the setup created by --all, which consists of objects generated with names crafted by utility. Said that we should also test the behaviour when such objects already exist. But I guess randomness in generated names is hard to control. -- Best Wishes, Ashutosh Bapat

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-20 Thread Ashutosh Bapat
subnames1[1]); +$node_u->wait_for_subscription_sync($node_p, $subnames1[2]); + This function just makes sure that initial sync is done. But it doesn't wait for replication to catch up with current state of the publisher. It's the latter which would make sure that the last INSERTS are visible. We should be using wait_for_slot_catchup() on the publisher. +# Check result in database 'postgres' of node U +$result = $node_u->safe_psql('postgres', 'SELECT * FROM tbl1'); +is($result, qq(first row), "logical replication works in database postgres"); + +# Check result in database $db1 of node U +$result = $node_u->safe_psql($db1, 'SELECT * FROM tbl2'); +is( $result, qq(first row +second row), + "logical replication works in database $db1"); + +# Check result in database $db2 of node U +$result = $node_u->safe_psql($db2, 'SELECT * FROM tbl3'); +is($result, qq(first row), "logical replication works in database $db2"); + These tests may not always pass if we use wait_for_slot_catchup above. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-03-19 Thread Ashutosh Bapat
On Thu, Mar 13, 2025 at 6:10 PM Ashutosh Bapat wrote: > > > > I think the fix is to explicitly pass --lc-monetary to the old cluster > > and the restored cluster. 003 patch in the attached patch set does > > that. Please check if it fixes the issue for you. > > &g

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-18 Thread Ashutosh Bapat
On Wed, Mar 19, 2025 at 8:22 AM Amit Langote wrote: > > On Tue, Mar 18, 2025 at 8:48 PM Ashutosh Bapat > wrote: > > On Tue, Mar 18, 2025 at 4:02 PM Amit Langote > > wrote: > > > > > > I ran a couple of benchmarks of my own as follows. > > > >

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-18 Thread Ashutosh Bapat
ing perf that create_join_clause() is a hotspot when > running without the patch, especially at high partition counts (> 500) > and the more join relations. That's an interesting observation. Any hotspot in planning would show up as hotspot in total execution time. So expected. > > Let me know if my methodology seems off or if the results look reasonable. Thanks for benchmarking using a different method. The results agree with my results. -- Best Wishes, Ashutosh Bapat

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-18 Thread Ashutosh Bapat
On Tue, Mar 18, 2025 at 4:25 PM vignesh C wrote: > > On Mon, 17 Mar 2025 at 16:51, Ashutosh Bapat > wrote: > > > > On Mon, Mar 17, 2025 at 4:02 PM vignesh C wrote: > > > > > > On Mon, 17 Mar 2025 at 11:28, Shubham Khanna > > > wrote: > > &

  1   2   3   4   5   6   7   8   9   10   >