Re: Proposal: Global Index for PostgreSQL

2025-06-10 Thread Dilip Kumar
On Wed, Jun 11, 2025 at 1:08 AM Bruce Momjian wrote: > > On Mon, Jun 9, 2025 at 05:51:25PM -0400, Bruce Momjian wrote: > > On Mon, Jun 9, 2025 at 03:28:38PM +0530, Dilip Kumar wrote: > > There are certainly use cases where this would be helpful, but I think > > the big question is whether it wou

Re: Proposal: Global Index for PostgreSQL

2025-06-10 Thread Dilip Kumar
On Tue, Jun 10, 2025 at 3:21 AM Bruce Momjian wrote: Thanks Bruce for your thoughts on this. > On Mon, Jun 9, 2025 at 03:28:38PM +0530, Dilip Kumar wrote: > > On Mon, Jun 9, 2025 at 2:03 PM Nikita Malakhov wrote: > > > Global Indexes is a very interesting functionality that has both > > > sig

Re: Proposal: Global Index for PostgreSQL

2025-06-10 Thread Bruce Momjian
On Mon, Jun 9, 2025 at 05:51:25PM -0400, Bruce Momjian wrote: > On Mon, Jun 9, 2025 at 03:28:38PM +0530, Dilip Kumar wrote: > There are certainly use cases where this would be helpful, but I think > the big question is whether it would have so many negatives that most > people who try to use it w

Re: Proposal: Global Index for PostgreSQL

2025-06-09 Thread Bruce Momjian
On Mon, Jun 9, 2025 at 03:28:38PM +0530, Dilip Kumar wrote: > On Mon, Jun 9, 2025 at 2:03 PM Nikita Malakhov wrote: > > Global Indexes is a very interesting functionality that has both > > significant advantages > > and drawbacks, and the community seems not ready to accept it without very > >

Re: Proposal: Global Index for PostgreSQL

2025-06-09 Thread Dilip Kumar
On Mon, Jun 9, 2025 at 2:03 PM Nikita Malakhov wrote: > > Hi Dilip! Thanks Nikita for your response and reading my proposal. > Global Indexes is a very interesting functionality that has both significant > advantages > and drawbacks, and the community seems not ready to accept it without very

Re: Proposal: Global Index for PostgreSQL

2025-06-09 Thread Nikita Malakhov
Hi Dilip! Global Indexes is a very interesting functionality that has both significant advantages and drawbacks, and the community seems not ready to accept it without very strong motivation. There was a more recent approach to Global index problem [1], please check it out. I've read you proposal

Re: Proposal: Global Index for PostgreSQL

2025-06-06 Thread Dilip Kumar
On Fri, Jun 6, 2025 at 1:01 PM wenhui qiu wrote: > > Hi Dilip Kumar >Thank you for your working on this ,I remember six years ago there was > talk about global index ,You can see if this mailing list has any references > to > (https://www.postgresql.org/message-id/CALtqXTcurqy1PKXzP9XO%3Dof

Re: Proposal: Global Index for PostgreSQL

2025-06-06 Thread wenhui qiu
Hi Dilip Kumar Thank you for your working on this ,I remember six years ago there was talk about global index ,You can see if this mailing list has any references to ( https://www.postgresql.org/message-id/CALtqXTcurqy1PKXzP9XO%3DofLLA5wBSo77BnUnYVEZpmcA3V0ag%40mail.gmail.com ) Thanks On Fri

Proposal: Global Index for PostgreSQL

2025-06-06 Thread Dilip Kumar
PostgreSQL’s table partitioning allows large tables to be broken into smaller, more manageable pieces for better performance. However, a key limitation currently is the absence of global indexes, which restricts using partitioned tables, especially when you need unique constraints on columns that a

Re: Proposal: Global Index

2021-01-18 Thread 曾文旌
> 2021年1月12日 02:37,Robert Haas 写道: > > On Mon, Jan 11, 2021 at 12:46 PM Bruce Momjian wrote: >>> For 1) The DETACH old child table can be finished immediately, global index >>> can be kept valid after DETACH is completed, and the cleanup of garbage >>> data in global index can be deferred to

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 12:05:43PM -0800, Peter Geoghegan wrote: > On Mon, Jan 11, 2021 at 11:25 AM Bruce Momjian wrote: > > Once you layer on all the places a global index will be worse than just > > creating a single large table, or a partitioned table with an index per > > child, there might no

Re: Proposal: Global Index

2021-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2021 at 11:25 AM Bruce Momjian wrote: > Once you layer on all the places a global index will be worse than just > creating a single large table, or a partitioned table with an index per > child, there might not be much usefulness left. A POC patch might tell > us that, and might a

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 11:01:20AM -0800, Peter Geoghegan wrote: > However, it probably would be okay if a global index feature performed > poorly in scenarios where partitions get lots of UPDATEs that produce > lots of index bloat and cause lots of LP_DEAD line pointers to > accumulate in heap pag

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 01:37:02PM -0500, Robert Haas wrote: > However, there is a VACUUM amplification effect to worry about here ... > That's not necessarily a death sentence for every use case, but it's > going to be pretty bad for tables that are big and heavily updated. Yeah, I had not really

Re: Proposal: Global Index

2021-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2021 at 10:37 AM Robert Haas wrote: > I actually think the idea of lazily deleting the index entries is > pretty good, but it won't work if the way the global index is > implemented is by adding a tableoid column. Perhaps there is an opportunity to apply some of the infrastructure

Re: Proposal: Global Index

2021-01-11 Thread Robert Haas
On Mon, Jan 11, 2021 at 12:46 PM Bruce Momjian wrote: > > For 1) The DETACH old child table can be finished immediately, global index > > can be kept valid after DETACH is completed, and the cleanup of garbage > > data in global index can be deferred to VACUUM. > > This is similar to the global

Re: Proposal: Global Index

2021-01-11 Thread Tom Lane
Bruce Momjian writes: > On Mon, Jan 11, 2021 at 07:40:18PM +0800, 曾文旌 wrote: >> This is indeed a typical scenario for a partitioned table. >> there are two basic operations >> 1) Monthly DETACH old child table >> 2) Monthly ATTACH new child table >> >> For 1) The DETACH old child table can be fin

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 07:40:18PM +0800, 曾文旌 wrote: > >> In addition you mentioned: "It is still unclear if these use-cases justify > >> the architectural changes needed to enable global indexes." > >> Please also describe the problems you see, I will confirm each specific > >> issue one by one.

Re: Proposal: Global Index

2021-01-11 Thread 曾文旌
> 2021年1月7日 23:04,Robert Haas 写道: > > On Thu, Jan 7, 2021 at 4:44 AM 曾文旌 wrote: >> I've been following this topic for a long time. It's been a year since the >> last response. >> It was clear that our customers wanted this feature as well, and a large >> number of them mentioned it. >> >> S

Re: Proposal: Global Index

2021-01-11 Thread 曾文旌
> 2021年1月8日 16:26,Julien Rouhaud 写道: > > On Fri, Jan 8, 2021 at 4:02 PM 曾文旌 wrote: >> >>> 2021年1月7日 22:16,Bruce Momjian 写道: >>> >>> On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: I've been following this topic for a long time. It's been a year since the last response.

Re: Proposal: Global Index

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 11:26:48AM +0800, 曾文旌 wrote: > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > >> I've been following this topic for a long time. It's been a year since the > >> last response. > >> It was clear that our customers wanted this feature as well, and a large > >> numb

Re: Proposal: Global Index

2021-01-08 Thread Julien Rouhaud
On Fri, Jan 8, 2021 at 4:02 PM 曾文旌 wrote: > > > 2021年1月7日 22:16,Bruce Momjian 写道: > > > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > >> I've been following this topic for a long time. It's been a year since the > >> last response. > >> It was clear that our customers wanted this feat

Re: Proposal: Global Index

2021-01-08 Thread 曾文旌
> 2021年1月7日 22:16,Bruce Momjian 写道: > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: >> I've been following this topic for a long time. It's been a year since the >> last response. >> It was clear that our customers wanted this feature as well, and a large >> number of them mentioned

Re: Proposal: Global Index

2021-01-07 Thread Robert Haas
On Thu, Jan 7, 2021 at 4:44 AM 曾文旌 wrote: > I've been following this topic for a long time. It's been a year since the > last response. > It was clear that our customers wanted this feature as well, and a large > number of them mentioned it. > > So, I wish the whole feature to mature as soon as

Re: Proposal: Global Index

2021-01-07 Thread Bruce Momjian
On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > I've been following this topic for a long time. It's been a year since the > last response. > It was clear that our customers wanted this feature as well, and a large > number of them mentioned it. > > So, I wish the whole feature to mature

Re: Proposal: Global Index

2021-01-07 Thread 曾文旌
I've been following this topic for a long time. It's been a year since the last response.It was clear that our customers wanted this feature as well, and a large number of them mentioned it.So, I wish the whole feature to mature as soon as possible.I summarized the scheme mentioned in the email and

Re: Proposal: Global Index

2019-12-19 Thread Bruce Momjian
On Thu, Dec 19, 2019 at 11:28:55AM -0800, Jeremy Schneider wrote: > On 12/19/19 08:12, Bruce Momjian wrote: > > I don't see lossy BRIN indexes helping with the uniqueness use-case, and > > I am not sure they would help with the rare case either. They would > > help for range-based partitions, but

Re: Proposal: Global Index

2019-12-19 Thread Bruce Momjian
On Thu, Dec 19, 2019 at 09:48:40AM +0100, Jose Luis Tallon wrote: > On 19/12/19 4:03, Bruce Momjian wrote: > > On Mon, Nov 25, 2019 at 03:44:39PM -0800, Jeremy Schneider wrote: > > > On 11/25/19 15:05, Jeremy Schneider wrote: > > > > ... the cost of doing the individual index lookups across 180 > >

Re: Proposal: Global Index

2019-12-19 Thread Jose Luis Tallon
On 19/12/19 4:03, Bruce Momjian wrote: On Mon, Nov 25, 2019 at 03:44:39PM -0800, Jeremy Schneider wrote: On 11/25/19 15:05, Jeremy Schneider wrote: ... the cost of doing the individual index lookups across 180 partitions (and 180 indexes) was very high, so they stored max and min txn id per par

Re: Proposal: Global Index

2019-12-18 Thread Bruce Momjian
On Mon, Nov 25, 2019 at 03:44:39PM -0800, Jeremy Schneider wrote: > On 11/25/19 15:05, Jeremy Schneider wrote: > > ... the cost of doing the individual index lookups across 180 > > partitions (and 180 indexes) was very high, so they stored max and min > > txn id per partition and would generate a q

Re: Proposal: Global Index

2019-10-31 Thread Tomas Vondra
On Thu, Oct 31, 2019 at 03:02:40PM -0400, Isaac Morland wrote: On Thu, 31 Oct 2019 at 14:50, Stephen Frost wrote: Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: [] Absolutely- our lack of such is a common point of issue when folks are considering using or migrating to PostgreSQL.

Re: Proposal: Global Index

2019-10-31 Thread Isaac Morland
On Thu, 31 Oct 2019 at 14:50, Stephen Frost wrote: > Greetings, > > * Peter Geoghegan (p...@bowt.ie) wrote: > [] > > Absolutely- our lack of such is a common point of issue when folks are > considering using or migrating to PostgreSQL. > Not sure how similar my situation really is, but I fi

Re: Proposal: Global Index

2019-10-31 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: > > Well, the *effects* of the feature seem desirable, but that doesn't > > mean that we want an implementation that actually has a shared index. > > As soon as you do that, you've thrown away mos

Re: Proposal: Global Index

2019-10-30 Thread Andres Freund
Hi, On 2019-10-30 13:05:57 -0400, Tom Lane wrote: > Peter Geoghegan writes: > > On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: > >> Well, the *effects* of the feature seem desirable, but that doesn't > >> mean that we want an implementation that actually has a shared index. > >> As soon as you

Re: Proposal: Global Index

2019-10-30 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: >> Well, the *effects* of the feature seem desirable, but that doesn't >> mean that we want an implementation that actually has a shared index. >> As soon as you do that, you've thrown away most of the benefits of >> having

Re: Proposal: Global Index

2019-10-30 Thread Peter Geoghegan
On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: > Well, the *effects* of the feature seem desirable, but that doesn't > mean that we want an implementation that actually has a shared index. > As soon as you do that, you've thrown away most of the benefits of > having a partitioned data structure i

Re: Proposal: Global Index

2019-10-30 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 30, 2019 at 10:13 AM Tom Lane wrote: >> I believe that the current design of partitioning is explicitly intended >> to avoid the need for such a construct. It'd be absolutely disastrous >> to have such a thing from many standpoints, including the breadth of >> l

Re: Proposal: Global Index

2019-10-30 Thread Robert Haas
On Wed, Oct 30, 2019 at 10:13 AM Tom Lane wrote: > I believe that the current design of partitioning is explicitly intended > to avoid the need for such a construct. It'd be absolutely disastrous > to have such a thing from many standpoints, including the breadth of > locking needed to work with

Re: Proposal: Global Index

2019-10-30 Thread Tom Lane
Ibrar Ahmed writes: > A global index by very definition is a single index on the parent table > that maps to many > underlying table partitions. I believe that the current design of partitioning is explicitly intended to avoid the need for such a construct. It'd be absolutely disastrous to have

Proposal: Global Index

2019-10-30 Thread Ibrar Ahmed
A global index by very definition is a single index on the parent table that maps to many underlying table partitions. The parent table itself does not have any underlying storage, so it must, therefore, retrieve the data satisfying index constraints from the underlying tables. In very crude terms,