Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Amit Kapila
On Sun, Jun 7, 2015 at 3:02 PM, Simon Riggs wrote: > > On 23 April 2015 at 17:24, Andres Freund wrote: >> > > > It's hard to see how to save space there without reference to a specific use case. I see different solutions depending upon whether we assume a low number of transactions or a high numb

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Simon Riggs
On 23 April 2015 at 17:24, Andres Freund wrote: > Split into a new thread, the other one is already growing fast > enough. This discussion started at > http://archives.postgresql.org/message-id/55391469.5010506%40iki.fi > > On April 23, 2015 6:48:57 PM GMT+03:00, Heikki Linnakangas < > hlinn...@i

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Peter Geoghegan
On Thu, Apr 30, 2015 at 6:54 AM, Robert Haas wrote: > The other, related problem is that the ordering operator might start > to return different results than it did at index creation time. For > example, consider a btree index built on a text column. Now consider > 'yum update'. glibc gets upda

Re: [HACKERS] Reducing tuple overhead

2015-06-06 Thread Amit Kapila
On Thu, Apr 23, 2015 at 9:54 PM, Andres Freund wrote: > > Split into a new thread, the other one is already growing fast > enough. This discussion started at > http://archives.postgresql.org/message-id/55391469.5010506%40iki.fi > > On April 23, 2015 6:48:57 PM GMT+03:00, Heikki Linnakangas < hlinn

Re: [HACKERS] Reducing tuple overhead

2015-05-01 Thread Jim Nasby
On 4/30/15 7:37 AM, Robert Haas wrote: On Thu, Apr 30, 2015 at 8:05 AM, Simon Riggs wrote: A much better idea is to work out how to avoid index bloat at cause. If we are running an UPDATE and we cannot get a cleanup lock, we give up and do a non-HOT update, causing the index to bloat. It seems

Re: [HACKERS] Reducing tuple overhead

2015-05-01 Thread Amit Kapila
On Thu, Apr 30, 2015 at 5:35 PM, Simon Riggs wrote: > > On 25 April 2015 at 01:12, Amit Kapila wrote: >> >> On Sat, Apr 25, 2015 at 1:58 AM, Jim Nasby wrote: >> > >> > On 4/23/15 10:40 PM, Amit Kapila wrote: >> >> >> >> I agree with you and what I think one of the major reasons of bloat is that

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Amit Kapila
On Thu, Apr 30, 2015 at 7:24 PM, Robert Haas wrote: > > On Thu, Apr 30, 2015 at 9:46 AM, Amit Kapila wrote: > > As the index expression contain table columns and all the functions > > or operators used in expression must be IMMUTABLE, won't that > > guarantee to avoid such a situation? > > The co

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Apr 27, 2015 at 5:01 PM, Jim Nasby wrote: > > The problem with just having the value is that if *anything* changes between > > how you evaluated the value when you created the index tuple and when you > > evaluate it a second time you'll corrupt your index. This is act

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 9:46 AM, Amit Kapila wrote: > As the index expression contain table columns and all the functions > or operators used in expression must be IMMUTABLE, won't that > guarantee to avoid such a situation? The concern is that they might be labeled as immutable but not actually

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Amit Kapila
On Thu, Apr 30, 2015 at 5:03 PM, Robert Haas wrote: > > On Thu, Apr 30, 2015 at 12:31 AM, Amit Kapila wrote: > > I think I am missing something here, but when this second > > evaluation is needed. Basically what I understand from index > > insertion is that it evaluates the value to be inserted

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 8:05 AM, Simon Riggs wrote: > A much better idea is to work out how to avoid index bloat at cause. If we > are running an UPDATE and we cannot get a cleanup lock, we give up and do a > non-HOT update, causing the index to bloat. It seems better to wait for a > short period

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Simon Riggs
On 25 April 2015 at 01:12, Amit Kapila wrote: > On Sat, Apr 25, 2015 at 1:58 AM, Jim Nasby > wrote: > > > > On 4/23/15 10:40 PM, Amit Kapila wrote: > >> > >> I agree with you and what I think one of the major reasons of bloat is > that > >> Index segment doesn't have visibility information due t

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 12:31 AM, Amit Kapila wrote: > I think I am missing something here, but when this second > evaluation is needed. Basically what I understand from index > insertion is that it evaluates the value to be inserted in index > before calling nbtree module and then nbtree just in

Re: [HACKERS] Reducing tuple overhead

2015-04-29 Thread Amit Kapila
On Tue, Apr 28, 2015 at 2:31 AM, Jim Nasby wrote: > > On 4/25/15 12:12 AM, Amit Kapila wrote: >> >> > ... which isn't possible. You can not go from a heap tuple to an >> index tuple. >> >> We will have the access to index value during delete, so why do you >> think that we need linkage between he

Re: [HACKERS] Reducing tuple overhead

2015-04-29 Thread Jim Nasby
On 4/29/15 12:18 PM, Robert Haas wrote: On Mon, Apr 27, 2015 at 5:01 PM, Jim Nasby wrote: The problem with just having the value is that if *anything* changes between how you evaluated the value when you created the index tuple and when you evaluate it a second time you'll corrupt your index. T

Re: [HACKERS] Reducing tuple overhead

2015-04-29 Thread Robert Haas
On Mon, Apr 27, 2015 at 5:01 PM, Jim Nasby wrote: > The problem with just having the value is that if *anything* changes between > how you evaluated the value when you created the index tuple and when you > evaluate it a second time you'll corrupt your index. This is actually an > incredibly easy

Re: [HACKERS] Reducing tuple overhead

2015-04-27 Thread Jim Nasby
On 4/25/15 12:12 AM, Amit Kapila wrote: > ... which isn't possible. You can not go from a heap tuple to an index tuple. We will have the access to index value during delete, so why do you think that we need linkage between heap and index tuple to perform Delete operation? I think we need to th

Re: [HACKERS] Reducing tuple overhead

2015-04-24 Thread Amit Kapila
On Sat, Apr 25, 2015 at 1:58 AM, Jim Nasby wrote: > > On 4/23/15 10:40 PM, Amit Kapila wrote: >> >> I agree with you and what I think one of the major reasons of bloat is that >> Index segment doesn't have visibility information due to which clearing of >> Index needs to be tied along with heap.

Re: [HACKERS] Reducing tuple overhead

2015-04-24 Thread Jim Nasby
On 4/23/15 10:40 PM, Amit Kapila wrote: I agree with you and what I think one of the major reasons of bloat is that Index segment doesn't have visibility information due to which clearing of Index needs to be tied along with heap. Now if we can move transaction information at page level, then we

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Amit Kapila
On Fri, Apr 24, 2015 at 1:03 AM, Jim Nasby wrote: > > On 4/23/15 11:45 AM, Petr Jelinek wrote: >> >> On 23/04/15 18:24, Andres Freund wrote: >>> >>> Whether that's feasible complexity wise is debatable, but it's certainly >>> possible. >>> >>> >>> I do wonder what, in realistic cases, is actually

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Jim Nasby
On 4/23/15 11:45 AM, Petr Jelinek wrote: On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liber

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Alvaro Herrera
Thanks for posting this. Joshua D. Drake wrote: > First of all I should note that I discussed the approach mentioned above to > pgsql-hackers and got a very interesting comment from Tom Lane that adding > sorting info to Var and TargetEntry nodes was not a very good idea because > it'd break stor

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Joshua D. Drake
On 04/23/2015 09:42 AM, Jim Nasby wrote: On 4/23/15 11:24 AM, Andres Freund wrote: I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... Assuming you're talking about padding between fields.

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Petr Jelinek
On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... The logical o

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Jim Nasby
On 4/23/15 11:24 AM, Andres Freund wrote: I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... Assuming you're talking about padding between fields... Several years ago Enova paid Command Pro

[HACKERS] Reducing tuple overhead

2015-04-23 Thread Andres Freund
Split into a new thread, the other one is already growing fast enough. This discussion started at http://archives.postgresql.org/message-id/55391469.5010506%40iki.fi On April 23, 2015 6:48:57 PM GMT+03:00, Heikki Linnakangas wrote: >Stop right there. You need to reserve enough space on the page