Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-10-30 Thread Peter Geoghegan
On Wed, Oct 30, 2019 at 1:25 PM Andres Freund wrote: > I assume you mean that the index would dynamically recognize when it > needs the wider tids ("for the higher portion")? If so, yea, that makes > sense to me. Would that need to encode the 6/8byteness of a tid on a > per-element basis? Or are y

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-10-30 Thread Andres Freund
Hi, On 2019-10-30 12:37:50 -0700, Peter Geoghegan wrote: > On Wed, Oct 30, 2019 at 12:03 PM Andres Freund wrote: > > I'd much rather not entrench this further, even leaving global indexes > > aside. The 4 byte block number is a significant limitation for heap > > tables too, and we should lift th

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-10-30 Thread Peter Geoghegan
On Wed, Oct 30, 2019 at 12:03 PM Andres Freund wrote: > I'd much rather not entrench this further, even leaving global indexes > aside. The 4 byte block number is a significant limitation for heap > tables too, and we should lift that at some point not too far away. > Then there's also other AMs t

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-10-30 Thread Andres Freund
Hi, On 2019-10-30 11:33:21 -0700, Peter Geoghegan wrote: > On Mon, Apr 22, 2019 at 9:35 AM Andres Freund wrote: > > On 2019-04-21 17:46:09 -0700, Peter Geoghegan wrote: > > > Andres has suggested that I work on teaching nbtree to accommodate > > > variable-width, logical table identifiers, such a

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-10-30 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 9:35 AM Andres Freund wrote: > On 2019-04-21 17:46:09 -0700, Peter Geoghegan wrote: > > Andres has suggested that I work on teaching nbtree to accommodate > > variable-width, logical table identifiers, such as those required for > > indirect indexes, or clustered indexes, w

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-24 Thread Peter Geoghegan
On Wed, Apr 24, 2019 at 10:43 AM Peter Geoghegan wrote: > The hard part is how to do varwidth encoding for space-efficient > partition numbers while continuing to use IndexTuple fields for heap > TID on the leaf level, *and* also having a > BTreeTupleGetHeapTID()-style macro to get partition numbe

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-24 Thread Peter Geoghegan
On Wed, Apr 24, 2019 at 5:22 AM Robert Haas wrote: > If you drop or detach a partition, you can either (a) perform, as part > of that operation, a scan of every global index to remove all > references to the former partition, or (b) tell each global indexes > that all references to that partition

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-24 Thread Robert Haas
On Mon, Apr 22, 2019 at 1:16 PM Peter Geoghegan wrote: > Yes, though that should probably work by reusing what we already do > with heap TID (use standard IndexTuple fields on the leaf level for > heap TID), plus an additional identifier for the partition number that > is located at the physical e

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 10:32 AM Stephen Frost wrote: > Yes, global indexes for partitioned tables could potentially be simpler > than the logical row identifiers, but maybe it'd be useful to just have > one implementation based around logical row identifiers which ends up > working for global ind

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Mon, Apr 22, 2019 at 8:36 AM Stephen Frost wrote: > > This seems like it would be helpful for global indexes as well, wouldn't > > it? > > Yes, though that should probably work by reusing what we already do > with heap TID (use standard Ind

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 9:35 AM Andres Freund wrote: > I, more generally, wonder if there's not a case to squeeze out more > padding than "just" what you describe (since we IIRC don't frequently > keep pointers into such tuples anyway, and definitely don't for byval > attrs). But that's very likel

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 8:36 AM Stephen Frost wrote: > This seems like it would be helpful for global indexes as well, wouldn't > it? Yes, though that should probably work by reusing what we already do with heap TID (use standard IndexTuple fields on the leaf level for heap TID), plus an addition

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Andres Freund
Hi, On 2019-04-21 17:46:09 -0700, Peter Geoghegan wrote: > Andres has suggested that I work on teaching nbtree to accommodate > variable-width, logical table identifiers, such as those required for > indirect indexes, or clustered indexes, where secondary indexes must > use a logical primary key v

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > Andres has suggested that I work on teaching nbtree to accommodate > variable-width, logical table identifiers, such as those required for > indirect indexes, or clustered indexes, where secondary indexes must > use a logical primary key value i

Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-21 Thread Peter Geoghegan
Andres has suggested that I work on teaching nbtree to accommodate variable-width, logical table identifiers, such as those required for indirect indexes, or clustered indexes, where secondary indexes must use a logical primary key value instead of a heap TID. I'm not currently committed to working