Re: Increasing IndexTupleData.t_info from uint16 to uint32

Hi, > > The overall trend in machine learning embedding sizes has been growing > > rapidly over the last few years from 128 up to 4K dimensions yielding > > additional value and quality improvements. It's not clear when this trend > > in growth will ease. The leading text embedding models gener

Re: Increasing IndexTupleData.t_info from uint16 to uint32

On Thu, 18 Jan 2024 at 13:41, Montana Low wrote: > > The overall trend in machine learning embedding sizes has been growing > rapidly over the last few years from 128 up to 4K dimensions yielding > additional value and quality improvements. It's not clear when this trend in > growth will ease.

Re: Increasing IndexTupleData.t_info from uint16 to uint32

I wrote: > On a micro level, this makes sizeof(IndexTupleData) be not maxaligned, > which is likely to cause problems on alignment-picky hardware, or else > result in space wastage if we were careful to MAXALIGN() everywhere. > (Which we should have been, but I don't care to bet on it.) A lot of >

Re: Increasing IndexTupleData.t_info from uint16 to uint32

Montana Low writes: > I've attached a patch that increases IndexTupleData.t_info from 16bits to > 32bits allowing for significantly larger index tuple sizes. I fear this idea is a non-starter because it'd break on-disk compatibility. Certainly, if we were to try to pursue it, there'd need to be

Increasing IndexTupleData.t_info from uint16 to uint32

The overall trend in machine learning embedding sizes has been growing rapidly over the last few years from 128 up to 4K dimensions yielding additional value and quality improvements. It's not clear when this trend in growth will ease. The leading text embedding models