Hi,
> > The overall trend in machine learning embedding sizes has been growing
> > rapidly over the last few years from 128 up to 4K dimensions yielding
> > additional value and quality improvements. It's not clear when this trend
> > in growth will ease. The leading text embedding models gener
On Thu, 18 Jan 2024 at 13:41, Montana Low wrote:
>
> The overall trend in machine learning embedding sizes has been growing
> rapidly over the last few years from 128 up to 4K dimensions yielding
> additional value and quality improvements. It's not clear when this trend in
> growth will ease.
I wrote:
> On a micro level, this makes sizeof(IndexTupleData) be not maxaligned,
> which is likely to cause problems on alignment-picky hardware, or else
> result in space wastage if we were careful to MAXALIGN() everywhere.
> (Which we should have been, but I don't care to bet on it.) A lot of
>
Montana Low writes:
> I've attached a patch that increases IndexTupleData.t_info from 16bits to
> 32bits allowing for significantly larger index tuple sizes.
I fear this idea is a non-starter because it'd break on-disk
compatibility. Certainly, if we were to try to pursue it, there'd
need to be
