Re: Adding vector search to SAI with heirarchical navigable small world graph index

[Benedict]

We probably at least need to bike shed naming as we already have FLOAT, DOUBLE, and LIST - which are similar/overlapping types, and we shoo on should be consistent. If we introduce FLOAT32 we probably need that to be an alias of FLOAT and introduce FLOAT64 to alias DOUBLE for consistency. DENSE seems to just be an array? So very similar to a frozen list, but with a fixed size? On 26 Apr 2023, at 06:53, Mick Semb Wever <m...@apache.org> wrote:  I was soooooo happy when I saw this, I know many users are going to be thrilled about it. On Wed, 26 Apr 2023 at 05:15, Patrick McFadin <pmcfa...@gmail.com> wrote: Not sure if this is what you are saying, Josh, but I believe this needs to be its own CEP. It's a change in CQL syntax and changes how clusters operate. The change needs to be documented and voted on. Jonathan, you know how to find me if you want me to help write it. :)  I'd be fine with just a DISCUSS thread to agree to the CQL change, since it: `DENSE FLOAT32` appears to be a minimal,  and the overall patch building on SAI. As Henrik mentioned there's other SAI extensions being added too without CEPs.  Can you elaborate on how you see this changing how the cluster operates? This will be easier to decide once we have a patch to look at, but that depends on a CEP-7 base (e.g. no feature branch exists). If we do want a CEP we need to allow a few weeks to get it through, but that can happen in parallel and maybe drafting up something now will be valuable anyway for an eventual CEP that proposes the more complete features (e.g. cosine_similarity(…)).