Re: Conneting Lucene with ChatGPT Retrieval Plugin

2023-05-10 Thread Gus Heck
Do you anticipate that the vector engine would be changed in a way that fundamentally precluded larger vectors (intentionally)? I would think that the ability to support larger vectors should be a key criteria for any changes to be made. Certainly if there are optimizations to be had at specific si

Re: Conneting Lucene with ChatGPT Retrieval Plugin

2023-05-10 Thread Jonathan Ellis
I did track down a weird bug I was seeing to our cosine similarity returning NaN with high dimension vectors. Fix is here: https://github.com/apache/lucene/pull/12281 On Tue, May 9, 2023 at 12:15 PM Jonathan Ellis wrote: > I'm adding Lucene HNSW to Cassandra for vector search. One of my test >

Re: New branch and feature freeze for Lucene 9.6.0

2023-05-10 Thread Alan Woodward
Thanks Ishan, turns out the error was between chair and keyboard - I’d told the wizard to use the gradle java plugin to sign things when I should have been using gpg. > On 2 May 2023, at 14:19, Ishan Chattopadhyaya > wrote: > > Don't remember the specifics, but I ran into GPG issues during So

[ANNOUNCE] Apache Lucene 9.6.0 released

2023-05-10 Thread Alan Woodward
The Lucene PMC is pleased to announce the release of Apache Lucene 9.6.0. Apache Lucene is a high-performance, full-featured search engine library written entirely in Java. It is a technology suitable for nearly any application that requires structured search, full-text search, faceting, neares

Re: Dimensions Limit for KNN vectors - Next Steps

2023-05-10 Thread Bruno Roustant
*Proposed option:* Move the max dimension limit lower level to a HNSW specific implementation. Once there, this limit would not bind any other potential vector engine alternative/evolution. *Motivation:* There seem to be contradictory performance interpretations about the current HNSW implementati