Tomoko -- this fits with my experience as well. I really like the idea of treating LUCENE-9322 as setting up a framework for experimentation + benchmarking (but not requiring us to commit a particular ANN implementation quite yet).
Julie On 2020/07/17 12:16:18, Tomoko Uchida <t...@gmail.com> wrote: > > would it make sense to create a separate Lucene module for ANN search ?> > > From a bit of my experience with LUCENE-9004, it is currently impossible to> > plug in or opt in custom codecs and indexing chain for aknn search without> > touching lucene-core module (plz correct that if it's wrong).> > I think LUCENE-9322 (unified low-level Codec/Format for dense vectors)> > would open the possibility for us to experiment different aknn algorithms> > on some sandbox modules or even separated jars from Lucene itself.> > > Tomoko> > > > 2020年7月17日(金) 17:00 Tommaso Teofili <to...@gmail.com>:> > > > would it make sense to create a separate Lucene module for ANN search ?> > > we could then experiment with the different approaches and compare them> > > across the same benchmarks.> > >> > > On Thu, 16 Jul 2020 at 23:14, Ali Akhtar <al...@ali.actor> wrote:> > >> > > > I’m a bit of a layman in this area, but if we are talking about formats> > > for> > > > vectors, I vote for the one used by FastAI word vectors. It’s pretty > > > easy> > > > to work with.> > > >> > > > If we are talking about the same / similiar things, if not just ignore > > > me> > > > 😀> > > >> > > > On Thu, 16 Jul 2020 at 7:06 PM, Michael Sokolov <ms...@gmail.com>> > > > wrote:> > > >> > > > > We have some prototype implementations in the issues you found. If> > > > > you want to try out the approaches in those issues, you could build> > > > > Lucene from source and patch it, but there is no release containing> > > > > KNN/vector support. We're still working to establish consensus on what> > > > > the best way forward is. I think the most fruitful thing we can do at> > > > > the moment is establish a format for storing and accessing vectors> > > > > that will support different approaches since there is such a rich> > > > > variety of algorithms and approaches in this area. The last issue you> > > > > pointed to is focused on the format.> > > > >> > > > > On Wed, Jul 15, 2020 at 11:20 AM Alex K <ak...@gmail.com> wrote:> > > > > >> > > > > > Hi Mikhail,> > > > > >> > > > > > I'm not sure about the state of ANN in lucene proper. Very > > > > > interested> > > > to> > > > > > see the response from others.> > > > > > I've been doing some work on ANN for an Elasticsearch plugin:> > > > > > http://elastiknn.klibisz.com/> > > > > > I think it's possible to extract my custom queries and modeling code> > > so> > > > > > that it's elasticsearch-agnostic and can be used directly in Lucene> > > > apps.> > > > > > However I'm much more familiar with Elasticsearch's APIs and> > > > > usage/testing> > > > > > patterns than I am with raw Lucene, so I'd likely need to get some> > > help> > > > > > from the Lucene community.> > > > > > Please LMK if that sounds interesting to anyone.> > > > > >> > > > > > - Alex> > > > > >> > > > > >> > > > > >> > > > > > On Wed, Jul 15, 2020 at 11:11 AM Mikhail <wm...@mail.ru.invalid>> > > > > wrote:> > > > > >> > > > > > >> > > > > > > Hi,> > > > > > >> > > > > > > I want to incorporate semantic search in my project,> > > which> > > > > uses> > > > > > > Lucene. I want to use sentence embeddings and ANN (approximate> > > > nearest> > > > > > > neighbor) search. I found the related Lucene issues:> > > > > > > https://issues.apache.org/jira/browse/LUCENE-9004 ,> > > > > > > https://issues.apache.org/jira/browse/LUCENE-9136 ,> > > > > > > https://issues.apache.org/jira/browse/LUCENE-9322 . I see that> > > there> > > > > > > are some related work and related PRs. What is the current state > > > > > > of> > > > > this> > > > > > > functionality?> > > > > > >> > > > > > > --> > > > > > > Thanks,> > > > > > > Mikhail> > > > > > >> > > > > > >> > > > >> > > > > ---------------------------------------------------------------------> > > > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org> > > > > For additional commands, e-mail: java-user-h...@lucene.apache.org> > > > >> > > > >> > > >> > >> > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
