Impressive! On Tue, Aug 5, 2025 at 5:07 PM Chris Hegarty <christopher.hega...@elastic.co.invalid> wrote:
> Just to close the loop on this. I say close, however the work is still > active, but worth an update. > > Trevor and I added a bulk scoring abstraction to RandomVectorScorer [1] > in. The API can be adapted a little too, in order to improve some common > use cases [2], but is otherwise “solid”. Trevor has prototyped a bulk > scorer implementation using rust, and I’ve verified similar with a native > scorer - both prefetch yet-to-be-scorer vectors from the given set of > ordinals (improvements can be seen in the linked PR comments). This is > great, but we can do better in Lucene. > > Now that we have a bulk scorer API, I’ve implemented an off-heap bulk > scorer for float32’s and orientated a trivial dot product implementation > using Panama vector around scoring 4 vectors at time, shaping the code in a > way to more easily help provoke the CPU to do overlap the memory fetches > (at least, that’s the idea!). > > Running luceneutil with 1M cohere 768d vectors, search latencies show an > improvement of ~33%, and merge times 40-50% [3]. I have some ideas about > how to further improve indexing, but they can be done separately. > > -Chris. > > [1] https://github.com/apache/lucene/pull/14978 > [2] https://github.com/apache/lucene/pull/15021 > [3] https://github.com/apache/lucene/pull/14980#issuecomment-3155502316 > > > > On 19 Jun 2025, at 20:37, Adrien Grand <jpou...@gmail.com> wrote: > > > > Thanks Mike, this is useful information. Then I'll try to reproduce this > benchmark to better understand what is happening. > > > > On Thu, Jun 19, 2025 at 8:16 PM Michael Sokolov <msoko...@gmail.com> > wrote: > > We've recently been comparing Lucene's HNSW w/FAISS' and there is not > > a 2x difference there. FAISS does seem to be around 10-15% faster I > > think? The 2x difference is roughly what I was seeing in comparisons > > w/hnswlib prior to the dot-product improvements we made in Lucene. > > > > On Thu, Jun 19, 2025 at 2:12 PM Adrien Grand <jpou...@gmail.com> wrote: > > > > > > Chris, > > > > > > FWIW I was looking at luceneknn ( > https://github.com/erikbern/ann-benchmarks/blob/f402b2cc17b980d7cd45241ab5a7a4cc0f965e55/ann_benchmarks/algorithms/luceneknn/Dockerfile#L15) > which is on 9.7, though I don't know if it enabled the incubating vector > API at runtime? > > > > > > I hope that mentioning ANN benchmarks did not add noise to this > thread, I was mostly looking at whether I could find another benchmark that > suggests that Lucene is significantly slower in similar conditions. Does it > align with other people's experience that Lucene is 2x slower or more > compared with other good HNSW implementations? > > > > > > Adrien > > > > > > Le jeu. 19 juin 2025, 18:44, Chris Hegarty > <christopher.hega...@elastic.co.invalid> a écrit : > > >> > > >> Hi Adrien, > > >> > > >> > Even though it uses Elasticsearch to run the benchmark, it really > benchmarks Lucene functionality, > > >> > > >> Agreed. > > >> > > >> > This seems consistent with results from > https://ann-benchmarks.com/index.html though I don't know if the cause of > the performance difference is the same or not. > > >> > > >> On ann-benchmarks specifically. Unless I’m looking in the wrong > place, then they’re using Elasticsearch 8.7.0 [1], which predates our usage > of the Panama Vector API for vector search. We added support for that in > Lucene 9.7.0 -> Elasticsearch 8.9.0. So those benchmarks are wildly out of > date, no ? > > >> > > >> -Chris. > > >> > > >> [1] > https://github.com/erikbern/ann-benchmarks/blob/f402b2cc17b980d7cd45241ab5a7a4cc0f965e55/ann_benchmarks/algorithms/elasticsearch/Dockerfile#L2 > > >> > > >> > > >> > On 19 Jun 2025, at 16:39, Adrien Grand <jpou...@gmail.com> wrote: > > >> > > > >> > Hello all, > > >> > > > >> > I have been looking at this benchmark against Vespa recently: > https://blog.vespa.ai/elasticsearch-vs-vespa-performance-comparison/. > (The report is behind an annoying email wall, but I'm copying relevant data > below, so hopefully you don't need to download the report.) Even though it > uses Elasticsearch to run the benchmark, it really benchmarks Lucene > functionality, I don't believe that Elasticsearch does anything that > meaningfully alters the results that you would get if you were to run > Lucene directly. > > >> > > > >> > The benchmark seems designed to highlight the benefits of Vespa's > realtime design, that's fair game I guess. But it also runs some queries in > read-only scenarios when I was expecting Lucene to perform better. > > >> > > > >> > One thing that got me curious is that it reports about 2x worse > latency and throughput for pure unfiltered vector search on a force-merged > index (so single segment/graph). Does anybody know why Lucene's HNSW may > perform slower than Vespa's HNSW? This seems consistent with results from > https://ann-benchmarks.com/index.html though I don't know if the cause of > the performance difference is the same or not. > > >> > > > >> > For reference, here are details that apply to both Lucene and > Vespa's vector search: > > >> > - HNSW, > > >> > - float32 vectors, no quantization, > > >> > - embeddings generated using Snowflake's Arctic-embed-xs model > > >> > - 1M docs > > >> > - 384 dimensions, > > >> > - dot product, > > >> > - m = 16, > > >> > - max connections = 200, > > >> > - search for top 10 hits, > > >> > - no filter, > > >> > - single client, so no search concurrency, > > >> > - purple column is force-merged, so single segment/graph like > Vespa. > > >> > > > >> > I never seriously looked at Lucene's vector search performance, so > I'm very happy to be educated if I'm making naive assumptions! > > >> > > > >> > Somewhat related, is this the reason why I'm seeing many threads > around bringing 3rd party implementations into Lucene, including ones that > are very similar to Lucene on paper? To speed up vector search? > > >> > > > >> > -- > > >> > Adrien > > >> > <vespa-vs-es-screenshot.png> > > >> > > --------------------------------------------------------------------- > > >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > >> > For additional commands, e-mail: dev-h...@lucene.apache.org > > >> > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > >> For additional commands, e-mail: dev-h...@lucene.apache.org > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > > -- > > Adrien > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
