Re: Latency and recall re HSWN: Lucene versus Vespa

2022-10-01 Thread Michael Wechner
Thanks for your feedback! I really do not know enough, so it is good to hear different opinions and I will try to understand better. But anyway, I see it positive in the sense, that one way or the other, one can learn from each other hopefully :-) Am 01.10.22 um 14:41 schrieb Michael Sokolo

Re: Latency and recall re HSWN: Lucene versus Vespa

2022-10-01 Thread Michael Sokolov
I'd agree with the main point re: the need to combine vector-based matching with term-based matching. As for the comparison with Lucene, I'd say it's a shallow and biased take. The main argument is that Vespa's mutable in-memory(?) data structures are superior to Lucene's immutable on-disk segment

Latency and recall re HSWN: Lucene versus Vespa

2022-10-01 Thread Michael Wechner
Hi Together I just read the following article, where the author compares Lucene and Vespa re HSWN https://bergum.medium.com/will-new-vector-databases-dislodge-traditional-search-engines-b4fdb398fb43 What is your take on "comparing Lucene and Vespa re HSWN latency and recall"? Thanks Micha

Re: Upgrading from 9.1.0. to 9.4.0: Old codecs may only be used for reading Lucene91HnswVectorsFormat.java

2022-10-01 Thread Michael Wechner
ah ok :-) I think I just set it, because I saw this in one of the code examples and assumed that it might be necessary. Yes, I am using HNSW, but I don't think I set any particular HNSW parameters. I will double-check and will try without setting the codec. Thanks Michael Am 01.10.22 um

Re: Upgrading from 9.1.0. to 9.4.0: Old codecs may only be used for reading Lucene91HnswVectorsFormat.java

2022-10-01 Thread Adrien Grand
The best practice is to not set the codec explicitly, and Lucene will make sure to always use the right one. Seeing the codec explicitly is considered expert. I guess you are doing this because you want to configure things like stored fields compression or HNSW parameters? If so, there is no bette

Re: Upgrading from 9.1.0. to 9.4.0: Old codecs may only be used for reading Lucene91HnswVectorsFormat.java

2022-10-01 Thread Michael Wechner
Hi Adrien Thank you very much for your help! That was it :-) I completely forgot that I set this somewhere hidden inside my code. I made a note in the pom file, such that I should not forget again during the next upgrade :-) Or what is the best practice re setting / handling the codec? Than