To my knowledge, FAISS isn't utilizing hand-rolled SIMD calculations. Do we
know if it was compiled with `--ffast-math`?
Vespa does utilize SIMD optimizations for vector comparisons.
Some more ways I think Lucene is slower (though, I am not sure the 2x is
fully explained):
- Reading floats onto
+1 SUCCESS! [1:06:35.392122]
On Tue, Jun 17, 2025 at 11:28 AM Mayya Sharipova
wrote:
> +1 SUCCESS! [0:48:22.366538]
>
> On Tue, Jun 17, 2025 at 8:48 AM Stefan Vodita
> wrote:
>
>> +1 SUCCESS! [0:42:50.959477]
>>
>> On Tue, 17 Jun 2025 at 12:49, Chris Hegarty
>> wrote:
>>
>>> Please vote for re
+ 1 SUCCESS! [0:54:41.406539]
On Tue, Jun 17, 2025 at 7:49 AM Chris Hegarty
wrote:
> Please vote for release candidate 1 for Lucene 9.12.2
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-9.12.2-RC1-rev-5a26234c834e4a33fd612ef8d13111ac0f0a1263
>
Heya Chris,
I *THINK* I have backported everything we need to 9.12.2, and 10.2.2.
Connected components disabling was done in 10.2.0 (note the commit tag):
https://github.com/apache/lucene/releases/tag/releases%2Flucene%2F10.2.0
As for Michael's test fix, that test doesn't exist in 9.12, so a bac
+1
SUCCESS! [1:03:30.451481]
On Mon, Apr 28, 2025 at 7:12 AM Dawid Weiss wrote:
>
> +1.
>
> SUCCESS! [0:56:17.417711]
>
>
>
> On Sat, Apr 26, 2025 at 10:59 AM Chris Hegarty
> wrote:
>
>> Please vote for release candidate 2 for Lucene 10.2.1
>>
>> The artifacts can be downloaded from:
>>
>> htt
+ 1
SUCCESS! [1:02:54.111488]
On Wed, Apr 9, 2025 at 1:14 PM Tomás Fernández Löbbe
wrote:
> +1
> SUCCESS! [1:07:37.187555]
>
> On Tue, Apr 8, 2025 at 8:02 PM Mayya Sharipova
> wrote:
>
>> +1 SUCCESS! [0:39:47.154199]
>>
>> On Tue, Apr 8, 2025 at 11:29 AM Robert Muir wrote:
>>
>>> issue for Uw
g it, maybe
> for 10.2 is safer to just disable it.
>
>
>
> On Thu, Apr 3, 2025 at 2:28 PM Michael Sokolov wrote:
> >
> > It makes sense to me. I think it's providing marginal benefits, and the
> downside is bad
> >
> > On Thu, Apr 3, 2025, 4:58 AM Benj
o a bugfix release there...but that is a discussion for a separate thread.
Thanks!
On Thu, Apr 3, 2025 at 7:55 AM Benjamin Trent wrote:
> Hey y'all,
>
> Unless there is strong dissenting opinion, I think we should revert the
> connected components work in HNSW for 10.2 a
Hey y'all,
Unless there is strong dissenting opinion, I think we should revert the
connected components work in HNSW for 10.2 as a bug fix.
https://github.com/apache/lucene/pull/14411
We found that when "connectedComponents" is most needed (e.g. a very
disconnected graph), it takes an inordinate
Hey y'all,
We ran into a strange postings merge error in production and I am not sure
if its how Elasticsearch is handling the merges or if it's a bug in Lucene.
The FST compiler reaches the "merge" line when merging some segments:
if (lastInput.length() == input.length && prefixLenPlus1 == 1
Welcome Michael!
On Thu, Mar 6, 2025 at 8:43 AM Michael McCandless
wrote:
> Welcome to another Michael!
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Thu, Mar 6, 2025 at 7:26 AM Mikhail Khludnev wrote:
>
>> Welcome, Michael!
>>
>> On Thu, Mar 6, 2025 at 12:52 PM Stefan Vodita
Good reminder Dawid!
That failure is mine. I will open an issue. It is likely a test set data
error.
On Tue, Feb 4, 2025 at 4:05 PM Dawid Weiss wrote:
>
> Hey, everyone,
>
> I'd like to remind folks that there is this mailing list:
> bui...@lucene.apache.org
>
> with archives at:
> https://list
gt; +super(wrapReader(core, r));
>
> On Thu, Jan 23, 2025 at 6:49 AM Benjamin Trent
> wrote:
>
>> From the vector search side of things, nothing immediately pops up as a
>> cause. https://lucene.apache.org/core/9_11_0/changes/Changes.html
>>
>> The given qu
>From the vector search side of things, nothing immediately pops up as a
cause. https://lucene.apache.org/core/9_11_0/changes/Changes.html
The given query is just a regular kNN query. So, its rewrite should behave
similarly as it did in 9.10.
One significant change for kNN search behavior did hap
+1 SUCCESS! [0:52:48.847123]
Thank you again Luca! Maybe two is the lucky number!
On Tue, Dec 17, 2024 at 3:01 PM Sami Siren wrote:
> +1 SUCCESS! [0:36:00.846742]
>
> ti 17.12.2024 klo 18.52 Luca Cavanna (java...@apache.org) kirjoitti:
> >
> > Please vote for release candidate 2 for Lucene 10.1
SUCCESS! [1:02:28.138493]
+1
Thank you Luca!
On Mon, Dec 16, 2024 at 12:20 PM Michael Sokolov wrote:
> SUCCESS! [1:54:16.305613]
>
> +1
>
> On Mon, Dec 16, 2024 at 4:32 AM Ignacio Vera wrote:
> >
> > SUCCESS! [1:16:40.906176]
> >
> > +1
> >
> > On Mon, Dec 16, 2024 at 9:05 AM Luca Cavanna wr
SUCCESS! [0:55:27.954415]
+1
Thank you Chris!
On Mon, Dec 9, 2024 at 2:16 PM Chris Hegarty
wrote:
> Please vote for release candidate 1 for Lucene 9.12.1
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-9.12.1-RC1-rev-7a97a05a239d6fb9f1f347aa09
SUCCESS! [0:56:23.261438]
+ 1!
On Thu, Oct 10, 2024 at 1:16 PM Dawid Weiss wrote:
>
> SUCCESS! [1:20:05.964755]
>
> +1.
>
> I've reviewed the artifacts, ran Luke, etc. Nothing caught my attention other
> than the NOTICE.txt file -
>
> Apache Lucene
> Copyright 2001-2022 The Apache Software Foun
+1 SUCCESS! [0:48:25.853452]
On Mon, Oct 7, 2024 at 8:31 AM Stefan Vodita wrote:
>
> +1 SUCCESS! [0:40:01.995275]
>
> On Sat, 5 Oct 2024 at 11:36, Luca Cavanna wrote:
>>
>> Please vote for release candidate 3 for Lucene 10.0.0
>>
>> I published a draft of the release notes at
>> https://cwiki.a
+1 SUCCESS! [0:56:38.403983]
On Thu, Oct 3, 2024 at 5:51 AM Stefan Vodita wrote:
>
> +1 SUCCESS! [0:39:04.597088]
>
>
> On Thu, 3 Oct 2024 at 07:48, Luca Cavanna wrote:
>>
>> Please vote for release candidate 2 for Lucene 10.0.0
>>
>> I published a draft of the release notes at
>> https://cwiki
+1 (binding)
SUCCESS! [0:43:49.760748]
Let's go Chris!
Mike's laptop is more than 2x faster than mine?!?!? Dang, do I need an upgrade?
On Wed, Sep 25, 2024 at 2:57 PM Jan Høydahl wrote:
>
> +1 (binding)
>
> Just ran smoke tester
>
> SUCCESS! [1:05:22.107463]
>
> Jan
>
> > 25. sep. 2024 kl. 18:
is small
> and merges are disabled, it's hard to imagine how 40 GB could've been
> consumed.
>
> Best,
> Gautam Worah.
>
>
> On Wed, Jun 12, 2024 at 9:42 AM Benjamin Trent wrote:
>>
>> Michael,
>>
>> Empirically, I am
SUCCESS! [0:40:46.898514]
+1
On Mon, Jun 24, 2024 at 1:29 AM Ignacio Vera wrote:
>
> Please vote for release candidate 1 for Lucene 9.11.1
>
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-9.11.1-RC1-rev-0c087dfdd10e0f6f3f6faecc6af4415e671a9e69
Anand,
In short, I think it's feasible, but I don't think it's simple. I also
don't think Lucene should directly provide an interface to the format
that says "Give me the graph". You could have a custom writer that
does this however.
All formats are nominally based, so if your GPU merge format wr
gary of the merge process happening
> to tip over a limit. It's also possible I messed something up in
> https://github.com/apache/lucene/pull/13469 which I am trying to use
> in order to index quantized vectors without building an HNSW graph.
>
> On Wed, Jun 12, 2024 at 10:24 A
Heya Michael,
> the first one I traced was referenced by vector writers involved in a merge
> (Lucene99FlatVectorsWriter.FieldsWriter.vectors). Is this expected?
Yes, that is holding the raw floats before flush. You should see
nearly the exact same overhead there as you would indexing raw
vector
The Lucene PMC is pleased to announce the release of Apache Lucene 9.11.0.
Apache Lucene is a high-performance, full-featured search engine library
written entirely in Java. It is a technology suitable for nearly any
application that requires structured search, full-text search, faceting,
nearest-
It's been >72h since the vote was initiated and the result is:
+1 12 (11 binding)
0 0
-1 0
This vote has PASSED
Thanks!
Ben Trent
On Thu, Jun 6, 2024 at 12:27 AM Patrick Zhai wrote:
> +1
>
> SUCCESS! [1:01:30.064666]
>
> On Wed, Jun 5, 2024 at 11:08 AM Houston Putman wrote:
>
>> +
Please vote for release candidate 1 for Lucene 9.11.0
The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-9.11.0-RC1-rev-d433394b292e3562e0bb34222f7dd4f307e2b8ca
You can run the smoke tester directly with this command:
python3 -u dev-tools/scripts/smokeTest
29, 2024 at 12:45 AM Stefan Vodita
> wrote:
>
>> Ben, I just merged #13414 <https://github.com/apache/lucene/pull/13414>,
>> so it's not a blocker for the release.
>> Thanks again for volunteering to be release manager!
>>
>> Stefan
>>
>
gt;
> > +1 the 9.11 changelog looks great!
> >
> > On Tue, May 14, 2024 at 4:50 PM Benjamin Trent
> wrote:
> > Hey y'all,
> >
> > Looking at changes for 9.11, we are building a significant list. I
> propose we do a release in the next couple of weeks.
Hey y'all,
Looking at changes for 9.11, we are building a significant list. I propose
we do a release in the next couple of weeks.
While this email is a little early (I am about to go on vacation for a
bit), I volunteer myself as release manager.
Unless there are objections, I plan on kicking of
Hey y'all,
I am confused about when we should supply a new format name (e.g.
Lucene911... vs. Lucene99) versus using a new metadata header version
(incrementing VERSION_CURRENT).
Are there general rules to follow?
At first glance, using a new Lucene format name prefix is functionally the
same as
urn lists all releases available at:
> https://archive.apache.org/dist/lucene/java/
>
> version 9.10.1 isn't there so it complains with:
> RuntimeError: tested version=9.10.1 but it was not released?
>
> I'm not sure how to deal with yet-unreleased minor versions myse
This seems related to us forgetting to make the back-compat indices &
versions when 9.10.1 was released and me adding them later.
I have since added the 9.10.1 to Version.java and version.txt in main and
9x. Now, both main and 9x have the back-compat indices (these changes were
not at the same tim
This is me. We missed the 9.10.1 version in the 9x branch and the main
branch. So, I added it. But, obviously, I didn't think about generating all
the bwc indices that we didn't generate when that release was pushed.
We can remove it, I would just need to adjust some new BWC tests I added
that wer
+1
On Fri, Feb 23, 2024 at 8:54 AM Adrien Grand wrote:
> +1
>
> On Fri, Feb 23, 2024 at 12:54 PM Uwe Schindler wrote:
> >
> > Here is my +1
> >
> > Uwe
> >
> > Am 23.02.2024 um 12:24 schrieb Chris Hegarty:
> > > Hi,
> > >
> > > Since the discussion on bumping the Lucene main branch to Java 21 i
+1
SUCCESS! [0:47:01.998711]
And I verified via a local monster test that this bug is fixed:
https://github.com/apache/lucene/pull/13027
I need to contribute back the monster integration test to fully exercise
that code path.
Thanks Chris!
On Thu, Jan 25, 2024 at 11:01 AM Michael McCandless <
SUCCESS! [1:06:02.232333]
+ 1!
On Wed, Dec 13, 2023 at 3:26 PM Greg Miller wrote:
> SUCCESS! [2:27:01.875939]
>
> +1
>
> Thanks!
> -Greg
>
> On Wed, Dec 13, 2023 at 3:58 AM Chris Hegarty
> wrote:
>
>> And (short) release note:
>>
>> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseN
SUCCESS! [0:44:05.132154]
+1
On Thu, Nov 30, 2023 at 1:09 PM Chris Hegarty
wrote:
> Please vote for release candidate 2 for Lucene 9.9.0
>
>
> The artifacts can be downloaded from:
>
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-9.9.0-RC2-rev-06070c0dceba07f0d33104192d9ac98ca16fc500
SUCCESS! [0:47:11.013106]
+1
On Thu, Nov 30, 2023 at 7:16 AM Ignacio Vera wrote:
> SUCCESS! [0:52:59.891964]
>
>
> +1
>
> On Thu, Nov 30, 2023 at 12:42 PM Michael McCandless <
> luc...@mikemccandless.com> wrote:
>
>> +1 to release.
>>
>> I hit a corner-case test failure and opened a PR to fix i
+1 9.9 will be a stellar release!
Thank you Chris!
On Tue, Nov 21, 2023 at 7:31 AM Adrien Grand wrote:
> +1 9.9 has plenty of great changes indeed! Thanks for volunteering as a
> RM, Chris.
>
> It would be good to try and fix the PKLookup regression that was
> introduced since 9.8:
> http://peo
Hey Michael,
In short, it's being worked on :).
Could you point to the LinkedIN post? Is Nils talking about the model
output quantized output or that their default output is easily compressible
because of how the embeddings are built?
I have done a bad job of linking back against that original i
TL;DR, forcing non-committers to squash things is a good idea. Enforcing
through some measure for committers is a bad idea.
Since this thread is now in Robert's spam, I am guessing it won't have any
impact :). I do not think Robert is actively trying hurt the project in any
way. It seems to me tha
Heya Patrick,
What version of Lucene Util are you using? There was a bug where
`forceMerge` was not actually using your configured maxConn & beamWidth.
See: https://github.com/mikemccand/luceneutil/pull/232
Do you have that commit and rebuilt the KnnGraphTester?
On Wed, Oct 11, 2023 at 10:10 AM
edges compared to the
> exact relative neighborhood graphs, allowing controlling the number of the
> connections which is important for search performance.
> ```
>
> On 2023/08/23 16:07:55 Benjamin Trent wrote:
> > Nitiraj,
> >
> > Good experimentation! Connectedness
Nitiraj,
Good experimentation! Connectedness within layers is indeed important. The
algorithm itself should ensure connectedness of disjoint NSWs as it
mutually connects nodes (selected over diversity).
However, if the data is extremely clustered, this can cause connectedness
to drop (few densely
My vote is for option 3. Prevents Lucene from having the limit increased.
Allows others who implement a different codec to set a limit of their
choosing.
Though I don't know the historical reasons for putting specific
configuration items at the codec level. This limit is performance related
and va
+1 !
You rock Alan!
On Wed, Apr 19, 2023, 9:54 AM Ignacio Vera wrote:
> +1
>
> Thanks Alan!
>
> On Wed, Apr 19, 2023 at 1:27 PM Alan Woodward
> wrote:
>
>> Hi all,
>>
>> It’s been a while since our last release, and we have a number of nice
>> improvements and optimisations sitting in the 9x b
>From all I have seen when hooking up JFR when indexing a medium number of
vectors(1M +), almost all the time is spent simply comparing the vectors
(e.g. dot_product).
This indicates to me that another algorithm won't really help index build
time tremendously. Unless others do dramatically fewer v
Hey y'all!
This is truly an honor!
Well, I am Ben Trent and have been writing code for over a decade now.
Which I know is not a very long time compared to most folks. I originally
wanted to do research and work in pure mathematics (my baccalaureate), but
quickly realized I am nowhere near smart e
51 matches
Mail list logo
