Hi Uwe
Thank you very much for confirming the code!
Yes, I only set it for the IndexWriter, but what I meant to ask was,
what if the default Codec gets updated
and I will update my implementation, then I guess I will have to reindex
from scratch, right?
Or can I assume that the default Codec
Hi Michael,
The version below looks correct. Of course the Solr version is able to
do much more. The code you posted limits it to the bare minimum:
* subclass default codec
* implement getKnnVectorsFormatForField() and return the wrapper with
other max dimension
Reading indexes still wor
Hi Uwe
Thanks again for your feedback, I got it working now :-)
I am using a simplified version, which I will post below, such that it
might help others, at least as long as this implementation makes sense.
Btw, when a new version of Lucene gets released, how do I best find out
that "Lucene
Hi Uwe
Thank you very much for your quick feedback, really appreciated!
Will change it as you describe below.
Thanks
Michael
Am 19.10.23 um 11:23 schrieb Uwe Schindler:
Hi Michael,
The max vector dimension limit is no longer checked in the field type
as it is responsibility of the codec
Hi Michael,
The max vector dimension limit is no longer checked in the field type as
it is responsibility of the codec to enforce it.
You need to build your own codec that returns a different setting so it
can be enforced by IndexWriter. See Apache Solr's code how to wrap the
existing KnnVec
I forgot to mention, that when using the custom FieldType and 1536
vector dimension does work with Lucene 9.7.0
Thanks
Michael
Am 19.10.23 um 10:39 schrieb Michael Wechner:
Hi
I recently upgraded Lucene to 9.8.0 and was running tests with
OpenAI's embedding model, which has the vector dim
