Hi, Apache Solr only supports fixed-length vectors. So you need to define the cardinality for your field and that's it. You can't pass variable length vectors. Each document, for the field, needs to have that vector dimension.
Cheers -------------------------- *Alessandro Benedetti* Director @ Sease Ltd. *Apache Lucene/Solr Committer* *Apache Solr PMC Member* e-mail: a.benede...@sease.io *Sease* - Information Retrieval Applied Consulting | Training | Open Source Website: Sease.io <http://sease.io/> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter <https://twitter.com/seaseltd> | Youtube <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github <https://github.com/seaseltd> On Fri, 12 Jul 2024 at 05:42, Derrick Cui <derrick...@gmail.com> wrote: > hi, > > I am planning to use DenseVectorField feature in solr 9.6.1, but I don't > know how to define it properly. > > for example, I have 1000 documents, I try to generate embedding for each > document from openAI, but the length is different for each document. > when I try to save it to Solr, I will get an error message, please help > org.apache.solr.common.SolrException: incorrect vector dimension. The > vector value has size 50688 while it is expected a vector with size 46080 > at > > org.apache.solr.util.vector.DenseVectorParser.checkVectorDimension(DenseVectorParser.java:104) > ~[?:?] > at > > org.apache.solr.util.vector.DenseVectorParser.parseIndexVector(DenseVectorParser.java:59) > ~[?:?] > at > > org.apache.solr.util.vector.DenseVectorParser.parseVector(DenseVectorParser.java:45) > ~[?:?] > at > > org.apache.solr.util.vector.FloatDenseVectorParser.getFloatVector(FloatDenseVectorParser.java:34) > ~[?:?] > at > > org.apache.solr.schema.DenseVectorField.createField(DenseVectorField.java:266) > ~[?:?] > at > > org.apache.solr.schema.DenseVectorField.createFields(DenseVectorField.java:230) > ~[?:?] > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:67) > ~[?:?] > at > > org.apache.solr.update.DocumentBuilder.addOriginalField(DocumentBuilder.java:304) > ~[?:?] > at > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:193) > ~[?:?] > > -- > Regards, > > Derrick Cui > Email: derrick...@gmail.com >
